FreeSide On The Web

http://freeside.co.uk/~ben/ubuntu/

Be Happy

Woot

http://wiki.freeside.co.uk/wiki/File:PartyTiamat.jpg

What a faff that was.. Haha

Look in the wp_sitemeta table

You want the meta key: site_admins

Its serialised PHP, or something…
a:2:{i:0;s:5:”admin”;i:1;s:7:”phoenix”;}

a being amount, i being index (ie which admin user), and s being the length

so add me…
a:3:{i:0;s:5:”admin”;i:1;s:7:”phoenix”;i:2;s:5:”reedy”;}

Job’s a good ‘un

So, FreeSide is nice and clean (Thanks to M Bell for sorting cleaners and such )

Most of the servers are back online, but Eye has a new home

Hidden server is hidden

Pretty much the same state as it was i last visited, but we do have a wasp (albeit mainly dead) problem

Loads all over the floor, and on the sides. A few alive ones about

And there’s a vending machine where we had our pile of computers

1
2
3
4
5
6

As many of you will probably know, the Department and the Robert Blackburn building are undergoing refurbishment, and therefore access is restricted

Some electrical work needed to be done, therefore power was lost to FreeSide on friday afternoon. I was the first avaliable admin to be able to get in today to get it all back online

I was greeted by a pigeon inside today (see pictures), and then a very different looking FreeSide from that of when i last saw it.

Some servers were already online, but the key server for our external access – M25 wasnt (probably a job for later, is to get this to automatically reboot if possible)

Bit of fiddling and poking, and the majority are online again. Notable exceptions are eye and bladeswitch, which had been unplugged and moved. Chances are they’ll get moved again, and being none crucial, i left them.

I couldnt actually find Gaia, it was at the other side of the room, and had been unplugged and moved, meaning our logging server is now offline for a bit. Not to us – MOAR LABELLING. It was one of the old fenner beige boxes, i think i knew which it was, but access to power cables and such, it wouldnt have been easy to get it online again

We’ve got some huge-ass cables not going over a gantry in the roof, and it would seem, some new lighting, which is nice

Get back home again and the website are down. Quick look seems that the webserver twins must’ve booted before mnemosyne had finished and the WWW directories hadnt been mounted. Quick restart, and still no websites. Buggar. Cause was Switchblade – Perlbal wasnt running on it

Part of Reboot! 2008 was to get some form of server monitoring installed and online, so we can work out whats what, rather than just poking in the dark.

So nearly, 2 months later, this is really only just starting to be properly implemented.

Phoenix made a start a couple of weeks ago, but then didnt have chance to finish it.

Putting a new server on nagios is certainly a treck round the houses (unfortunately, we cant just do a basic apt-get install nrpe nagiosplugins )

So, after phoenix finally giving me some basic instructions on how to setup a box for monitoring, and also a link to the proper manual, i gave it a go

And it took ages! Loads of commands to enter, 2 guides to partially follow – Not Fun!

The logical answer, was therefore, to write a basic bash install script (to at least semi-automate the process). A bit of fiddling, and combining the 2 guides, has brought about a shell script that works

It is listed at http://wiki.freeside.co.uk/index.php/Admin:Nagios_Plugin_Install

So, copy and paste that into a script on the server, make it executable, and run it

Currently, unless something goes wrong, you’ll only need to enter 2 bits of detail, 1, being Y to accept the extra apt-get package installs, and the nagios user password (enter it twice)

It will then do all the client side configuration (to a basic form). Any other services to be monitored, must be manually added (see wiki for more details)

Then, when thats done, you’ve just got to hop onto gaia, using a copy of the basic client monitoring config, change the few needed parameters, paste into the config (as root), check it, and restart nagios

Give it a few minutes and you’ll have the server listed as monitored in Nagios

Jobs a good ‘un

Reedy

It seems that Reboot! 2008 has been and gone, leaving in it’s wake a trail of destruction and broken limbs … well, not really, but ‘we moved some shelves and tidied up’ doesn’t sound as exciting now, does it?

We all started to trickle in from around 9:30AM, some of us stopping off to fuel ourselves (see the pics at the end of the post).

A few of us were dispatched to assemble a rack, which was quickly dubbed ‘FailRack’ due to our inability to get it to actually resemble a rack … having completely ballsed that task up we moved onto noting down specs of the current servers and powering them down (which I’m pleased to report we managed to do correctly), moving them into the lecture theatre next door … most of this stuff worked, honest.

It was at this point we all decided we’d had enough and torched the lot broke for lunch … this may have been a somewhat extended lunch break, but hey, we totally deserved it!

After lunch we set about the easy task (I say easy because I was the one filming it) of moving the shelving units about. How these managed to withstand Hull’s recent earthquake we have no idea. With the the wobbly shelves in their new position to the left of the room, we started to move all of the machines back in and hook it all up, except for a few of the big-iron servers which Jonty and Wes seemed to be hitting with hammers (but not beans) in the lecture theatre.

After this came the networking cabinet (or was it before? I was jumped up on caffeine by this point), perhaps the most professional-looking part of the setup at the moment … which is saying something.

At the end of the day pretty much everything was working, we had managed to install a few new servers and the room looked a hell of a lot better … there was still the huge pile of broken machines around the corner, but that horror deserves an entry all of it’s own.

Huge thanks to those that came and gave up their weekend to make FreeSide rock (more) and … hey, anyone feel like doing it again next year? …

 

FreeSide have recently acquired a grand total of 27 P4 2.4GHz PC which have been “saved” from being disposed of. Many thanks to the department and all involved in making this happen.

With 27 machines (Even without memory) we can keep FreeSide running for a long time to come. However, with new machines come new ideas. It has already been mentioned by Jonty to create a Hadoop cluster (http://hadoop.apache.org/) a scalable distributed system. Sound cool, but we’ll see :).

From the new machines, we have put 2 more into use and finished installing a couple of others. These include gaia, the replacement for dagger, terra, the new FreeSide console machine, yxorp, which will soon be our new proxy and finally, bladeswitch has finally been completed as out redundent web load balancer.