FreeSide On The Web

Part of Reboot! 2008 was to get some form of server monitoring installed and online, so we can work out whats what, rather than just poking in the dark.

So nearly, 2 months later, this is really only just starting to be properly implemented.

Phoenix made a start a couple of weeks ago, but then didnt have chance to finish it.

Putting a new server on nagios is certainly a treck round the houses (unfortunately, we cant just do a basic apt-get install nrpe nagiosplugins :()

So, after phoenix finally giving me some basic instructions on how to setup a box for monitoring, and also a link to the proper manual, i gave it a go

And it took ages! Loads of commands to enter, 2 guides to partially follow - Not Fun!

The logical answer, was therefore, to write a basic bash install script (to at least semi-automate the process). A bit of fiddling, and combining the 2 guides, has brought about a shell script that works

It is listed at http://wiki.freeside.co.uk/index.php/Admin:Nagios_Plugin_Install

So, copy and paste that into a script on the server, make it executable, and run it

Currently, unless something goes wrong, you’ll only need to enter 2 bits of detail, 1, being Y to accept the extra apt-get package installs, and the nagios user password (enter it twice)

It will then do all the client side configuration (to a basic form). Any other services to be monitored, must be manually added (see wiki for more details)

Then, when thats done, you’ve just got to hop onto gaia, using a copy of the basic client monitoring config, change the few needed parameters, paste into the config (as root), check it, and restart nagios

Give it a few minutes and you’ll have the server listed as monitored in Nagios

Jobs a good ‘un

Reedy

It seems that Reboot! 2008 has been and gone, leaving in it’s wake a trail of destruction and broken limbs … well, not really, but ‘we moved some shelves and tidied up’ doesn’t sound as exciting now, does it?

We all started to trickle in from around 9:30AM, some of us stopping off to fuel ourselves (see the pics at the end of the post).

A few of us were dispatched to assemble a rack, which was quickly dubbed ‘FailRack’ due to our inability to get it to actually resemble a rack … having completely ballsed that task up we moved onto noting down specs of the current servers and powering them down (which I’m pleased to report we managed to do correctly), moving them into the lecture theatre next door … most of this stuff worked, honest.

It was at this point we all decided we’d had enough and torched the lot broke for lunch … this may have been a somewhat extended lunch break, but hey, we totally deserved it!

After lunch we set about the easy task (I say easy because I was the one filming it) of moving the shelving units about. How these managed to withstand Hull’s recent earthquake we have no idea. With the the wobbly shelves in their new position to the left of the room, we started to move all of the machines back in and hook it all up, except for a few of the big-iron servers which Jonty and Wes seemed to be hitting with hammers (but not beans) in the lecture theatre.

After this came the networking cabinet (or was it before? I was jumped up on caffeine by this point), perhaps the most professional-looking part of the setup at the moment … which is saying something.

At the end of the day pretty much everything was working, we had managed to install a few new servers and the room looked a hell of a lot better … there was still the huge pile of broken machines around the corner, but that horror deserves an entry all of it’s own.

Huge thanks to those that came and gave up their weekend to make FreeSide rock (more) and … hey, anyone feel like doing it again next year? …

FreeSide have recently acquired a grand total of 27 P4 2.4GHz PC which have been “saved” from being disposed of. Many thanks to the department and all involved in making this happen.

With 27 machines (Even without memory) we can keep FreeSide running for a long time to come. However, with new machines come new ideas. It has already been mentioned by Jonty to create a Hadoop cluster (http://hadoop.apache.org/) a scalable distributed system. Sound cool, but we’ll see :).

From the new machines, we have put 2 more into use and finished installing a couple of others. These include gaia, the replacement for dagger, terra, the new FreeSide console machine, yxorp, which will soon be our new proxy and finally, bladeswitch has finally been completed as out redundent web load balancer.