Servers Ginger and Izzy down

[David I]

Both servers are cururrently running a fsck. Updates will follow.

[David I]

Ginger and Izzy servers are up now and working pretty fine.

[Matt]

Izzy took a digger again.

It looks as if the main drive is rapidly failing and needs to be replaced. Unfortunately there was no warning of pending drive failure and when the drive goes south, there is not much we can do but replace it. Normally we would do this in the middle of the night not during business hours on a Monday but the drive is completely dead so it needs to be replaced. Any of you who have replaced a primary hardrive and installed an operating system know that this process takes several hours to complete.

We are working on that now and when the machine is back online we will work on restoring our backups for you.

[sergey]

The restore process on Izzy is under way.

[sergey]

The restoration has completed.

[jmarcv]

Note:
It appears that mysql was configured to use new style passwords only, where before it ad the old style directive in place.
You may experience login problems in your scripts due to the old style encrypted passwords stored in the DB not matching the new style being passed by MySQL now.