Downtime: Data corruption, Round 2

[Granger]

I wonder: why not invest into a UPS (=uninterruptible power supply)?

Because they cost more than I think we have donations to spare (though Jorb knows that better than I do).

Then re-read this:

Of course, it isn't actually the machine that crashes, but the game server program itself. Just for the record, if it weren't for the power outage the other day, the machine would've had a lot more uptime that than.

My offer to donate you a working APC SmartUPS (you'll need new batteries for it though to keep shipping costs reasonable) still stands.

3rd and last time i'm posting this.

Batteries are ~40€ (2x 12V 7Ah) and should be available locally. And it also ties nicely into Linux so system can perform clean shutdown when main power goes away to long.

[loftar]

Batteries are ~40€ (2x 12V 7Ah) and should be available locally. And it also ties nicely into Linux so system can perform clean shutdown when main power goes away to long.

Are you really sure? Every time I've been looking around, batteries have seemed to be about as expensive as a new UPS. What battery model would fit? Or is it possible to have the batteries external to the UPS itself (in that case, I can see that it would be possible to find cheaper batteries)?

[Lothaudus]

The only reason is really with I/O bandwidth. Taking a backup requires quite some of it, which makes the game laggy, and therefore I try to do the manually at times when the load is low. The "manually" part of course requires me to remember to do them also, which is the second part of the problem.

Take the server offline for scheduled down-time every day for an hour and do the backup then (or once every 2 or 3 days). EVE Online does something similar with daily down-time at a regular time. Makes it fairly easy to work and plan around. May piss off some people in certain time zones.

[Granger]

Batteries are ~40€ (2x 12V 7Ah) and should be available locally. And it also ties nicely into Linux so system can perform clean shutdown when main power goes away to long.

Are you really sure? Every time I've been looking around, batteries have seemed to be about as expensive as a new UPS. What battery model would fit? Or is it possible to have the batteries external to the UPS itself (in that case, I can see that it would be possible to find cheaper batteries)?

http://www.mtk-handel.de/rbc5-kompatibler-akkusatz-p-41.html?ref=3

Will last at least 2 years, longer if you have low room temperature.

[Granger]

The most recent backup was 4 days ago? What keeps you from having daily backups?

The only reason is really with I/O bandwidth. Taking a backup requires quite some of it, which makes the game laggy, and therefore I try to do the manually at times when the load is low. The "manually" part of course requires me to remember to do them also, which is the second part of the problem.

Then put some of the donations into a SSD as main disk (to get throughput and IOPS to keep game server served), use LVM snapshots (or ZFS clones or whatever) to freeze a filesystem copy and then rsync it to a normal RAIDed volume for backup (using hardlinks against an old backup will offer multiple generations cheaply).

EDIT: typo fixed

[loftar]

Are you really sure? Every time I've been looking around, batteries have seemed to be about as expensive as a new UPS. What battery model would fit? Or is it possible to have the batteries external to the UPS itself (in that case, I can see that it would be possible to find cheaper batteries)?

http://www.mtk-handel.de/rbc5-kompatibler-akkusatz-p-41.html?ref=3

Will last at least 2 years, longer if you have low room temperature.

In that case, I would like to gracefully accept. I have to check if they are available some place local first, though.

[Granger]

The only reason is really with I/O bandwidth. Taking a backup requires quite some of it, which makes the game laggy, and therefore I try to do the manually at times when the load is low. The "manually" part of course requires me to remember to do them also, which is the second part of the problem.

Take the server offline for scheduled down-time every day for an hour and do the backup then (or once every 2 or 3 days). EVE Online does something similar with daily down-time at a regular time. Makes it fairly easy to work and plan around. May piss off some people in certain time zones.

Given the scale of H&H (single server) an actual downtime of ~1 minute will do it (to create a clone/snapshot/whatever), actual backup can then be niced and done in background.

[jaydee0004]

Our high quality farm has been reset to the lower quality and it went to the begininng :'(

Please do that daily backup. Like just one hour per day just to prevent dreadful rollbacks :'(

[loftar]

Then put some of the donations into a SSD as main disk (to get throughput and IOPS to keep game server served), use LVM snapshots (or ZFS clones or whatever) to freeze a filesystem copy and then rsync it to a normal RAIDed volume for backup (using hardlinks against an old backup will offer multiple generations cheaply).

Well, true enough, there are ways around it. I haven't exerted any efforted for it, however, since I've mainly just kept a backup around for the utmost disaster of a hard drive failure, though. I hadn't expected XFS to regularly corrupt data on power failures. Reasonably, I guess I should switch back to ext3 instead (I used XFS on the new hard drive mainly for its ability to be extended on-line, and because I find it performs rather well).

[nossr50]

I lost my hearth fire
I'm guessing this is normal?
Kind of sucks because I put a lot of work into the area I claimed