Planned downtime: Filesystem test (2019-04-11 22:00 UTC)

[loftar]

As a next installment in the lagspike saga, I'm going to do a little experiment tomorrow night (see subject for exact date). I'm not sure whether it will help or not, but it might at least provide some debugging data. To be more precise, I want to try putting the game data on an ext4 filesystem (rather than XFS, as currently), so the experiment involves copying all the game data over, so that's what's going to take some time. It shouldn't be too bad, though; the data transfer itself should take under half an hour. There's some administrivia to be performed as well as part of the operation, so I'd expect it to take 30-60 minutes in toto.

[shubla]

Old and proven technologies surprisingly work!

[MagicManICT]

Thanks for the heads up. At least I can make sure I don't log in 3 minutes before the downtime like I've done every week for the last month.

[Omnipotent]

Kinda spooked but glad to hear you guys are trying something. Praying for the best!

[loftar]

Old and proven technologies surprisingly work!

Which technology is it that you refer to as "old and proven" in this case? I'd say both ext4 and XFS are pretty old and proven.

[shubla]

Old and proven technologies surprisingly work!

Which technology is it that you refer to as "old and proven" in this case? I'd say both ext4 and XFS are pretty old and proven.

I think there was some discussion about some new cool filesystem that was being used earlier but maybe I remember wrong.

[Granger]

I think there was some discussion about some new cool filesystem that was being used earlier but maybe I remember wrong.

The discussion was about bcache, which isn't a filesystem but a caching layer (when given drives of different speed characteristics).
Regarding a cool filesystem I tried to pitch the use of ZFS to loftar a while back.

@loftar: good luck, please keep up posted on the results. Though I must admit that I don't see a reason why switching to ext could help.
Fun fact: had to discard an otherwise intact SSD (Intel datacenter class) today as writes to it took 240ms on average...

[loftar]

Though I must admit that I don't see a reason why switching to ext could help.

I've generally gotten the feeling that XFS behaves a bit weird on something. Like, when one program does a [f]sync operation that takes some time, that seems to very easily influence unrelated I/O operations in other processes, in ways that I haven't seen on ext4. Therefore, I just want to compare and see what happens. It vaguely seems to me that XFS has some kind of global bottleneck causing that (though it does sound weird, as I know XFS has separate parallel allocation pools precisely to avoid that kind of thing; I dunno).

[Headchef]

Let's hope it results in progress and the downtime stays as mentionned... Good luck

[borka]

Honestly bad time of day choosen to do such ... but good to try with Ext4