Planned downtime: OS upgrade

[irdoom88]

it might be intentional, or if not lets hope it generates a report of some type.

[strpk0]

this could be good, maybe not, we shall see.

inb4w7
Lol jk, don't wanna get banned

[toshirohayate]

this could be good, maybe not, we shall see.

inb4w7
Lol jk, don't wanna get banned

Lol, you won't get banned for bringing it up. You just can't request it, or go off topic about how it should happen.

[Berdy]

The first restart had similar lag, probabbly even worse. But Loftar fixed it by running on linux 3.2


=some of the more important Quotes for you people who don't care to look through the comments=

This is the first time I've ever rage quit from lag. Logged on, spent 15 minutes trying to put in my curios and walk out of my house. GG server -.-

For some reason, the upgrade seems to have wasted I/O performance completely (as if it weren't bad enough before).

My initial thought is to blame the kernel in Debian 6.0, because I started experiencing similar problem on my own home server after having upgraded it to the same kernel. I will try the kernel from Testing, but need to reboot again for that.

A little problems with the website and Php.

Saw this at the top of page after viewing this post and new post XD
[phpBB Debug] PHP Notice: in file /includes/auth/auth_haven.php on line 11: dba_open(/srv/haven-data/web/sessions.db,c): Driver initialization failed for handler: db4: Invalid argument
[phpBB Debug] PHP Notice: in file /includes/auth/auth_haven.php on line 12: dba_exists() expects parameter 2 to be resource, boolean given

Always a pleasure to have to deal with PHP.

So there, running again, on Linux 3.2. Let's hope that does away with the problem.

This fixed it, and there was little lag after this. But....

After this most recent restart, i was one of the first people back online, There was vitually no lag at all, Better then i have seen in over a year


And then when it hit 100-150+ The lag came back, and was worse then it was before the OS upgrade

P.S. ;D

just a reminder about the rules:

• Stop requesting character wipes or map resets. They will only ever be introduced alongside major developments and there are no signs of those developments happening anytime soon. You have no say in the matter.

consider this your warning.

Edit: Look below if you think you can help (:

[loftar]

Is you game server app multithread?

Nope.

Did you try deadline scheduler?

Yes, but I can't say I notice much of a practical difference. The lag is sometimes distributed somewhat differently, but it doesn't seem to actually worsen or better it.

iostat -x 30

Sure; this is what it might look like during particularly bad lag:

Code: Select all

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           2.24   11.31    0.78   24.37    0.00   61.30

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sdb               0.80    37.97    7.23  124.73   965.33  3189.90    31.49     4.64   36.81   2.41  31.75
sda               0.87    38.03    3.63  138.00   417.07  4680.67    35.99    68.38  491.85   5.53  78.39
md0               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
md1               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
md2               0.00     0.00    6.83  121.77   481.60  2178.20    20.68     0.00    0.00   0.00   0.00
dm-0              0.00     0.00    2.30   47.57    28.00   379.73     8.18    22.41  451.41  14.47  72.13
dm-1              0.00     0.00    5.63   23.90   899.47  1729.63    89.02    13.59  464.62  19.21  56.72
dm-2              0.00     0.00    4.53   73.47   453.60  1798.20    28.87   239.20 3523.69   7.49  58.43


Notice, particularly, the absurdly high waiting times on dm-2, which is the LV holding the game data. It, in turn, is backed by md2, which is a md-RAID mirror backed by sda and sdb. dm-0 is the root filesystem, backed by the same VG as dm-2, and dm-1 is on a separate VG (backed by sda and sdb directly, without redundancy) and holds the minimap data. All filesystems are XFS. I suspect the lag spikes happen because of either XFS logging flushing, or by XFS' interaction with sync()-ing, but I have no hard proof. The practical effect, either way, is that some process (like the game server) might have to wait sometimes upwards of 10-20 seconds for some simple I/O operation like an open(), rename() or close() to complete.

mpstat -P ALL 30

I don't think it's going to tell you a whole lot, but here you go:

Code: Select all

Linux 3.2.0-0.bpo.2-amd64 (moltke)      06/09/2012      _x86_64_        (8 CPU)

08:41:17 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest   %idle
08:41:47 PM  all    1.82   12.49    0.47   21.44    0.00    0.46    0.00    0.00   63.32
08:41:47 PM    0    4.06    0.00    0.93   57.54    0.00    3.59    0.00    0.00   33.88
08:41:47 PM    1    3.50    0.00    0.83   39.35    0.00    0.03    0.00    0.00   56.28
08:41:47 PM    2    0.00  100.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
08:41:47 PM    3    2.76    0.00    0.80   22.78    0.00    0.03    0.00    0.00   73.63
08:41:47 PM    4    1.67    0.00    0.43   23.88    0.00    0.03    0.00    0.00   73.99
08:41:47 PM    5    1.00    0.00    0.47    5.20    0.00    0.00    0.00    0.00   93.33
08:41:47 PM    6    0.13    0.00    0.03    0.00    0.00    0.00    0.00    0.00   99.83
08:41:47 PM    7    1.40    0.00    0.27   22.71    0.00    0.03    0.00    0.00   75.59


You can note, of course, the high iowait times, but that should come as no particular surprise to anyone. :)

[Kaoru]

note: this is not my actual account. it seems every time i make an account, it is eaten by the internet darkness... tried everything, but it doesent seem to have an answer. (still able to log in to the game though)

just wanted to tell you, loftar, thank you very much for h&h, and your continued dedication to the game its unlike anything else.

[irdoom88]

and just going off what everyone else is saying about W7...I personally like the way it is at the moment, i made a random alt to travel around on via boat and what not and its kinda interesting seeing what has been going on around the world, you just see lots of ruins and stuff

[Guattro]

Anyone else getting disconnected?
A minute or two after logging in, I get disconnected, it's the first time I experience a problem like this.
Help!

[Zluka52]

1) you have some disbalance on sda and sdb, i think from dm-1 (don't split read/write (data) on both disk)
2) did you test ext4 for this project?
3) how many files/dirs with data?
4) on disk data size?
5) mem on server?
6) in memory server process size?
7) did you defragment xfs with xfs_fsr?

[Kaoru]

yes guattro, it is being looked at. if you are still having the problem after the server is fixed, then complain.