Planned downtime: OS upgrade

[DDDsDD999]

And Sarchi, I don't think that would help haha

He's the expert, he saw a video.

[Kaoru]

IT IS NOT HARDWARE RELATED. he said that hardware additions or modifications could possibly help(possibly), but that the problem is in the programming. the root of the problem, which he has mentioned before is somewhere in the CODE, was only exacerbated in some way by the OS upgrade.

basically, for him to fix the problem, he is going to have to go back through the ENTIRE programming of the game and find the magical needle in the haystack that is causing the problem in the first place. he probably simply doesent have the time for it right now, what with salem and whatever else he has going on. its been said already that the problem is in the I/O taking too much time to write to disk and leaving the player's character to moonwalk or whatever for the duration.

that is the problem in a nutshell. the sudden change when he upgraded the OS CANNOT be caused by anything hardware related, and the more perminant problem of the lag in the first place, before the OS upgrade, is something that is going to take most likely a good long time to really fix.

if you dont know how to create this game over again, and program it from scratch, most likely, you DO NOT have the magical solution that will make our lag problems go away. while im sure he appreciates your attempts, they are not going to help... and regardless of anything, this problem is not going away any time soon... so basically, learn to deal with it, or dont play. its unfortunate, but life happens. i dont think anybody here really thinks he is going to abandon H&H, so rest assured in the knowledge that it will EVENTUALLY be fixed.

a note to people saying the game is completely unplayable: try and give it a good chance. commit yourself to playing for 1 hour. if you cant get used to it after that, you have given it a good old college try. I did, and while the lag is worse than before the upgrade, ofc, it is playable, if you have patience.

my point? the problem isnt going to be solved any time very soon, so might as well learn to deal.

[Frycu]

That was nicely said.

[loftar]

IT IS NOT HARDWARE RELATED. he said that hardware additions or modifications could possibly help(possibly), but that the problem is in the programming. the root of the problem, which he has mentioned before is somewhere in the CODE, was only exacerbated in some way by the OS upgrade.

basically, for him to fix the problem, he is going to have to go back through the ENTIRE programming of the game and find the magical needle in the haystack that is causing the problem in the first place.

Mind if I ask where you got that idea? :)

I know exactly where in my code the problem lies -- it is where I call the operating system to actually do I/O. What I don't know is why the operating system takes so long to complete it. What I would like most of all is some tool that could fetch me a backtrace of the requesting thread in kernel space while the syscall is in progress, so that I could find out what exactly the request is waiting on, and then find out what other process is holding that resource busy.

[Kaoru]

well now i feel like a jackass. i got that info from this very thread. i got my info from other people who got there info from other people i suppose. who knows where it originated, but i should have kept my nose out of it in the first place. my apologies.

::Edit::
my intention was to say that the person who has the biggest grasp on the issue is loftar, and unless somebody knows exactly how the server and programming works, they arent going to help the situation... but then again, i guess they cant hurt it either.

[TeckXKnight]

well now i feel like a jackass. i got that info from this very thread. i got my info from other people who got there info from other people i suppose. who knows where it originated, but i should have kept my nose out of it in the first place. my apologies.

::Edit::
my intention was to say that the person who has the biggest grasp on the issue is loftar, and unless somebody knows exactly how the server and programming works, they arent going to help the situation... but then again, i guess they cant hurt it either.

Don't fret about it too much, you're still more helpful than the great bulk of the posters in this thread. =)

[Nummy]

IT IS NOT HARDWARE RELATED. he said that hardware additions or modifications could possibly help(possibly), but that the problem is in the programming. the root of the problem, which he has mentioned before is somewhere in the CODE, was only exacerbated in some way by the OS upgrade.

basically, for him to fix the problem, he is going to have to go back through the ENTIRE programming of the game and find the magical needle in the haystack that is causing the problem in the first place.

Mind if I ask where you got that idea?

I know exactly where in my code the problem lies -- it is where I call the operating system to actually do I/O. What I don't know is why the operating system takes so long to complete it. What I would like most of all is some tool that could fetch me a backtrace of the requesting thread in kernel space while the syscall is in progress, so that I could find out what exactly the request is waiting on, and then find out what other process is holding that resource busy.

Go for it, find bastard for once and for all and get rid of a problem for good. If you don't have time, get some friends to help you. Salem is not as good as HnH, so some of us prefer it over Salem.

[HarryDresden]

This all seems to be a bit over my head, but I'll keep looking for an answer or a tool that can help you Loftar, however, I found this paper interesting at least, maybe it'll help a bit?

/noob

http://219.239.227.109/zhong/paper/vmm.pdf

[yllet]

Loftar, I assume you've already checked and poked most of the premade systemtap scripts out there (not to mention making your own) but I'm posting some anyway: http://sourceware.org/systemtap/example ... ex.html#IO

Have you tried moving the xfs journal to a device thats not under such a heavy load? (If you believe this might be the rootcause.)
There is afaik no longer any way to disable the journaling.
And would you please post an iostat with some lower time to average both when things are running smoothly and when the lagbeast shows up. (ex. iostat -x 5 )

echo w > /proc/sysrq-trigger
echo d > /proc/sysrq-trigger
Might also give some info during the io lockups.

While I might just have wasted a good few min of your time, atleast you know we care =P

[borka]

The server is a SHARED server on a farm somewhere, it's not even dedicated.

What gave you that idea?

Not looking for EX10 Yummy And also he might not know about the condition that you have to care for the OS yourself there ....

I know exactly where in my code the problem lies -- it is where I call the operating system to actually do I/O. What I don't know is why the operating system takes so long to complete it. What I would like most of all is some tool that could fetch me a backtrace of the requesting thread in kernel space while the syscall is in progress, so that I could find out what exactly the request is waiting on, and then find out what other process is holding that resource busy.

internal Journal log writing? no delayed allocation?

-snip-
Have you tried moving the xfs journal to a device thats not under such a heavy load? (If you believe this might be the rootcause.)
There is afaik no longer any way to disable the journaling.

There have been suggestions to use an extra ssd - just for internal Journal log writing