Map Storage Optimization

[sabinati]

i didn't give him the warning. i told him to fuck off, obviously that is enough.

and potjeh, i was under the impression that they already used caching for the pngs. are maps really a bandwidth problem? aren't the files only a few kilobytes each?

[Thurrok]

A few kb's, that's right. Multiply that by 1.2k players accessing them simultaneously and despair.

[Moogie]

Sorry, didn't mean to confuse you. I actually did receive a proper mod warning PM from Jackard for that post. I don't dispute it at all (I've already replied to it with an apology), I just think it's a really weird and harmless post to get my first warning from, lol.

Anyway, I've long suspected that the reason the minimap occasionally 'breaks' (i.e. stops loading altogether) or takes a long time to show up is because of their filesize. It does seem like an awful lot of data to be sending out to hundreds of people, even if individually they are relatively small. I think they could get away with some amount of compression, without compromising the quality of the images much at all. Could perhaps even use gifs?

[sabinati]

A few kb's, that's right. Multiply that by 1.2k players accessing them simultaneously and despair.

under 10mb, not really a huge problem i think. there is surely some optimization that could be done, whether through improved caching, smaller files (removing the textures and having only flat colors and outlines would help a lot), or better serving method. i think the method in the OP would probably offer worse performance for both client and server, but i think only loftar could really say.

[Sarge]

Well, everyone agrees that it is resource intensive as it is and not efficient, I think we have for quite a while. Rich coming from me knowing so little about this, but principle being that there seems to be suggestions/ideas on how to improve this could could reduce bandwith use, decrease the amount of repeated data download by each user and make minimaps & maps operate as efficiently as intended.

This would be a welcome 'fix' in my books.

[Nemerle]

Yes CPU cycles *are* in higher demand than RAM, but bandwidth is in higher demand than CPU cycles. That and unless frequently transmitted map data (not the minimap) is cached in RAM the queries on the database must have a significant overhead. All the more reason to use shared seed keys, cache client side, and only update the client cache when data changes server side.

Of course this presumes Jorb and Loftar wouldn't mind going through the headache of piecing together a usable world map. A simpler solution would be to break the map data into layers (ground, trees, rocks, plants, etc) apply run length encoding, transmit it to the client, and have the map reassembled client side for display.

[Potjeh]

I did say that maps should be cached client-side. Anyway, the only difference is in the initial transfer, updating works the same, and I just don't think that the savings on bandwith would be enough to justify the client-side CPU cost of generating maps from seed. Especially since a seed wouldn't work at all for maps with significant terraforming, so you'd basically have to transfer the whole map again after generating it to "patch" it to reflect the actual state of the game world.

[Goom]

My understanding was the Devs were going to look into revamping the map generator at some point, but it was quite low priority at the moment, especially with new world just started and their Salem project.