Chunk Unloading

[HeroOfTheWinds]

After a while of making mods that alter the map (CaveRealms, SkyLands, my unreleased biome map loader), I realized there is one thing Minetest is very inefficient with (imho): Memory. Constantly, Minetest will crash with the error "Out of Memory" after loading mapchunks for several minutes.

So I propose an idea: have Minetest either have an arbitrary (and configurable) MAX_LOADED_CHUNKS setting or else monitor RAM usage, and when benchmarks are reached, unload chunks that are either the least recently loaded or the farthest from current players. On a technical level, this is nowhere near impossible, unless there is no way to force unload chunks. If this could be implemented, it would be a wonderful help to both modders and server owners.

Just a couple thoughts.

[Krock]

Minetest will crash with the error "Out of Memory" after loading mapchunks for several minutes.

Look at those settings:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

active_block_range = 1
active_object_send_range_blocks = 1
client_unload_unused_data_timeout = 300
server_unload_unused_data_timeout = 30

[Morn76]

Recently I tried to reduce MT's memory hogging in singleplayer with jp's NYC map. I played with client_unload_unused_data_timeout, but that did not help at all. Chunks still only got unloaded after 10 minutes, which is way too slow when flying over a large map.

So +1 from me for dynamic chunk loading limits based on current RAM usage. MT using 4 GB RAM or more is a bit crazy as default behavior IMO.

[Wuzzy]

I agree with this proposal. The settings Krock mentioned have a problem because these are static limits so this is not an alternative. The problem is that these settings don’t at all take the used memory into account.
Dynamic memory management sounds like the way to go. I think taking the actual used memory into account is more reasonable than just the number of loaded chunks, because you wouldn’t need to play around with the settings for long until you find a reasonable value. You know how much memory your system has, but you can’t really know how many chunks you need to run out of memory. Well, unless the size of chunks is always the same (I don’t know, sorry).

If Minetest can really eat 4 GB RAM, then this sounds like a serious problem.

[webdesigner97]

I agree with this proposal. The settings Krock mentioned have a problem because these are static limits so this is not an alternative. The problem is that these settings don’t at all take the used memory into account.
Dynamic memory management sounds like the way to go. I think taking the actual used memory into account is more reasonable than just the number of loaded chunks, because you wouldn’t need to play around with the settings for long until you find a reasonable value. You know how much memory your system has, but you can’t really know how many chunks you need to run out of memory. Well, unless the size of chunks is always the same (I don’t know, sorry).

If Minetest can really eat 4 GB RAM, then this sounds like a serious problem.

+1 I also experienced this problem on the NYC map. (4gb RAM)

[Sokomine]

That's a bit odd, considering how small the chunks are.

What I experience a lot with view set to far (machine is a Celeron G1610 with 8 GB RAM and integrated graphics) is that fps more or less collapse when I look at faraway places where not all the chunks inbetween have arrived/been rendered yet. When suddenly the chunks in front of the camera are rendered and block part of the view, fps recover more or less immediately, and it becomes playable again.

When playing on a server,chunks arrive so slowly that memory on the client side cannot be a consideration. Unless it's a phone/tablet/antique computer with too few memory anyway.

[HeroOfTheWinds]

That's a bit odd, considering how small the chunks are.

What I experience a lot with view set to far (machine is a Celeron G1610 with 8 GB RAM and integrated graphics) is that fps more or less collapse when I look at faraway places where not all the chunks inbetween have arrived/been rendered yet. When suddenly the chunks in front of the camera are rendered and block part of the view, fps recover more or less immediately, and it becomes playable again.

When playing on a server,chunks arrive so slowly that memory on the client side cannot be a consideration. Unless it's a phone/tablet/antique computer with too few memory anyway.

That odd-ness is also part of the reason why this should be brought to attention. Perhaps there's a memory leak that just makes it seem as if chunk loading is at fault. But if nothing is investigated, it won't get fixed, so might as well start by looking at the affected areas. I have 12 GB RAM and external graphics, and STILL get the error after about 8-15 minutes of loading floatlands/caverealms/custom mapgens...

[Krock]

I stay at ~300 MB RAM usage (and 99% CPU) when playing MT on a server, no crashes when playing regular, but this at 10 - 30 fps.
Regular playing means for me to not use something with a lot minetest.remove_node(pos), because that causes an access violation earlier or later.
Also, when testing with my own mapgen, I noticed this fps drop when chunks load and that's because it takes a time to draw the terrain.

[4aiman]

My 2 cents:
Singleplayer: below 800 Mb*
Server: below 500 Mb*

*Total RAM = 2Gb

CPU usage is between 48-78%% on any of my devices other than RPi (I have really slow ones).