Page 1 of 1

[Help] Mapgen causes OOM crashes [mod] [snow]

PostPosted: Thu Sep 15, 2016 13:13
by Hybrid Dog
The snow mod's mapgen somehow takes too much memory resulting in crashes.
l didn't experience one yet, but someone reported crashes:
https://github.com/Splizard/minetest-mod-snow/issues/22

l couldn't figure out the reason for the memory overfilling yet, please help me.

Here's the mapgen code:
https://github.com/Splizard/minetest-mo ... 6.lua#L175

Re: [Help] Mapgen causes OOM crashes [mod] [snow]

PostPosted: Sat Sep 17, 2016 09:11
by paramat
There are 2 noise optimisations that can be done:
1. Don't recreate noise objects in every mapchunk.
2. Use the new noise buffers in 'get perlin map'.
See this mod for code example https://github.com/paramat/stability/blob/master/init.lua

PostPosted: Sat Sep 17, 2016 10:08
by Hybrid Dog
paramat wrote:There are 2 noise optimisations that can be done:
1. Don't recreate noise objects in every mapchunk.

Done, thanks.

2. Use the new noise buffers in 'get perlin map'.

What is the difference between putting the noise values into an existing table and storing them in a new table?
https://github.com/minetest/minetest/co ... 769b65R316

Re: [Help] Mapgen causes OOM crashes [mod] [snow]

PostPosted: Sat Sep 17, 2016 18:49
by paramat
The Lua tables that store noise values are big, especially for 3D noise, and garbage collection is slow, creating new tables makes unnecessary duplicate tables build up in number. So a big reduction in memory use.

PostPosted: Sat Sep 17, 2016 19:57
by Hybrid Dog
If l use a buffer localized outside the on_generated and a bit map becomes generated but then no more mapgen happens because the players don't load new mapchunks, does the garbage collector remove the buffer because it's unused or does it keep it until shutdown?

Re: [Help] Mapgen causes OOM crashes [mod] [snow]

PostPosted: Sun Sep 18, 2016 05:02
by paramat
No idea, hmmmm might know.

Re:

PostPosted: Mon Sep 19, 2016 10:32
by qwertymine3
Hybrid Dog wrote:If l use a buffer localized outside the on_generated and a bit map becomes generated but then no more mapgen happens because the players don't load new mapchunks, does the garbage collector remove the buffer because it's unused or does it keep it until shutdown?


You mean like this?
Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.
Code: Select all
local buffer = {}
minetest.register_on_generated(function()
    --Stuff
end)


It can't be freed as it can be accessed the next time the function is called.

PostPosted: Mon Sep 19, 2016 16:24
by Hybrid Dog
If somewhere it is marked as weak it (e.g. if the buffer table is stored in a weak table) can be collected, can't it?
And how much memory does buffer use?

Re:

PostPosted: Mon Sep 19, 2016 17:45
by qwertymine3
Hybrid Dog wrote:If somewhere it is marked as weak it (e.g. if the buffer table is stored in a weak table) can be collected, can't it?
And how much memory does buffer use?


IDK precisely, it would depend on how large you are making the table (minp/maxp or emin/emax).
I made a quick script which created and stored a table of size 112^3 (emin/emax) and called collectgarbage("count");
it reported ~33mb

PostPosted: Thu Oct 06, 2016 19:17
by Hybrid Dog
Obviously using noise buffers didn't fix the crash.