Lua PerlinNoiseMap memory efficiency improvements

hmmmm
Member
 
Posts: 47
Joined: Tue Apr 02, 2013 04:04

Lua PerlinNoiseMap memory efficiency improvements

by hmmmm » Sun May 17, 2015 16:16

Hello Minetest Modding Community,

Recently I've had some complaints about extremely high memory usage in mods that make extensive use of Perlin noise maps (core.get_perlin_map/PerlinNoiseMap). I investigated and found that the most memory-intensive part of noise is, by far, placing the resulting computation into a Lua table. Optimizing memory usage was possible, but taking advantage of the improvements required some additions to the Lua API.

The first (and simplest) of the changes is to pass along a pre-existing table to receive the results of the noise calculations. This not only saves CPU time as all entries have already been allocated, but it minimizes Lua object creation so memory usage stays stable throughout execution. It can be used like so:
Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.
Code: Select all
local noise_buffer = {}
local noise = nil
...
function do_noise_thing(pmin, pmax)
   ...
   -- Notice that we reuse the same noise object instead of creating a new one each call
   noise = noise or core.get_perlin_map(noise_params, chunk_size)

   -- Here, the variable 'noisevals' is merely a reference to the table noise_buffer, not a copy.
   local noisevals = noise:get3dMap_flat(pmin, noise_buffer)
   ...
end


The second change requires the use of new PerlinNoiseMap methods calc2dMap, calc3dMap, and getMapSlice.
First, calc2dMap or calc3dMap is called to compute the noise. Then, when a chunk of noise is needed, it can be retrieved from the internal buffer using getMapSlice(). getMapSlice() takes a slice of noise at the specified coordinates, relative to the position inside the buffer (not absolute map coordinates!) and starts at 1, as is standard for Lua. If a coordinate is not specified in the slice offset parameter, then all noise along that axis is written to the output table. Therefore, a horizontal plane would be {y=}, a vertical column would be {x=, z=}, a single row spanning the X axis would be {y=, z=}, the whole buffer would be {}, and a single point would be {x=, y=, z=}. getMapSlice's second parameter, the slice size, is the length along that specified axis to retrieve.

Here is an example of these new methods in use:
Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.
Code: Select all
local noise_buffer = {}
local noise = nil
...
function do_noise_thing(pmin, pmax)
   ...
   noise = noise or core.get_perlin_map(noise_params, chunk_size)

   -- Calculate the noise starting at pmin, storing the result internally.
   noise:calc3dMap(pmin)

   -- Generate the terrain
   local slice_count = 0
   local nvals
   local ni
   for z = pmin.z, pmax.z do
      -- Get an 80x80x2 slice of the noise result, if needed
      if slice_count == 0 then
         slice_count = 2
         nvals = noise:getMapSlice({z=z-pmin.z+1}, {z=slice_count}, noise_buffer)
         ni = 1
      end

      for y = pmin.y, pmax.y do
      for x = pmin.x, pmax.x do
         if nvals[ni] > 0.5 then
            ...
         end
         ni = ni + 1
      end
      end

      slice_count = slice_count - 1
   end
   ...
end


With the above code, execution time is 1.8% slower than the previous example, but uses a whole 40x less memory. The table noise_buffer consumes 200KB instead of 8000KB.

I hope you all find this information useful in minimizing the memory footprint of your mods making use of noise maps.

- hmmmmm
 

paramat
Member
 
Posts: 2662
Joined: Sun Oct 28, 2012 00:05
GitHub: paramat

Re: Lua PerlinNoiseMap memory efficiency improvements

by paramat » Sun May 17, 2015 20:47

Excellent work and extremely useful, thanks so much.
 

User avatar
Krock
Member
 
Posts: 3598
Joined: Thu Oct 03, 2013 07:48
GitHub: SmallJoker

Re: Lua PerlinNoiseMap memory efficiency improvements

by Krock » Mon May 18, 2015 15:58

Thanks for pointing to this. Does it count for Lua and LuaJIT?
Newest Win32 builds - Find a mod - All my mods
ALL YOUR DONATION ARE BELONG TO PARAMAT (Please support him and Minetest)
New DuckDuckGo !bang: !mtmod <keyword here>
 

paramat
Member
 
Posts: 2662
Joined: Sun Oct 28, 2012 00:05
GitHub: paramat

Re: Lua PerlinNoiseMap memory efficiency improvements

by paramat » Tue May 26, 2015 05:30

Yes both.

See the issue at github for more detail on this subject https://github.com/minetest/minetest/issues/2665
 


Return to Modding Discussion

Who is online

Users browsing this forum: No registered users and 5 guests

cron