How fast or slow are get_node/set_node?

[burli]

HybridDog adds some kind of cache for get_node in this thread

Is get_node really that slow that a cache like this worth it?

[Krock]

Test code:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

function minetest.dummy_jump(pos)
   return {}
end

local total_time = 0

minetest.register_globalstep(function(dtime)
   total_time = total_time + dtime
   if total_time < 5 then
      return
   end
   total_time = 0
   
   local player = minetest.get_player_by_name("singleplayer")
   assert(player ~= nil, "Mod designed for singleplayer testing only.")
   
   local pos = player:getpos()
   local minp, maxp = vector.add(pos, -16), vector.add(pos, 16)
   
   -- Measure time used for looping and counting nodes
   local start_looptime_clock = os.clock()
   for z = minp.z, maxp.z do
   for y = minp.y, maxp.y do
   for x = minp.x, maxp.x do
      local node = minetest.dummy_jump({x = x, y = y, z = z})
   end
   end
   end
   local loop_time = os.clock() - start_looptime_clock
   
   local start_clock = os.clock()
   for z = minp.z, maxp.z do
   for y = minp.y, maxp.y do
   for x = minp.x, maxp.x do
      local node = minetest.get_node({x = x, y = y, z = z})
      -- do nothing
   end
   end
   end
   local delta = os.clock() - start_clock - loop_time
   print("It took ".. (delta * 1000) .." ms to get ".. (32^3) .." different node positions, "..
         " without looptime of ".. (loop_time * 1000) .." ms")
end)

Logger output:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

It took 140.00000000001 ms to get 32768 different node positions,  without looptime of 0 ms
It took 204.00000000001 ms to get 32768 different node positions,  without looptime of 0 ms
It took 264.99999999999 ms to get 32768 different node positions,  without looptime of 0 ms
It took 250 ms to get 32768 different node positions,  without looptime of 0 ms
It took 250 ms to get 32768 different node positions,  without looptime of 0 ms
It took 265.99999999999 ms to get 32768 different node positions,  without looptime of 0 ms
It took 265.00000000001 ms to get 32768 different node positions,  without looptime of 0 ms

So if you plan to call the same node thousands of times, it's surely helpful. For huge landscape modifications (set_node) better use the voxelmanipulator - it only updates the map once.

[burli]

Thx for your example. I tried it myself because the 0 ms confuses me. I got this values

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

It took 49.897000000001 ms to get 32768 different node positions,  without looptime of 0.22400000000022 ms
It took 48.852 ms to get 32768 different node positions,  without looptime of 0.45500000000054 ms
It took 57.516 ms to get 32768 different node positions,  without looptime of 0.17899999999926 ms
It took 84.904000000002 ms to get 32768 different node positions,  without looptime of 0.37700000000029 ms
It took 50.235000000008 ms to get 32768 different node positions,  without looptime of 0.3369999999947 ms
It took 81.295000000011 ms to get 32768 different node positions,  without looptime of 0.24599999998998 ms
It took 44.851999999992 ms to get 32768 different node positions,  without looptime of 0.066000000003896 ms
It took 42.079999999999 ms to get 32768 different node positions,  without looptime of 0.072000000002959 ms
It took 39.323999999993 ms to get 32768 different node positions,  without looptime of 0.14500000000339 ms
It took 38.707999999986 ms to get 32768 different node positions,  without looptime of 0.15700000000152 ms
It took 94.920000000002 ms to get 32768 different node positions,  without looptime of 0.30900000000145 ms
It took 104.876 ms to get 32768 different node positions,  without looptime of 0.33500000000686 ms
It took 52.299999999988 ms to get 32768 different node positions,  without looptime of 0.30900000000145 ms
It took 126.40500000001 ms to get 32768 different node positions,  without looptime of 0.36000000000058 ms
It took 51.929000000001 ms to get 32768 different node positions,  without looptime of 0.67199999999445 ms
It took 46.590999999992 ms to get 32768 different node positions,  without looptime of 0.4820000000052 ms
It took 46.210000000016 ms to get 32768 different node positions,  without looptime of 0.43199999998933 ms
It took 41.869999999989 ms to get 32768 different node positions,  without looptime of 0.10199999999827 ms


I think, even for a few dozen nodes a cache has a benefit, especially if additional code is needed like this. Each ms counts

[Krock]

Thx for your example. I tried it myself because the 0 ms confuses me.

Maybe you're using the regular Lua. LuaJIT precompiles all the code, thus it is very fast for loops and other heavy tasks. But in total I get a higher execution time, that could be caused by some time loss in the Lua -> C -> C++ bridge (and backwards) and the CPU speed.

[burli]

Maybe you're using the regular Lua. LuaJIT precompiles all the code, thus it is very fast for loops and other heavy tasks.

No, USE_LUAJIT = 1

I get similar times with a self compiled Minetest and with the daily build from Ubuntu PPA. The only reason could be that the LuaJit is running in interpreted mode and not compiled. This is possible

http://luajit.org/performance_x86.html

LuaJit as interpreter is faster than the regular interpreter, but still much slower than the Jit. Need to check that

[burli]

I compiled Minetest without LuaJIT and got significant slower values. Between 30 and 40 ms instead of 0.4 ms.

Still confused about your 0 ms

However, I think a node cache for get_node is a good idea for that what I want to do

[burli]

I modified your code a little bit. I add a simple cache and I measure the average time

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

local node_cache = {}

local function get_cached_node(pos)
   local hash = minetest.hash_node_position(pos)
   local node = node_cache[hash]
   if node then
      return node
   else
      node = minetest.get_node(pos)
      node_cache[hash] = node
      return node
   end
end

local function clear_cache()
   node_cache = {}
   minetest.after(20, clear_cache)
end

minetest.after(20, clear_cache)


local total_time = 0
local timer = 0
local counter = 0

minetest.register_globalstep(function(dtime)
   total_time = total_time + dtime
   if total_time < 1 then
      return
   end
   total_time = 0

   local player = minetest.get_player_by_name("singleplayer")
   assert(player ~= nil, "Mod designed for singleplayer testing only.")

   local pos = vector.round(player:getpos())
   local minp, maxp = vector.add(pos, -16), vector.add(pos, 16)
   local start_clock = minetest.get_us_time()
   for z = minp.z, maxp.z do
   for y = minp.y, maxp.y do
   for x = minp.x, maxp.x do
      local node = get_cached_node({x = x, y = y, z = z})
      --local node = minetest.get_node({x = x, y = y, z = z})
   end
   end
   end
   timer = timer + minetest.get_us_time() - start_clock
   counter = counter + 1
   
   print("It took ".. (timer / 1000 / counter) .." ms to get ".. (32^3).. " nodes")
end)


On my system the get_node() function needs an average time of 38ms for 32^3 nodes and ~8ms with cache. I will think about a more dynamic cache, but I'm afraid that will kill the performance.

[paramat]

get / set node are slow, each is an individual read or write to the world. For any large amount use the lua voxel manipulator https://forum.minetest.net/viewtopic.php?f=18&t=6396
Lua mapgens used to take 1min per chunk to generate using set node, now 1s with LVM.

[burli]

I need to read nodes most time. Voxel manip does not help me. But the cache works pretty well

[burli]

I tried my cache with a few mobs. I had 4 or 5 mobs walking around and just read one node ahead each on_step call. I had over 2000 cache hits within 10 seconds. And because I want to read more than one node I think it worth it

[rubenwardy]

You can read with LVMs

[burli]

What is a LVM?

You mean Voxel Manip? Isn't that a little bit too much just to read a few nodes? And I just get the node id. For further usage I may have to get the node definition.

Edit: I'm sorry, I'm tired today. paramat already mentioned the LVM. But I still think for my case a cache for get_node is the better solution