[0.4.8] LuaVoxelManipulator

[hmmmm]

Hello modding community,
I am pleased to announce the addition of the LuaVoxelManip to Minetest's Lua API. Ever since the beginning, modders have been looking for a way to manipulate nodes on the map in bulk. Just until recently with the addition of the Schematic API, this was not possible. However, now with VoxelManip, there is a fast, complete, and official way to accomplish this task. It may seem that this API is much more technical and less user friendly to use than others, but this is necessary in order to maximize speed - its central purpose.
Within on_generate callbacks, the same exact VoxelManip object used to generate the chunk being handled with the internal map generator can be obtained using the new minetest.get_mapgen_object API. Paired with the singlenode map generator, this is especially convenient for creating completely Lua-based mapgens.

For testing purposes, I made a very simple 3d noise-based map generator, which can also be used as a decent reference of how to use VoxelManip:


How well does it perform?
Setup:
CPU: Intel Xeon E3-1230v2
OS: FreeBSD/amd64 9.1-RELEASE
CC: g++ 4.2.2, -g, -O1

On average, to generate an 80x80x80 chunk of this, it takes:
minetest.env:set_node: 6854.4ms
VoxelManip: 153.5ms
Mapgen in core: 60ms (estimated)

So, generating map with VoxelManip in pure Lua is only about 2.5x slower than a mapgen in the core, but a whole 45x faster than using set_node for everything.


I strongly encourage everybody to use this interface and *not* the Schematic API for VoxelManip tasks. The primary purpose of the Schematic API is to define discrete, relatively static structures such as trees, spikes, huts, etc. so they can be randomly placed on the map at generation time. minetest.create_structure is to make a new structure. minetest.place_structure is not exactly a necessary API, but convenient for checking to see that the correct structure was created and did not take much additional effort at all to implement; an added bonus was that it could be used in the same way WorldEdit schematics were - this was not the intent however. Further, defining the schematic completely within Lua via the node table was again a feature added for flexibility, and because it did not take much effort to add.
VoxelManip is even faster than the Schematic API, adds vastly more flexibility, is not hackish, and allows reading the map as well, which is needed for many node placement tasks. There is really no reason not to use it.

Also:
Apparently, having no way to find the real name of a node was a long-standing problem in mods.
As luck would have it, two API that were added needed by VoxelManip, minetest.get_content_id() and minetest.get_name_from_content_id(), can be used to find the real name of a node alias as follows:
local c = minetest.get_content_id(alias_name)
local real_name = minetest.get_name_from_content_id(c)

Enjoy!

[paramat]

*faints*

[Nore]

Where does the VoxelArea thing come from? It gives me a nil value error when I try to run that code...

[MirceaKitsune]

Very nice! Glad to see this idea is finally reality and in upstream, it will certainly help a lot. I still wouldn't trust making a whole mapgen with it, but at least it will make that possible in theory which could be interesting.

I'll switch my Structures mod to it as soon as it's safe. There's one thing I don't understand though: How do you send a list of nodes to the engine for spawning? read_from_map(p1, p2) has two corners you can specify like I thought, but write_to_map() takes no parameters. Instead it takes another data writing function which I wasn't able to understand. How would I handle a list of the form { { pos1, node1 }, { pos2, node2 } }?

[Casimir]

I need to buy sunglasses, so I can wear them while using the LuaVoxelManipulator.

[hmmmm]

Where does the VoxelArea thing come from? It gives me a nil value error when I try to run that code...

VoxelArea was my own helper class; though as of today, builtin/voxelarea.lua is part of upstream.

I'll switch my Structures mod to it as soon as it's safe. There's one thing I don't understand though: How do you send a list of nodes to the engine for spawning? read_from_map(p1, p2) has two corners you can specify like I thought, but write_to_map() takes no parameters. Instead it takes another data writing function which I wasn't able to understand. How would I handle a list of the form { { pos1, node1 }, { pos2, node2 } }?

read_from_map() and write_to_map() deal with the internal VoxelManipulator object pulling and pushing data to the actual map. To get and set the array of nodes in Lua, you need to use local data = get_data() and set_data(data).
To handle that sort of list, you would do something like:
local vi = area:indexp(yourtable.pos1)
data[vi] = minetest.get_content_id(yourtable.node1)

(although I would prefetch the results from minetest.get_content_id() and not call it in a loop if i were you)
Setting node strings instead of content IDs is an option coming soon. It's easier in some circumstances, but would be slower. This was designed for being fast.

[ch98]

Is this the thing that makes large spaceships possible? If it is, Minetest is now close to complete!

[Zeg9]

Is this the thing that makes large spaceships possible? If it is, Minetest is now close to complete!

If you mean moving spaceships, no. This is a just a way to set nodes faster.

[oxenfreedan]

AWESOME!

[PilzAdam]

Also:
Apparently, having no way to find the real name of a node was a long-standing problem in mods.
As luck would have it, two API that were added needed by VoxelManip, minetest.get_content_id() and minetest.get_name_from_content_id(), can be used to find the real name of a node alias as follows:
local c = minetest.get_content_id(alias_name)
local real_name = minetest.get_name_from_content_id(c)

It turned out this is wrong. There is a non-hacky way of solving it:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

ItemStack("alias"):get_name()

This will return the real itemname.

[Casimir]

For testing purposes, I made a very simple 3d noise-based map generator, which can also be used as a decent reference of how to use VoxelManip

Thank you. But some comments in that code would help to understand it. I think that's the reason nobody used it until now, because nobody understands it (except you).

[paramat]

There is a branch of PilzAdam's nether mod that uses LVM, also latest worldedit mod, 2 more you can study.

[Splizard]

Dev version of Snow mod uses it now,
https://github.com/Splizard/minetest-mod-snow/blob/master/mapgen_v6.lua (see line 238 and below)
I understood it, you just need to try it out.

[Casimir]

OK cut my last sentence. I got it to work the way I wanted too. But it would be nice if there would be some explanation somewhere.

[paramat]

It works on unloaded chunks too apparently.

[miner65536]

Whats the easiest way to load regions prior to writing to them with setnode or using this? In singleplayer I'm doing something similar and all i can think of moving the player to the position and waiting a bit for the server to load or generate that region.

[rubenwardy]

Can some one give me an example that does set node using this?

[paramat]

Set node = add node and is done like this:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

data[vi] = c_dirt

If you want another example mod see my LVM version of moonrealm mod.

[george90867]

can we download it?

[paramat]

Yes you can :)

[Prestotron562]

Can't wait for this. This will help my mod "More Biomes"

[Evergreen]

Can't wait for this. This will help my mod "More Biomes"

There will be a biome api in mg v7.

[leetelate]

it is helpful to work through a chunk column by column, and working down each column instead of up

yep, y, then either x or z first, and z+ is north if your biomes are n/s/ or e/w oriented - following you nicely - i don't use perlin noise anymore, but if you think of it as a basically horizontal tube and work the area with that in mind, it gets easier to visualize

don't know if you've hit it yet but watch out for missing content id's - you specify c_snow but you fail to do the declare to say that c_snow is default:snow, that is where the fire comes from, since a missed content id = fire

enjoy, the lua voxel manip is the best thing about MT

[leetelate]

http://www.scribblethink.org/Work/Crunge/crunge.pdf gives an overview of the Weiner interpolation - so much faster!
i am still working on this one, but here's the latest attempt:
make a noise 1 and a noise 2, then make a bunch of random points based on the average of noise1+noise2 at those points
expand each point(like doing a simple gaussian blur) until you hit a non-0
interesting, though i'm still getting artifacts - cool mountain ranges! - they even have ridge lines!

wiaworld

cliffs

overhangs


let me work on it a bit more and i'll put the code up, maybe even on github

[paramat]

That looks surprisingly good considering it's simplicity, i like these gem-like facets at various angles.

[leetelate]

the water should flow downward at the joins, make rivers, and where there is no water is a desert

bah, i got side-tracked with the idea of mapping on-the-fly (which is almost solved, btw - a super-simple routine creates an overhead map via the noise and is built as the chunks are generated - sweet, though the output is only in pgm format at the moment - waiting for 0.4.8)

shupagen is coming along nicely:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

 for x=minp.x, maxp.x, 1 do
  for z=minp.z, maxp.z, 1 do
    local n = noise:get2d({x=x,y=z})
    local q = math.abs(math.floor(n*128))
    local c=q+128
    for y=minp.y, maxp.y, 1 do
    ...  build it based on the height data from the 2d noise


seed=679072 octaves=9 persistance=0.9 scale=100 gives a rpg-friendly terrain of: (white = mountains, black = ravines, gray = flat)
now i just need to clamp c to a decent value, then smooth it out a bit with a fft (bigger hammer)

[paramat]

Looks like crazy terrain :)
Some advice on your use of perlin noise:
The first octave creates detail roughly on the scale of 'scale' = 100 nodes, each extra octave creates detail on a scale half as big, so 100 50 25 12.5 6.25 nodes ...
Any detail with a scale smaller than roughly 4-8 nodes is lost in the 'blockiness' of the individual nodes. So you dont need 9 octaves only 5, this will greatly reduce processing and speed up mapgen.
Also, having the largest scale of terrain as 100 nodes will create terrain with no variation larger than 100 nodes, i recommend at least 250 nodes, which is the value used in default MT mapgen.
I use 2^n numbers for scale because they always divide by 2, so an MT scale mapgen would be scale = 256, octaves 6, creating detail on these scales: 256 128 64 32 16 8 nodes. Personally i always stop at 8 nodes and dont go to 4 because the detail is lost.
Persistence is roughness of terrain, a 'medium' / noise-industry standard value for this is 0.5.
MT default is 0.6 = fairly detailed, 0.4 = smooth, 0.7 = crazy.
0.9 means the small scale variation is too strong and drowns-out the large scale noise, i suggest 0.4-0.7 for most uses.

[leetelate]

oh, i see what you mean - making it porcupine then having to smooth back again it wastes the horses eh? - thanks a lot for the data, it will help - and my mapscale is obviously wrong too - i was thinking it was a big distance but the flat areas between the mountains and ravines are tiny lol - it is always something we don't know about causing the error! - early days - i will go back and look at the octaves/scale again in a bit - i really like that contiguous terrain with mountains->flats->ravines, which is basically why i was using 9/100 as 'fill' instead of doing it myself - sharpness of the mountains was a plus too - yeah i see what you mean now about the scale and it getting lost
5,0.4,256 gives the first map file
5,0.6,256 gives the first map file
and the mapper code (makes a pgm 500*500 overhead map from a 2d noise specified by the values)

[Neon]

Some advice on your use of perlin noise......

This information should totally be added to the wiki

[leetelate]

yes it should - also there are some things that cannot (as far as i can figure out) be used with lvm

chests - forget about param2, there is no meta data, so chests no work when you use it - have to dig it up and place it again to put the metadata

furnaces - crashes mintest with some error about "fuel" - i didn't care at that point and changed it to a stone block - metadata again

torches, signs (anything wallmounted?) - floats harry potter style until disturbed then falls as a lua object lol

everything else seems to work great - i think the lvm is wonderful!

shupagen is finished (no on-the-fly) and working in 500*500 with a pop-up map - very sweet - one more cleanup then i will put it up, probably as a christmas rpg mod


note: for those that didn't see the thread, on-the-fly mapping from the noise during the mapgen now works but is useless! as there is no way to push the map texture to the client. i got drunk the day i found that out. Very sad.