Noob question: simulating entire map

[dnebeker]

I'm very excited about having found Minetest -- how awesome!

Something I'd like to try, and after reading forums and docs for two hours, it doesn't seem to have been done yet:
- have a server (core?) process simulate activity across the entire map, whether there is a player there or not.

This will of course need a lot of memory and a lot of CPU power, but that's OK -- I'm thinking ahead 5 years from now :)

The idea is to combine the concept of Minetest/Minecraft and Civilization -- so there are AI bots doing things in other parts of the map that you can eventually find.

I assume this would be much more than a mod.

On a somewhat related note, I was reading about some of the flowing water work going on. Is that done in the server/core as well? Any hints about where in the source to look at it?

Thanks
New Minetester!

[sfan5]

You could use the Pull Request that adds a "force_load" flag to entities, then you could place such entities all over the map, and all ABMs, Nodetimers, everything else will happen (e.g. grass grow)

[Calinou]

This will of course need a lot of memory and a lot of CPU power, but that's OK -- I'm thinking ahead 5 years from now :)

It's not like CPU power will be multiplied by 10 in 5 years, at least at the x86 side of things. Lack of competition. 8)

Note that not everyone is willing to buy a new CPU in 5 years, too.

[PilzAdam]

How much was a the size of a complete generated Minetest world? Was it peta- or exabyte?

[sfan5]

How much was a the size of a complete generated Minetest world? Was it peta- or exabyte?

asdf7389, dont try to generate a full Minetest map at once. No really, don't.

The SQLITE map file can only go upto 4GB and the file becomes unstable, not to mention that the file size will grow to about 14TB.

Good Luck.

Assuming every block is one byte its 216 TB.
As every block has a light level, param1 and param2 that is 4 bytes per block.
That makes 864 TB.
But since almost 50% of the map is air (~ 0 -- +30 000) and 30% of the rest should be easily compressable, a complete minetest map would occupy 172.8 TB.
Comparisons by Wolfram Alpha:

• ~~ ( 0.086 ~~ 1/12 ) × identifiable storage capacity of the human brain (~~ 2 PB )
• ~~ ( 0.086 ~~ 1/12 ) × approximate data content of the Internet Archive Wayback Machine as of 2006 (~~ 2 PB )
• ~~ estimated data content of the surface web (~~ 170 TB )

That file size also shouldn't break any system as it is expressable (in bytes) with a 64-bit integer (signedness doesn't matter)

http://forum.minetest.net/viewtopic.php?pid=82558#p82558

[dnebeker]

lol - I think I've found a home based on these replies :)

Yes, I had been doing some back of the envelope calculations too. I realized I shouldn't have said "the whole map". Probably just what would fit in 8GB of RAM for example (100 square miles???).

Before coming across MineTest, I was also thinking of having a separate thread (or two) that would work on processing updates to the rest of the map for everything outside of what a user could see. Hopefully that keeps it running fast. Feasible?

Finally, I'm very interested in playing with water flow. When I thought about writing my own world generator (which I really don't want to do), I thought about adding a tiny amount of water to the entire landscape, and then just letting it flow down -- that would define where rivers, lakes and oceans ended up being. It doesn't need to be done now that I've found MineTest, but I'm still interested in running the simulation and being able to fly around and watch it happen.

With that in mind, where can I look in the code to find existing water flow dynamics?

Thanks for the replies.

[dnebeker]

Oh, and about Jordach's comment about SQLite, I use it a LOT in my day job. We regularly see databases over 4GB. It does get slower as it grows, just from pure file I/O required, but it's never gotten unstable or corrupted in my experience. One thing we do is have a simple database layer that supports SQLite or MS SQL Server. For people that want to run a big server, having the option for MS SQL Server might be interesting. For us, it performs better at large sizes for SELECTs (probably because it uses so much more memory and can thus cache better).

[twoelk]

..

With that in mind, where can I look in the code to find existing water flow dynamics?

...

I once used a program called Scapemaker. It simulated erosion by calculating "waterdrops". Version 1.3 is still on the net but the old homepage seems broken. It could import greyscale hightmaps saved as png and export results as png again. I imported some maps to WorldPainter and exported that to MC. Was fun, got some epic failures until I figured out how to tweak the maps.

If I remember correctly this wasn"t the only program to produce pixelmap based hightmaps. Seems the format for this for the unproffesional. Don't know what Terragen can export to but there is a script to get the results into Blender.

To design own maps we would need something along the line of WorldPainter (search MC forums). Maybe the makers can be convinced to export to Minetest as well. Is there an import/export library somewhere? A whole map in WE format might be to large. Maybe a tool to feed a large zipped we file into sqlite? Could a hightmap be fed to the terraingenerator which adds its other magic over and underground but keeps the hight information?

As for the ingame terrain generating with water might prove difficult. At the moment the map is generated chunkwise as you visit the area. Seeing what happens to the trees of the more trees mod when they generate near chunk borders with one chunk loaded and the other not when calculated, I can imagine a lot of problems with water flowing over chunk borders. Water would not be restricted like the flowing lava in caves. One sollution would be to generate all chunks which are connected by flowing water. You would also have to generate chunks to see if your chunk might recieve water from a new one. This could go on for ever, so generating would have to be limited by producing island worlds devided by at least chunk wide oceans. Hmmm ... oceans .... I would love that ... buildable ships ... not one node boats ... real ships ... I'm off dreaming ... er there are threads on rivers somewhere on the forum ... I'm of to the seaside ...

[tinoesroho]

You could convert with Worldpainter and scapemaker to minecraft, and use the mc_import mod to bring it into Minetest!