Guide to create fast minetest server on Linux

[lag01]

This guide assumes that the one who try to follow it, have basic knowledge in installing and using Linux (i was beginner in Linux when i started this). Also some of programming knowledge will be needed.
This is not complete guide, so use other guides to configure port forwarding, bash commands, Linux installing etc.

So, my big idea about making fast fast server is to store map file and player files in ram. It increases ram requirement and makes server theoretically less stable, but it is not a big deal if you have much ram and UPS.
I know that minetest can be compiled with different database engine, but i feel it would be harder for me and you need ramdisk for player profiles anyway (at least until developers have fixed intensive read/write to profiles).

I have tested this setup on my "just test" server for two months and it can easily handle 30 players and more.

1. You need computer with enough ram - 4GB or more. Also, you better have UPS.
2. Install Debian or Ubuntu. Better do not install graphical interface or other modules, except SSH. Better do not install swap partition.
   Later you can use Putty and WinSCP to connect to your server.
3. If you installed Debian stable, add unstable repository to get lates minetest. To file /etc/apt/sources.list add line "deb http://ftp.de.debian.org/debian jessie main"
4. Install minetest:
   Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

   Code: Select all

   apt-get install minetest-server minetest-data

5. Define ramfs to store map and folder with user profiles in ram: in file /etc/fstab add lines like these:
   
   Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

   Code: Select all

   ramfs   /home/user/.minetest/worlds/test0/players       ramfs   noauto,rw,users,sync,size=100M   0       0
ramfs   /home/user/.minetest/worlds/test0/map      ramfs   noauto,rw,users,sync,size=2000M        0       0

6. Create folder /home/user/.minetest/worlds/test0/map/
   Then in folder /home/user/.minetest/worlds/test0/ create symbolic link with name "map.sqlite" pointing to file /home/user/.minetest/worlds/test0/map/map.sqlite
7. Make startup script "/etc/init.d/minetest" to run minetest as service, use /etc/init.d/skeleton or https://gist.github.com/MarkTraceur/998260 as base. Use command "/ets/init.d/minetest start" to test it.
8. Create folders to safely save map and players, something like:
   /home/user/.minetest/worlds/test0/players.safe/ and /home/user/.minetest/worlds/test0/map.safe/
9. To keep all in ram, while server is working, and save to disk, when it shuts down, add the following to your startup script:
   In "start" part add lines before daemon start:
   
   Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

   Code: Select all

   mount /home/user/.minetest/worlds/test0/players
chown user /home/user/.minetest/worlds/test0/players/
cp -rp /home/user/.minetest/worlds/test0/players.safe/* /home/user/.minetest/worlds/test0/players/
mount /home/user/.minetest/worlds/test0/map
chown user /home/user/.minetest/worlds/test0/map/
cp -rp /home/user/.minetest/worlds/test0/map.safe/map.sqlite /home/user/.minetest/worlds/test0/map/map.sqlite

   
   In "stop" part add lines after daemon stop:
   
   Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

   Code: Select all

   cp -rp /home/user/.minetest/worlds/test0/players/* /home/user/.minetest/worlds/test0/players.safe/
umount /home/user/.minetest/worlds/test0/players
cp -rp /home/user/.minetest/worlds/test0/map/map.sqlite /home/user/.minetest/worlds/test0/map.safe/map.sqlite
umount /home/user/.minetest/worlds/test0/map

10. Make minetest to always run and gracefully shutdown, i used chkconfig:
    
    Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

    Code: Select all

    apt-get install chkconfig
chkconfig --add minetest

11. Configure minetest by changing file /home/user/.minetest/minetest.conf
    Add something like this:
    
    Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

    Code: Select all

    server_map_save_interval = 500.0
sqlite_synchronous = 0
max_simultaneous_block_sends_server_total = 250
max_simultaneous_block_sends_per_client = 6
max_clearobjects_extra_loaded_blocks = 30000

    
    Also spend some time to configure other options.
12. On your local computer create new map and copy your map files to server. Copy map.sqlite to map.safe folder.
13. Make your server restart regularly, for example every night, so map and players can be saved to disk.
14. Restart server and test if game works now.
15. Add your mods(to folder /usr/share/games/minetest/games/minetest_game/mods/) and improvements, make good game, which people could play happily for long time :)

Notes:

I used name "user" for user on my Debian server. For map, i used name "test0", take into account if yours is different.
I heard that maps can easily grow more than 4GB, be aware.
Player profiles could take more than 100MB too.
On Windows you can use ImDisk application to create ramdisks.
Too much bones do heavy load on mobile clients, either disable bones, or remove them regularly with custom mod.
It is possible that map generator take some CPU to generate new bloks, after some time map could load faster and with less CPU usage.

Please, write here if you succeed!

Andrey.

[Calinou]

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

active_block_range = 1
max_block_generate_distance = 6
max_block_send_distance = 6

Saves quite a lot of bandwidth of CPU, for almost no downsides (few people use a high enough viewing range to enjoy high distance block sends anyway).

The first line helps a lot when using mobs or mods that heavily rely on entities and ABMs.

[twoelk]

... (few people use a high enough viewing range to enjoy high distance block sends anyway).
....

I for one love to climb to some high place after a long journey and then switching on far view look back at the path I have taken. I have loads of screenshots of such views. I have experimented with how far "far view" actually is on VanessaE's creative server. Turned out at the time to be limited to something like 3500 nodes. At least that was how far away you could be from MichaelEh's space station and still see it. The pictures I made of Mt.Meru on Brandon's Landrush server also made use of a viewing distance of nearly 2000 blocks. Some objects or features are only half the fun if you cannot use "far view" once in a while and have that view display a little more the just the immediate surroundings of the places you have just been. I don't know what the viewing range on the Landrush server was at the time but climbing the mountain to have most of it's blocks displayed and kept in memory was some hell of a job. I don't know if roaming the landscape on the "Just Test" server wouldn't be too difficult if I couldn't see my own feet. ;-P

[lag01]

I don't know if roaming the landscape on the "Just Test" server wouldn't be too difficult if I couldn't see my own feet. ;-P

Well, recently i cut down max_block_send_distance from 9 blocks to 6 blocks (96 nodes/meters), i think, performance improvement which server gained, was worth it.

[twoelk]

...
Well, recently i cut down max_block_send_distance from 9 blocks to 6 blocks (96 nodes/meters), i think, performance improvement which server gained, was worth it.

hehe - so now nobody can see my castle retreat when aproaching the little hill it is perching on and scraping the clouds.

[Joz]

If you just use redis, which stores many blocks in RAM per default (or leveldb maybe) I think it is very fast also. Nearly as fast as RamFS and more-often backups.
Minetest world settings for redis: http://wiki.minetest.net/Database_backends#Redis
Redis quick start, also for debian: http://redis.io/topics/quickstart
Alternatively sudo apt-get redis-tools or something like this

[cesarpachon]

@lag01: thank you for the detailed description! I am new to minetest and had been playing for a couple of days with a amazon ec2 instance and this info is really useful!

[amadin]

deleted

[TuxerP]

@lag01
I see that you are running debian jessie which runs systemd. I am running ubuntu 15.10 which also runs systemd. When I add the mount option before start en after the stop daemon in my /etc/init.d/minetest-server script and restart. Nothing happens.
Can you help out?

This is part of my /etc/init.d/minetest-server file:

# Include defaults if available
if [ -f /etc/default/$NAME ] ; then
. /etc/default/$NAME
fi

mount /var/games/minetest-server/.minetest/worlds/world/players
chown Debian-minetest:games /var/games/minetest-server/.minetest/worlds/world/players/
cp -rp /var/games/minetest-server/.minetest/worlds/world/players.safe/* /var/games/minetest-server/.minetest/worlds/world/players/

minetest_start() {
start-stop-daemon \
--start \
--quiet \
--pidfile $PIDFILE \
--oknodo \
--background \
--exec $BINARY \
--startas $DAEMON \
--make-pidfile --chuid $USER \
-- $DAEMON_OPTS > /dev/null 2>&1 || return 1
return 0
}

minetest_stop() {
start-stop-daemon \
--stop \
--quiet \
--pidfile $PIDFILE \
--oknodo \
--exec $BINARY || return 1
rm -f $PIDFILE
return 0
}

#cp -rp /var/games/minetest-server/.minetest/worlds/world/players/* /var/games/minetest-server/.minetest/worlds/world/players.safe/
#umount /var/games/minetest-server/.minetest/worlds/world/players

case "$1" in
start)
log_begin_msg "Starting $DESC: $NAME"
minetest_start
log_end_msg $?
;;

And my fstab file

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/mapper/minetest--vg-root / ext4 errors=remount-ro 0 1
# /boot was on /dev/sda1 during installation
UUID=42d02e8c-e515-4162-b1d1-4e57f77f11fe /boot ext2 defaults 0 2
/dev/mapper/minetest--vg-swap_1 none swap sw 0 0
ramfs /var/games/minetest-server/.minetest/worlds/world/players ramfs defaults 0 0

[trukoil]

Don't know how LevelDB runs for a server (goes well in singleplayer) but it's just a changed line in world.mt

By the way:

Well, recently i cut down max_block_send_distance from 9 blocks to 6 blocks (96 nodes/meters), i think, performance improvement which server gained, was worth it.

Didn't thought that block != node.

[lag01]

Don't know how LevelDB runs for a server (goes well in singleplayer) but it's just a changed line in world.mt

By the way:

Well, recently i cut down max_block_send_distance from 9 blocks to 6 blocks (96 nodes/meters), i think, performance improvement which server gained, was worth it.

Didn't thought that block != node.

Would be nice to have another good option, instead of randisk... But, as i read in wikipedia about LevelDB, there is nothing about casching in memory and there is no indexing.

"Blocks" and "nodes" terminology is used in minetest mapformat description, so i try to honor that naming convention.

[ExeterDad]

But, as i read in wikipedia about LevelDB, there is nothing about casching in memory

No need to cache in memory. The database IS in memory. That's why it's so fast. But... you must have enough memory to accommodate the database or you will run out of memory for the server to run on.

[TuxerP]

I am trying to run a server that you can play from a mobile device. Mobile devices have limited amounts of cpu and memory, so server optimization is a thing that is important. I have tuned my minetest.conf settings so the amount of data send to the mobile device is not to much to have good fps and so little lag as possible.
Running on leveldb, redis or running players and maps in memory will be a faster option than running sqlite I think.
Also I try to run stable mods.

[ExeterDad]

I am trying to run a server that you can play from a mobile device. Mobile devices have limited amounts of cpu and memory, so server optimization is a thing that is important. I have tuned my minetest.conf settings so the amount of data send to the mobile device is not to much to have good fps and so little lag as possible.
Running on leveldb, redis or running players and maps in memory will be a faster option than running sqlite I think.
Also I try to run stable mods.

From my limited experience using Android on our server. Less is more. Using games or mods heavy in textures/sounds will kill our devices the fastest. And as you said, it boils down to limited memory. All media is downloaded to the devices when logging in. Even textures for nodes that you may never see the entire time you are logged in with the device. My Android, and it's not too shabby... will crash while loading the media when our server is running a BFD instance. It plays a default game map like a dream. But the heavier it gets, the more likely it will crash.

[amadin]

I think this guide the developers must put in wiki, because with default minetest.conf - parts of the world loading very-very slow if more than 30 people online on powerfull server.

[lag01]

Right now i am working on improving this ramdisk setup.
Here i attached halfway ready code for using ramdisk.
Also other useful functions, like backup, clear player profiles, keep service up.

To use this install, minetest must be installed, and there must be correct user and world folder names. Also map.sqlite and debug.txt must be present in the right places.

Can somebody help me improve install.sh to automatically add crontab entries (like in crontab.txt)?

[amadin]

Do you can add parameters of your server (processor, hard drive (ssd, sata or ata), ram (ddr or sdram), network speed) or add it to your post here viewtopic.php?p=162449#p162449?

[Calinou]

If you have spare CPU and bandwidth, you can use this:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

dedicated_server_step = 0.05

This will make player/mob movement appear to be smoother and more accurate (and any other action that occurs every server step).