Xanadu Server

[Bobr]

Grails test is having the same problem if you want I'm sure afflatus would appreciate it if you or ten go on grails test nd see if it's the same problem

[neozerosv]

Xanadu server runs on a script that does restart when server crashes or something happens... but... this STALL glitch does just that, it stalls server and makes it think everything is ok, but meanwhile no-one can do anything or sign on... I've given a solution for this on the github issue page so we'll see what can be fixed.

Just as an idea, it would'nt be better for the script check minetest job runing and the server list in http://www.minetest.net/servers/ so it can restart minetest when its not listed?.

Just an idea, i know you and Shinji work hard keeping the server online 24/7.

Thanks a lot.

[auouymous]

Just as an idea, it would'nt be better for the script check minetest job runing and the server list in http://www.minetest.net/servers/ so it can restart minetest when its not listed?.

The server is probably using a looping script that restarts the server each time it exits.

The following script runs separately and checks the server list every 30 minutes (1800 seconds). The script kills the server if address not found. This should cause the loop to restart the server. Change it to "killall -9 minetestserver" if the server doesn't want to die.

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

#!/bin/sh
ADDRESS="98.166.90.48"
SECONDS=1800
while true; do
    sleep $SECONDS
    LIST=`wget -qO- http://servers.minetest.net/list`
    if [ "$?" = "0" ]; then
        ENTRY=`echo "$LIST"|sed -e $'s:}:}\\\n:g'|grep "\"address\": \"$ADDRESS\""`
        if [ "$ENTRY" = "" ]; then
            echo "Server not found, killing it"
            killall minetestserver
        fi
    else
        echo "Unable to get server list"
    fi
done


[mystik]

You have to be careful with that --- sometimes the server list service does not respond correctly ...

I use a wrapper to connect to mine test servers and once in a great while the server list is down, or invalid, and it can't find the right server to connect to.

*FOR THIS ISSUE*, a better solution might be to (a) make a test connection to the server with an invalid account, and make sure the server responds in time (b) monitor traffic in/out of the server (xanadu is pretty busy, it'd be rare if it's idle), (b) montor CPU useage (when this bug triggers, it seems to spin into an infinite loop?)

*BUT* Killing the server violently like this can't possibly be good for the DB integrity. Doing this too frequently is probably an easy way to introduce corruption :(

[Kilarin]

I hope the devs get this fixed quickly. I finally talked my Sister-in-law into trying minetest and... Xanadu and half the servers are all down. NOT a great introduction...

[auouymous]

You have to be careful with that --- sometimes the server list service does not respond correctly

The script handles it correctly if the list server is down or not responding (the $?=0 is success). But I will modify the script to validate the json in case the server returns garbage.

a better solution might be to (a) make a test connection to the server with an invalid account

I actually started writing a little program that would send the initial connection packet to the server and wait for an ACK. That is the best way, but I didn't feel like looking up the values... It is also possible that a fake connecton could crash or stall the server, and the server has enough problems to deal with already.

Killing the server violently like this can't possibly be good for the DB integrity.

Commands don't work when the server is stalled, doesn't matter if the admin runs the kill command or the script does. If the server is run with --terminalit might accept the /shutdown command from there, I don't know. If it does, it might be possible to have the script pipe the /shutdown command to it.

[auouymous]

This does a little verification on the json and has better output for logging.

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

#!/bin/sh
ADDRESS="98.166.90.48"
SECONDS=1800
while true; do
    sleep $SECONDS
    NOW="$(/bin/date +%Y)-$(/bin/date +%m)-$(/bin/date +%d) $(/bin/date +%H:%M:%S)"
    LIST=`wget -qO- http://servers.minetest.net/list`
    if [ "$?" = "0" ]; then
        ENTRIES=`echo "$LIST"|sed -e $'s:}:}\\\n:g'`
        V1=`echo "$ENTRIES"|head -n1|grep -E '{"total_max": {"servers": [0-9]+, "clients": [0-9]+}'`
        V2=`echo "$ENTRIES"|tail -n2|grep -E '], "total": {"servers": [0-9]+, "clients": [0-9]+}'`
        if [ "$V1" != "" -a "$V2" != "" ]; then
            ENTRY=`echo "$ENTRIES"|grep "\"address\": \"$ADDRESS\""`
            if [ "$ENTRY" = "" ]; then
                echo "[$NOW] Server not found, killing it!!!"
                killall minetestserver
            else
                echo "[$NOW] Server is okay"
            fi
        else
            echo "[$NOW] Server list is corrupt!"
        fi
    else
        echo "[$NOW] Unable to get server list!"
    fi
done


[minetestjosh]

I'm not sure if restarting scripts are the answer since the update happened there are certain areas that are affected. Anyone logging in in those areas will always make the server crash and unless they quickly type /spawn and log off it could result in a crashing loop. Just my theory, anyway. Like mystik said, you don't want to corrupt the DB either, Xanadu has had some serious building going on on all the paths, hundreds if not thousands of player hours spent making stuff.

[amadin]

I'm not sure if restarting scripts are the answer since the update happened there are certain areas that are affected. Anyone logging in in those areas will always make the server crash and unless they quickly type /spawn and log off it could result in a crashing loop. Just my theory, anyway. Like mystik said, you don't want to corrupt the DB either, Xanadu has had some serious building going on on all the paths, hundreds if not thousands of player hours spent making stuff.

Xanadu have backups of DB, i think if server stall then TenPlus1 killing it with "killall minetestserver" or "killall -s KILL minetestserver".

[afflatus]

I'm not sure if restarting scripts are the answer since the update happened there are certain areas that are affected. Anyone logging in in those areas will always make the server crash and unless they quickly type /spawn and log off it could result in a crashing loop.

A restart script is not the solution, it's a temporary workaround so we can all continue playing. Polling servers.minetest.net/list seems like taking a sledgehammer to a small moving target, I prefer amadin's idea of pinging the port directly, but it's a question of what works ultimately.

AFA restarts are concerned I run my own server via systemctl, so I would use systemctl restart, which I trust does the right thing. If minetest-server is running from a script you don't have much choice but to kill it. I don't know what the possibilities are of corrupting the db this way. That would be good to know.

What you're describing is an engine bug AFAICT, it does seem like the result of map corruption and also seems to affect areas with multiple ownership / protection, although there are exceptions to that. I also suspect it is related to this stall glitch, but so far I can't prove it; I'm calling this the "Bermuda Triangle effect". We need to provide enough information to reproduce these errors in order to get them fixed and unfortunately this one seems to slip away if you look at it too hard.

It would be useful if everyone would provide details of OS and minetest build (or clone) when reporting bugs plus a clear explanation of what they did to trigger the problem. That would give devs a chance to build a clearer picture of what is happening. There was an idea that stalling was related to the increase in mobile clients but I've seen it triggered by PC users too. By 'triggered' I don't mean 'caused' btw. I'm currently wondering whether the Bermuda triangle effect has something to do with protection or lighting.

[mystik]

A restart script is not the solution, it's a temporary workaround so we can all continue playing. Polling servers.minetest.net/list seems like taking a sledgehammer to a small moving target, I prefer amadin's idea of pinging the port directly, but it's a question of what works ultimately.

AFA restarts are concerned I run my own server via systemctl, so I would use systemctl restart, which I trust does the right thing. If minetest-server is running from a script you don't have much choice but to kill it. I don't know what the possibilities are of corrupting the db this way. That would be good to know.

I may investigate this, but a patch to call sd_notify() in the main loop is arguably the right(tm) way to catch hangs of this nature, and with a sensible WatchdogSec= parameter, it can allow systemd to properly sense a hang, and trigger a restart. But to re-iterate your point -- this is a workaround, not a fix.

[auouymous]

this is a workaround, not a fix.

Not sure where the idea of it being a fix came from, the server would still stall and be down for up to 30 minutes when it does. This just increases the server's currently poor uptime.

[~Xhole]

I just wanted to chat with my frands :(: ........lol will try later

[amadin]

... You may write script wich will restart server every 3 hours (with "killall -9 minetestserver" or "killall -s KILL minetestserver") if you don't want use Auouymous's script.

[Phiwari123]

The last few days the server crashed whenever i joined, so I guess i will take a break from Minetest until the problem is solved.

[auouymous]

Tenplus1 why you not added script from Auouymous? You may write script wich will restart server every 3 hours (with "killall -9 minetestserver" or "killall -s KILL minetestserver") if you don't want use Auouymous's script.

That is a VERY BAD idea. It would cause rollbacks and increase chances of data corruption.

[Milan*]

just for interest: why no recording for youtube?

[Glorfindel]

just for interest: why no recording for youtube?

I have never heard a reason for this rule, but I don't record so I don't care one way or the other

[amadin]

Tenplus1 why you not added script from Auouymous? You may write script wich will restart server every 3 hours (with "killall -9 minetestserver" or "killall -s KILL minetestserver") if you don't want use Auouymous's script.

That is a VERY BAD idea. It would cause rollbacks and increase chances of data corruption.

It was just simply example, script may run "killall -s TERM minetestserver" then after 60 seconds checking process tree for "minetestserver" and if "minetestserver" still running then run "killall -s KILL minetestserver". TERM signal is analog ctrl+c and it safely shutdown server. After minetestserver was killed script must wait 120 seconds and run minetestserver again. I can publish this script here if needed.

[Texan]

The "No YouTube Recording Without Permission" I believe, is there for other players' privacy. I, for one, do not want people simultaneously recording my gaming environment and chat, especially without my knowing. It would pretty much be someone recording me and my conversations with my friends. Of course, we do not share personal information in-game, but to have our chat and builds posted on the internet for whoever to see... I don't think it is right. This rule also could help the server avoid a whole lot of disputes between players who enjoy the private environment, versus the players who record it.

[amadin]

1. I propose to make crafting "Smartshop" more expensive, because: in real life, this requires a lot of money and a lot of time, so that players will not sell resources for next to nothing. Also propose replace "locked chest" by "barter shop" in "Smartshop" recipe.
2. I propose add sound for using /spawn and /home wich will hear for nearest players. Will be good if you add visual effect for this commands (change player screen color for 1 second or some effects near player model).

[Glorfindel]

Why should it cost more to be able to sell stuff?

If you don't like being able to get stuff cheaply I believe you are in the minority.

[amadin]

Why should it cost more to be able to sell stuff?

If you don't like being able to get stuff cheaply I believe you are in the minority.

Because game is more interesting if player get most resources and things by himself.

[Glorfindel]

Why should it cost more to be able to sell stuff?

If you don't like being able to get stuff cheaply I believe you are in the minority.

Because game is more interesting if player get most resources and things by himself.

I see. Well, feel free to not use the shops :) but I prefer to be able to buy a stack or two for a build rather than spending hours collecting materials myself.

[amadin]

Why should it cost more to be able to sell stuff?

If you don't like being able to get stuff cheaply I believe you are in the minority.

Because game is more interesting if player get most resources and things by himself.

I see. Well, feel free to not use the shops :) but I prefer to be able to buy a stack or two for a build rather than spending hours collecting materials myself.

Well, feel free to play on servers with creative mode :)

[Glorfindel]

Well, feel free to play on servers with creative mode :)

That's the point. I am free to play on a creative server. But why should I be limited in being able to buy stuff cheaply because you think it should be expensive??

[firefox]

Why should it cost more to be able to sell stuff?

If you don't like being able to get stuff cheaply I believe you are in the minority.

Because game is more interesting if player get most resources and things by himself.

the prices in shops are made by the players. it's their decision to sell stuff for low or high prices. based on my observations most players are greedy and make high prices. (like 1potato for 1 diamond. potatoes are the most common plant in every jungle, diamonds are the second most rare ore.)

[TenPlus1]

The network stall patch has been applied to today's build of minetest, so tonight we update server and <fingers crossed> no more problems :)

[Glorfindel]

Yay!!!

[TenPlus1]

Okie, network patch didnt do much it seems, still stalling so we ran in debug mode and here's the txt file if it helps anyone fix this:

update: ran debug build of minetest using verbose and gdb, attached is the output: