Page 1 of 1

How I learned to start worrying and hate above-water cities

PostPosted: Thu Mar 02, 2017 20:55
by Fixerol
I've noticed that some server owners and people like to build large cities and even spawns on large seas, it consists of thin layer of block above water with people buildings on it and below. What can go wrong with such innocent and nice idea?

When I played on original "Just Test", spawn and pretty large area around was plagued by slow fps and constant jitter. Back then I was worried but ehh.

After I learned you can enable debug wireframe mode by editing source code I've tried it and start studying various servers, its layout above and underground and how it renders.

I realised something that was quite obvious but often forgotten by everyone: minetest has special algorithm that decides what 16x16x16 areas to render and what not, if something is hidden from view - it is not rendered. You feel comfy at first, algorithm will sort everything out.

Then you go into wireframe mode and realise that algorithm is so inefficient that your lovely water city spawn with 4 buildings is rendered like a fps-eating monster world, you can see deepest parts of the sea with tons of garbage blocks made by people over the years, underwater tunnels, mines, trenches, terrain with cheese-like appearance.

The truth is: you render almost TWO worlds at the same time: city part (above), and sea floor with everything on it (below) and even deeper, you don't see it, yet it is rendered almost fully. That is the reason of abysmally low fps in such water cities. Every day sea floor accumulates more and more stuff, fps gets lower with time, players digging deeper expose more land that renders more etc etc.

How much it affects fps? Depends, if you take "just test"-like water city spawn and turn everything from 0 to -20 to solid stone, you can almost double your fps and remove a lot of server lag due to calculating liquid mechanics.

There's another problem with any big city with tall buildings in general (even on solid ground), LinuxGaming2 is one of the best examples. Algorithm is inefficient and you end up rendering a looooooot of stuff you don't even realise, like part of building behind building behind building. Another example is Vanessa Dreambuilder Survival server - that has nice and huge underground service network just below the roads with pipes and other stuff - everything is rendered, even if you can't see it.

Conclusions:
+ minetest occlusion culling algorithm is not very efficient
+ in water city practically everything below thin floor level down to sea floor is rendered
+ in land city a lot of tunnels and underground floors are rendered, but less aggressively.
+ in city with tall buildings a lot of stuff is rendered behind buildings
+ building city spawn over water is particularly bad idea, if sea floor is unprotected - already low fps degrades with time.
+ building spawn on land can offer much better fps, depending on design, you can achieve good results, say, you can make custom protected spawn with people building outside, you can protect at least underground parts of spawn to avoid overloading it with tunnels and such, you can forbid tall buildings on spawn, sky cities near spawn should be located high to avoid going into range (300 nodes is safe bet), limit number of underground tunnels etc etc

Image

Re: How I learned to start worrying and hate above-water cit

PostPosted: Thu Mar 02, 2017 21:24
by ExeterDad
+100
Very well done write up.

Re: How I learned to start worrying and hate above-water cit

PostPosted: Thu Mar 02, 2017 22:05
by webdesigner97
This is really interesting! I always thought that the low FPS at VE-Survival were because of the many different nodes, meshes and textures there, but according to what you wrote here, I know better now...

Re: How I learned to start worrying and hate above-water cit

PostPosted: Fri Mar 03, 2017 00:20
by Fixerol
webdesigner97 wrote:This is really interesting! I always thought that the low FPS at VE-Survival were because of the many different nodes, meshes and textures there, but according to what you wrote here, I know better now...

Occlusion culling inefficiency just makes it worse. A lot of low fps and jitter on VE-S spawn comes from changing nodes (mesh gen and metadata updates): technic, mesecons, road semaphores etc.
Here is example of what is rendered underground, it does also depend on your point and angle of view: https://i.imgur.com/3Rk2mgY.jpg and https://i.imgur.com/R8Gh7Zs.jpg

Re: How I learned to start worrying and hate above-water cit

PostPosted: Fri Mar 03, 2017 00:39
by Jordach
Image
fps_max = 60

Image
Try me.

Stats:

Image
Image

Re: How I learned to start worrying and hate above-water cit

PostPosted: Fri Mar 03, 2017 00:46
by Fixerol
Thats kinda offtopic :p, Jordach has high-end PC with i7 and AMD RX480 (8gb)

Re: How I learned to start worrying and hate above-water cit

PostPosted: Fri Mar 03, 2017 02:59
by Jordach
Fixerol wrote:Thats kinda offtopic :p, Jordach has high-end PC with i7 and AMD RX480 (8gb)

Is it?

Modern hardware has better IPC and, late 2000s hardware don't stand up as well as they used to.

Minetest is more dependent on CPU than GPU due to Irrlicht doing all mesh operations in RAM, not VRAM.

This is why pre AMD Ryzen CPUs are terrible with Minetest specifically. This also extends to Intel Pentium (recent LGA models lower than the i3) and Celeron / Atom series.

I would potentially have even better performance if I was using a native Linux machine. However, due to poor nonfree and free drivers the gain would be worthless overall. AMD RX 480 drivers are bad in both Radeon and fglrx.

I remember RBA complaining about Irrlicht not using the GPU for meshes in memory. VBOs help, but not a lot overall due to Minetests poor occlusion culling.

Addendum: As Minetest isn't multi threaded, AMDs "moar cores" meme doesn't apply. This also goes for HT Intel CPUs too.

Re: How I learned to start worrying and hate above-water cit

PostPosted: Fri Mar 03, 2017 06:48
by webdesigner97
Jordach wrote:
Fixerol wrote:Thats kinda offtopic :p, Jordach has high-end PC with i7 and AMD RX480 (8gb)

Is it?

Modern hardware has better IPC and, late 2000s hardware don't stand up as well as they used to.

Minetest is more dependent on CPU than GPU due to Irrlicht doing all mesh operations in RAM, not VRAM.

This is why pre AMD Ryzen CPUs are terrible with Minetest specifically. This also extends to Intel Pentium (recent LGA models lower than the i3) and Celeron / Atom series.

I would potentially have even better performance if I was using a native Linux machine. However, due to poor nonfree and free drivers the gain would be worthless overall. AMD RX 480 drivers are bad in both Radeon and fglrx.

I remember RBA complaining about Irrlicht not using the GPU for meshes in memory. VBOs help, but not a lot overall due to Minetests poor occlusion culling.

Addendum: As Minetest isn't multi threaded, AMDs "moar cores" meme doesn't apply. This also goes for HT Intel CPUs too.

I wonder why Irrlicht does this. I'm running the game on a poor FX-4130@4.1GHz, while my GPU is basically idling. That's like pushing a car on your own instead of using the engine :/

Re: How I learned to start worrying and hate above-water cit

PostPosted: Fri Mar 03, 2017 07:58
by rubenwardy
It's how minetest is designed, not irrlicht

Re: How I learned to start worrying and hate above-water cit

PostPosted: Fri Mar 03, 2017 14:22
by Jordach
Didn't RBA complain about Minetest's specific usage of Irrlicht?

This wasn't a shaders issue - it was more like how monolithic and un-modular the graphics back end was. More time in my screenshots was doing rendering than the game logic.

Re: How I learned to start worrying and hate above-water cit

PostPosted: Fri Mar 03, 2017 14:29
by Fixerol
Addendum: As Minetest isn't multi threaded, AMDs "moar cores" meme doesn't apply. This also goes for HT Intel CPUs too.


Did you read first message at all? It is not about AMD or anything, it is about occlusion culling inefficiency.

Re: How I learned to start worrying and hate above-water cit

PostPosted: Fri Mar 03, 2017 16:10
by burli
I can see the inefficiency on the PC from my daughter. She has an old core2duo with 1,8GHz, 4GB RAM and a Nvidia GT630, if I remember. There is Ubuntu Mate 16.04 installed

She plays Starstable with wine and has relatively good and constant 30FPS, with a nice view range, shadows and other nice stuff. If I compare that with Minetest I have to disable shaders and need to reduce the view range to get playable frame rates

Edit: it is just a Nvidia GeForce 210

Re: How I learned to start worrying and hate above-water cit

PostPosted: Fri Mar 03, 2017 22:34
by Robsoie
I hope a dev will find a solution to improve this heavy problem.

Re: How I learned to start worrying and hate above-water cit

PostPosted: Sat Mar 04, 2017 01:46
by cheapie
Fixerol wrote:
Addendum: As Minetest isn't multi threaded, AMDs "moar cores" meme doesn't apply. This also goes for HT Intel CPUs too.


Did you read first message at all? It is not about AMD or anything, it is about occlusion culling inefficiency.


It sounds like we have two issues - broken occlusion culling, and using the CPU/RAM for things that should be done with the GPU/VRAM?

Re: How I learned to start worrying and hate above-water cit

PostPosted: Sat Mar 04, 2017 01:49
by Jordach
cheapie wrote:
Fixerol wrote:
Addendum: As Minetest isn't multi threaded, AMDs "moar cores" meme doesn't apply. This also goes for HT Intel CPUs too.


Did you read first message at all? It is not about AMD or anything, it is about occlusion culling inefficiency.


It sounds like we have two issues - broken occlusion culling, and using the CPU/RAM for things that should be done with the GPU/VRAM?

Minetest generates meshes and textures on CPU. Not using modern GPU geometry shaders either.

Re: How I learned to start worrying and hate above-water cit

PostPosted: Tue Mar 14, 2017 05:06
by Robsoie
Found a link to this article that reminds me a lot of that underground rendering problem Minetest has and how that developer worked for a solution :
https://tomcc.github.io/2014/08/31/visibility-2.html