LevelDB Test Server

[BrandonReese]

I have setup a sever running the new levelDB backend. It's basically a copy of my LandRush Server using LevelDB so if you are already on LandRush you already have a username, password, etc... on this server. The landrush mod has been removed, this is not for serious building. Report any map anomalies or other issues in this topic. This server is for testing so I don't know how long it will be up.

Address: 192.95.62.241
Port: 32000

[fairiestoy]

I played about 10 - 20 min and checked parts from the main town to sicdonia and kept walking south to generate new chunks. The first thing that I encountered was: All sign rotations were wrong. Then i encountered some strange tree generation errors while exploring new chunks (half cut off trees, some trees missed the trunks but generated leaves, some trees were mixed like rubber + pine ). But anything else seem to work as it should.
If you need screenshots, i have made a few.

[BrandonReese]

I played about 10 - 20 min and checked parts from the main town to sicdonia and kept walking south to generate new chunks. The first thing that I encountered was: All sign rotations were wrong. Then i encountered some strange tree generation errors while exploring new chunks (half cut off trees, some trees missed the trunks but generated leaves, some trees were mixed like rubber + pine ). But anything else seem to work as it should.
If you need screenshots, i have made a few.

I got the sign rotation problem too, That has happened before but I don't remember what I did to fix it. I don't think it's related to LevelDB. I'll see if I can check out the mapgen problems you saw.

Did you feel it was faster, slower, or about the same speed when loading chunks?

[fairiestoy]

Its hard to say without many user traffic. To determine if already generated chunks are loading faster from the map, a few more people should start a test visit on this server (in my opinion). Since most problems occur, when the server is nearly full.
Except the problems when generating chunks, everything loaded smooth. I didn't ran into any grey areas of not loading chunks. Is there a possibility to add some time values to the new backend to see the differences in seconds between the old versions and this one?

[Sokomine]

Regarding how "fast" chunks are loaded on Redcrabs server, and regarding how "fast" fast running on this server here is, I'm afraid the speed of the database backend may be utterly insignificant here. It may play a role regarding possible map size. AFAIK sqlite based databases can grow only to 4 GB, while leveldb does not have such limits, so leveldb is a good investment into the future.

[fairiestoy]

I did a quick research because i couldn't remember any file size limitations. And i found this:

Maximum Number Of Pages In A Database File

SQLite is able to limit the size of a database file to prevent the database file from growing too large and consuming too much disk space. The SQLITE_MAX_PAGE_COUNT parameter, which is normally set to 1073741823, is the maximum number of pages allowed in a single database file. An attempt to insert new data that would cause the database file to grow larger than this will return SQLITE_FULL.

The largest possible setting for SQLITE_MAX_PAGE_COUNT is 2147483646. When used with the maximum page size of 65536, this gives a maximum SQLite database size of about 140 terabytes.

The max_page_count PRAGMA can be used to raise or lower this limit at run-time.

Maximum Database Size

Every database consists of one or more "pages". Within a single database, every page is the same size, but different database can have page sizes that are powers of two between 512 and 65536, inclusive. The maximum size of a database file is 2147483646 pages. At the maximum page size of 65536 bytes, this translates into a maximum database size of approximately 1.4e+14 bytes (140 terabytes, or 128 tebibytes, or 140,000 gigabytes or 128,000 gibibytes).

This particular upper bound is untested since the developers do not have access to hardware capable of reaching this limit. However, tests do verify that SQLite behaves correctly and sanely when a database reaches the maximum file size of the underlying filesystem (which is usually much less than the maximum theoretical database size) and when a database is unable to grow due to disk space exhaustion.

Maybe i missunderstood it, but it looks like the file could easily grow out more then 4GB (correct me please if this is wrong)

[Pitriss]

Brandon: I don't think that this test have some value without some mods, especially landrush one.. Even without landrush, homedecor and maybe other missing features I had speed almost not changed. I tried also fall into my mine and speed of chunks loading was circa the same as on normal landrush server. There was online 2 other ppl.

[sfan5]

I did a quick research because i couldn't remember any file size limitations. And i found this:

Maximum Number Of Pages In A Database File

SQLite is able to limit the size of a database file to prevent the database file from growing too large and consuming too much disk space. The SQLITE_MAX_PAGE_COUNT parameter, which is normally set to 1073741823, is the maximum number of pages allowed in a single database file. An attempt to insert new data that would cause the database file to grow larger than this will return SQLITE_FULL.

The largest possible setting for SQLITE_MAX_PAGE_COUNT is 2147483646. When used with the maximum page size of 65536, this gives a maximum SQLite database size of about 140 terabytes.

The max_page_count PRAGMA can be used to raise or lower this limit at run-time.

Maximum Database Size

Every database consists of one or more "pages". Within a single database, every page is the same size, but different database can have page sizes that are powers of two between 512 and 65536, inclusive. The maximum size of a database file is 2147483646 pages. At the maximum page size of 65536 bytes, this translates into a maximum database size of approximately 1.4e+14 bytes (140 terabytes, or 128 tebibytes, or 140,000 gigabytes or 128,000 gibibytes).

This particular upper bound is untested since the developers do not have access to hardware capable of reaching this limit. However, tests do verify that SQLite behaves correctly and sanely when a database reaches the maximum file size of the underlying filesystem (which is usually much less than the maximum theoretical database size) and when a database is unable to grow due to disk space exhaustion.

Maybe i missunderstood it, but it looks like the file could easily grow out more then 4GB (correct me please if this is wrong)

IIRC SQLite files larger than 4 GB don't perform very stable on some systems

Anyway, LevelDB is still faster and is more stable in general.

[trukoil]

I've read somewhere that on SQLite is better (~= faster) to insert, e.g., 10 values with a single INSERT than 10 separate INSERT with a single value; doing this would require a deep rewrite of the map engine, or the implementation of a queue system to perform the query after a determined number of actions. BTW, i don't know if Minetest already does that, but the best thing to do with databases is to adapt the workload to their faster, or best, work style.

[PilzAdam]

The end-user on a server wont notice any speed difference with leveldb, the network latency is just too high and changes too much to measure the milliseconds that leveldb might be faster or slower.

[trukoil]

Theoretically, a faster map backend should lower latency?

But practically it wouldn't be noticed.