Page 1 of 1

DB Class

PostPosted: Sun Sep 09, 2012 20:43
by wieszak
Hello.

I decided to make some maybe small but needed for my next step change to code. I moved all database function to class Database, and create class Database_SQLite3 as implementation to currently used sqlite3. Howewer run into small problem - i need in this class link to ServeMap which is awful... Maybe you are able to patch it further to remove this. But i want to keep current API - functions from class Database takes and returns MapBlock - i want to keep serialization in this class, so i could later use another style of serialization for another db. My next step is to provide MySQL implementation and test if it would behave better for multiplayer servers.

Here is the patch against current 0.4.3 http://para.zonk.pl/patch-db.diff

If you like it maybe include this into mainstream?

PostPosted: Sun Sep 09, 2012 22:57
by cornernote
+1 for a mysql option

PostPosted: Tue Sep 11, 2012 05:34
by wieszak
Ok, i managed to run server with mysql. Unfortunately now database credentials are hardcoded, so i need to find out how to move them to config files and some similiar stuff. First impressions (as singleplayer) - i have feeleing, that server runs more smoothly: with sqlite i have some short hungups then and then, with mysql there is none. But maybe i need play longer... Anyway, there is some work to do before i realese full patch, and of course i'm not dropping support for sqlite, i plan leave it as default especially for single player - it is much easier to use for enduser.

PostPosted: Fri Sep 21, 2012 23:38
by cornernote
Nice, I'm keen to have a play with it.

PostPosted: Mon Sep 24, 2012 16:43
by wieszak
Ok. Here it is first 'public' version of patch for mysql support:

http://para.zonk.pl/mysql-0.1.1.patch

Server is sqlite3 by default, to use mysql you need to put in minetest.conf values for mysql_server, mysql_user, mysql_password and mysql_database. Database need to be created, tables are created by server, user need to get select, update, insert, delete and create priviliges. Each worlds data is located in table <world_name>_blocks, so better do not use fancy characters in name - it is not checked yet ;)

In few (or more...) days i will announce 24/7 server running on mysql with this patch for multiplayer testing.

TODO:
- maybe move config data to world config not server so user could choose storage at world creation...
- maybe move worlds metadata and players to mysql also
- change serialization for mysql - it would be helpful for next:
- write some external online tools to admin world ;)

PostPosted: Mon Sep 24, 2012 17:02
by rubenwardy
+1

Two questions:

How do you install/compile/build/add the changes in the above link?

Do you need to installing anything (ie: MySQL) to run in MySQL version?

PostPosted: Mon Sep 24, 2012 17:03
by xyz
wieszak wrote:Ok. Here it is first 'public' version of patch for mysql support:

http://para.zonk.pl/mysql-0.1.patch

Server is sqlite3 by default, to use mysql you need to put in minetest.conf values for mysql_server, mysql_user, mysql_password and mysql_database. Database need to be created, tables are created by server, user need to get select, update, insert, delete and create priviliges. Each worlds data is located in table <world_name>_blocks, so better do not use fancy characters in name - it is not checked yet ;)

In few (or more...) days i will announce 24/7 server running on mysql with this patch for multiplayer testing.

TODO:
- maybe move config data to world config not server so user could choose storage at world creation...
- maybe move worlds metadata and players to mysql also
- change serialization for mysql - it would be helpful for next:
- write some external online tools to admin world ;)

Nice.
BTW, line 260 of your patch seems to be wrong
Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.
Code: Select all
+        if (mysql_select_db(m_database,"minetest")) {

And there is also no FindMySQL.cmake

It works, but I can't determine whether it works faster or not.

PostPosted: Mon Sep 24, 2012 19:55
by wieszak
Yes, this line is even unnecessary, removed from patch (link edited)

FindMySQL.cmake i've got from mysql package. I know that it is not standart module for cmake, but it is not related witch patch itself. I will provide link to donwload it as soon as i remember from where i exectly got it.

For me is slighty faster but in singleplayer. Will see if it makes any differences in multiplayer when i (or somebody) setup server for people to test...

rubenwardy:
- download sources and patch, unpack sources
- enter mintetest directory
- apply patch with : patch -p1 </somewhere/mysql-0.1.1.patch
- compile minetest as in minetest's readme

If problems i see you sometimes at redcrabs server, ask me there (i'm smoki) or here .
Patch requires at least mysql client libraries and mysql headers to compile, then any mysql server to connect.

PostPosted: Tue Oct 30, 2012 08:44
by fgr
Hi,

is this still in development? I am searching a simple function to store events into a mysql db but i rather wont change the whole minetest backend db from a sqlite to mysql if there is no benefit on performance...

PostPosted: Tue Oct 30, 2012 21:33
by Dragonop
what is this mod adds

PostPosted: Wed Oct 31, 2012 03:30
by Alfino
Dragonop wrote:what is this mod adds

this isn't mods

PostPosted: Wed Oct 31, 2012 19:28
by fgr
any hint how to change the backend db in config? just still patched it and the database is still sqlite... no readme nor anything else, could you provide some more infos to it?

also i get the followin error:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.
Code: Select all
patching file src/CMakeLists.txt
patching file src/client.cpp
patching file src/clientserver.h
patching file src/database-mysql.cpp
patch: **** malformed patch at line 571: diff -ruN celeron55-minetest-9cadaf8/src/database.cpp celeron55-minetest-9cadaf8-mysql/src/database.cpp


any clue?

thanks

PostPosted: Wed Oct 31, 2012 19:53
by xyz
fgr wrote:any hint how to change the backend db in config? just still patched it and the database is still sqlite... no readme nor anything else, could you provide some more infos to it?

also i get the followin error:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.
Code: Select all
patch: **** malformed patch at line 571: diff -ruN celeron55-minetest-9cadaf8/src/database.cpp celeron55-minetest-9cadaf8-mysql/src/database.cpp


any clue?

thanks

I've already included that patch with two more backends (dummy and leveldb) into my db_backends branch

PostPosted: Wed Oct 31, 2012 20:29
by fgr
ho thats nice, so i need only compile it and create the mysql db, or does it by its own (the db)? because if i run the minetestserver (even in the config i postet the right mysql settings) there will no mysql db be used....???? The server creates the sqlite db as usual....

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.
Code: Select all
 
Server creating detached inventory "creative"
creative inventory size: 111
21:46:19: INFO[ServerThread]: ServerMap: SQLite3 database structure was createdServerMap: SQLite3 database opened

PostPosted: Tue Nov 27, 2012 11:29
by wieszak
Sory, been off some time.

fgr: default is still sqlite3, mysql is used when in minetest.conf you provide database credential - it is described in post with patch. However i dont know how xyz it did, need to look into it.

PostPosted: Sat Aug 17, 2013 00:00
by Pitriss
i want to ask, is there any progress in this patch? can be applied on latest client?

PostPosted: Fri Oct 25, 2013 19:03
by fgr
why isnt this considered anymore? is there any possibility to get short attention to it? thanks a lot!!!!!!

PostPosted: Fri Oct 25, 2013 19:44
by xyz
fgr wrote:why isnt this considered anymore? is there any possibility to get short attention to it? thanks a lot!!!!!!

It's merged already. MySQL backend isn't present though.

PostPosted: Sat Oct 26, 2013 08:05
by fgr
xyz wrote:
fgr wrote:why isnt this considered anymore? is there any possibility to get short attention to it? thanks a lot!!!!!!

It's merged already. MySQL backend isn't present though.


So it's not fully integrated because of different opinions about mysql. The game will probably have lack of speed when lot of mods loaded. This wont get better with leveldb.

PostPosted: Sat Oct 26, 2013 10:44
by xyz
fgr wrote:
xyz wrote:
fgr wrote:why isnt this considered anymore? is there any possibility to get short attention to it? thanks a lot!!!!!!

It's merged already. MySQL backend isn't present though.


So it's not fully integrated because of different opinions about mysql. The game will probably have lack of speed when lot of mods loaded. This wont get better with leveldb.

I don't see how this is related to not having MySQL backend.

PostPosted: Sat Nov 02, 2013 03:49
by aldobr
Have you ever tought about NoSQL ?

PostPosted: Tue Mar 11, 2014 17:26
by lightonflux
What is the current state of your MySQL support? Is it dead or just stagnating?

PostPosted: Tue Mar 11, 2014 17:36
by sfan5
You could use lua to communicate between Minetest and MySQL (or any other way of saving things).
1) Compile this branch
2) Use the lua backend in world.mt
3) Make the database_lua_scriptfile setting point at a Lua file that provides the following functions:
Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.
Code: Select all
function db_init()
    --Initialize database
end

function db_beginSave()
    --Is called before multiple transactions take place
end

function db_endSave()
    --Is called after multiple transactions have taken place
end

function db_saveBlock(x, y, z, blobdata)
    --Save MapBlock
end

function db_loadBlock(x, y, z)
    --Load MapBlock (return string)
    -- "" means the MapBlock could not be loaded (e.g. Not existant, Error occurred)
end

function db_listAllLoadableBlocks()
    --Return list of positions of all existing MapBlocks
end

function db_destroy()
    --Close database
end

PostPosted: Tue Mar 11, 2014 18:22
by twoelk
At some time in the future an open db api that allows easy coding of plugins to use any database out there now and maybe to come in the future? That is if Lua isn't allready sufficient for the job.

Re: DB Class

PostPosted: Tue Oct 14, 2014 17:22
by Johnny Joy
This is a great patch. What ever happened to it? Why did it not progress to the next level?