3D meshes for players and creatures [Models are now upstream!]

[jin_xi]

you also need to clone minetest_game from same github into games folder

[mito551-guest]

oh, ok, thx!

[Mito551]

it really works :) nice!
looking forward to seeing this in mainstream ;)

[MirceaKitsune]

Alright, I've been working for another full day on this code. There is some good news and some bad news.

The bad news is that several features are either not implemented yet or being buggy and not working. Although I might make changes to the code, I've reached the amount of knowledge and energy I have to continue, and from this point require help from other developers. If no one offers there's a high chance this might not get fully finished... instructions on what I need further below.

The good news is that the basics work and are finished. Objects can be set to nodes which can be animated from LUA, and there should be enough LUA hooks to get a player walk and mine animation at least. For this reason I might begin working on a player model soon, which will be rigged animated and textured. The .blend source will of course be available in GIT as well, so anyone can edit the model later on. But I need to see where the rest of the code is going before I get to this.

Issues list:

This is a list with the things left to do, either because they're not implemented or are buggy. They are things I can't figure out on my own, and without help on them I might not be able to continue. If anyone is familiar with the MineTest code (both the C and lua part) any help is greatly appreciated.

* Attachment system not implemented:

- Bug: I could not implement the attachment system so far, which aims to allow attaching objects to other objects on skeletal bones (so players can visibly hold items for example).

- Cause: I don't know how to network (serialize or send via genericobject) the object reference of the parent we're attaching to. Therefore I have no way in client_cao.cpp to know what we're attaching on. I don't know if there are other issues past that.

- Code: The framework for the attachment system is done, but no data goes through it. See how setAnimations or setBonePosRot are done for info on what to add (they're typically above the setAnimations framework in each file). The function in which the effect will be applied is content_cao.cpp line 1194: void updateAttachment() Also, a very useful reference on how it seems attachments should be done with Irrlicht: http://irrlicht.sourceforge.net/forum/viewtopic.php?t=7514

* Bone positioning crashes the client at render time:

- Bug: If attempting to change the position or rotation of a skeletal bone, the MineTest client crashes when looking at the model.

- Cause: Unknown, normally it should work. The console doesn't say a lot so GDB might be needed.

- Code: The lua call to set bone origin / rotation is of the form: obj:set_bone_posrot("bone", {x=0,y=0,z=0}, {x=0,y=0,z=0}) The C++ lines which set this are content_cao.cpp lines 1188 and 1189, bone->setPosition(bone_pos); and bone->setRotation(bone_rot);

* Material colors not working:

- Bug: I implemented a feature which should allow setting the color of each material, including on sprites and cubes. Although it should be fully implemented, it doesn't work in practice, and neither material alpha or color are affected.

- Cause: Irrlicht doesn't have a lot of ways to change the color of a material (just the full mesh, but not materials which is what I want it to do). The current method I left in the code requires objects to be lit, which is not the case with MineTest's lighting system. I believe an Irrlicht alternative of sorts needs to be found, or a change to the lighting system must be done.

- Code: In lua, when doing obj:set_properties, add the following property to change the color of a material (works the same way as textures): colors = {{255, 255, 255, 255}, } The C++ code is located in various places around the content_cao.cpp file. Look where m_prop.colors is used in this file.

* Animation blending not working:

- Bug: If blending is enabled on an animation, the model no longer animates. It stays stuck at the first frame and its normals also appear to become flipped (mesh is rendered inside out).

- Cause: Unknown. Might be an Irrlicht problem or an issue with the object render settings in MineTest.

- Code: In the animation LUA command (eg: obj:set_animations({x=1, y=50}, 35, 0) ) the last parameter represents blending. The issue occurs when that value is greater than 0. The C++ line which sets it is contant_cao.cpp, line 1175: m_animated_meshnode->setTransitionTime(m_frame_blend);

End of list

I'm asking for anyone who's willing to look into this and help fix the four remaining issues. You are free to fork my GIT repositories on github and work on your own versions, I can look at them and merge them to mine as soon as they're ready. If you're unsure about something ask anytime, I'd rather functionality is done in the same way as the working code if possible. If you can't use github, even sending a GIT diff would help if you find solutions. My GIT branches are:

Code branch: https://github.com/MirceaKitsune/minetest/tree/models
Game branch: https://github.com/MirceaKitsune/minetest_game/tree/models

To look at the code and compare it to existing MineTest (if you can't use GIT this makes it easy to see my changes):

Code branch: https://github.com/MirceaKitsune/minetest/compare/models
Game branch: https://github.com/MirceaKitsune/minetest_game/compare/models

Irrlicht documentation is available here: http://irrlicht.sourceforge.net/docu/ Almost all model settings use the Irrlicht functions so those are very important. I hope someone will volunteer and maybe in the next days we will finish this code, I can work on making a player model then :)

[MirceaKitsune]

Another update with more good news and bad news about attachments. Good news is that I was able to figure out how to identify the parent object client-side, using the same Id as the server. I added that to the animation framework and completed it... now I just need to finish the code detecting bone location to set origin and rotation to that.

Bad news is that I just discovered something obvious: Although object entities and their position exist on both the client and server, the animated model is client-side. That means I have to take a choice (as far as the code is concerned): If I attach server-side, the server knows of the attachment's position / rotation and things like punching the attachment or reading its real location with LUA are possible. However, the server does not know of the model, and therefore can't tell where the position of a skeletal bone at a certain animation frame is. If I attach client-side instead, I can tell where the position of any skeletal bone on the parent is, at a certain animation frame of the parent model. But the server doesn't know where the attachment is being moved, so it becomes only a visual effect on clients. Players for instance cannot be attached to minecarts this way... others would see them as such but the player in cause wouldn't even notice a difference, because on the server he's still in the same spot.

So I'm going to take the least painful approach. The attachment will always be attached server-side to its parent, and copy its origin and rotation. However, if skeletal animation is also enabled, then in the client setting it to that bone. In this case the server sees the attachment at the same origin as its parent, but the client will show it on the specified bone (only as a visual effect). It's sadly the only way I can think of, so I'll see how this works out.

In the meantime, feel free to look at the other bugs. I'm still far from solving these, and it might be some time until attachments will be fully working.

[Temperest]

It seems to me that with a bit of server side math, it would be possible to use the client side attachment and apply the transformation matrices to the attached object to get its current position.

But then again, I'm not sure how useful getting the world coordinates of an animated bone is. In my opinion, server side attachment is a great solution.

[MirceaKitsune]

It seems to me that with a bit of server side math, it would be possible to use the client side attachment and apply the transformation matrices to the attached object to get its current position.

But then again, I'm not sure how useful getting the world coordinates of an animated bone is. In my opinion, server side attachment is a great solution.

No. Because when multiple players are connected to one server, each client might be seeing their own version of an animated model. The server wouldn't know which one to get. Most gentle way is copying object origin and rotation server-side, and if a bone attachment is set then send the message to the client. A parent's bone modifier shouldn't position the model too far away from the origin of the parent, so no one should normally feel or notice any difference (attachments will likely be made un-clickable either way for other reasons too).

[MirceaKitsune]

Today I managed to get *server side* attachments working. Only issue there is that when a player is attached, he will not notice a difference and be able to walk around as usual, but others see him fine. That's probably because the player's client needs to read camera position from the server while he's is attached, not the other way around. Note that you won't see attachments working because I disabled networking their position (they'll be attached client-side so there's no need to waste bandwidth). I tested them and their server position is updated properly.

Client side attachments are still not functional because I've not found a way to implement them. I am however sending all needed data and have the function that handles them... just need a few lines of code to do the attaching of our object to its parent (via Irrlicht). The empty function contains a code comment explaining what needs to be done in detail, as well as the functions (even links to what I used). If anyone with a bit more knowledge in MineTest and Irrlicht could look into it, a solution can likely be found. If you wish to try and help, read the comment in client_cao.cpp line 1214, void updateAttachments(). I'll also be quoting it here:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

        // REMAINING ATTACHMENT ISSUES:
        // We get to this function when the object is an attachment that needs to
        // be attached to its parent. If a bone is set we attach it to that skeletal
        // bone, otherwise just to the object's origin. Attachments should not copy parent
        // position as that's laggy... instead the Irrlicht function(s) to attach should
        // be used. If the parent object is NULL that means this object should be detached.
        // This function is only called whenever a GENERIC_CMD_SET_ATTACHMENT message is received.
       
        // We already attach our entity on the server too (copy position). Reason we attach
        // to the client as well is first of all lag. The server sends the position
        // of the child separately than that of the parent, so even on localhost
        // you'd see the child lagging behind. Models are also client-side, so this is
        // needed to read bone data and attach to joints.
       
        // Functions:
        // - m_attachment_parent is ClientActiveObject* for the parent entity.
        // - m_attachment_bone is std::string of the bone, "" means none.
        // - m_attachment_position is v3f and represents the position offset of the attachment.
        // - m_attachment_rotation is v3f and represents the rotation offset of the attachment.
       
        // Implementation information:
        // From what I know, we need to get the AnimatedMeshSceneNode of m_attachment_parent then
        // use parent_node->addChild(m_animated_meshnode) for position attachment. For skeletal
        // attachment I don't know yet. Same must be used to detach when a NULL parent is received.

        //Useful links:
        // http://irrlicht.sourceforge.net/forum/viewtopic.php?t=7514
        // http://www.irrlicht3d.org/wiki/index.php?n=Main.HowToUseTheNewAnimationSystem
        // http://gamedev.stackexchange.com/questions/27363/finding-the-endpoint-of-a-named-bone-in-irrlicht
        // Irrlicht documentation: http://irrlicht.sourceforge.net/docu/

GIT branch for code: https://github.com/MirceaKitsune/minetest/tree/models
GIT branch for game: https://github.com/MirceaKitsune/minetest_game/tree/models

If you have any knowledge in the code please take some time to look at this. I tried nailing it down as closely as possible so anyone can understand what I have so far and what's left. I'm learning both C++ and the MineTest code as I go, so I can't do everything on my own. If you want 3D players to happen I will require help with the last touches on this code, and especially testing after.

[GloopMaster]

now make player skins. client-based, sent to server. :D

[Dragonop]

what is the program wheresn't you use to made 3D things?

[MirceaKitsune]

now make player skins. client-based, sent to server. :D

Not something I'll be doing. It would require a reverse file stream first of all, which is beyond any of my abilities to code. Sending stuff from a client to the server can also be risky and make griefing more easy. I also don't see a point in allowing custom skins if we don't allow custom models too... if this ever gets added I'd like to see it done for both.

As far as the code is concerned however, you could override the model of any entity locally (as models are client-side). So if you have a player model you want everyone to appear as on your end, that would work out. But currently that's not a priority, and there's also no client-side lua API to allow this so it would have to be a setting.

what is the program wheresn't you use to made 3D things?

Not sure what you meant there. Did you mean my 3D modeling program? Only one I know and use is Blender, http://www.blender.org/

[Dragonop]

oh that is exactly what i whant thanks

[Dragonop]

THIS MOD MAKES PLAYER 3D! http://minetest.net/forum/viewtopic.php?id=897

[Menche]

THIS MOD MAKES PLAYER 3D! http://minetest.net/forum/viewtopic.php?id=897

That is a source patch, not a mod, and can only be applied to a very old version of minetest.

[MirceaKitsune]

THIS MOD MAKES PLAYER 3D! http://minetest.net/forum/viewtopic.php?id=897

That is a source patch, not a mod, and can only be applied to a very old version of minetest.

That and it uses hard-coded node boxes, not actual meshes with skeletal animations. I think MineCraft does that too, and IMO that's a reason why some detail is very limited. There are a lot of advantages to using rigged models, both in terms of flexibility and features, but I also think this should be faster and better in terms of performance as well.

[Dragonop]

Menche wrote:

Dragonop wrote:
THIS MOD MAKES PLAYER 3D! http://minetest.net/forum/viewtopic.php?id=897

That is a source patch, not a mod, and can only be applied to a very old version of minetest.

[Dragonop]

Menche wrote:

Dragonop wrote:
THIS MOD MAKES PLAYER 3D! http://minetest.net/forum/viewtopic.php?id=897

That is a source patch, not a mod, and can only be applied to a very old version of minetest.

NO IT WORKS FINE IN 0.4.3 !

[Menche]

Menche wrote:

Dragonop wrote:
THIS MOD MAKES PLAYER 3D! http://minetest.net/forum/viewtopic.php?id=897

That is a source patch, not a mod, and can only be applied to a very old version of minetest.

NO IT WORKS FINE IN 0.4.3 !

How? I tried applying it, it wouldn't build with the patched files.

[MirceaKitsune]

If that's ok, I'd prefer discussion about that patch is kept in its own topic. Not that I mind, but I'm trying to get focus on finishing the models code, and since I posted about needing help no one has taken one look at the code yet. Otherwise, I think the nodebox player patch would still work after this too, but IMO I see no point in it now that we can use real meshes.

[Mito551]

IMO I see no point in it now that we can use real meshes.

+1

[PilzAdam]

For those who cant wait for 3D players: http://minetest.net/forum/viewtopic.php?id=3585
Its a simple mod (yes, a mod; you can install it like other mods) that adds simple 3D players.

[MirceaKitsune]

There are some technical updates on this (no new functionality yet). I managed to get the mesh node of the child object to know what the mesh node of the parent is (client-side), which has been the biggest issue in the last days. Attachments are also hidden and ignored when attached to the local player, which was the reason for the last segmentation fault. Credits for the help here go to thexyz, and TheRedMood also offered to look into the code and help which I appreciate.

Right now I just finished coding in the Irrlicht functions to attach the object (mainly child->setParent(parent)), but for some reason they don't work yet. This should be the last major issue with attachments... why isn't the child being attached visually for clients?



The image above is a random screenshot I took during testing. The object to the left is me (I connect with 2 clients to test) and the one to the right is the attachment (lua entity) which is displaying at origin 0 0 0 instead of sticking to my player's mesh. Tested with both bone and direct object attachments and neither work.

[MirceaKitsune]

Today was some big progress on the code. Client attachments and detachments should technically work now. They're properly attached to the parent, but for an unknown reason their origin changes a few ticks after. Will try to find out why tomorrow. Bone attachments were not tested yet but should work too, and other major bugs likely exist. I still get some segmentation faults as well and must find out why.

Another important change is that all features in this code are now sent to new clients when initializing entities. Previously, a player would only see attachments or animations working if he was near the object when the script was triggered. With the latest GIT, if a client logs in after those settings are set the server will send them. Therefore the script only needs to run once per entity (or whenever it wishes to make a change), not every X seconds to update clients that might be seeing our object for the first time.

There's still stuff left to do, but the code is slowly reaching testing point. Once that happens I'll probably make a more solid test script and post a list of what's left to fix and check. If all goes well I'll get working on the player model too, but for now the code stays the biggest concern.

[MirceaKitsune]

If anyone's been wondering what happened in the last days, yes... I'm still working on the code. And I'm happy to announce big progress.

I've been fixing the remaining segmentation faults, which were caused by a lot of weird and complicated factors. After analyzing the code for a day, I realized that the bad memory reads were due to children remaining parented to an object in Irrlicht while that object's mesh node was removed (including to update its visuals). I addressed that by implementing a new way of storing children and parents, and detaching all children of a parent when updating its visuals then attaching back. This was the biggest and hardest issue still present in the code.

Not only that the segmentation faults should now be gone. But with today's changes, client-side attachments are finally working. I can attach a lua object to a player, and other players properly see it attached. No jittering, no lag, all good.

But it's not all that perfect and working either: If the player disconnects, its attachment goes to origin 0,0,0 which should have been addressed in my code. Tomorrow I will check why. Also, bone attachments don't work as they should. When the child is parented to a bone, it properly copies its origin and rotation. But as the parent mesh is animated, the attachment doesn't keep following the bone. This too needs to be investigated.

Things are going well pretty fast, so expect more good news soon. Here's a screenshot of attachments in action:

[Dragonop]

HO GOD SO COOL!

[MirceaKitsune]

The last two days have once again brought a lot of progress. Yesterday, all remaining issues with attaching and detaching lua entities have been fixed. Today, attached players have also been completed and are fully functional. An attached player will be unable to move and will see himself smoothly following the parent, without any additional data needing to be sent by the server. As of the last commit, object to object attachments should work flawlessly. Vanessa's patch for anisotropic filtering and mip-mapping was also merged today, so I enabled those features on models too.

At this moment, there's only two known issues with the models code: Attempting to manually animate joints doesn't work and breaks animations instead. Also, bone attachments don't work properly either. Those are problems regarding the Irrlicht functions, and I'm currently waiting for input from the Irrlicht people about them. If anyone has experience with this engine, feel free to look at the code and figure out what's happening. Here are the topics on the Irrlicht forum about those issues in detail:

http://irrlicht.sourceforge.net/forum/viewtopic.php?f=1&t=47579

http://irrlicht.sourceforge.net/forum/viewtopic.php?f=1&t=47580

I'm currently waiting on this last problem. But in a few days, I'm planning to announce the code as finished and ready for testing. I'll post information here as to what and how needs to be tested, so stay tuned.

[Mito551]

thank you! doing great job there!

[jin_xi]

hi there, ive been testing your latest code along with PilzAdams carts mod. Its so cool to see how this works smoothly, and i cant wait to see all the cool rides people are gonna build. One thing tho is that the mesh of the player attached to (riding) a cart is not visible to other players.

[MirceaKitsune]

hi there, ive been testing your latest code along with PilzAdams carts mod. Its so cool to see how this works smoothly, and i cant wait to see all the cool rides people are gonna build. One thing tho is that the mesh of the player attached to (riding) a cart is not visible to other players.

It is, but the player in the cart becomes smaller. That's because the visual_size of the cart is not 1,1 and in Irrlicht children also copy the size of the parent. This can't be helped... the cart will either need to resize itself in another way, or set the visuals of the player to be larger too in order to compensate. I'd prefer the first method... currently testing this to see what can be done.

[MirceaKitsune]

Doing some final tests on the bone functions, until I get news from Irrlicht users about the issues I can't solve. It seems bone transformations do work, although enabling them breaks animations and inverts the mesh normals. Here's me giving a large value to the position of a joint for the test player mesh, via the lua script. The result is funny :)



[EDIT] W00t! Just tested bone attachments again, and they also work! I attached the second player to the animated bone of the first player. The third player can see him leaning together with the bone as the mesh animates. This means that visibly hand-held tools should now be possible. But once again, enabling joint control does not work properly and breaks everything, which is the last remaining issue. Here is a screenshot showing this in action... the child player leans back and forth as the parent mesh animates: