[Mod] QA-Block [qa_block]

[bell07]

This is a developer helper mod, allow run any lua code for testing reason. The mod can list and run lua-scripts placed in checks subfolder. some check scrips provided.
A second future is a browser trough lua variables/global tables.




Features

• redirection of print() output to minetest chat. (can be disabled by changing code line "print_to_chat")
• robust call of the scripts trough "pcall" does not crash the game in case of syntax- or runtime errors
• all functionality available trough chat commands and the QA-Block node
• refresh and list the checks script list at runtime
• edit the code before calling them
• type code and run them (with smartfs only)
• browser trough lua variables/global tables.

Dependencies

• smartfs - https://github.com/minetest-mods/smartfs
  GUI for check selection and manipulation. Optional, but without smartfs there is limited functionality available.
  smartfs is provided again (my version) because of missed futures in upstream

Available check modules

• empty - Empty file for your own checks
• global_variables - List suspicious global variables
• is_ground_content - This checker lists all nodes for which is_ground_content == true
• list_entities - Lists all the registered entities (except builtin)
• list_spawning_mobs - List entities that are mobs from mobs_redo or compatible framework
• no_item_description - Lists all items without description
• same_recipe - Find duplicate crafting recipes
• unobtainable_items - Lists items which seem to be unobtainable
• useless_items - Lists all items which are probably useless
• get_items_csv - Write a csv-file from minetest.registered_items to the world folder that can be pivot-analyzed in a spreadsheet like localc or excel

How to use:

• add the mod to the game you like to test

Using chat command

• /qa help - print available chat commands
• /qa ls - list all available check modules
• /qa ui - display and run check using the selection dialog (smartfs only)
• /qa checkname - run check
• /qa set checkname - set default check
• /qa - run default check

Using the block

• get the QA-Block from creative inventory
• place the block somewhere
• - without smartfs - wait till the default check is finished and the block disappears
• - with smartfs - start the check using selection dialog

In all cases - check the debug.txt for test results

License GPL-V3: https://www.gnu.org/licenses/gpl-3.0.html
Github: https://github.com/bell07/minetest-qa_block

Credits: Wuzzy, thanks for ideas, code optimizations and the most check scripts

[Wuzzy]

Interesting idea, although I think if the mod would include more checking scripts, it will become more useful.
Just puking the entire script output into chat does not really seem useful to me. Even your screenshot shows that.
Although it is useful later to extract it from the log files. But not when using Minetest.
I have an idea: Post any script output in a separate formspec window. Honestly I am not sure if this is a good idea.

The list_spawning_mobs module has an error: If the mobs variable is not present, the module will show a runtime error.

I have ideas for some other scripts:
- List all global variables (important to hunt down “suspicious” variables which should never be global)
- List all items/nodes/tools/craftitems
- List all recipes

I would also like to have an option to disable redirection of print() at runtime. I suppose there are situations even for developers where this redirection may cause trouble. I have not tested it but maybe this could interfere with the luacmd mod.

The default dependency should be removed no matter what. This is a development tool after all, it should not hard depend on any mod. Or the dependency should at least be optional. Ideally, just draw a simple texture (the default texture is confusing anyways). Sounds are not that important.

Oh, and depends.txt does not support comments. If a dependency is not present, just don't put it there. Everything else will just confuse Minetest.

[bell07]

Thank you for your remarks. The main goal of this mod was previously to get a way to run lua code snippets without restart the MT on each change and that does not crash on syntax errors. It was my way to learn lua and minetest-API. But now it "might" be a collection of helpers code snippets that can anyone need for quality checks. So all code snippets welcome! Even if one line check!

The check output should be read from debug.txt or from console. Usually I start the MT trough the command line in windowed mode so I can work on scripts and read the output without leaving MT. I updated the screenshot to reflect this.
I did not found any way to redirect the check output only, the redirection print ALL messages to the chat. So I see no way to redirect the print messages (from a check run) to something like a formspec. The current redirection to the chat is just for fun, I like the background noise in chat that where I can see if something happens. If you don't like the noise in chat it or if it interferes with any other mod, just disable them in the source code.

The list_spawning_mobs module has an error: If the mobs variable is not present, the module will show a runtime error.

fixed

List all global variables (important to hunt down “suspicious” variables which should never be global)

developed. The other ideas you need more specify or just paste the code snippets for them ;)

I would also like to have an option to disable redirection of print() at runtime

Please disable it in source code if you don't need it.

The default dependency should be removed no matter what.

Dependency removed. An own png added for the tool.

[Wuzzy]

Thank you very much! With the globals check, this becomes very useful. If I find the time, I will contribute additional checkes for this block. I can recall that some people around these forums posted some checking snippets, it might be useful to gather these.
The more checker scripts are added, the more useful the node will become.

The globals checker still shows many globals even if I use this node in the Empty subgame (subgame without any mods) with only qa_block and origin mod installed.
I figured that some globals are part of the Lua API and can be safely blacklisted:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

   dump2 = true,
   PerlinNoise = true,
   PerlinNoiseMap = true,
   VoxelManip = true,
   VoxelArea = true,
   AreaStore = true,
   SecureRandom = true,
   PcgRandom = true,
   PseudoRandom = true,
   ItemStack = true,
   Settings = true,


However, even after clearing this up, the globals check still shows many globals, namely:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

   cleanup_path
   gcinfo
   on_placenode
   hack_nodes
   file_exists
   nodeupdate
   check_attached_node
   drop_attached_node
   newproxy
   get_last_folder
   spawn_falling_node
   on_dignode
   basic_dump
   nodeupdate_single
   INIT
   DIR_DELIM
   PLATFORM


Those are mostly functions. The all-caps variables are strings. I have checked, those variables seem to be leaked into the global namespace by builtin and they are not documented in lua_api.txt. I am not sure if those should be blacklisted (also, I have not checked every single name) since this is clearly a violation of the Minetest Lua styleguide (“Avoid globals like the plague.”). I think I have to post a Minetest issue about this soon to make sure that builtin does not leak so many globals.

I think this just shows that your mod is pretty cool already, although it has its rough edges. Just shortly after you added the globals check, I already found a style problem in builtin. :-)

I have tweaked the output code of the globals check a bit and applied some pretty-printing:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

local first = true
for name, val in pairs(_G) do
   if minetest.get_modpath(name) then
--      print(name, "mod exists")
   elseif blacklist[name] then     -- skip blacklisted
--      print(name, "builtin")
   else
      if first then
         print("Suspicious global variables:")
         first = false
      end
      local t = type(val)
      if t == "number" or t == "string" or t == "boolean" then
         local sval = dump(val)
         if string.len(sval) < 64 then
            print(name .. ": type \""..t.."\", value: " .. sval)
         else
            print(name .. ": type \""..t.."\", long value")
         end
      else
         print(name .. ": type \""..t.."\"")
      end
   end
end
if first then
   print("No suspicious global variables found.")
end


Feel free to use it in your mod. :-)

Oh, your qa_block mod leaks one global variable itself: It is old_print in init.lua. Please make it local.


Unrelated sidenode: I suggest to add the group “dig_immediate=2” for the node, to make sure it can always be dug, even without pickaxe.

Another unrelated sidenode: Why are you using multiple arguments in print()? The strings will appear concatenated in Minetest's chat output, which makes it hard to read. :-/

[bell07]

I added a second blacklist for the "maybe bug in MT builtin" globals and set them enabled by default

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

local enable_second_blacklist = true

The second blacklist can be disabled to check the MT-core/builtins, but for usual modder this globals should not spam the output. So I can see next time if the qa_block leaks any globals oO.

I added your code to the check and updated the block as proposed.

Why are you using multiple arguments in print()?

The main reason is I do not need to check if the variables are "nil" that I like to print. Concatenating works with strings only. Nice effect, the output is tabbed in case of multiple variables. And as I said, the chat output is just noise for me so I did not seen the issue. Do you know a way to get tabbed output in the chat? So I can adjust the redirection.

[Wuzzy]

I have written 7 new checkers for this mod. See here for more information:
https://github.com/bell07/minetest-qa_block/pull/2

Hints: If you don't know the type of a variable, but still want to print it, you can always shove it through tostring first. Also look up Minetest's dump function. In strings you can add a horizontal tab with “\t”.

PS: Don't forget to update the dependencies in the first post.

PPS: What exactly is this smartfs thing and why may this be needed?

[bell07]

Pull request merged, and your "filesys" change too.

If you don't know the type of a variable, but still want to print it, you can always shove it through tostring first.

Mostly I use the "print" for debugging output, and "print(a,b)" is smarter for me as "print(tostring(a)..tostring(b))"

In strings you can add a horizontal tab with “\t”.

I know and use it in the print redirection. But "\t" does nothing in chat :/

What exactly is this smartfs thing and why may this be needed?

Smartfs is a framework for easy creation of formspecs: https://forum.minetest.net/viewtopic.php?f=11&t=7553&p=116028
I did some enhancements on them primary for my "townchest" mod, tested the enhancement with qa_block and it seems to be usable. Agree the framework is oversized for qa_block needs. The in qa_block used version is not in upstream currently: https://github.com/minetest-mods/smartfs/pull/12. rubenwardy does the hard job to get my "it should work"-changes upstream ready.
If you interested you can have a look to my last enhanced version in the "view" branch: https://github.com/bell07/minetest-smartfs/tree/views. Just uncomment the "example-views.lua" in init.lua and try the /sfs_v and /sfs_t chat commands ;)
The smartfs dependency is optional in qa_block, but without them you have the chat-command only to select and run scripts. The block just calls the "defaultmodule" if placed and then removes the block if done. It's really a "light" version. I do not want to miss the selection screen any more (that can be called with /qa_block sel" too).

[bell07]

Today I updated the mod description to the current git version. You could say it is a new release ;) Thanks to Wuzzy for many contributions in this version!

By the way I have a question: which way should I prefer to handle the used smartfs lib. All needed changes are merged to upstream last days and for now I can go two ways with qa_block and smartfs
1: As is: qa_block provide a copy of smartfs library.
Pro: There is no dependency defined to other mod because all needed is provided. Provided version is in a known working version.
Contra: If any other mod uses smartfs the library is loaded twice.

2: remove the copy and set a dependency to smartfs library mod.
Pro: Proper way, no redundancy. This way was intended originally. A fallback mode (without gui) is implemented already in chat commands
Contra: Additional dependency, maybe not tested or working combination to smartfs version in feature possible (in theory each smartfs change needs a qa_block retest)

[Wuzzy]

I'd go for option 2. This is the proper way to do it, IMO this is the whole point of providing a library.
But please make rubenwardy promise not to break the smartfs API too often. ;-)

Formspecs in Minetest themvelves are still very much subject to change anyway, so either way you have maintenance work. :-/

[bell07]

Ok, I did it. For now there is no smartfs copy provided. If you like the gui, you need to install the https://github.com/minetest-mods/smartfs mod in the current git version (Nov, 13, 2016 or later)
To get more functionality without smartfs I added /qa help and /qa set <modname> chat commands. /qa sel is renamed to /qa ui.

[bell07]

The smartfs is provided again :-| I enhanced the QA-Block and used new futures that are not in upstream.

For now the QA-Block does have a Lua-Tables Explorer. See screenshot in updated first post.

[Byakuren]

newproxy is an undocumented part of Lua by the way, not from Minetest.

[bell07]

I do not see a relation qa-block to newproxy oO
by the way if I trust google, the newproxy is deprecated. Still present in lua-5.1 that is used in minetest, but is removed from lua-5.2.

[Byakuren]

Wuzzy mentioned it as an unaccounted global.

[bell07]

Thanks for the hint. I moved the newproxy from "to clarify"-Blacklist to the "clarified"-Blacklist inside the check script.

[bell07]

Mod updated:
- The globals/lua tables browser does sort the variables now
- Search/Filter field added to the globals/lua tables browser
- Check light_source_15 removed, since MT-0.4.15 correct the values by self and therefore the list is always empty
- New check: get_items_csv: Write a csv-file from minetest.registered_items to the world folder that can be pivot-analyzed in a spreadsheet like localc or excel. Item groups expanded in the list.

[Wuzzy]

The new features sound awesome, I would have suggested them at one point or another.

Sadly, just by putting a QA block from the creative inventory into my player inventory, Minetest fails:

2017-02-19 19:25:53: WARNING[Server]: Undeclared global variable "arg" accessed at /home/wuzzy/.minetest/mods/qa_block/init.lua:57
2017-02-19 19:25:53: ERROR[Main]: ServerError: Lua: Runtime error from mod '??' in callback detached_inventory_OnTake(): /home/wuzzy/.minetest/mods/qa_block/init.lua:57: bad argument #1 to 'ipairs' (table expected, got nil)

Tested with Minetest 0.4.14 on MineClone 2 0.10.0 and qa_block commit ID 02a5e4b96c741326619990c18d67a6c4777d86de.

The fact that there's an undeclared variable warning makes me believe it's not caused by MineClone 2.

[bell07]

Hi Wuzzy,

I know about the bug report, but I do not see the root reason for the issue and therefore no solution. :/ "arg" is a lua-builtin table and should be allways there, defined by "..." in function interface. Maybe any one other in forum know the issue and the solution? I cannot reproduce the issue.

As a workaround you can disable the nice-to-have only functionality "print-to-chat" till the solution is found. Just change the line 2 in "init.lua" to

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

local print_to_chat = false

[Byakuren]

Hi Wuzzy,

I know about the bug report, but I do not see the root reason for the issue and therefore no solution. :/ "arg" is a lua-builtin table and should be allways there, defined by "..." in function interface. Maybe any one other in forum know the issue and the solution? I cannot reproduce the issue.

As a workaround you can disable the nice-to-have only functionality "print-to-chat" till the solution is found. Just change the line 2 in "init.lua" to

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

local print_to_chat = false

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

ipairs({...})
-- instead of
ipairs(arg)


[bell07]

Thanks!

In addition I found the http://stackoverflow.com/a/7630202. It seems the "arg" syntax is a relict of Lua-5.0 and does not work right on lua-5.1. In Lua 5.1 the new syntax should be used as proposed.

I changed it in git. @Wuzzy, can you please test if the change does fix the issue for you?

[bell07]

Small update, both on globals explorer page:
- form will not be closed on enter in search field. Take the filter instead
- If the qa_block node is used, the path to the selected variable is saved to the nodemeta. So it is possible to come back to specific variable even after restart the game. Just come back to the placed node.

EDIT: 19.03.
- store more data to nodemeta (selected tab, function-hide in explorer)
- Globals-Explorer: search string per node - more intuitive. Search strings are stored now too