[Mod] Internationalization Library [intllib]

[Xanthin]

Sorry for that, brunob.santos. I only tried to understand what 4aiman meant with his/her question. but my english is quite limited and except some little experiences how to use this mod, I don´t have any knowledge about programing things. Bad sciolism, I shouldn´t have wrote so much, better asking simply what is meant. Damn.
But I will try to explain my thought/question better.

Because my answer to brunob.santos is very long I made it in three parts and "hide" it in the spoilers, so no one have to scroll down tons of text lines only to read the post after it.

Part 1:
[spoiler]My first thought was:" Eh? I am totaly independent from the english grammar when I want to translate with this mod.
The short example I gave than comes here now detailed:

If a mod supports translation by intllib you can find a folder called "locale" in the mods folder. Therein should be a file called template.txt. Therein you can find the english words you are confronted in the game. After each word/sentence is an equals sign. Behind that you can type in your translation for that word/sentence. But it doesn´t matter what you write.
As an example I take the pt.txt file from homedecor mod with already made brasilian portuguese translation. In the middle of the file you can find this line:

Glass Skylight = Claraboia de vidro
This should translate the description of that (really nice and usefull!) item in the game from english to brasilian portuguese.

But you can also write:
Glass Skylight = vidro de Claraboia
or this:
Glass Skylight = Não falo lusitano. :D

Another example from this pt.txt in homedecor:
Reading cached character database. = Lendo banco de dados de caracteres no cache.
but you can write:
Reading cached character database. = Hoje está fazendo tempo ruim. :(

As you can see, at this point you are totaly independent from the english grammar. You can use the one of your own language, you must not even really translate what stands before the equal sign.

(Now a short digression needed for the second part.)
How does this work, how could you tell the game to not use the english words? Mostly it´s not done with simply creating a text file and diligent write down all the english words you want to translate. You must look at the lua files and modify the right code lines ( ! if the mod is not already supporting intllib, cause than somebody else have already done the following steps and luckily you only have to take the template.txt file, copy it, rename it with the two letters for your language and happily start to figure out what the translation may be :D).
Using our example from homedecor we look at the shingles.lua file. Right at the top you can find:

local S = homedecor.gettext

minetest.register_node("homedecor:skylight", {
description = S("Glass Skylight"),
drawtype = "raillike",
tiles = { "default_glass.png" },
... and so on

After description is written which text should be displayed for this item in the game. To be translatable with the intllib mod someone wrote S() to it. And the first line is a short command to use the "Boilerplate to support localized strings [...]" you read in the intllib mod description from kaeza. Look at the init.lua of homedecor, right at the top you can find the full one. That´s it. You put what you want to translate in S(...) and then translate it in the grammar and spelling of your language.

That´s a superficial explanation, don´t ask me how kaeza does it to tell the program that it should look in the right text file for your language to translate what stands in the S(....).
I don´t know it, my aim was a localized game not to learn a computer language. ;)
[/spoiler]

Part 2:
[spoiler]For trying to explain my second thought I want to use the hardenedclay mod. Look at the init.lua file from this mod. There is a column of english colour names:

local colors = {
"White", "Black", "Light Blue", "Green", "Red",
"Light Gray", "Purple", "Lime", "Magenta", "Orange",
"Brown", "Blue", "Yellow", "Pink", "Cyan", "Gray"
}

then you can find:
[...]
description = S("Hardened Clay"),
[...]

but than:
[...]
description = S("%s Hardened Clay"):format(S(color)),
[...]

Before this mod was modified to support intllib the last line was:
[...]
description = color.." Hardened Clay",
[...]

What means the step with "%s" instead of "color.." and the addition with ":format(S(color)"?
"color.." seems to be (remember, I´m no programer) a command to fill in, at this place, the words from "local colors" to be displaed in the game as: "White Hardened Clay", "Black Hardened Clay"... and so on.
This "%s" is like a joker in rummy and with ":format(color)" you say what the joker should be: the words from "local colors." To flank "color" with S() means you want these colour names to be translated togehther with "%s Hardenedclay", so you have to flank this bit with S() too.
In the text file you create for brasilian portuguese (pt.txt) you write:

%s Hardened Clay = %s <your translation for Hardened Clay >
or
%s Hardened Clay = <Your translation for Hardened Clay> %s

Then you have to translate all the colour names.
Now you see in the game: "<your translation for the colour> <your translation for Hardened Clay>" or the other way.

That way to deal with the "color.." comand is what I meant with "eas". You only have one line with the joker and then some colours.
But you are a little bit forced to be creative with your grammar and the translation. For example:

In german you can make an addition for the translation like: "%ser <german translation>". That means that ijn german every colour name gets the gender of the german noun "Ton" (the german translation of clay, the gender is male so the adjective must end with "er"). For the most colour names this is the right way, but not for all.

Example:
Red Hardened Clay = Roter gebrannter Ton.
But not:
Pink Hardened Clay = Rosaer gebrannter Ton.

In this case you have to write it like "Rosa farbener ..." (means: "pink coloured ...").

I don´t know the russian grammar, I just had a little look at an online russian lerning course for possissve adjective. Generally speaking it seems to be a similar situation like the the example with the german translation for "pink" in the example. I don´t know if there is a similar solution in russian. When not, I can understand the problem.
But we can change the the whole way of the translation.
I mean:
Instead of description = S("%s Hardened Clay"):format(S(color)),

we can use (in this situation): description = S(color.." Hardened Clay"),

That means we translate every combination en bloc. Example:
White Hardened Clay = <translation of the whole description, now like in part 1>
Black Hardened Clay = <translation of the whole description, now like in part 1>
...
and so on

That´s what I wanted to ask 4aiman, cause when this is the solution for the problem he/she meant, we could change the translation way. It´s a little bit more work to write, but it works.
[/spoiler]

Part 3:
[spoiler]As kaeza mentioned in the mod description, the translation is limited to latin characters. Cyrillic uses non latin characters. Even the transliteration uses non latin characters.
So, apart from thinking about the grammar of the russian, there could be a problem to use cyrillic characters for the translation, as I understand.
[/spoiler]

That´s a huge explanation of my original ten lines question/thougts. I hope I could help you for understanding what I wanted to know and not confuse you more than I´ve done. And please, don´t misunderstand my detailed answer as I´m treating you like a silly person. I only tried to structure myself to be as little as possible confusing and correct enough to be not beaten from all the well studied programer in the forum. :D

[editation made for a silly typo]

[4aiman]

This is going to be loooong :)
Sorry for making you wait.

Apart from talking about non-latin characters which are perfectly supported if we are paying attention to locales there are other problems. I'll try to explain it with Russian language as an example, 'cause it's my native and 'cause I know little about Deutsch.

Did I understand you correctly? Or did you have an example?

Sorry for being unclear, I'm so familiar with the problems of translation that sometimes forget that others may not know what "case" (linguistics) is. *German "Kasus"*

Note: to fully understand the info given below one may need to read about linguistic typology, and one of it's approaches to distinguish languages into different types.

Here we are to talk about Russian (as a Fusional or Inflecting type of language) and English (as an Analytic type of language).

To be shorter (which I WILL fail), "Isolating" means that the language uses bare words with no inflections. "Fusional" language can have 1 inflection. "Agglutinative" language can have more than 1 inflection.
"Analytic" language like Isolating can NOT create new words and new forms of a word with the help of inflections. It does it via concatenation of roots (here it's not essential to bring in morphemes).

Note: While no language of just one type exists nowadays, we still can say things like "The X language is of Y type by ~Z %". This means that English DOES have ways to behave like inflective language and it's inflective by a few %.
Examples are: the "s" semaphore that shows us that it's 3rd person singular (walks, does); and the " 's " semaphore that shows us that something belongs to someone (Queen's Palace, author's handwriting).
But that's almost all of it! Usage of inflections is inferior in comparison to other ways to show grammatical relationships (like prepositions).
So, English is more like Analytic or Isolating language.

On the other hand Russian is more like Fusional/Inflective language, cause it's natural way for Russian language to create new words and forms of a word by the means of inflection.

Compare:
English: your, yours. (that's what we are told in schools, but there are more)
Russian: твой, твоего, твоему, твоего, твоим, о твоём. (and that's only what we are told in schools - there are more of them, but just like in English those additional 4-5 cases almost always use the same form as one of the above ones)

So, modern English for the most of us have only 2 cases. The Russian have 6. Deutsch have 4 that we were told of at school.

Now for some examples, finally.

Moreores.
This mod defines tools, ores, blocks and ingots.
It also gives us opportunity to change the name of the material and the name of an item.
Now we are to compare.
Note, that there's only 1 substitute for 1 word. We can't

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

+-----------------------+----------------------------+
|        English        |           Russian          |
+-----------------------+----------------------------+
|        silver         |          серебр[о]           |
|                       |                            |
|     silver pick       |     серебрянная кирк[а]    |
|     silver ore        |    серебрянный самородок[] |
|     silver ingot      |    слиток серебр[а]        |
|     silver spade      |    лопата из серебр[а]     |
|     silver sword      |    меч из серебр[а]        |
+-----------------------+----------------------------+

Note, please, that "лопата из серебра" can be changed to "серебрянная лопата" rather safely. In fact, the second form is colloquial and spoken so widely, that most of Russians would say it's "ok" to substitute the first one with the second one.
Minetest is not an RPG/JRPG after all.
Bookish language is much more vivid and bright, though.

But we just can't substitute "серебрянный самородок" with "серебрянная самородок". That would be like speaking to a man in a way we usually speak to woman. That's not nice and what's more important it's not even colloquial/spoken.
So here we are dealing with something inappropriate for Russian ear. Every 5 year old child will tell you it's not "серебрянная самородок" but "серебрянный самородок". "Ая" and "ый" are inflections used to show gender: male (he) and female (she) respectively. There's the "ое" inflection also. It is used for undefined gender ("it" in English).

In Russia, if we want to parody some Asian we would use the fact that their languages are NOT Inflective. Thus they don't know there should be some particular inflection and use the first they think would fit nicely:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

Моя хотеть иметь бизнеса с твоя.

I'll try to translate it, but it won't be the same of course:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

Me wishes to have a business with yours.

In English it's hard to distinguish whether someone is male or female if a person would refer to him/her~self as "I" or "me".
But in Russian, Deutsch and many other languages this in not a problem: "Я бежал" means that a person has been running and he's a man; "Я бежала" means that a person has been running and she's a woman.

One may think that "бежал" has no inflection. Bzzzt! WRONG! It had zero inflection.

But in terms of translation "male" gender gives us possibility to avoid too much inflections - verbs are likely to have zero inflection, thus making it possible to translate phrases defined like

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

player:get_player_name() .. " has killed 4aiman"

as

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

"Чайман был убит. Это сделал ".. player:get_player_name()

which in English is the equivalent of

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

"4aiman was killed. " .. player:get_player_name() .. " has done it "

instead of more precise form of translation

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

player:get_player_name() .. " убил Чайман[а]"

But when gender kicks in, we can't do the same. BOTH versions are inappropriate if killer was a female. Our workaround for males won't help us anymore:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

"Чайман был убит. Это сделал[а] ".. player:get_player_name()

Solution:
Make it possible to supply additional variables that would store at least gender and case.

[BrunoMine]

My first thought was:" Eh? I am totaly independent from the english grammar when I want to translate with this mod.

We are never completely independent of the English.

Long explanation. But Okay!

[Sokomine]

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

|     silver spade      |    лопата из серебр[а]     |
|     silver sword      |    меч из серебр[а]        |


The extra "из" in there almost looks like what in German might be "Schwert aus Silber" (Silberschwert would be grammatically correct as well, but it would sound odd somehow - perhaps because silver is not a very convincing material for a sword). In English, that would be "sword made out of silver". Both languages would allow for constructs like "sword (silver)" (or "Schwert (Silber)"). That's a possible fallback for some situations.

As far as those tools and materials go, "<material> <tool>" works in English, and "<material><tool>" as one word in German. We do get into trouble if it's something like "<color> <object>" - which works fine in English, as <color> does not give a damn about <object>, but at least in German, <color> requires a suffix depending on the grammatical gender of <object>. The suffix is also changed if <object> is in plural form. In case of the hardened clay, that was easy - there's just the clay, and clay (Ton) is always grammatically male in German. If the hardened clay mod would start to introduce colord walls (<color> wall = <color>e wand) or roofs (<color> roof = <color>es Dach), translating individual words will fail. However, the "<object> (<color>)" schem could be used, i.e. Ton (rot), Wand (rot), Dach (rot) - instead of roter Ton, rote Wand, rotes Dach.

stead of description = S("%s Hardened Clay"):format(S(color)),

we can use (in this situation): description = S(color.." Hardened Clay"),

That means we translate every combination en bloc. Example:
White Hardened Clay = <translation of the whole description, now like in part 1>
Black Hardened Clay = <translation of the whole description, now like in part 1>

Unfortionately, that does require some understanding of other languages. Whoever
writes the first translation needs to know where to break up expressions and
to use general placeholders for all of it and where not to.

But in terms of translation "male" gender gives us possibility to avoid too much inflections - verbs are likely to have zero inflection, thus making it possible to translate phrases defined like

There's another problem: There is no way to tell weather a player is male or female. English will have this problem as well. The workaround is usually to assume that "the player" is male.

Another problem arises from the way of adressing the reader. In English, it's just "you" everywhere. In German, there's a distinction between adressing people as "du" (informal, for family, children and friends) and formal "Sie" (everyone else). Some older people may still feel insulted if a game would adress them as "du" (but those are probably too old to be intrested in games anyway), and in a manual, adressing the reader that way would be an indication of very low regard for the reader (=bad translation). With English having become more prominent amongst younger people, "du" has become the standard form of adressing on the internet.

But when gender kicks in, we can't do the same. BOTH versions are inappropriate if killer was a female. Our workaround for males won't help us anymore:

Hm. I can only talk about German here. The language is quite flexible as far as grammar goes, so there are usually ways of rephrasing sentences when that's needed due to the position of placeholders. The rephrased sentence may however sound strange. Gender is less of a problem in this case. Or, rather, gender is a problem in all languages. Imagine the following:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

X killed Y with his sword. He ran away afterwards.


Now - that's English, and the problem's the same: How do we know which gender X had? Solution: We don't. We assume everyone's male until proven otherwise. Minetest has no idea about the gender of its players. The best workaround is to avoid sentences like the above one. It just sounds odd if I (as a female) have to read that "Sokomine died. He ran into lava." (no, I didn't! And it'd have to be "she"!).

Solution:
Make it possible to supply additional variables that would store at least gender and case.

That won't work either. In the "<color> <material>" case, the German translation would require knowledge of the grammatical gender of <material>. That grammatical gender may be a diffrent one in other languages! I can't just take the grammatical gender of a German word and assume it to be the same in French. And German does not seem to have that many cases and complexity in
that regard as Russian seems to have.

The only solution I see is to use something like this:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

S(adjective,object)


..which then could be translated depending on the language - if there's nothing that fits "adjective,object" completely, then translate each part individually; else take the complete translation.

Of course there are ways to construct grammatically correct sentences in German (to a degree - I know of no solution that covers all) with an algorithm if you know gender and plural/singular of all the parties/objects involved. However, even complex algorithms will have to know about all of the exceptions to the general rules - else people will start laughing when the algorithm fails and constructs something wrong :-) And there are many exceptions. One French teacher once told us (when we where complaining about how difficult it was to remember grammatical gender of nouns) that German's not better there - and that the only rule without exception was nouns ending with -chen (i.e. "das Häuschen" - "the small (cute) house") - those are always of neutral gender (English: it).

It'd be intresting to hear how this turns out in other languages. There seem to be quite a lot of players from Brazil recently. I can't really talk to them unless they speak English :-(

Another issue are server rules. It would be great to have a set of the standard rules in the most common languages so that children which did not have English in school yet do have a chance of understanding what the server's about.

[Xanthin]

Thank you both for the detailed explanation. After some days reading I see there is more to pay attention than I thought. I think, with my limited knowledge I was a little bit too much enthusiastic.

[Sokomine]

The topic is very intresting. I hope speakers of less common languages tell us a bit about problems when translating into their languages :-)

[Hybrid Dog]

l like having translated descriptions, but l also like that l can learn English words, e.g. l need to use that mod of Bas080 which adds Hopfen, l always forget the English word for it, but if l do sth with that in minetest, l quickly know its meaning and don't forget it for at least 3 years.
Can you make intllib show the translated words only as subtitle?

[MrIbby]

l like having translated descriptions, but l also like that l can learn English words, e.g. l need to use that mod of Bas080 which adds Hopfen, l always forget the English word for it, but if l do sth with that in minetest, l quickly know its meaning and don't forget it for at least 3 years.
Can you make intllib show the translated words only as subtitle?

Is this what you want?

+ Spoiler

You can download it here. Make sure you rename the folder to "intllib".

[BrunoMine]

This mod is very cool.
I will use it on all my mods from today.

[Hybrid Dog]

Is this what you want?

+ Spoiler

You can download it here. Make sure you rename the folder to "intllib".

Actually l wanted it the other way round, but l can easily change that, thanks.It aborts with error: moreores/init.lua:99: bad argument #2 to 'format' (no value)

[MrIbby]

Actually l wanted it the other way round, but l can easily change that, thanks.It aborts with error: moreores/init.lua:99: bad argument #2 to 'format' (no value)

I'm glad to help.Try this.

Also, I fixed a bug, so make sure you re-download my version of intllib.

[Hybrid Dog]

it doesn't work, moreores is not the only mod using that %s

[MrIbby]

I'll try another idea. For each mod that gives an error, disable the mod and record its name. When it stops crashing, message the list of mods.

[Hybrid Dog]

you could also disable subtitle for that text

[MrIbby]

All right, I added a blacklist in init.lua. Download (Further discussion about my version of intllib should be in a private conversation.)

[amadin]

This mod don't work for me. Maybe i need to compile server or/and client with ENABLE_FREETYPE or/and ENABLE_GETTEXT ???
Can this mod to show different translations at the same time for different players?

[kaeza]

It has nothing to do with gettext, but it may cause problems without FreeType on the client (just jumbled text or missing chars I guess; nothing "serious").

This mod currently translates strings server-wide, not per-user.

WRT you original question, can you elaborate? What "doesn't work"? Did you set `language` in `minetest.conf` (or `LANG` in the OS environment)?

Also, do the mods you are testing with support `intllib` at all?

[amadin]

Deleted

[ABJ]

Shame about not being able to use non-ASCII characters. I was going to make a translation into my own language when I saw that. Any way to fix?

[Hybrid Dog]

Shame about not being able to use non-ASCII characters. I was going to make a translation into my own language when I saw that. Any way to fix?

on my pc non-ASCII characters work in minetest

[Martno]

Shame about not being able to use non-ASCII characters. I was going to make a translation into my own language when I saw that. Any way to fix?

on my pc non-ASCII characters work in minetest

They work OK for me too (menu, chat), except in case of mods that I translated with Intllib: non-ASCII are not showing, or (on Linux) I get this: "invalid UTF-8 string". Casimir's translate mod work OK by the way (but that translates only the item descriptions).
If there's any way to fix this, that would be awesome.

[BrunoMine]

If I want to concatenate a variable is very complicated.

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

LUA script
S("Ola ")..playername..S(" Bem vindo ao Servidor")
locale folder
Ola\32 = Hello\32
#playername
\32Bem vindo ao Servidor = \32Welcome to Server


It should be possible to facilitate concatenation of variables with strings.

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

LUA script
S("Ola ", playername, " Bem vindo ao Servidor")
locale folder
Ola 'VAR' Bem vindo ao Servidor = Hello 'VAR' Welcome to Server


Just a mod of example that can be used in various servers.

[kaeza]

It should be possible to facilitate concatenation of variables with strings. [...]

init.lua:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

S("Hello @1 Welcome to Server", playername)


pt.txt:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

Hello @1 Welcome to Server = Ola @1 Bem vindo ao Servidor


The `@N` mark is replaced by the Nth argument to the `S()` call (after the format string itself).

For example:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.

Code: Select all

S("@2 @1", "hello", "world") --> world hello


[kaeza]

[...] If there's any way to fix this, that would be awesome.

Which text editor are you using? Are you sure you're saving the file with UTF-8 encoding?

[BrunoMine]

Thanks.
I want to create a PDF guide to Portuguese language.
You would include a link in README.txt?

[Martno]

[...] If there's any way to fix this, that would be awesome.

Which text editor are you using? Are you sure you're saving the file with UTF-8 encoding?

Oh, thanks. That was the problem. I used the notepad on windows, which saved the text with ANSI encoding. I saved it as UTF-8 encoding, and works fine!

[Soultest]

Hi, Kaeza.

Currently, when one tries to edit some of the target texts (i.e. S(target text)), it does not seem to update the template file automatically.

In certain cases, if one deletes the template file, it just does not regenerate it at all?

Thanks for your time.

[Glory!]

For any modders out there... I've made this:

http://i.imgur.com/1bjbxmD.png - Feel free to add to your mod's page.

[kaeza]

Hi, Kaeza.

Currently, when one tries to edit some of the target texts (i.e. S(target text)), it does not seem to update the template file automatically.

In certain cases, if one deletes the template file, it just does not regenerate it at all?

Thanks for your time.

The mod/library does not handle template generation/updates in any way. It is up to mod developers to do it manually.

For any modders out there... I've made this:

http://i.imgur.com/1bjbxmD.png - Feel free to add to your mod's page.

Woah. Nice!

What's the license for the image? If you release it as something open source, I'll link it in the first post (with due credit, of course).

[Glory!]

What's the license for the image? If you release it as something open source, I'll link it in the first post (with due credit, of course).

WTFPL.