Investigate rendering of Arabic

[keithw]

ECMA-48 doesn't support RTL Unicode and neither do we, but if screen, tmux, gnome-terminal and xterm all support Arabic in the same way, we should too.

[kmcallister]

I tested some UTF-8 Arabic text in gnome-terminal, urxvt, and screen-in-urxvt. They all displayed the text left-to-right but with letter joining. I get the same results in Mosh. (The text has no explicit directional markers, but the bidi algorithm should recognize it as RTL, which happens correctly in e.g. gedit.)

I tried mlterm but couldn't get it to use any of the Arabic fonts on my system. However the unknown character boxes seem to be rendered right-to-left...

[keithw]

Do you want to take a look at the input sample we got from the user? They claimed it worked outside mosh, but maybe it's not our bug. Their terminal was apparently 128x30.

[lexande]

I tried running "lynx ar.wikipedia.org" in mlterm locally and in mlterm+mosh on my system (packages from debian sid). Both give the same output, which looks correct to me (the words I recognise are correctly RTL).

[kmcallister]

Okay, I got mlterm to use Arabic fonts by running it as

LC_ALL=ar_EG.UTF-8 mlterm

Then I tested some text from ar.wikipedia.org and found a discrepancy between mlterm and mosh in mlterm. The text file base64-encodes as

2KfZhNiv2Kgg2KfZhNmC2LfYqNmKINmH2Ygg2YbZiNi5INmF2YYg2KPZhtmI2KfYuSDYp9mE2K/Y
qNio2Kkg2YrYqtmI2KfYrNivINmB2Yog2YXZhti32YLYqSDYp9mE2YLYt9ioINin2YTYtNmF2KfZ
hNmKINin2YTZhdmF2KrYr9ipINi52KjYsSDYtNmF2KfZhNmKINin2YTYp9iz2YPYp9iMINmD2YbY
r9in2Iwg2LHZiNiz2YrYp9iMINin2YTZhtix2YjZitis2Iwg2YjYrNix2YrZhtmE2KfZhtivINmI
2YXYpwo=

[kmcallister]

screen in mlterm looks the same as mlterm, while tmux in mlterm does something else entirely.

[kmcallister]

Tagging this bitesize because somebody who can read/write a little Arabic can probably help us a lot, even if they don't want to write code.

[andersk]

gnome-terminal has no bidi support so far, and at least one developer says that’s WONTFIX. It’s really unclear to me how this even is supposed to work.

[paulproteus]

Per @andersk 's remark, is it still bitesize? If so, can someone provide a sequence of actions you want the helper to do? Thanks!

[keithw]

Basically, we want somebody who regularly uses an RTL language in a terminal emulator to tell us if the Mosh user experience is acceptable.

[sohaeb]

Are you still looking for an RTL language guy ? Just before I get my self in trouble(I don't know how to code :-P) . Do you want me to check if the Arabic language is being displayed correctly ? If yes, then I am willing to help.

[paulproteus]

Hey @sohaeb ! I think that yes, a report will be useful! Even if the other developers think the problem is fixed, getting another report will be valuable as it may have since been re-broken...!

[textshell]

Gnome Terminal meanwhile got a implementation of Egmont Koblinger's bidi for terminals spec:
https://gitlab.freedesktop.org/terminal-wg/bidi

Which is very unlikely to be bite sized or even in scope for a bug. From what i understand adding support for that (an he's got very good argument that that's needed for decent bidi support) is not a small project.