emacs and other programs do not display special characters

OpenSubmitted by quiliro.
Details
4 participants
  • Bengt Richter
  • quiliro
  • Ricardo Wurmus
  • Miguel Arruga Vivas
Owner
unassigned
Severity
normal
Q
Q
quiliro wrote on 10 Sep 2019 01:27
(address . bug-guix@gnu.org)
fcd8b47e1a9093ed62acca7845113b01.squirrel@sm.riseup.net
Hello Guix:
I am reporting this because there are no other similar cases on themailing list and because I think this might be a bug and not my error.
Emacs Magit and Emacs shell don't dispaly special characters (such as ñ,í, ó) on their output. It is strange because a command that includes aspecial character is displayed. But a special character from the outputwill not be displayed correcly. Those special characters are displayedcorrectly on afairs such as opening a file with those characters.
With 'emacs -Q' I did not have that problem. When copying .emacs.d toanother directory, setting that directory as HOME and running emacs with'mkdir ~/temp', 'cp ~/.emacs.d ~/temp/' and 'HOME="~/temp" emacs', itwould not use my configurations. But it would not have the problem withEmacs shell. Emacs Magit would not be available either. The same situationis with 'emacs -Q' as with 'HOME="~/temp" emacs'.

Sample from BASh displaye correctly:
quiliro@GSD3 ~/magit/prueba0$ git logcommit 0904ec46cb737d2116d59b0b7c4f0c21a74feb70 (HEAD -> master)Author: quiliro <quiliro+commits@fsfla.org>Date: Sun Sep 8 15:43:09 2019 -0500
Modificación remota
commit 5024f6d525b1b61cd2691aaaa60dde07ae6f489f (origin/master)Author: ramiro.ordonez <quiliro+fablab@fsfla.org>Date: Sun Sep 8 13:20:11 2019 -0500
Añadí a mi amorquiliro@GSD3 ~/magit/prueba0$

Same command sample from Emacs shell displayed incorrectly:
quiliro@GSD3 ~/magit/prueba0$ git logWARNING: terminal is not fully functional- (press RETURN)commit 0904ec46cb737d2116d59b0b7c4f0c21a74feb70 (HEAD -> master)Author: quiliro <quiliro+commits@fsfla.org>Date: Sun Sep 8 15:43:09 2019 -0500
Modificación remota
commit 5024f6d525b1b61cd2691aaaa60dde07ae6f489f (origin/master)Author: ramiro.ordonez <quiliro+fablab@fsfla.org>Date: Sun Sep 8 13:20:11 2019 -0500
Añadí a mi amorquiliro@GSD3 ~/magit/prueba0$

I am not sure if this is related that in Icecat I sometimes see squareboxes with numbers inside them in place of characters. But other specialcharacters are displayed. That is probably a missing font. It could be aseparate problem.

Happy hacking!
Q
Q
quiliro wrote on 10 Sep 2019 02:13
(address . 37363@debbugs.gnu.org)
5b7a274019a6659a19b1193a154fd6c4.squirrel@sm.riseup.net
As per nckx's question on IRC, this is the output to locale on both Emacsshell and BASh:
quiliro@GSD3 ~/magit/prueba0$ localeLANG=es_EC.UTF-8LC_CTYPE="es_EC.UTF-8"LC_NUMERIC="es_EC.UTF-8"LC_TIME="es_EC.UTF-8"LC_COLLATE="es_EC.UTF-8"LC_MONETARY="es_EC.UTF-8"LC_MESSAGES="es_EC.UTF-8"LC_PAPER="es_EC.UTF-8"LC_NAME="es_EC.UTF-8"LC_ADDRESS="es_EC.UTF-8"LC_TELEPHONE="es_EC.UTF-8"LC_MEASUREMENT="es_EC.UTF-8"LC_IDENTIFICATION="es_EC.UTF-8"LC_ALL=
Q
Q
quiliro wrote on 10 Sep 2019 11:53
(name . 37363)(address . 37363@debbugs.gnu.org)
54101b276c9500b5d28fa5ca4cf1fb56.squirrel@sm.riseup.net
On M-x shell with (locale "es_ES.utf8"):quiliro@GSD3 ~$ ñbash: $'quiliro@GSD3 ~$ ñbash: $'\361': command not found\361': command not foundquiliro@GSD3 ~$
Also with (locale "es_ES.utf8"), the system is all in English and GnomeTerminal will not start:quiliro@GSD3 ~$ gnome-terminal# Locale not supported by C library.# Using the fallback 'C' locale.# Error constructing proxy fororg.gnome.Terminal:/org/gnome/Terminal/Factory0: Error callingStartServiceByName for org.gnome.Terminal:GDBus.Error:org.freedesktop.DBus.Error.Spawn.ChildExited: Processorg.gnome.Terminal exited with status 9quiliro@GSD3 ~$
I have also tested unknown command error output. On Emacs' M-x shell with(locale "es_EC.UTF-8"):quiliro@GSD3 ~$ ñbash: $'\361': no se encontró la ordenquiliro@GSD3 ~$
On BASh with (locale "es_EC.UTF-8"):quiliro@GSD3 ~$ ñbash: ñ: no se encontró la ordenquiliro@GSD3 ~$
I have tested this with:guix (GNU Guix) 1d03a9198db6f3656a34d62eb89e5f7d5a99e76aand with:guix (GNU Guix) 83aa6562173e06e8fafd4857a2e708feabfe56f8
R
R
Ricardo Wurmus wrote on 10 Sep 2019 18:40
(address . quiliro@riseup.net)(name . 37363)(address . 37363@debbugs.gnu.org)
874l1kqf4k.fsf@elephly.net
quiliro@riseup.net writes:
Toggle quote (6 lines)> Also with (locale "es_ES.utf8"), the system is all in English and Gnome> Terminal will not start:> quiliro@GSD3 ~$ gnome-terminal> # Locale not supported by C library.> # Using the fallback 'C' locale.
Do you have “glibc-locales” installed?Is GUIX_LOCPATH set?
-- Ricardo
B
B
Bengt Richter wrote on 11 Sep 2019 04:15
(address . quiliro@riseup.net)(address . 37363@debbugs.gnu.org)
20190911021503.GA1118@PhantoNv4ArchGx.localdomain
On +2019-09-09 19:13:08 -0500, quiliro@riseup.net wrote:
Toggle quote (20 lines)> As per nckx's question on IRC, this is the output to locale on both Emacs> shell and BASh:> > quiliro@GSD3 ~/magit/prueba0$ locale> LANG=es_EC.UTF-8> LC_CTYPE="es_EC.UTF-8"> LC_NUMERIC="es_EC.UTF-8"> LC_TIME="es_EC.UTF-8"> LC_COLLATE="es_EC.UTF-8"> LC_MONETARY="es_EC.UTF-8"> LC_MESSAGES="es_EC.UTF-8"> LC_PAPER="es_EC.UTF-8"> LC_NAME="es_EC.UTF-8"> LC_ADDRESS="es_EC.UTF-8"> LC_TELEPHONE="es_EC.UTF-8"> LC_MEASUREMENT="es_EC.UTF-8"> LC_IDENTIFICATION="es_EC.UTF-8"> LC_ALL=> >
Hi,I have been having locale-related problems too,so maybe we can bounce enough clues around thatwe can advance a little.
[ later ... I'll have to come back to locale per se, but hope the following is useful for poking around with fonts and unicode character and their glyphs ]
[ To the advanced, please don't be insulted by my posting obvious stuff, as if you didn't know how to use grep and sed and especially guix better than my examples show -- it is motivated by wanting to exchange helpful methods and info with others also coming to guix, who might benefit from my recent newbie experiences trying to find my way into guix city, in the Commonwealth of FOSS :)
Hm, I wonder if we could use postgresql plus postgis to do an openstreetmap map of guix city stores and pubs -- and potholes and contruction blockages ;-)] Anyway, I have a little script which may be helpfulin generating utf8 characters for display in yourvarious contexts (what this (emacs) context is I'll show below):
$ uchr 229 10å$ which -a uchr/home/bokr/bin/uchr$ cat ~/bin/uchr#!/home/bokr/.guix-profile/bin/bash# 2019-08-19 22:25:34 ## was: #!/usr/bin/bash# ~/bin/uchr -- print unicode characters from numeric args# uchr 65 67 10 | od -a -t x1# 0000000 A C nl# 41 43 0a# 0000003cc="$( printf '\\u%x' "$@" )"echo -en "$cc"
Those last two lines do all the work ;-)(printf is a bash built-in -- type -"help printf"at the bash command line.
(By -"foo" I mean "foo" minus the quotes :)
printf re-uses its format for each arg it encounters, so itconverts all the integers according to '\\u%x' above in uchr.
$ uchr {192..255} 10ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ$What do you get if you try that from your login console, not gnome?
To make various fonts available to the console, you should beable to use setfont from console bash -- see -"man setfont"(remembering my minus-the-quotes notation :)
The {FIRST_INTEGER..LAST_INTEGER} of course generates individualinteger arguments including first and last. The 10 is a newline.
I'm pretty sure I did -"setfont sun12x22" from the console bash,which gives you most of the 12x22 font built into the kernel.
It's got 256 character cells for its 12x22 pixel glyphs, eachrepresented by 22 16-bit integers using the ms12 bits with 1as foreground, IIRC.
The sun12x22 font is pretty good, with box-drawing charactersas well most things you need in European languages (I'm familiarwith it because I wrote a little script to display the glyphson the frame buffer, in the pursuit of independence from hugeblobs of gooey GUI software :)
After having done -"setfont sun12x22" you can do-"setfont -ou glyph-code-to-unicodepoint.txt" which willgive you a tab-delimited table starting ... ending like:
0x00 U+00000x20 U+00200x21 U+00210x22 U+00220x23 U+00230x24 U+00240x25 U+0025...0xdf U+25800xdc U+25840xdb U+25880xdd U+258c0xde U+25900xb0 U+25910xb1 U+25920xb2 U+25930x01 U+263a0x5f U+f804
(BTW, this would be really easy to snarf and convert toan assoc list mapping unicode code points to glyph indices)
That glyph 0x01 has a unicode we can discover, even thoughthe console font you get from -"setfont sun12x22" does nothave the glyph that is in the kernel version's glyph table:
$ $ unicode-info "$(uchr 0x263a)"
"☺": glyph codepoint .....int name... _☺_ +U00263a 9786 WHITE SMILING FACE $
The glyph is in the kernel's 256-glyph bit-map forsun12x22 though, and it should be visible in a gui browserwith good unicode coverage.
You can find the kernel's bitfont defined in kernel sources.../linux-4.14.3/lib/fonts/font_sun12x22.c (or changethe kernel version -- 4.14.3 is the last one I grop^H^Hepped around inlooking for stuff to "steal" :)
Ok, back to$ uchr {192..255} 10ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ$
The first four A's above are really not As, as another little scriptof mine that shows unicode information will show:
(you can find the source archived in a recent post of mine https://lists.gnu.org/archive/html/guix-devel/2019-09/msg00115.htmlif curious)
$ uchr {192..202} | unicode-info
"ÀÁÂÃÄÅÆÇÈÉÊ": glyph codepoint .....int name... _À_ +U0000c0 192 LATIN CAPITAL LETTER A WITH GRAVE _Á_ +U0000c1 193 LATIN CAPITAL LETTER A WITH ACUTE _Â_ +U0000c2 194 LATIN CAPITAL LETTER A WITH CIRCUMFLEX _Ã_ +U0000c3 195 LATIN CAPITAL LETTER A WITH TILDE _Ä_ +U0000c4 196 LATIN CAPITAL LETTER A WITH DIAERESIS _Å_ +U0000c5 197 LATIN CAPITAL LETTER A WITH RING ABOVE _Æ_ +U0000c6 198 LATIN CAPITAL LETTER AE _Ç_ +U0000c7 199 LATIN CAPITAL LETTER C WITH CEDILLA _È_ +U0000c8 200 LATIN CAPITAL LETTER E WITH GRAVE _É_ +U0000c9 201 LATIN CAPITAL LETTER E WITH ACUTE _Ê_ +U0000ca 202 LATIN CAPITAL LETTER E WITH CIRCUMFLEX $
The above is copy/pasted from a shell window I got by -"M-x shell"To show the pid genealogy of that shell context, I'll -"C-x o"over there and output about the context and paste it back here:
$ $ # typing pidgeny (for pid genealogy :) we get:$ pidgenypidgeny pts/0 23549 S+ /home/bokr/.guix-profile/bin/bash /home/bokrbash pts/0 16231 Ss /home/bokr/.guix-profile/bin/bash --noeditin.emacs-26.3-rea tty1 16204 Sl+ /gnu/store/ki85c221k56y6hnp7qyx42q2qmra4w4s-mutt tty1 1118 S muttbash tty1 537 Ss -bashlogin ? 521 Ss login -- bokrsystemd ? 1 Ss /sbin/init \EFI\PhantoNv4ArchGx\vmlinuz-linu$ which pidgeny|xargs realpath/home/bokr/bin/pidgeny$ ## --- pidgeny ---$ which pidgeny|xargs cat#!/home/bokr/.guix-profile/bin/bash# 2019-08-19 07:16:38 -- was: #!/usr/bin/bash# ~/bin/pidgeny
pid=${1:-$$} #this process if no pid specified as $1
while [ $(($pid)) -gt 0 ]; do ps h -p $pid -o comm,tt,pid,stat,args pid=$(ps -q $pid -o ppid=)done$ ## hm, monkeyed with that hashbang too, need a better idea :)$ $ ## pidgeny output doesn't show full path on mutt 1118,bash 537,$ ## or login 521, but we can get them easily:$ realpath /proc/1118/exe/gnu/store/bsd34k2v78mi0wxk85rz32xaminls9nb-mutt-1.12.1/bin/mutt$ ## that was a guix version$ realpath /proc/537/exe/usr/bin/bash$ ## that was the inital shell of the "foreign distro" -- in my case:$ uname -aLinux PhantoNv4ArchGx 5.2.9-arch1-1-ARCH #1 SMP PREEMPT Fri Aug 16 11:29:43 UTC 2019 x86_64 GNU/Linux$ realpath /proc/521/exerealpath: /proc/521/exe: Permission denied$ ## need permission$ su -c 'realpath /proc/521/exe'/usr/bin/login$ ## also built by the foreign distro's building tool chain and libraries -- I guess I will feel better$ ## when I replace the foreigner with linux-libre ;-)$
Well, HTH you to probe the state of your system vis-a-visutf8, glyphs, fonts etc.
I'll come back with some locale mystery, which will probablywind up being something I thought I did but didn't ;-/
But now I need to go do some things IRL ;-)
Regards,Bengt Richter
Q
Q
quiliro wrote on 11 Sep 2019 17:32
(name . 37363)(address . 37363@debbugs.gnu.org)
3607ee9b9427d5d654e27f7dbb56c7b1.squirrel@sm.riseup.net
El Mar, 10 de Septiembre de 2019, 11:40 am, Ricardo Wurmus escribió:
Toggle quote (12 lines)>> quiliro@riseup.net writes:>>> Also with (locale "es_ES.utf8"), the system is all in English and Gnome>> Terminal will not start:>> quiliro@GSD3 ~$ gnome-terminal>> # Locale not supported by C library.>> # Using the fallback 'C' locale.>> Do you have “glibc-locales” installed?> Is GUIX_LOCPATH set?
Not on my user. Probably on my system. I did not know I had to add thoseon Guix System. Should I?
M
M
Miguel Arruga Vivas wrote on 21 Oct 2019 19:52
20191021195257.1f8b2d83@gmail.com
I'm running on guix and spanish symbols work properly, on emacs and theterminal.
---miguel@unfall ~/src/gnu/guix$ localeLANG=es_ES.utf8LC_CTYPE="es_ES.utf8"LC_NUMERIC="es_ES.utf8"LC_TIME="es_ES.utf8"LC_COLLATE="es_ES.utf8"LC_MONETARY="es_ES.utf8"LC_MESSAGES="es_ES.utf8"LC_PAPER="es_ES.utf8"LC_NAME="es_ES.utf8"LC_ADDRESS="es_ES.utf8"LC_TELEPHONE="es_ES.utf8"LC_MEASUREMENT="es_ES.utf8"LC_IDENTIFICATION="es_ES.utf8"LC_ALL=---
I've seen some es_EC localizations instead of es_ES, couldyou check that's not your issue?
Best regards,Miguel
?