Page 2 of 2

Re: special characters (åäö)

Posted: 10 Jun 2011, 09:41
by ryz
If i do an ls on a file name åäö I do get for utf-8 encoding


Code: Select all

~$ ls åäö | xxd -g 1
0000000: c3 a5 c3 a4 c3 b6
and for is0-8859-1 it would be

0000000: e5 e4 f6

Re: special characters (åäö)

Posted: 13 Jun 2011, 06:02
by d_rylndr
Thank you all.
I've tried your hex-number solutions but I can't figure it out.
Perhaps I'm on the wrong track altogether. It might be an easier fix to just replace those strangely displayed characters to the right ones and not care about encodings at all?

How would this be easiest achieved? Search and replace through the filesystem with sed? (about 500Gb of data on this disk...)

Re: special characters (åäö)

Posted: 14 Jun 2011, 04:09
by d_rylndr
No go.
Did some testing. Filenames created or changed (on server) to contain any of åäöÅÄÖ from the one machine that had trouble reading looks good on that machine.
Read from another computer connected to the share filname however file names are said to have invalid encoding. Looks like this through nautilus: "testfile�.txt (invalid encoding)"

All machines report sv_SE.utf8 with echo $LANG.

So it does seem to be some ecoding issue after all and not just an issue with old files copied from the old server.

I just can't figure out where the problem lies.

added info:
the testfile above is read right by the b3-server also. From terminal in other machine (comp.02 from now on) that has problems with this testfile wrong name is still displayed. ssh:ing into b3 from this machine in terminal looking at the file it is displayed correctly.

locale | grep LC_CTYPE gives LC_CTYPE="sv_SE.UTF-8" from both machine creating file (comp.01 from now on) and b3-server (b3 from now on).
LC_CTYPE for comp.02 is "sv_SE.utf8"

Re: special characters (åäö)

Posted: 14 Jun 2011, 13:25
by ryz
A quick google gives that Nautilus uses UTF8 as default http://osdir.com/ml/gnome.nautilus/2004 ... 00233.html

And that there has been some bug in some libsambaclient that resets the locale in 2005 https://bugzilla.samba.org/show_bug.cgi?id=2926

If you create a file called aåäö in a empty directory from the comp2 machine what does then

Code: Select all

ls a* | xdd -g1
from b3 give?

My guessing here is that it is comp2 that is incorrect.

Re: special characters (åäö)

Posted: 16 Jun 2011, 09:39
by d_rylndr
I dont have the xdd package installed, neither can it be found with apt-get install from b3. Googling it wasn't very easy either (try man xdd :))...
ls from b3 shows the file with strange characters.

Re: special characters (åäö)

Posted: 16 Jun 2011, 09:51
by ryz
Sorry should have bin xxd

Re: special characters (åäö)

Posted: 16 Jun 2011, 09:58
by d_rylndr
ok, created a file called testfil2åäöÅÄÖ from comp.2.

ls from b3 gives testfil2åäöÅÄÖ

ls | xdd -g1 gives

Code: Select all

0000000: 74 65 73 74 66 69 6c 32 e2 94 9c c3 91 e2 94 9c  testfil2........
0000010: c3 b1 e2 94 9c e2 95 a2 e2 94 9c c3 a0 e2 94 9c  ................
0000020: c3 a4 e2 94 9c c3 bb 0a                          ........

Re: special characters (åäö)

Posted: 17 Jun 2011, 02:01
by Ancan
Kinda offtopic, but saw this on a tee...

I � Unicode