Bug 1063 - Wrong encoding on automatically mounted existing partitions (Live KDE) if setting Time Zone other than languages on CD
Summary: Wrong encoding on automatically mounted existing partitions (Live KDE) if set...
Status: RESOLVED DUPLICATE of bug 3723
Alias: None
Product: Mageia
Classification: Unclassified
Component: Release (media or process) (show other bugs)
Version: Cauldron
Hardware: i586 Linux
Priority: Normal normal
Target Milestone: ---
Assignee: Olivier Blin
QA Contact:
URL:
Whiteboard:
Keywords: Triaged
Depends on: 2415
Blocks:
  Show dependency treegraph
 
Reported: 2011-04-29 20:58 CEST by Filip Komar
Modified: 2012-05-09 20:51 CEST (History)
6 users (show)

See Also:
Source RPM:
CVE:
Status comment:


Attachments

Description Filip Komar 2011-04-29 20:58:33 CEST
Description of problem:
Browsing in Dolphin reveals wrong encoding in ext3 and NTFS partitions. This behaviour is not expected. This could also lead to some strange problems when opening files on such partitions with different programs.


In my case it would probably need to be utf-8. Result bellow should be "PoÄakej" instead of "PoÃkej"
[live@localhost PoÃkej]$ pwd
/media/Dom/filip/Dokumenti/PoÃkej

On NTFS should be "Anže" instead of "Anþe"
/media/Podatki/Dokumenti Anþe

[live@localhost ~]$ mount
none on / type unionfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
gvfs-fuse-daemon on /home/live/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=live)
/dev/sdb6 on /media/Podatki type fuseblk (rw,nosuid,nodev,allow_other,blksize=4096,default_permissions)
/dev/sdb5 on /media/Dom type ext2 (rw,nosuid,nodev,uhelper=udisks)

On working Mandriva I have in fstab:
# Entry for /dev/sdb5 :
UUID=3c257aa0-e9d5-4a91-8669-8117dc463574 /home ext2 defaults 1 2
# Entry for /dev/sdb6 :
UUID=7C941B18941AD50C /media/podatki ntfs-3g defaults,umask=000 0 0


Version-Release number of selected component (if applicable):
Live KDE CD

How reproducible:
When booting Live KDE CD.

Steps to Reproduce:
1. Boot Live KDE CD
2. Open Dolphin
3. Open (and AutoMount) partition
4. Find folder with national characters
5. Result: Wrong encoding on those folders
Comment 1 Ahmad Samir 2011-04-30 19:21:03 CEST
What's the output of:
locale

LC_ALL=C pwd

?

Keywords: (none) => NEEDINFO

Ahmad Samir 2011-04-30 19:21:12 CEST

Component: Installation => RPM Packages

Comment 2 Filip Komar 2011-05-01 17:16:54 CEST
$ LC_ALL=C pwd
/home/live
Comment 3 Ahmad Samir 2011-05-01 23:12:36 CEST
And the output of 'locale' ?

Also, 'LC_ALL=C pwd' _after_ entering PoÄakej (just like in the example you posted in the bug description).
Comment 4 Filip Komar 2011-05-02 17:34:08 CEST
Forgot to mention that I choose British English for language and Slovenian keyboard.

[live@localhost PoÃakej]$ LC_ALL=C pwd
/media/Dom/filip/Glasba/PoÃakej

and

[live@localhost Dokumenti Anþe]$ LC_ALL= pwd
/media/Podatki/Dokumenti Anþe


[live@localhost PoÃakej]$ locale
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_GB.UTF-8
LC_CTYPE=en_GB.UTF-8
LC_NUMERIC=sl_SI.UTF-8
LC_TIME=en_GB.UTF-8
LC_COLLATE=en_GB.UTF-8
LC_MONETARY=sl_SI.UTF-8
LC_MESSAGES=en_GB.UTF-8
LC_PAPER=sl_SI.UTF-8
LC_NAME=sl_SI.UTF-8
LC_ADDRESS=sl_SI.UTF-8
LC_TELEPHONE=sl_SI.UTF-8
LC_MEASUREMENT=sl_SI.UTF-8
LC_IDENTIFICATION=sl_SI.UTF-8
LC_ALL=
[live@localhost PoÃakej]$
Filip Komar 2011-05-02 17:36:27 CEST

Summary: Wrong encoding on automatically mounted existing partitions => Wrong encoding on automatically mounted existing partitions (Live KDE)

Comment 5 Filip Komar 2011-05-02 18:18:52 CEST
If I choose British English for language AND keyboard encoding is fine:
[live@localhost ~]$ locale
LANG=en_GB.UTF-8
LC_CTYPE=en_GB.UTF-8
LC_NUMERIC=en_GB.UTF-8
LC_TIME=en_GB.UTF-8
LC_COLLATE=en_GB.UTF-8
LC_MONETARY=en_GB.UTF-8
LC_MESSAGES=en_GB.UTF-8
LC_PAPER=en_GB.UTF-8
LC_NAME=en_GB.UTF-8
LC_ADDRESS=en_GB.UTF-8
LC_TELEPHONE=en_GB.UTF-8
LC_MEASUREMENT=en_GB.UTF-8
LC_IDENTIFICATION=en_GB.UTF-8
LC_ALL=


[live@localhost PoÄakej]$ LC_ALL=C pwd
/media/Dom/filip/Glasba/PoÄakej

[live@localhost Dokumenti Anže]$ LC_ALL=C pwd
/media/Podatki/Dokumenti Anže
Comment 6 Ahmad Samir 2011-05-02 19:59:15 CEST
- If the locale is partially set to sl_SI.UTF-8, this will definitely not work with the current live CD, as the locales-sl packages isn't on the ISO at all (the plan if for the final Mageia 1 release to have multiple live CD ISO's each containing a specific set of locales to try and support more locales for live CD users).

- I tested:
  - Selecting British English as language
  - Setting the keyboard layout to Slovenian
but I didn't find any of the locales (LC_*) set to sl_SI.UTF-8, so I don't see why it happened in your case.
Comment 7 Filip Komar 2011-05-03 20:24:40 CEST
True. Sorry. Didn't think of influence of Time Zone.

Time Zone Ljubljana for sure gives wrong results. I used it together with British English for language and keyboard.
In that case encoding was messed up again.
Ahmad Samir 2011-05-03 20:35:53 CEST

Keywords: NEEDINFO => Triaged
Source RPM: ? auto mount of existing partitions => draklive-install

Comment 8 Manuel Hiebel 2011-10-11 12:25:16 CEST
Hello, is this bug still valid ?
Comment 9 Filip Komar 2011-10-11 21:25:53 CEST
Unfortunately there is no difference in released version.

To sum up again.

Steps to Reproduce:
1. Boot Live KDE CD
2. Select (British) English as language
3. Choose Time Zone Ljubljana
4. Setting the keyboard layout to Slovenian
5. Open Dolphin
6. Open (and AutoMount) partition
7. Find folder with national characters
8. Result: Wrong encoding on those folders


$ locale
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_GB.UTF-8
LC_CTYPE=en_GB.UTF-8
LC_NUMERIC=sl_SI.UTF-8
LC_TIME=en_GB.UTF-8
LC_COLLATE=en_GB.UTF-8
LC_MONETARY=sl_SI.UTF-8
LC_MESSAGES=en_GB.UTF-8
LC_PAPER=sl_SI.UTF-8
LC_NAME=sl_SI.UTF-8
LC_ADDRESS=sl_SI.UTF-8
LC_TELEPHONE=sl_SI.UTF-8
LC_MEASUREMENT=sl_SI.UTF-8
LC_IDENTIFICATION=sl_SI.UTF-8
LC_ALL=

Nice solution would be resolved together with https://bugs.mageia.org/show_bug.cgi?id=2415

CC: (none) => filip.komar

Comment 10 Filip Komar 2011-11-29 23:22:56 CET
 
Unfortunately same results also in Mageia 2 alpha 1.

If I add locales-sl in size of 129 KB and set language to Slovenian is OK. See bellow:
[live@localhost ~]$ locale
LANG=sl_SI.UTF-8
LC_CTYPE=sl_SI.UTF-8
LC_NUMERIC=sl_SI.UTF-8
LC_TIME=sl_SI.UTF-8
LC_COLLATE=sl_SI.UTF-8
LC_MONETARY=sl_SI.UTF-8
LC_MESSAGES=sl_SI.UTF-8
LC_PAPER=sl_SI.UTF-8
LC_NAME=sl_SI.UTF-8
LC_ADDRESS=sl_SI.UTF-8
LC_TELEPHONE=sl_SI.UTF-8
LC_MEASUREMENT=sl_SI.UTF-8
LC_IDENTIFICATION=sl_SI.UTF-8
LC_ALL=


[live@localhost Dokumenti Anže]$ LC_ALL=C pwd
/media/Podatki/Dokumenti Anže
Comment 11 Manuel Hiebel 2011-11-29 23:35:21 CET
(In reply to comment #6)
> - If the locale is partially set to sl_SI.UTF-8, this will definitely not work
> with the current live CD, as the locales-sl packages isn't on the ISO at all
> (the plan if for the final Mageia 1 release to have multiple live CD ISO's each
> containing a specific set of locales to try and support more locales for live
> CD users).
> 
> - I tested:
>   - Selecting British English as language
>   - Setting the keyboard layout to Slovenian
> but I didn't find any of the locales (LC_*) set to sl_SI.UTF-8, so I don't see
> why it happened in your case.

(for the maintainer)

Assignee: bugsquad => mageia

Comment 12 Filip Komar 2011-11-30 10:42:05 CET
To reproduce this bug this is also crucial in comment #9:
3. Choose Time Zone Ljubljana




I have an idea. Is it possible to add all locales in all LIVE-CD and remove one language from each and put removed language into a new LIVE-CD and add also other languages like Slovenian (https://bugs.mageia.org/show_bug.cgi?id=2415) or Catalan (ca)? I can offer my assistance in testing them.

It is a bit sad that some languages with much less work done in Transifex are present in LIVE-CD while Slovenian with 99% for mga1 and 98% for mga2 and is not. It's not easy to encourage people to try Mageia if our language is missing in LIVE-CD.

CC: (none) => oliver.bgr

Manuel Hiebel 2011-12-05 19:45:56 CET

CC: (none) => sysadmin-bugs
Component: RPM Packages => Release (media or process)
Source RPM: draklive-install => (none)

Comment 13 Filip Komar 2011-12-19 10:57:14 CET
Is this connected with bug 3723?
Comment 14 Filip Komar 2012-01-03 07:30:20 CET
Same in Mageia 2 Alpha2 Live KDE 32.
Marja Van Waes 2012-04-20 17:12:45 CEST

CC: (none) => marja11
Depends on: (none) => 2415

Comment 15 Filip Komar 2012-05-01 18:05:20 CEST
Unfortunately no difference in Mageia 2 Beta3 Live KDE 32.
Comment 16 Filip Komar 2012-05-02 11:12:37 CEST
Actually setting Time Zone to Ljubljana is enough to trigger this bug.

[live@localhost Dokumenti Anþe]$ LC_ALL=C pwd
/media/Podatki/Dokumenti Anþe


# urpmi
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = "en_GB.UTF-8:en_GB:en",
        LC_ALL = (unset),
        LC_PAPER = "sl_SI.UTF-8",
        LC_ADDRESS = "sl_SI.UTF-8",
        LC_MONETARY = "sl_SI.UTF-8",
        LC_SOURCED = "1",
        LC_NUMERIC = "sl_SI.UTF-8",
        LC_TELEPHONE = "sl_SI.UTF-8",
        LC_MESSAGES = "en_GB.UTF-8",
        LC_COLLATE = "en_GB.UTF-8",
        LC_IDENTIFICATION = "sl_SI.UTF-8",
        LC_MEASUREMENT = "sl_SI.UTF-8",
        LC_CTYPE = "en_GB.UTF-8",
        LC_TIME = "en_GB.UTF-8",
        LC_NAME = "sl_SI.UTF-8",
        LANG = "en_GB.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").


So this is duplicate of bug 3723.

*** This bug has been marked as a duplicate of bug 3723 ***

Status: NEW => RESOLVED
Resolution: (none) => DUPLICATE
Summary: Wrong encoding on automatically mounted existing partitions (Live KDE) => Wrong encoding on automatically mounted existing partitions (Live KDE) if setting Time Zone other than languages on CD

Comment 17 Marja Van Waes 2012-05-02 13:05:32 CEST
I think your bug is more severe, because there is no LiveCD with Slovenian locales. TBH, I think the other guy should have used the LiveCD for Eastern Europe, because he'd need the locales for his currency and such, too. I didn't want to close that one as a duplicate of this one, because for me that one is 50% user error, while this one isn't.

@ Anssi

I read your comment here:
https://qa.mandriva.com/show_bug.cgi?id=31639#c23

Is it very hard to make such a localisation bundle? If not, would you be willing to make one for Slovenia(n), or do you know of someone who could do that?

CC: (none) => anssi.hannula

Comment 18 Anssi Hannula 2012-05-02 16:23:29 CEST
@Marja, generating such a bundle is relatively easy (just put all %lang(fi) marked files there from all (or only Live) rpms), but properly handling the installation of such a bundle is more difficult (localizations for non-installed package should be skipped, and preferably (i.e. if not too hard) rpmdb should be updated so that the new localization files are assigned to the packages where they came from).

In theory I could look at it, but I don't think it will be in Mageia 2 timeframe... Maybe e.g. pterjan, tmb or blino could also take a look, but I'd guess they are pretty busy right now with the release medias.

The manual way to switch to a new locale and internationalization right now is to install locales-xx, then enable it in draklocale, and then reinstall needed packages to get the new translation files, by using e.g. Eric Piel's script:
https://bzattachment.mandriva.com/attachment.cgi?id=8509
Comment 19 Marja Van Waes 2012-05-02 17:48:58 CEST
(In reply to comment #18)
> @Marja, generating such a bundle is relatively easy (just put all %lang(fi)
> marked files there from all (or only Live) rpms), but properly handling the
> installation of such a bundle is more difficult (localizations for
> non-installed package should be skipped, and preferably (i.e. if not too hard)
> rpmdb should be updated so that the new localization files are assigned to the
> packages where they came from).
> 
> In theory I could look at it, but I don't think it will be in Mageia 2
> timeframe... Maybe e.g. pterjan, tmb or blino could also take a look, but I'd
> guess they are pretty busy right now with the release medias.
> 
> The manual way to switch to a new locale and internationalization right now is
> to install locales-xx, then enable it in draklocale, and then reinstall needed
> packages to get the new translation files, by using e.g. Eric Piel's script:
> https://bzattachment.mandriva.com/attachment.cgi?id=8509

Thanks a lot, Anssi :)

cc'ing doc team, because we need to document the manual method.

Anssi, am I correct to think that what you call "translation files" is about a lot more than just translation, so about other localisation stuff too?

CC: (none) => doc-bugs

Comment 20 Filip Komar 2012-05-07 10:58:03 CEST
(In reply to comment #17)
> I think your bug is more severe, because there is no LiveCD with Slovenian
> locales. TBH, I think the other guy should have used the LiveCD for Eastern
> Europe, because he'd need the locales for his currency and such, too. I didn't
> want to close that one as a duplicate of this one, because for me that one is
> 50% user error, while this one isn't.

I tried setting Time Zone to Rome and encoding was fine. So I think that Thomas is right about Time Zone influence (see bug 3723, comment 7).

@Marja: Maybe I shouldn't close this one as duplicate but I'm sure that this is the case.  But please you reopen it if you agree with it, I don't want to mess it up more than I did ;).

Is this worth of being mentioned in errata? In my limited knowledge I think that data or filenames corruption of can occur.
Comment 21 Filip Komar 2012-05-09 15:37:59 CEST
As expected Live KDE CD for Europe 1 works fine in Slovenian.

Note:
I didn't test yet if wrong encoding on automatically mounted existing partitions when setting Time Zone other than languages on CD still stand on M2RC.
Comment 22 Marja Van Waes 2012-05-09 15:40:37 CEST
(In reply to comment #21)
> As expected Live KDE CD for Europe 1 works fine in Slovenian.
> 

Great :)

> Note:
> I didn't test yet if wrong encoding on automatically mounted existing
> partitions when setting Time Zone other than languages on CD still stand on
> M2RC.

No problem ;)
Comment 23 Filip Komar 2012-05-09 18:31:12 CEST
@Marja (In reply to comment #20)
Can you clean the mess after my closing this one as a duplicate of bug 3723?
THX.
Comment 24 Marja Van Waes 2012-05-09 20:51:44 CEST
(In reply to comment #23)
> @Marja (In reply to comment #20)
> Can you clean the mess after my closing this one as a duplicate of bug 3723?
> THX.

There is no mess :)

About adding this issue to the errata:

Already in Mandriva when setting your time zone, in fact not only the time zone, but also a bunch of other locales were set. If this step had been about time zone only, we'd be given the choice to pick "CET/CEST" instead of over 20 different choices in this time zone, like "Ljubljana", "Amsterdam", "Rome", and so on. 

It feels weird to start adding this to the errata now, moreover very few users are affected: most will use a LiveCD for their country/language

If I could think of a better name for the Time Zone setting, that would reflect all the other locale settings that are influenced by it, I would open an enhancement request for it.

Thomas Backlund might see a way to solve this, from his reply here https://bugs.mageia.org/show_bug.cgi?id=3723#c7 I get the impression he wants to make "setting the time zone" set nothing but the time zone

Note You need to log in before you can comment on or make changes to this bug.