[partitionmanager] [Bug 384321] Mess with encoding in kpmcore

[Andrius Štikonas]

https://bugs.kde.org/show_bug.cgi?id=384321

Andrius Štikonas  changed:

   What|Removed |Added

  Latest Commit|https://commits.kde.org/kpm |https://invent.kde.org/syst
   |core/4bf5bb467f6c029da7871f |em/kpmcore/commit/0c25d1f9c
   |8bd2a78410a5f03e70  |42e94efb5403a2652b62f145d2b
   ||24c6
 Status|REOPENED|RESOLVED
 Resolution|--- |FIXED

--- Comment #8 from Andrius Štikonas  ---
Git commit 0c25d1f9c42e94efb5403a2652b62f145d2b24c6 by Andrius Štikonas.
Committed on 01/10/2020 at 22:51.
Pushed by stikonas into branch 'master'.

Use QFile::decodeFileName for decoding device node names.

M  +3-2src/core/partition.cpp
M  +3-2src/fs/filesystem.cpp

https://invent.kde.org/system/kpmcore/commit/0c25d1f9c42e94efb5403a2652b62f145d2b24c6

-- 
You are receiving this mail because:
You are watching all bug changes.

[partitionmanager] [Bug 384321] Mess with encoding in kpmcore

[Andrius Štikonas]

https://bugs.kde.org/show_bug.cgi?id=384321

--- Comment #7 from Andrius Štikonas  ---
Git commit d27455b6f08e42470f375de679b1751a4972d73b by Andrius Štikonas.
Committed on 04/12/2017 at 21:19.
Pushed by stikonas into branch 'sfdisk'.

Avoid unnecessary QString encoding/decoding.

M  +1-1src/fs/luks.cpp
M  +1-1src/plugins/libparted/libpartedbackend.cpp
M  +3-3src/plugins/sfdisk/sfdiskbackend.cpp
M  +1-1src/plugins/sfdisk/sfdiskpartitiontable.cpp

https://commits.kde.org/kpmcore/d27455b6f08e42470f375de679b1751a4972d73b

-- 
You are receiving this mail because:
You are watching all bug changes.

[partitionmanager] [Bug 384321] Mess with encoding in kpmcore

[Andrius Štikonas]

https://bugs.kde.org/show_bug.cgi?id=384321

--- Comment #6 from Andrius Štikonas  ---
Ok, thanks  for explaining. Getting a bit clearer (I didn't have much
experience with this encoding stuff before.

I think KPM will never work on Windows or Mac OS. But better be safer now. In
any case I would like to make it more portable, e.g. FreeBSD support might be
possible in not too distant future.

-- 
You are receiving this mail because:
You are watching all bug changes.

[partitionmanager] [Bug 384321] Mess with encoding in kpmcore

[Pali Rohár]

https://bugs.kde.org/show_bug.cgi?id=384321

--- Comment #5 from Pali Rohár  ---
Block device name is just ordinary file. In most cases people use only ascii
characters (because non-ascii names are lot of times broken in CLI or GUI
tools), but there is no restrictions for 8bit. On linux you can call mknod and
create block device on file system with any file name (all 8 bit characters
expect slash and nul are allowed)...

Now looking at QFile::decodeName() implementation (it is inlined in header file
qfile.h) and there is #ifdef. On linux it just call QString::fromLocal8Bit()
and on other platforms it doing some UTF-8 normalization.

So this filename stuff depends on you, if you are going to write application
which is linux-only then QString::fromLocal8Bit() is for sure enough (as
QFile::decodeName() is doing this on linux). But if you are going to support
also other systems (Mac OS, Windows, ...) then it is really a good idea to use
Qt functions which deals with those platform dependent problems...

But problem with LC_ALL=C is a real problem for linux, because all application
which uses wide characters (in C it is wchar_t and in C++ it is also
std::wstring) uses encoding specified in LC_CTYPE (resp. LC_ALL). And "C" means
7bit ASCII. Not Latin1, not UTF-8. And both C and C++ libraries uses LC_CTYPE
for converting char*/std::string to wchar_t*/std::wstring according to LC_CTYPE
(resp. LC_ALL). So with restriction to 7bit ASCII (as "C" is ASCII) you just
kill whole UTF-8/Unicode support.

To test current encoding you can use this simple C program:

  #include 
  #include 
  #include 
  int main() { setlocale(LC_CTYPE,""); puts(nl_langinfo(CODESET)); return 0; }

Also it is important that some programs works in UTF-8 independently of how is
LC_ALL configured (e.g mkudffs when is called with --utf8 param, but there are
probably others too). And so for these programs it is needed to use fromUtf8
and toUtf8, not fromLocal8bit/toLocal8bit (as those functions decode/encode
according to LC_CTYPE/LC_ALL).

-- 
You are receiving this mail because:
You are watching all bug changes.

[partitionmanager] [Bug 384321] Mess with encoding in kpmcore

[Andrius Štikonas]

https://bugs.kde.org/show_bug.cgi?id=384321

--- Comment #4 from Andrius Štikonas  ---
(In reply to Pali Rohár from comment #3)
> > -if 
> > (QFileInfo(QString::fromUtf8(storage.device())).canonicalFilePath() == 
> > canonicalDeviceNode ) {
> > +if 
> > (QFileInfo(QString::fromLocal8Bit(storage.device())).canonicalFilePath() == 
> > canonicalDeviceNode ) {
> 
> This does not look like a correct change. As I wrote in first comment, for
> decoding and encoding file names QFile::decodeName() resp.
> QFile::decodeName() should be used:
> 
> http://doc.qt.io/qt-5/qfile.html#decodeName
> http://doc.qt.io/qt-5/qfile.html#encodeName

Hmm, but can block device name even be encoded? I was looking for some
documentation, but it looks to be always ASCII. I was indeed looking at those
decodeName functions but aren't they unnecessary for ASCII.

-- 
You are receiving this mail because:
You are watching all bug changes.

[partitionmanager] [Bug 384321] Mess with encoding in kpmcore

[Pali Rohár]

https://bugs.kde.org/show_bug.cgi?id=384321

Pali Rohár  changed:

   What|Removed |Added

 Resolution|FIXED   |---
 Status|RESOLVED|REOPENED

--- Comment #3 from Pali Rohár  ---
> -if 
> (QFileInfo(QString::fromUtf8(storage.device())).canonicalFilePath() == 
> canonicalDeviceNode ) {
> +if 
> (QFileInfo(QString::fromLocal8Bit(storage.device())).canonicalFilePath() == 
> canonicalDeviceNode ) {

This does not look like a correct change. As I wrote in first comment, for
decoding and encoding file names QFile::decodeName() resp. QFile::decodeName()
should be used:

http://doc.qt.io/qt-5/qfile.html#decodeName
http://doc.qt.io/qt-5/qfile.html#encodeName

And when starting external program with env LC_ALL=C then
QString::fromLocal8Bit() and toLocal8Bit() is also wrong as LC_ALL=C define
ASCII encoding. So some equivalent of "to 7bit ASCII" and "from 7bit ASCII"
should be used (seems that QString does not provide ASCII transformation
functions yet).

-- 
You are receiving this mail because:
You are watching all bug changes.

[partitionmanager] [Bug 384321] Mess with encoding in kpmcore

[Andrius Štikonas]

https://bugs.kde.org/show_bug.cgi?id=384321

--- Comment #2 from Andrius Štikonas  ---
Git commit f09cb6435ec8ec6175d098b6bb5cb5bfb5871f54 by Andrius Štikonas.
Committed on 03/12/2017 at 12:40.
Pushed by stikonas into branch 'sfdisk'.

Fix conversion to/from QByteArray in sfdisk backend.

M  +3-3src/plugins/sfdisk/sfdiskbackend.cpp
M  +1-1src/plugins/sfdisk/sfdiskdevice.cpp
M  +1-1src/plugins/sfdisk/sfdiskpartitiontable.cpp

https://commits.kde.org/kpmcore/f09cb6435ec8ec6175d098b6bb5cb5bfb5871f54

-- 
You are receiving this mail because:
You are watching all bug changes.

[partitionmanager] [Bug 384321] Mess with encoding in kpmcore

[Andrius Štikonas]

https://bugs.kde.org/show_bug.cgi?id=384321

Andrius Štikonas  changed:

   What|Removed |Added

 Resolution|--- |FIXED
  Latest Commit||https://commits.kde.org/kpm
   ||core/4bf5bb467f6c029da7871f
   ||8bd2a78410a5f03e70
 Status|CONFIRMED   |RESOLVED

--- Comment #1 from Andrius Štikonas  ---
Git commit 4bf5bb467f6c029da7871f8bd2a78410a5f03e70 by Andrius Štikonas.
Committed on 03/12/2017 at 12:37.
Pushed by stikonas into branch 'master'.

Fix conversion to/from QString/QByteArray

M  +1-1src/core/partition.cpp
M  +3-3src/core/smartstatus.cpp
M  +2-2src/fs/filesystem.cpp
M  +4-4src/fs/luks.cpp
M  +9-9src/plugins/libparted/libpartedbackend.cpp
M  +3-3src/plugins/libparted/libparteddevice.cpp
M  +7-7src/plugins/libparted/libpartedpartitiontable.cpp
M  +1-1src/util/externalcommand.cpp
M  +1-1src/util/htmlreport.cpp

https://commits.kde.org/kpmcore/4bf5bb467f6c029da7871f8bd2a78410a5f03e70

-- 
You are receiving this mail because:
You are watching all bug changes.

[partitionmanager] [Bug 384321] Mess with encoding in kpmcore

[Andrius Štikonas]

https://bugs.kde.org/show_bug.cgi?id=384321

Andrius Štikonas  changed:

   What|Removed |Added

 Ever confirmed|0   |1
 Status|UNCONFIRMED |CONFIRMED

-- 
You are receiving this mail because:
You are watching all bug changes.