Re: [GRASS-dev] [GRASS GIS] #2525: Unable to open sqlite database if path contains non-latin letters

[GRASS GIS]

#2525: Unable to open sqlite database if path contains non-latin letters
--+-
  Reporter:  marisn   |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  major|  Milestone:  7.6.2
 Component:  wxGUI|Version:  svn-releasebranch70
Resolution:   |   Keywords:  attribute
   CPU:  Unspecified  |   Platform:  MSWindows Vista
--+-

Comment (by marisn):

 Replying to [comment:13 sbl]:
 > In 7.6, the state is actually worse, if the mapset contains non-ascii
 characters...

 Mapset names should not contain non-ascii chars. This issue was about
 cases when mapset name is OK, but path leading to it contains non-ascii
 chars.

 Unfortunately I could not test the issue as GRASS failed to start on
 Windows at all due to #3837

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #2525: Unable to open sqlite database if path contains non-latin letters

[GRASS GIS]

#2525: Unable to open sqlite database if path contains non-latin letters
--+-
  Reporter:  marisn   |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  major|  Milestone:  7.6.2
 Component:  wxGUI|Version:  svn-releasebranch70
Resolution:   |   Keywords:  attribute
   CPU:  Unspecified  |   Platform:  MSWindows Vista
--+-
Changes (by sbl):

 * milestone:  7.0.7 => 7.6.2


Comment:

 In 7.6, the state is actually worse, if the mapset contains non-ascii
 characters...

 With mapset "tøst", GUI does not start up. Also maps with name tøst cannot
 be created:

 {{{
 v.edit map=test tool=create
 WARNING: Illegal filename . Character  not allowed.
 }}}

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #2525: Unable to open sqlite database if path contains non-latin letters

[GRASS GIS]

#2525: Unable to open sqlite database if path contains non-latin letters
--+-
  Reporter:  marisn   |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  major|  Milestone:  7.0.7
 Component:  wxGUI|Version:  svn-releasebranch70
Resolution:   |   Keywords:  attribute
   CPU:  Unspecified  |   Platform:  MSWindows Vista
--+-

Comment (by martinl):

 What is status of this ticket?

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #2525: Unable to open sqlite database if path contains non-latin letters

[GRASS GIS]

#2525: Unable to open sqlite database if path contains non-latin letters
--+-
  Reporter:  marisn   |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  major|  Milestone:  7.0.3
 Component:  wxGUI|Version:  svn-releasebranch70
Resolution:   |   Keywords:  attribute
   CPU:  Unspecified  |   Platform:  MSWindows Vista
--+-
Changes (by neteler):

 * keywords:   => attribute
 * milestone:  7.0.0 => 7.0.3


--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #2525: Unable to open sqlite database if path contains non-latin letters

[GRASS GIS]

#2525: Unable to open sqlite database if path contains non-latin letters
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  major|   Milestone:  7.0.0
Component:  wxGUI| Version:  svn-releasebranch70  
 Keywords:   |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--

Comment(by glynn):

 Replying to [comment:3 hellik]:

 
 {{{
   File C:\OSGeo4W\apps\Python27\lib\encodings\cp1252.py,
 }}}

 Again, that character doesn't exist in cp1252. This isn't specific to
 GRASS; any portable C code will have exactly the same problems. The only
 way that you can open that file is to use Windows-specific functions (e.g.
 _wfopen() or CreateFileW()). Even passing it as an argument requires using
 wmain() instead of main().

 I'm only interested in whether it works in a locale which uses cp1257. The
 fact that it doesn't work with cp1252 is a wontfix.

-- 
Ticket URL: http://trac.osgeo.org/grass/ticket/2525#comment:5
GRASS GIS http://grass.osgeo.org

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #2525: Unable to open sqlite database if path contains non-latin letters

[GRASS GIS]

#2525: Unable to open sqlite database if path contains non-latin letters
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  major|   Milestone:  7.0.0
Component:  wxGUI| Version:  svn-releasebranch70  
 Keywords:   |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--

Comment(by hellik):

 Replying to [comment:2 glynn]:
  Replying to [ticket:2525 marisn]:
 
   Seems that any operation touching attribute database fails if path
 contains non-latin letter.
   It is a single instance of a general problem of passing file names as
 arguments between GUI and modules.
 
  Do you have the same problem with filenames other than database files?
 E.g. do r.in.* or r.out.* work?

 {{{
 r.out.gdal --verbose input=MRVBF4@user1 output=C:\wd\eudata\Māris\test.tif
 format=GTiff
 }}}

 {{{
 Exception in thread Thread-278:
 Traceback (most recent call last):
   File C:\OSGeo4W\apps\Python27\lib\threading.py, line
 810, in __bootstrap_inner
 self.run()
   File C:\OSGeo4W\apps\grass\grass-7.1.svn\gui\wxpython\cor
 e\gconsole.py, line 155, in run
 self.resultQ.put((requestId, self.requestCmd.run()))
   File C:\OSGeo4W\apps\grass\grass-7.1.svn\gui\wxpython\cor
 e\gcmd.py, line 575, in run
 env = self.env)
   File C:\OSGeo4W\apps\grass\grass-7.1.svn\gui\wxpython\cor
 e\gcmd.py, line 161, in __init__
 args = map(EncodeString, args)
   File C:\OSGeo4W\apps\grass\grass-7.1.svn\gui\wxpython\cor
 e\gcmd.py, line 92, in EncodeString
 return string.encode(_enc)
   File C:\OSGeo4W\apps\Python27\lib\encodings\cp1252.py,
 line 12, in encode
 return
 codecs.charmap_encode(input,errors,encoding_table)
 UnicodeEncodeError: 'charmap' codec can't encode character
 u'\u0101' in position 21: character maps to undefined
 }}}

-- 
Ticket URL: http://trac.osgeo.org/grass/ticket/2525#comment:3
GRASS GIS http://grass.osgeo.org

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #2525: Unable to open sqlite database if path contains non-latin letters

[GRASS GIS]

#2525: Unable to open sqlite database if path contains non-latin letters
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  major|   Milestone:  7.0.0
Component:  wxGUI| Version:  svn-releasebranch70  
 Keywords:   |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--

Comment(by hellik):

 Replying to [comment:2 glynn]:
  Replying to [ticket:2525 marisn]:
 
   Seems that any operation touching attribute database fails if path
 contains non-latin letter.
   It is a single instance of a general problem of passing file names as
 arguments between GUI and modules.
 
  Do you have the same problem with filenames other than database files?
 E.g. do r.in.* or r.out.* work?

 it doesn't work with

 {{{
 r.in.gdal --verbose input=C:\wd\eudata\Māris\test.tif output=testfile
 }}}

-- 
Ticket URL: http://trac.osgeo.org/grass/ticket/2525#comment:4
GRASS GIS http://grass.osgeo.org

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #2525: Unable to open sqlite database if path contains non-latin letters

[GRASS GIS]

#2525: Unable to open sqlite database if path contains non-latin letters
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  major|   Milestone:  7.0.0
Component:  wxGUI| Version:  svn-releasebranch70  
 Keywords:   |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--

Comment(by glynn):

 Replying to [ticket:2525 marisn]:

  Note: could CommandLineToArgvW be helpful?

 What we need is the reverse: something which reliably converts argv[] to a
 command string.

 We actually have one of those (make_command_line() in lib/gis/spawn.c),
 and Python also has one (list2cmdline() in the subprocess module). The
 problem is that both of these only reverse the parsing which is done by
 the executable itself, not that done by the shell. The shell's parsing
 rules are even less well documented than those of the executable, and even
 less sane.

 The other issue is that the shell uses two different encodings
 (codepages): ANSI and OEM. Most of the time this doesn't matter; you
 can just pass the byte strings straight through. But there are cases (such
 as using the FOR command with backticks to take process output and use it
 as an argument) where this doesn't work, and any character which doesn't
 have the same codepoint in both encodings will cause problems (problems
 which can't realistically be solved).

 As for filenames, the main issues are

  1. If you use byte strings (i.e. char*) (e.g. fopen()), you can't access
 any file whose name isn't representable in the current codepage. Those
 files effectively don't exist in the char* world.

  2. The only supported encoding for Japanese is Shift-JIS (cp932), which
 has the unfortunate feature of not being entirely compatible with ASCII.
 Specifically, 0x5c is used both for the directory separator (normally
 backslash, but actually prints as a yen (¥) sign in Japanese locales) and
 as the second byte of some multi-byte sequences. Meaning that any code
 which tries to parse filenames as byte strings with 0x5c as a directory
 separator will often fail on Japanese filenames.

 Neither of these have any simple solution (not even unreliable hacks).
 The only effective solution is to use the Unicode (i.e. wchar_t*) API.

 In practical terms, that would mean writing a compatibility layer which
 re-implements all of the standard ANSI C and POSIX filesystem calls,
 taking UTF-8 char* arguments, converting to UTF-16 wchar_t*, then using
 the Windows-specific wchar_t* functions. Anything which uses third-party
 library functions which take filenames as char* won't work.

 We'd also need custom startup code which used main16(int argc, wchar_t
 **argv) as the entry point, converted all arguments to UTF-8, then called
 main(). We'd still have issues with reading filenames from files, stdin,
 or output from child processes, as these would either have to be in UTF-8
 or would need to be converted to UTF-8 (which means that we'd need to know
 the encoding).

-- 
Ticket URL: http://trac.osgeo.org/grass/ticket/2525#comment:1
GRASS GIS http://grass.osgeo.org

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #2525: Unable to open sqlite database if path contains non-latin letters

[GRASS GIS]

#2525: Unable to open sqlite database if path contains non-latin letters
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  major|   Milestone:  7.0.0
Component:  wxGUI| Version:  svn-releasebranch70  
 Keywords:   |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--

Comment(by glynn):

 Replying to [ticket:2525 marisn]:

  Seems that any operation touching attribute database fails if path
 contains non-latin letter.
  It is a single instance of a general problem of passing file names as
 arguments between GUI and modules.

 Do you have the same problem with filenames other than database files?
 E.g. do r.in.* or r.out.* work?

-- 
Ticket URL: http://trac.osgeo.org/grass/ticket/2525#comment:2
GRASS GIS http://grass.osgeo.org

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev