[GRASS-dev] RE : [GRASS GIS] #1293: Creating mapset with non-latin letter gives python ascii error

[Robert Lagacé]

Can you give me an example?

Thank,

Robert Lagacé

De : grass-dev-boun...@lists.osgeo.org [grass-dev-boun...@lists.osgeo.org] de 
la part de GRASS GIS [t...@osgeo.org]
Date d'envoi : 3 avril 2013 09:50
Objet : Re: [GRASS-dev] [GRASS GIS] #1293: Creating mapset with non-latin 
letter gives python ascii error

#1293: Creating mapset with non-latin letter gives python ascii error
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…
 Type:  defect   |  Status:  new
 Priority:  normal   |   Milestone:  6.4.1
Component:  wxGUI| Version:  svn-trunk
 Keywords:   |Platform:  Unspecified
  Cpu:  Unspecified  |
-+--
Changes (by torsti):

 * cc: torsti (added)
  * keywords:  wingrass =>
  * version:  6.4.1 RCs => svn-trunk
  * platform:  MSWindows Vista => Unspecified


Comment:

 This bug persists in a fairly recent grass7 on Linux. But it's really two
 separate issues: Unicode handling in the python GUI (lost of other tickets
 on that...) and the fact that the validity of the mapset name isn't being
 checked.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Re: [GRASS GIS] #1293: Creating mapset with non-latin letter gives python ascii error

[GRASS GIS]

#1293: Creating mapset with non-latin letter gives python ascii error
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  normal   |   Milestone:  6.4.1
Component:  wxGUI| Version:  6.4.1 RCs
 Keywords:   |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--

Comment(by marisn):

 Seems that not only startup screen is failing with non-latin mapsets. No
 idea where this error comes from. Got it in WXGUI "Command console" by
 using mapset named "šaursliežu dzelzceļš".

 {{{
 C:\Program
 Files\GRASS-64\etc\wxpython\gui_modules\goutput.py:748:
 UnicodeWarning: Unicode equal comparison failed to convert
 both arguments to Unicode - interpreting them as being
 unequal
   if mapName in mapLayers:
 }}}

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Re: [GRASS GIS] #1293: Creating mapset with non-latin letter gives python ascii error

[GRASS GIS]

#1293: Creating mapset with non-latin letter gives python ascii error
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  normal   |   Milestone:  6.4.1
Component:  wxGUI| Version:  6.4.1 RCs
 Keywords:   |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--

Comment(by marisn):

 Another place, that fails: g.help

 {{{
 Traceback (most recent call last):
   File "c:/osgeo4w/usr/src/grass-6.4.1RC1/dist.i686-pc-
 mingw32/etc/wxpython/gui_modules/prompt.py", line 777, in
 OnItemSelected
   File "C:\Program
 Files\GRASS-64\etc\wxpython\gui_modules\menuform.py", line
 2022, in ParseInterface

 tree = etree.fromstring(getInterfaceDescription(cmd[0]))
   File "C:\Program
 Files\GRASS-64\etc\wxpython\gui_modules\menuform.py", line
 1975, in getInterfaceDescription

 "Details: %s") % (cmd, cmderr)
 UnicodeDecodeError
 :
 'ascii' codec can't decode byte 0xd0 in position 30: ordinal
 not in range(128)

 }}}

 And afterwards things are going only downhill from there. Starting a new
 GRASS WXGUI session greets with python crash and skipping of startup
 screen:

 {{{
 KŽ█DA:KARąU KOPA ┼Īaurslie┼Šu dzelzce─╝┼Ī nav atrasta
 C:/Program Files/GRASS-64/etc/wxpython/gis_set.py:623: UnicodeWarning:
 Unicode e
 qual comparison failed to convert both arguments to Unicode - interpreting
 them
 as being unequal
   if mapset not in self.listOfMapsetsSelectable or \
 Traceback (most recent call last):
   File "C:/Program Files/GRASS-64/etc/wxpython/gis_set.py", line 858, in
 
 GRASSStartUp = StartUp(0)
   File "C:\OSGeo4W\apps\Python25\lib\site-packages\wx-2.8-msw-
 unicode\wx\_core.p
 y", line 7935, in __init__
   File "C:\OSGeo4W\apps\Python25\lib\site-packages\wx-2.8-msw-
 unicode\wx\_core.p
 y", line 7509, in _BootstrapApp
   File "C:/Program Files/GRASS-64/etc/wxpython/gis_set.py", line 829, in
 OnInit
 StartUp = GRASSStartup()
   File "C:/Program Files/GRASS-64/etc/wxpython/gis_set.py", line 166, in
 __init_
 _
 self._set_properties()
   File "C:/Program Files/GRASS-64/etc/wxpython/gis_set.py", line 241, in
 _set_pr
 operties
 (utils.UnicodeString(mapset))
   File "C:\Program Files\GRASS-64\etc\wxpython\gui_modules\utils.py", line
 669,
 in UnicodeString
 return unicode(string, enc)
   File "C:\Program Files\GRASS-64\Python25\lib\encodings\cp1257.py", line
 15, in
  decode
 return codecs.charmap_decode(input,errors,decoding_table)
 UnicodeDecodeError: 'charmap' codec can't decode byte 0xa1 in position 1:
 charac
 ter maps to 
 }}}

 Contents of .grassrc6 ("šaursliežu dzelzceļš" is a mapset in location
 "spearfish60" and not "demo"):

 {{{
 GISDBASE: C:\Users\tests\Documents\GIS_DataBase
 LOCATION_NAME: demolocation
 MAPSET: šaursliežu dzelzceļš
 GRASS_GUI: wxpython
 MAPSET : PERMANENT
 }}}

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Re: [GRASS GIS] #1293: Creating mapset with non-latin letter gives python ascii error

[GRASS GIS]

#1293: Creating mapset with non-latin letter gives python ascii error
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  normal   |   Milestone:  6.4.1
Component:  wxGUI| Version:  6.4.1 RCs
 Keywords:   |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--

Comment(by glynn):

 Replying to [ticket:1293 marisn]:

 Note that 8-bit characters are not valid within names of locations,
 mapsets or maps. It should refuse to create the mapset, albeit with a more
 meaningful error message.

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Re: [GRASS GIS] #1293: Creating mapset with non-latin letter gives python ascii error

[GRASS GIS]

#1293: Creating mapset with non-latin letter gives python ascii error
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  normal   |   Milestone:  6.4.1
Component:  wxGUI| Version:  6.4.1 RCs
 Keywords:   |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--

Comment(by martinl):

 Replying to [comment:3 glynn]:

 > Note that 8-bit characters are not valid within names of locations,
 mapsets or maps. It should refuse to create the mapset, albeit with a more
 meaningful error message.

 How complicated would be to allow 8-bit characters in the
 location/mapset/map names? BTW, on Linux, I am able to create location,
 mapset with 8-bit characters, start GRASS with created mapset and display
 maps from this mapset.

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Re: [GRASS GIS] #1293: Creating mapset with non-latin letter gives python ascii error

[GRASS GIS]

#1293: Creating mapset with non-latin letter gives python ascii error
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  normal   |   Milestone:  6.4.1
Component:  wxGUI| Version:  6.4.1 RCs
 Keywords:   |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--
Changes (by martinl):

 * cc: martinl (added)


-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Re: [GRASS GIS] #1293: Creating mapset with non-latin letter gives python ascii error

[GRASS GIS]

#1293: Creating mapset with non-latin letter gives python ascii error
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  normal   |   Milestone:  6.4.1
Component:  wxGUI| Version:  6.4.1 RCs
 Keywords:   |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--

Comment(by marisn):

 Replying to [comment:3 glynn]:
 > Note that 8-bit characters are not valid within names of locations,
 mapsets or maps. It should refuse to create the mapset, albeit with a more
 meaningful error message.
 URL to documentation? RFC?

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Re: [GRASS GIS] #1293: Creating mapset with non-latin letter gives python ascii error

[GRASS GIS]

#1293: Creating mapset with non-latin letter gives python ascii error
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  normal   |   Milestone:  6.4.1
Component:  wxGUI| Version:  6.4.1 RCs
 Keywords:   |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--

Comment(by glynn):

 Replying to [comment:4 martinl]:

 > How complicated would be to allow 8-bit characters in the
 location/mapset/map names?

 It's not particularly complicated, just a lot of work. We would have to
 audit the whole of GRASS for any issues, then fix them.

 For code which simply passes strings around within GRASS, it isn't an
 issue. It becomes an issue when code tries to interpret strings or pass
 them outside of GRASS.

 The main issue is that once you use characters outside of ASCII, encoding
 becomes an issue. Many functions (e.g. case conversions) are affected by
 the current locale. Also, if the map name is written to a file, some file
 formats require the use of specific encodings, so we would have to convert
 the text to that encoding to avoid creating invalid files. E.g. if a map
 name is in ISO-8859-* and it gets written directly to a file format which
 uses UTF-8, the result is an invalid file.

 If you stick to ASCII, none of this matters. All commonly-used encodings
 are supersets of ASCII, so conversion between ASCII and such encodings is
 a no-op. Once you move outside of ASCII you have to perform conversions,
 which means knowing which encoding you're converting to/from.

 Another significant issue is that it's unspecified whether "char" is
 signed or unsigned. Signed is more common, which can break code which
 asssumes that a char ranges from 0 to 255.

 > BTW, on Linux, I am able to create location, mapset with 8-bit
 characters, start GRASS with created mapset and display maps from this
 mapset.

 That indicates that G_legal_name() isn't being called when it should. This
 function tests for invalid characters:
 {{{
 if (*s == '/' || *s == '"' || *s == '\'' || *s <= ' ' ||
 *s == '@' || *s == ',' || *s == '=' || *s == '*' || *s > 0176)
 {
 G_warning(_("Illegal filename <%s>. Character <%c> not
 allowed.\n"), name, *s);
 return -1;
 }
 }}}

 This prohibits all 8-bit characters (due to either the "*s > 0176" or the
 "*s <= ' '", depending upon whether char is signed or unsigned).

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Re: [GRASS GIS] #1293: Creating mapset with non-latin letter gives python ascii error

[GRASS GIS]

#1293: Creating mapset with non-latin letter gives python ascii error
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  normal   |   Milestone:  6.4.1
Component:  wxGUI| Version:  6.4.1 RCs
 Keywords:  wingrass |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--
Changes (by hellik):

  * keywords:  => wingrass


-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Re: [GRASS GIS] #1293: Creating mapset with non-latin letter gives python ascii error

[GRASS GIS]

#1293: Creating mapset with non-latin letter gives python ascii error
-+--
 Reporter:  marisn   |   Owner:  grass-dev@…  
 Type:  defect   |  Status:  new  
 Priority:  normal   |   Milestone:  6.4.1
Component:  wxGUI| Version:  6.4.1 RCs
 Keywords:  wingrass |Platform:  MSWindows Vista  
  Cpu:  Unspecified  |  
-+--

Comment(by glynn):

 Replying to [comment:8 hellik]:
 > keywords wingrass added

 Is this actually Windows-specific?

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev