#3925: winGRASS 7.8.1dev: 'charmap' codec can't decode byte 0x9d - issue in vector attribute data handling (e.g. opening attribute table, v.report, etc) ----------------------+------------------------------------ Reporter: hellik | Owner: grass-dev@… Type: defect | Status: new Priority: blocker | Milestone: 7.8.1 Component: Vector | Version: git-releasebranch78 Resolution: | Keywords: python3, py3, wingrass CPU: x86-64 | Platform: MSWindows ----------------------+------------------------------------
Comment (by hellik): now tested in a windows 10 box with a german locale: {{{ GRASS Version: 7.8.1dev Code revision: f5bfe545c Build date: 2019-10-27 Build platform: x86_64-w64-mingw32 GDAL: 2.4.1 PROJ: 5.2.0 GEOS: 3.8.0 SQLite: 3.29.0 Python: 3.7.0 wxPython: 4.0.3 Platform: Windows-10-10.0.18362-SP0 (OSGeo4W) }}} download geonames data of an [http://download.geonames.org/export/dump/AT.zip AT dump] {{{ v.in.geonames input=D:\temp\geonames\AT\AT.txt output=at_geonames Converting 51999 place names... Scanne die Eingabe zur Ermittelung der Spaltentypen... Number of columns: 19 Number of rows: 51999 WARNING: Spalte Nummer 13 <admin3code> ist als string definiert und hat nur Integer-Werte. WARNING: Spalte Nummer 14 <admin4code> ist als string definiert und hat nur Integer-Werte. Importiere Punkte... Fülle Tabelle... Erstelle Topologie für die Vektorkarte <at_geonames@data>... Registriere Primitive... GRASS_INFO_PROGRESS: 10000 GRASS_INFO_PROGRESS: 20000 GRASS_INFO_PROGRESS: 30000 GRASS_INFO_PROGRESS: 40000 GRASS_INFO_PROGRESS: 50000 }}} clicking on a point with a german umlaut {{{ east, north: 11.31700474110463, 47.23295282435024 at_geonames@data: Type: Point Id: 10136 Layer: 1 Category: 10136 Driver: sqlite Database: D:\grassdata\loc_test_vingeonames\data\sqlite\sqlite.db Table: at_geonames Key_column: cat Attributes: cat: 10136 geonameid: 2762446 name: Vellenberg asciiname: Vellenberg latitude: 47.23333 longitude: 11.31667 featureclass: S featurecode: FRM countrycode: AT admin1code: 07 admin2code: 703 admin3code: 70312 population: 0 gtopo30: 865 timezone: Europe/Vienna modification: 2014-05-03 at_geonames@data: Type: Point Id: 25781 Layer: 1 Category: 25781 Driver: sqlite Database: D:\grassdata\loc_test_vingeonames\data\sqlite\sqlite.db Table: at_geonames Key_column: cat Attributes: cat: 25781 geonameid: 2778215 name: Götznerberg asciiname: Goetznerberg latitude: 47.23333 longitude: 11.31667 featureclass: S featurecode: FRM countrycode: AT admin1code: 07 admin2code: 703 admin3code: 70312 population: 0 gtopo30: 865 timezone: Europe/Vienna modification: 2014-05-03 }}} trying v.report {{{ v.report map=at_geonames@data option=coor Traceback (most recent call last): File "C:\OSGEO4~1\apps\grass\grass78/scripts/v.report.py", line 226, in <module> main() File "C:\OSGEO4~1\apps\grass\grass78/scripts/v.report.py", line 108, in main cols = decode(line).rstrip('\r\n').split('|') File "C:\OSGEO4~1\apps\grass\grass78\etc\python\grass\scri pt\utils.py", line 193, in decode return bytes_.decode(enc) File "C:\OSGEO4~1\apps\Python37\lib\encodings\cp1252.py", line 15, in decode return codecs.charmap_decode(input,errors,decoding_table) UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 195: character maps to <undefined> }}} or trying to open the vector attribute table {{{ Traceback (most recent call last): File "C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\lmgr\frame.py", line 2060, in OnShowAttributeTable selection=selection) File "C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\dbmgr\ma nager.py", line 112, in __init__ self.CreateDbMgrPage(parent=self, pageName='browse') File "C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\dbmgr\base.py", line 811, in CreateDbMgrPage parent=parent, parentDbMgrBase=self, onlyLayer=onlyLayer) File "C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\dbmgr\base.py", line 1095, in __init__ self.AddLayer(layer) File "C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\dbmgr\base.py", line 1138, in AddLayer self.dbMgrData, layer, self.pages) File "C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\dbmgr\base.py", line 113, in __init__ keyColumn = self.LoadData(layer) File "C:\OSGEO4~1\apps\grass\grass78\gui\wxpython\dbmgr\base.py", line 278, in LoadData record = decode(outFile.readline().strip()).replace('\n', '') File "C:\OSGEO4~1\apps\grass\grass78\etc\python\grass\scri pt\utils.py", line 193, in decode return bytes_.decode(enc) File "C:\OSGEO4~1\apps\Python37\lib\encodings\cp1252.py", line 15, in decode return codecs.charmap_decode(input,errors,decoding_table) UnicodeDecodeError : 'charmap' codec can't decode byte 0x9d in position 219: character maps to <undefined> }}} opening the table freezes the attribute table window it seems to be an encoding issue of attribute data handling {{{ 'charmap' codec can't decode byte 0x9d }}} -- Ticket URL: <https://trac.osgeo.org/grass/ticket/3925#comment:2> GRASS GIS <https://grass.osgeo.org>
_______________________________________________ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev