Hello All, I'm an absolute newbie with web2py so I'm not sure if the issue lies in web2py/3.6 or my own mistakes... My apologies in advance.
I encounter this error in a db.my_table.update_or_insert(..) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position 96328: invalid start byte' What I'm trying to do: I have a text file that is the representation of a datamodel in a specific COTS format. I want to parse it, list the tables to be created and create them actually. I know that the file countains non-utf-8 characters, that's why I open it in binary: dpm_file = gzip.open(filepath,'rb') ... parse it using binary regex: (I imported the 'regex' module) regexp = regex.compile(b':columns (\(((?>[^()]+)|(?1))*\))',regex.DOTALL) header_classes = regexp.findall(one_format_text)[0] ... and insert the results in db blob fields: (the error occurs here) db.dpm_header.update_or_insert(db.dpm_header.dpm == my_dpm_id, dpm = my_dpm_id, header_classes = header_classes, header_objects = header_objects) I defined blob fields because I thought that it was the way to write directly in binary in database. I really wanted to use binary preciselly to avoid to manage the encoding/decoding issues. In addition, this project is an attempt to transport in web2py an older project that was working perfectly in pure python 3.6.3 (unix and windows) (except that instead of creating tables in a DB, I created text files in a filer...). Is it a issue on web2py/python 3.6? Or my misunderstanding on database management... Thank you for your help! Here is the trace Traceback (most recent call last): File "D:\web\zz\web2py\gluon\restricted.py", line 219, in restricted exec(ccode, environment) File "D:/web/zz/web2py/applications/dpm_manager/controllers/default.py" <http://127.0.0.1:8000/admin/default/edit/dpm_manager/controllers/default.py>, line 175, in <module> File "D:\web\zz\web2py\gluon\globals.py", line 419, in <lambda> self._caller = lambda f: f() File "D:\web\zz\web2py\gluon\tools.py", line 3982, in f return action(*a, **b) File "D:/web/zz/web2py/applications/dpm_manager/controllers/default.py" <http://127.0.0.1:8000/admin/default/edit/dpm_manager/controllers/default.py>, line 131, in analyze header_classes = header_classes) File "D:\web\zz\web2py\gluon\packages\dal\pydal\objects.py", line 811, in update_or_insert record = self(_key) File "D:\web\zz\web2py\gluon\packages\dal\pydal\objects.py", line 565, in __call__ orderby_on_limitby=False).first() File "D:\web\zz\web2py\gluon\packages\dal\pydal\objects.py", line 2250, in select return adapter.select(self.query, fields, attributes) File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\sqlite.py", line 82, in select return super(SQLite, self).select(query, fields, attributes) File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line 762, in select return self._select_aux(sql, fields, attributes, colnames) File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line 741, in _select_aux return processor(rows, fields, colnames, cacheable=cacheable) File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line 305, in parse for row in rows File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line 302, in <listcomp> self._parse( File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line 231, in _parse value = self.parse_value(value, fit, ft, blob_decode) File "D:\web\zz\web2py\gluon\packages\dal\pydal\adapters\base.py", line 198, in parse_value return self.parser.parse(value, field_itype, field_type) File "D:\web\zz\web2py\gluon\packages\dal\pydal\parsers\__init__.py", line 101, in parse return self.registered[field_itype](value, field_type) File "D:\web\zz\web2py\gluon\packages\dal\pydal\parsers\__init__.py", line 76, in __call__ return self.call(value, field_type) File "D:\web\zz\web2py\gluon\packages\dal\pydal\parsers\__init__.py", line 73, in _call return self.f(self.parser, value) File "D:\web\zz\web2py\gluon\packages\dal\pydal\parsers\base.py", line 37, in _blob return to_native(b64decode(to_bytes(value))) File "D:\web\zz\web2py\gluon\packages\dal\pydal\_compat.py", line 82, in to_native return obj.decode(charset, errors) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position 96328: invalid start byte -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.