I've not done much with reflection, but perhaps you could use the column_reflect event:
http://docs.sqlalchemy.org/en/rel_0_8/core/events.html#sqlalchemy.events.DDLEvents.column_reflect Simon On Tue, Feb 4, 2014 at 11:28 AM, Erich Blume <blume.er...@gmail.com> wrote: > Thanks Simon, > > Do you know how I might use that with reflection? There's several hundred of > these columns, I'd hate to have to override each one individually - that > sort of defeats the purpose of reflection. > > One thought I just had was perhaps I could subclass the Text type and then > override the ischema_names for SQLite for TEXT type. That'd do the trick, I > suspect! > > > On Tue, Feb 4, 2014 at 3:26 AM, Simon King <si...@simonking.org.uk> wrote: >> >> On Tue, Feb 4, 2014 at 10:15 AM, Erich Blume <blume.er...@gmail.com> >> wrote: >> > I am working on a binding to a SQLite database that I do not control the >> > creation of, with the aid of reflection. I'm running in to what I >> > believe >> > are very basic UTF-8 decoding errors. For instance, a TEXT cell has the >> > byte >> > '0x92' in it and is causing an OperationalError. Presumably, this is >> > because >> > 0x92 (by itself) is not a valid encoding for any Unicode code point. I >> > would >> > prefer that the decoding from UTF-8 to be forced, perhaps by dropping >> > the >> > bad byte. How can I do this? >> > >> > The database has a table with a column called 'description', which is of >> > type TEXT. The "PRAGMA encoding" is left at 'UTF-8', thank goodness. One >> > of >> > the rows, however, contains within its otherwise ascii byte contents the >> > singleton byte '0x92'. Based on the context of the sentence, it seems >> > that >> > this was intended to be encoded as a single quotation mark, some >> > googling >> > suggests 'RIGHT SINGLE QUOTATION MARK' in unicode, which is '0xE2 0x80 >> > 0x99'. I gather that MSSQL (which was the original source of the data in >> > this database) uses Microsofts' infernal web encodings sometimes and >> > that is >> > probably the source of this byte. >> > >> > The issue is this: I really need to read this data! It would be *ideal* >> > to >> > have the aid of something like python's 'replace' decoding handler but >> > failing that just eliding the byte would do fine in a pinch. >> > >> > When fetching this row in Python 3.3 with SQLAlchemy 0.9.1 my session >> > looks >> > vaguely like this (with the text and stack trace truncated out for >> > brevity). >> > >> > File >> > >> > "/usr/local/Cellar/python3/3.3.3/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/sqlalchemy/engine/result.py", >> > line 760, in <listcomp> >> > return [process_row(metadata, row, processors, keymap) >> > sqlalchemy.exc.OperationalError: (OperationalError) Could not >> > decode >> > to UTF-8 column 'description' with text <...> >> > >> > Is there some way to accomplish this? >> > >> >> The String-related column types have a "unicode_error" parameter which >> sounds like it might be what you want: >> >> >> http://docs.sqlalchemy.org/en/rel_0_9/core/types.html#sqlalchemy.types.String.params.unicode_error >> >> Note the various warnings around it though... >> >> Hope that helps, >> >> Simon >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "sqlalchemy" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/sqlalchemy/T--Ftk5EVZg/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> sqlalchemy+unsubscr...@googlegroups.com. >> >> To post to this group, send email to sqlalchemy@googlegroups.com. >> Visit this group at http://groups.google.com/group/sqlalchemy. >> For more options, visit https://groups.google.com/groups/opt_out. > > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at http://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
