That's a great point. I've made a bug card for it here: https://trello.com/c/LsZoPtwv/1053-jobs-better-unicode-handling-in-stderr
Thanks for reporting this. Carl On Sat, Jul 6, 2013 at 11:33 PM, Kyle Ellrott <kellr...@soe.ucsc.edu> wrote: > This also took out my job handlers (exception below). So the introduction > of non-ascii characters into the table (via the job stdout/stderr capture) > can make a galaxy instance pretty useless. > > I was able to find the offending records using "SELECT count(*) FROM job > WHERE stderr similar to '%\x8b%';" > Turns out it was the byproduct of doing a path paste of some gzipped fastq > files (so they where never decompressed, just passed along), and then > fastq_groomer.py complains about the file having an invalid header (line > 22?) reports back an invalid header and prints it out, thus the non-ascii > characters in the stderr. > > I 'cleaned' the database with 'update job set stderr = > regexp_replace(stderr, '\x8b', '\x5f');'. But there should probably be some > safe guards put in place to stop this from happening. > > Kyle > > > > galaxy.jobs.handler INFO 2013-07-06 12:35:32,033 job handler stop queue > started > Traceback (most recent call last): > File "/inside/depot4/galaxy/lib/galaxy/webapps/galaxy/buildapp.py", line > 35, in app_factory > app = UniverseApplication( global_conf = global_conf, **kwargs ) > File "/inside/depot4/galaxy/lib/galaxy/app.py", line 164, in __init__ > self.job_manager = manager.JobManager( self ) > File "/inside/depot4/galaxy/lib/galaxy/jobs/manager.py", line 36, in > __init__ > self.job_handler.start() > File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 34, in > start > self.job_queue.start() > File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 77, in > start > self.__check_jobs_at_startup() > File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 92, in > __check_jobs_at_startup > & ( model.Job.handler == self.app.config.server_name ) ): > File > "/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py", > line 2341, in instances > fetch = cursor.fetchall() > File > "/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", > line 3204, in fetchall > l = self.process_rows(self._fetchall_impl()) > File > "/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", > line 3171, in _fetchall_impl > return self.cursor.fetchall() > UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 404: > ordinal not in range(128) > > > > On Fri, Jul 5, 2013 at 11:43 PM, Kyle Ellrott <kellr...@soe.ucsc.edu>wrote: > >> I'm getting an exception when trying to look at the admin job management >> screen. It looks like SQLAlchemy doesn't like non-ascii characters. Any >> ideas about what to do? >> >> >> Error - <type 'exceptions.UnicodeDecodeError'>: 'ascii' codec can't >> decode byte 0x8b in position 404: ordinal not in range(128) >> URL: http://pk.kilokluster.ucsc.edu:8079/admin/jobs >> File >> '/inside/depot4/galaxy/lib/galaxy/web/framework/middleware/error.py', line >> 149 in __call__ >> app_iter = self.application(environ, sr_checker) >> File >> '/inside/depot4/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', >> line 84 in __call__ >> return self.application(environ, start_response) >> File >> '/inside/depot4/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', >> line 633 in __call__ >> return self.application(environ, start_response) >> File '/inside/depot4/galaxy/lib/galaxy/web/framework/base.py', line 132 >> in __call__ >> return self.handle_request( environ, start_response ) >> File '/inside/depot4/galaxy/lib/galaxy/web/framework/base.py', line 190 >> in handle_request >> body = method( trans, **kwargs ) >> File '/inside/depot4/galaxy/lib/galaxy/web/framework/__init__.py', line >> 221 in decorator >> return func( self, trans, *args, **kwargs ) >> File '/inside/depot4/galaxy/lib/galaxy/web/base/controllers/admin.py', >> line 1053 in jobs >> for job in jobs: >> File >> '/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py', >> line 2341 in instances >> File >> '/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py', >> line 3204 in fetchall >> File >> '/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py', >> line 3171 in _fetchall_impl >> UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 404: >> ordinal not in range(128) >> >> > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ >
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
