Bjoern, The following patch should put out the fire...
https://bitbucket.org/galaxy/galaxy-central/commits/d781366bc120787e201b73a4dd99b56282169d86 TL;DR - After this patch $output.extra_files_path is the same thing as $output.files_path - which is to say not broken in a bunch of corner cases and slightly less efficient in the most vanilla Galaxy deployments. $input.extra_files_path however is a bit tricky if the path doesn't exist (probably not the case with your deployment Bjoern) - for disk object stores it just returns the non-existent path - for the nested object store it throws an error and the tool won't run. I had a patch to synchronize the behavior (https://gist.github.com/jmchilton/d05bcf02874762cdb829) - but I cannot say for sure the current behavior isn't intentional (Dannon, Nate - want to weigh in on this?). -John On Thu, Jun 19, 2014 at 5:16 PM, Björn Grüning <bjoern.gruen...@gmail.com> wrote: > Nate, John, > > that bug bites me with so many tools. Next one is MACS2 > > > Traceback (most recent call last): > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", > line 153, in prepare_job > > job_wrapper.prepare() > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line > 712, in prepare > self.command_line, self.extra_filenames = tool_evaluator.build() > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line > 348, in build > self.__build_command_line( ) > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line > 364, in __build_command_line > command_line = fill_template( command, context=param_dict ) > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/template.py", line 9, > in fill_template > return str( Template( source=template_text, searchList=[context] ) ) > File > "/usr/local/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs2.egg/Cheetah/Template.py", > line 1004, in __str__ > return getattr(self, mainMethName)() > File "cheetah_DynamicallyCompiledCheetahTemplate_1403081606_9_13081.py", > line 253, in respond > > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/wrappers.py", line > 203, in __getattr__ > return getattr( self.dataset, key ) > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line > 1493, in extra_files_path > return self.dataset.extra_files_path > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line > 1349, in extra_files_path > return self.object_store.get_filename( self, dir_only=True, > extra_dir=self._extra_files_path or "dataset_%d_files" % self.id ) > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", > line 416, in get_filename > return self.__call_method('get_filename', obj, ObjectNotFound, True, > **kwargs) > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", > line 436, in __call_method > % ( method, str( obj ), str( kwargs ) ) ) > ObjectNotFound: objectstore, __call_method failed: get_filename on > <galaxy.model.Dataset object at 0x7eff89510310>, kwargs: {'dir_only': True, > 'extra_dir': 'dataset_118783_files'} > > Is there anything I can do? It is affecting our productive server right now. > Any quick fix would be very much appreciated. The object store is not a beta > feature or? I'm wondering who is using it at the moment? > > Cheers, > Bjoern > > > Am 16.06.2014 16:04, schrieb John Chilton: > >> I can confirm this is error. Switching the tool to use >> $outfile.files_path instead of $outfile.extra_files_path improves the >> situation in my testing. >> >> I thought for sure .extra_files_path had been deprecated in terms of >> files_path - but the only Trello card I could find indicates the >> opposite may be true - https://trello.com/c/ar9zF40y? I think that for >> some more exotic features such as parameter rewriting by setting >> outputs_to_working_directory (required for running such tools are real >> users I suspect) and the LWR, usage of files_path in tool wrappers >> will be reworked for the job and extra_files_path will not be. >> >> Hopefully someone with a longer history on the project will comment on >> usage of .files_path versus .extra_files_path. >> >> -John >> >> >> >> On Fri, Jun 13, 2014 at 7:43 AM, bjoern.gruen...@googlemail.com >> <bjoern.gruen...@gmail.com> wrote: >>> >>> Hi Nate, >>> >>> we encountered an addition bug related to the object-store: >>> >>> Traceback (most recent call last): >>> File "/usr/local/galaxy/galaxy- >>> dist/lib/galaxy/jobs/runners/__init__.py", line 153, in prepare_job >>> job_wrapper.prepare() >>> File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line >>> 712, in prepare >>> self.command_line, self.extra_filenames = tool_evaluator.build() >>> File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", >>> line >>> 348, in build >>> self.__build_command_line( ) >>> File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", >>> line >>> 364, in __build_command_line >>> command_line = fill_template( command, context=param_dict ) >>> File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/template.py", line >>> 9, >>> in fill_template >>> return str( Template( source=template_text, searchList=[context] ) ) >>> File >>> >>> "/usr/local/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs2.egg/Cheetah/Template.py", >>> line 1004, in __str__ >>> return getattr(self, mainMethName)() >>> File >>> "cheetah_DynamicallyCompiledCheetahTemplate_1402663002_65_42228.py", >>> line 94, in respond >>> File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/wrappers.py", >>> line >>> 203, in __getattr__ >>> return getattr( self.dataset, key ) >>> File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", >>> line >>> 1493, in extra_files_path >>> return self.dataset.extra_files_path >>> File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", >>> line >>> 1349, in extra_files_path >>> return self.object_store.get_filename( self, dir_only=True, >>> extra_dir=self._extra_files_path or "dataset_%d_files" % self.id ) >>> File >>> "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", >>> line 416, in get_filename >>> return self.__call_method('get_filename', obj, ObjectNotFound, True, >>> **kwargs) >>> File >>> "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", >>> line 436, in __call_method >>> % ( method, str( obj ), str( kwargs ) ) ) >>> ObjectNotFound: objectstore, __call_method failed: get_filename on >>> <galaxy.model.Dataset object at 0x7f7ee103fb90>, kwargs: {'dir_only': >>> True, >>> 'extra_dir': 'dataset_118453_files'} >>> >>> It should be reproducible with the latest Galaxy stable release and the >>> latest NCBI Blast+ wrappers if you are trying to generate a Blast+ >>> database. >>> >>> Thanks, >>> Bjoern >>> >>> >>> >>> 2014-06-12 16:50 GMT+02:00 Björn Grüning <bjoern.gruen...@gmail.com>: >>> >>>> Hi Nate, >>>> >>>> yes that is fixing it for me! >>>> >>>> Thanks! >>>> Bjoern >>>> >>>> Am 12.06.2014 16:26, schrieb Nate Coraor: >>>> >>>>> writeable (e.g. what you have new_files_path set to), that'd confirm >>>>> it. >>> >>> >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> http://lists.bx.psu.edu/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
