Thanks for the information. I believe I have seen a similar problem before
where we get a bunch of empty histories. We also use MySQL and MyISAM and
Separate web/handlers but are moving to InnoDB in mysql.
Regards,
Iyad Kandalaft
Microbial Biodiversity Bioinformatics
Agriculture and Agri-Food Canada | Agriculture et Agroalimentaire Canada
960 Carling Ave.| 960 Ave. Carling
Ottawa, ON| Ottawa (ON) K1A 0C6
E-mail Address / Adresse courriel iyad.kandal...@agr.gc.ca
Telephone | Téléphone 613-759-1228
Facsimile | Télécopieur 613-759-1701
Teletypewriter | Téléimprimeur 613-773-2600
Government of Canada | Gouvernement du Canada
From: galaxy-dev-boun...@lists.bx.psu.edu
[mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of John Chilton
Sent: Thursday, July 03, 2014 4:33 PM
To: Yves Gagnon
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Job execution order mixed-up
To close this loop - Ilya Sytchev was experiencing this problem as well and
did a bunch of tests at the recent GCC Hackathon that pretty clearly
demonstrated that this problem occurs when all of the following three
conditions hold - 1) Galaxy targets MySQL 2) MyISAM tables are used and 3)
Galaxy runs separate web and handler processes (even with 1 and 1).
Ilya's work has been documented here: https://trello.com/c/uVYR3IHc and he
kindly updated the wiki to reflect these problems. The Galaxy team already
strongly recommends Postgres over MySQL - but if you have to use MySQL please
use InnoDB tables (the new default) instead of MyISAM.
-John
On Sat, Dec 7, 2013 at 9:36 AM, John Chilton
chil...@msi.umn.edumailto:chil...@msi.umn.edu wrote:
Hello Yves,
duplicates the history the same amount of times as the number of
threadpool_workers we have configured for the job handlers
Well that shouldn't be happening. Do each of these multiple histories have all
of the datasets for the workflow run populated as well?
I assume then that Send results to a new history are checked when running the
workflow. Does this problem happen when that is not checked?
I have couple of related questions - but perhaps if you could send a version of
your universe_wsgi.ini to galaxy-b...@bx.psu.edumailto:galaxy-b...@bx.psu.edu
with sensitive information redacted (database passwords, id_secret,
admin_users) it would answer all of them.
It would be great to rule out tool related problems (though they are seeming
more and more unlikely as you describe the symptoms) - can you recreate this
with a big workflow that just uses standard Galaxy tools bundled with the
distribution? Eitherway, if you could send us a workflow example that
demonstrates the issue along with tool xml files for any custom tools that
would help. Again - you can send that to
galaxy-b...@bx.psu.edumailto:galaxy-b...@bx.psu.edu if it is sensitive.
Also, this workflow is being run through the GUI right - I want to rule out
this being an API related problem.
... as you can probably tell I am still flummoxed. Sorry I don't have answers
and thanks for your patience :(.
-John
On Wed, Dec 4, 2013 at 10:09 AM, Yves Gagnon
yves.gag...@dnalandmarks.camailto:yves.gag...@dnalandmarks.ca wrote:
Dear Galaxy mailing list,
I will be taking over this issue here now.
Here is an update on this issue and our observations. I would like to know
your insight on the matter if you can think of anything!
We tried to revert back to a Galaxy instance running only one job handler.
With that configuration, we observed that the out of order execution problem
still occured, but MUCH less frequently than when using three handlers. On
multiple workflow runs, only one job started while its input was not ready in
only one run of the workflow. When using three job handlers, it occured on all
workflow runs and on multiple jobs inside each workflow run.
Our poweruser also noticed that his workflow, when taking too much time to
prepare (which is always the case I think since it's a huge workflow),
duplicates the history the same amount of times as the number of
threadpool_workers we have configured for the job handlers. Now I am not sure
both are related at all since I do not know much what is the effect of running
a handler on multiple thread_workers.
In any case, as of now we are staying with a single handler configuration since
it in part fix the out of order execution problems, but since we have many
regular users and some powerusers, that is not the ideal solution.
Hope somebody can shed some light on this!
Cheers!
Yves Gagnon
LIMS/ELN Developer
Phone: +1 450 357-3370tel:%2B1%20450%20357-3370 Fax: +1 450
358-1154tel:%2B1%20450%20358-1154 E-Mail:
yves.gag...@dnalandmarks.camailto:yves.gag...@dnalandmarks.ca
Postal Address: DNA LandMarks Inc., 84 Rue Richelieu, Saint-Jean-Sur-Richelieu,
Quebec, CANADA, J3B 6X3
DNA LandMarks - une compagnie de BASF Plant Science / a BASF Plant Science
company
Confidentiality notice: The information contained in this e-mail is
