The docker version is 1.1.2. I'm not entirely sure how to get the galaxy version, but hg sum resulted in parent: 14955:c25df9f3d31e tip Update tag latest_2014.10.06 for changeset ff6e36d7a238 branch: stable
So docker may be a bit too old. To get some more control over my OS, I did a fresh Ubuntu install on a separate laptop and got docker (1.41) and galaxy (downloaded stable release on Friday). Now I can get the docker image to run, but I STILL get two jobs in the history panel. One results in the concatenated files, the other has the text 'Work dir output' (and nothing else). Screenshot attached. I'll try to talk the sysadmins into upgrading docker and see if that fixes things. Thanks! -Jeltje On Fri, Jan 30, 2015 at 10:22 AM, John Chilton <jmchil...@gmail.com> wrote: > Based on the screenshot you are calling the program correctly. > > Can you tell me what version of Docker you are running and Galaxy you > are running? There have been a number of fixes in both programs over > the last six months to a year. Docker is known to consume a lot of > space in /var - it probably doesn't have much to do with Galaxy or > this tool right. It looks like you Docker command line does not > include -rm - which was added recently to have Docker dispose of > instances after it done with them. This might help with your /var > space issue - though in general Docker does use a lot a relatively > large amount of disk under /var - it is the nature of the technology > and many traditional HPC clusters for instance may allocate relatively > small /var partitions. > > Upgrading Galaxy might also help the permission thing - I don't see a > '-u <uid>' argument in your command-line - which is another newer > default to help ensure the resulting files have the correct > permissions (especially when sudo is disabled - which it seems it is > in your configuration). > > Hope this helps, > > -John > > > > On Mon, Jan 26, 2015 at 7:02 PM, Jeltje van Baren > <jeltje.van.ba...@gmail.com> wrote: > > Our sysadmin just notified me docker was taking up large amounts of > space in > > /var/lib/docker/devicemapper/devicemapper/data > > and > > /var/lib/docker/devicemapper/devicemapper/metadata > > > > I noticed the following DEBUG message in paster.log: > > > > galaxy.jobs.runners.local DEBUG 2015-01-23 13:30:19,503 (12) executing > job > > script: > > > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/galaxy_12.sh > > galaxy.jobs DEBUG 2015-01-23 13:30:19,552 (12) Persisting job > destination > > (destination id: docker_local) > > > > Are these related? (the files I'm trying to concatenate are small text > files > > so that's not it) > > > > > > On Mon, Jan 26, 2015 at 11:47 AM, Jeltje van Baren > > <jeltje.van.ba...@gmail.com> wrote: > >> > >> See attached for screenshot! > >> > >> Thanks for the links. > >> > >> > >> > >> On Mon, Jan 26, 2015 at 11:37 AM, John Chilton <jmchil...@gmail.com> > >> wrote: > >>> > >>> Can you send me a screenshot of the Tool form right before you hit > >>> submit. It is odd that two commands are executed barely differing at > >>> all. There were some recent UI changes and I want to make sure that > >>> you are passing two inputs once and not submitting one input twice in > >>> the newly named "batch mode"? > >>> > >>> container.sh is generated in command_factory.py using containers.py - > >>> here are some relevant links. > >>> > >>> > >>> > https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/jobs/command_factory.py?at=default > >>> > >>> > https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/containers.py?at=default > >>> > >>> > https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/docker_util.py?at=default > >>> > >>> -John > >>> > >>> > >>> On Mon, Jan 26, 2015 at 2:17 PM, Jeltje van Baren > >>> <jeltje.van.ba...@gmail.com> wrote: > >>> > I'm still trying to debug this. > >>> > > >>> > How do I change the working directory? I can't seem to find out how > all > >>> > those directory mounts get passed to docker - the catDocker.xml > appears > >>> > to > >>> > generate a 'cat' command that I assume ends up in > >>> > ()/job_working_directory/000/12/container.sh > >>> > but I can't find where that's happening - container.sh isn't listed > in > >>> > any > >>> > file under galaxy-dist. > >>> > job_conf.xml lists the directories that get passed as variables - > >>> > again, > >>> > when and where are those defined? > >>> > > >>> > Thanks, > >>> > > >>> > -Jeltje > >>> > > >>> > > >>> > On Fri, Jan 23, 2015 at 1:35 PM, Jeltje van Baren > >>> > <jeltje.van.ba...@gmail.com> wrote: > >>> >> > >>> >> I tried the second solution (setting everything to rw) and while I > can > >>> >> confirm that the command is changed accordingly, the results are the > >>> >> same: > >>> >> Two empty output files and the same error message. > >>> >> > >>> >> Command from paster.log: > >>> >> docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v > >>> >> > >>> >> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:rw > >>> >> -v > >>> >> > >>> >> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:rw > >>> >> -v > >>> >> > >>> >> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:rw > >>> >> -v > >>> >> > >>> >> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw > >>> >> -w > >>> >> > >>> >> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12 > >>> >> --net none busybox:ubuntu-14.04 > >>> >> > >>> >> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/container.sh; > >>> >> return_code=$?; if [ -f > >>> >> > >>> >> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file > >>> >> ] ; then cp > >>> >> > >>> >> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file > >>> >> > >>> >> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_19.dat > >>> >> ; fi; sh -c "exit $return_code" > >>> >> > >>> >> -Jeltje > >>> >> > >>> >> On Fri, Jan 23, 2015 at 12:24 PM, John Chilton <jmchil...@gmail.com > > > >>> >> wrote: > >>> >>> > >>> >>> My first thought is the bug mentioned here - > >>> >>> > >>> >>> > >>> >>> > https://lists.galaxyproject.org/pipermail/galaxy-dev/2014-November/020892.html > >>> >>> along with potential work arounds. Are you able to confirm that > this > >>> >>> is or is not the problem at all? > >>> >>> > >>> >>> -John > >>> >>> > >>> >>> On Fri, Jan 23, 2015 at 2:37 PM, Jeltje van Baren > >>> >>> <jeltje.van.ba...@gmail.com> wrote: > >>> >>> > Hi, > >>> >>> > > >>> >>> > I'm following instructions at > >>> >>> > https://github.com/apetkau/galaxy-hackathon-2014. > >>> >>> > > >>> >>> > When I try to run the concatenate-datasets on two input files, > two > >>> >>> > odd > >>> >>> > things happen. First, TWO nearly identical commands are generated > >>> >>> > in > >>> >>> > the > >>> >>> > History panel, only differing in their output filename. In > >>> >>> > paster.log, > >>> >>> > only > >>> >>> > the second one shows up. Second, the program fails, with this > info > >>> >>> > in > >>> >>> > the > >>> >>> > History: > >>> >>> > > >>> >>> > An error occurred with this dataset: > >>> >>> > Galaxy slots passed through contain as 1 > >>> >>> > > >>> >>> > > >>> >>> > > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh: > >>> >>> > line 2: can't create > >>> >>> > > >>> >>> > > >>> >>> > > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_14.dat > >>> >>> > > >>> >>> > This happens for both commands in the History panel, only the > >>> >>> > output > >>> >>> > filename in the other error is dataset_15.dat > >>> >>> > > >>> >>> > Oddly, the dataset_14.dat and dataset_15.dat are both created > >>> >>> > during > >>> >>> > this > >>> >>> > command, they just end up empty. > >>> >>> > > >>> >>> > Paster.log: > >>> >>> > > >>> >>> > galaxy.jobs.runners DEBUG 2015-01-23 11:08:35,973 (10) command > is: > >>> >>> > docker > >>> >>> > inspect busybox:ubuntu-14.04 > /dev/null 2>&1 > >>> >>> > [ $? -ne 0 ] && docker pull busybox:ubuntu-14.04 > /dev/null 2>&1 > >>> >>> > > >>> >>> > docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v > >>> >>> > > >>> >>> > > >>> >>> > > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:ro > >>> >>> > -v > >>> >>> > > >>> >>> > > >>> >>> > > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:ro > >>> >>> > -v > >>> >>> > > >>> >>> > > >>> >>> > > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:rw > >>> >>> > -v > >>> >>> > > >>> >>> > > >>> >>> > > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw > >>> >>> > -w > >>> >>> > > >>> >>> > > >>> >>> > > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10 > >>> >>> > --net none busybox:ubuntu-14.04 > >>> >>> > > >>> >>> > > >>> >>> > > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh; > >>> >>> > return_code=$?; if [ -f > >>> >>> > > >>> >>> > > >>> >>> > > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file > >>> >>> > ] ; then cp > >>> >>> > > >>> >>> > > >>> >>> > > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file > >>> >>> > > >>> >>> > > >>> >>> > > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_15.dat > >>> >>> > ; fi; sh -c "exit $return_code" > >>> >>> > galaxy.jobs.runners.local DEBUG 2015-01-23 11:08:36,040 (10) > >>> >>> > executing > >>> >>> > job > >>> >>> > script: > >>> >>> > > >>> >>> > > >>> >>> > > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/galaxy_10.sh > >>> >>> > galaxy.jobs DEBUG 2015-01-23 11:08:36,091 (10) Persisting job > >>> >>> > destination > >>> >>> > (destination id: docker_local) > >>> >>> > > >>> >>> > > >>> >>> > What could be happening here, and how do I fix it? > >>> >>> > > >>> >>> > Thanks, > >>> >>> > > >>> >>> > -Jeltje > >>> >>> > > >>> >>> > > >>> >>> > > >>> >>> > ___________________________________________________________ > >>> >>> > Please keep all replies on the list by using "reply all" > >>> >>> > in your mail client. To manage your subscriptions to this > >>> >>> > and other Galaxy lists, please use the interface at: > >>> >>> > https://lists.galaxyproject.org/ > >>> >>> > > >>> >>> > To search Galaxy mailing lists use the unified search at: > >>> >>> > http://galaxyproject.org/search/mailinglists/ > >>> >> > >>> >> > >>> > > >> > >> > > >
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
