[galaxy-user] Flagstat crashes in non-linear workflows with TORQUE
Hello everyone
I observed an issue when flagstat is incorporated in a workflow in which
the BAM file it works on is also used by another program (generate
pileup for instance) and is NOT the input dataset (generated by sam to
bam within the workflow).
I tested it with the local job runner and with TORQUE (with the pbs
scheduler and Maui).
- With the local job runner, it works just fine
- With TORQUE I get the following error message:
pbs_submit failed, PBS error 15033: No free connections
Surprisingly, other non-linear workflows work fine. I only observed this
error with flagstat. Moreover, when flagstat is in a linear workflow, it
works fine too. Ad if it is non-linear but the input dataset is the bam
file flagstat works on, it works fine too.
Please find attached one of the test workflow where I found the error.
The input dataset is a sam file.
Any clue?
Cheers,
LA
{
"a_galaxy_workflow": "true",
"annotation": "to see if it fails if not forked",
"format-version": "0.1",
"name": "test flagstat",
"steps": {
"0": {
"annotation": "",
"id": 0,
"input_connections": {},
"inputs": [
{
"description": "",
"name": "Input Dataset"
}
],
"name": "Input dataset",
"outputs": [],
"position": {
"left": 200,
"top": 200
},
"tool_errors": null,
"tool_id": null,
"tool_state": "{\"name\": \"Input Dataset\"}",
"tool_version": null,
"type": "data_input",
"user_outputs": []
},
"1": {
"annotation": "",
"id": 1,
"input_connections": {
"source|input1": {
"id": 0,
"output_name": "output"
}
},
"inputs": [],
"name": "SAM-to-BAM",
"outputs": [
{
"name": "output1",
"type": "bam"
}
],
"position": {
"left": 274.5,
"top": 307
},
"tool_errors": null,
"tool_id": "sam_to_bam",
"tool_state": "{\"source\": \"{\\\"index_source\\\":
\\\"cached\\\", \\\"input1\\\": null, \\\"__current_case__\\\": 0}\",
\"__page__\": 0}",
"tool_version": "1.1.1",
"type": "tool",
"user_outputs": []
},
"2": {
"annotation": "",
"id": 2,
"input_connections": {
"input1": {
"id": 1,
"output_name": "output1"
}
},
"inputs": [],
"name": "flagstat",
"outputs": [
{
"name": "output1",
"type": "txt"
}
],
"position": {
"left": 396.5,
"top": 445
},
"tool_errors": null,
"tool_id": "samtools_flagstat",
"tool_state": "{\"__page__\": 0, \"input1\": \"null\"}",
"tool_version": "1.0.0",
"type": "tool",
"user_outputs": []
},
"3": {
"annotation": "",
"id": 3,
"input_connections": {
"refOrHistory|input1": {
"id": 1,
"output_name": "output1"
}
},
"inputs": [],
"name": "Generate pileup",
"outputs": [
{
"name": "output1",
"type": "tabular"
}
],
"position": {
"left": 519,
"top": 340
},
"tool_errors": null,
"tool_id": "sam_pileup",
"tool_state": "{\"__page__\": 0, \"c\": \"{\\\"consensus\\\":
\\\"no\\\", \\\"__current_case__\\\": 0}\", \"indels\": \"\\\"no\\\"\",
\"refOrHistory\": \"{\\\"input1\\\": null, \\\"reference\\\": \\\"indexed\\\",
\\\"__current_case__\\\": 0}\", \"lastCol\": \"\\\"no\\\"\", \"mapCap\":
\"\\\"60\\\"\"}",
"tool_version": "1.1.1",
"type": "tool",
"user_outputs": []
}
}
}___
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org. Please keep all replies on the list by
using "reply all" in your mail client. For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy li
[galaxy-user] suggestion for multithreading
Hello everyone,
I'm using TORQUE with Galaxy, and we noticed that if a tool is
multithreaded, the number of needed cores is not communicated to pbs,
leading to job crashes if the required resources are not available when
the job is submitted.
Therefore I modified a little the code as follows in
lib/galaxy/jobs/runners/pbs.py
256 # define PBS job options
257 attrs.append( dict( name = pbs.ATTR_N, value = str( "%s_%s_%
s" % ( job_wrapper.job_id, job_wrapper.tool.id, job_wrapper.user ) ) ) )
258 mt_file = open('tool-data/multithreading.csv', 'r')
259 for l in mt_file:
260 l = string.split(l)
261 if ( l[0] == job_wrapper.tool.id ):
262 attrs.append( dict( name = pbs.ATTR_l,
resource = 'nodes', value = '1:ppn='+str(l[1]) ) )
263 attrs.append( dict( name = pbs.ATTR_l,
resource = 'mem', value = str(l[2]) ) )
264 break
265 mt_file.close()
266 job_attrs = pbs.new_attropl( len( attrs ) +
len( pbs_options ) )
(sorry it didn't come out very well due to line breaking)
The csv file contains a list of the multithreaded tools, each line
containing:
\t\t\n
And it works fine, the jobs wait for their turn properly, but
information is duplicated. Perhaps there would be a way to include
something similar in galaxy's original code (if it is not already the
case, I may not be up-to-date) without duplicating data.
I hope that helps :)
Best regards,
L-A
___
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org. Please keep all replies on the list by
using "reply all" in your mail client. For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists,
please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-user] suggestion for multithreading
Just one little fix on line 261:
261 if ( len(l) > 1 and l[0] == job_wrapper.tool.id ):
Otherwise it pathetically crashes when non-multithreaded jobs are
submitted. Sorry about that.
Regards,
L-A
Le mardi 19 avril 2011 à 14:33 +0200, Louise-Amélie Schmitt a écrit :
> Hello everyone,
>
> I'm using TORQUE with Galaxy, and we noticed that if a tool is
> multithreaded, the number of needed cores is not communicated to pbs,
> leading to job crashes if the required resources are not available when
> the job is submitted.
>
> Therefore I modified a little the code as follows in
> lib/galaxy/jobs/runners/pbs.py
>
> 256 # define PBS job options
> 257 attrs.append( dict( name = pbs.ATTR_N, value = str( "%s_%s_%
> s" % ( job_wrapper.job_id, job_wrapper.tool.id, job_wrapper.user ) ) ) )
> 258 mt_file = open('tool-data/multithreading.csv', 'r')
> 259 for l in mt_file:
> 260 l = string.split(l)
> 261 if ( l[0] == job_wrapper.tool.id ):
> 262 attrs.append( dict( name = pbs.ATTR_l,
> resource = 'nodes', value = '1:ppn='+str(l[1]) ) )
> 263 attrs.append( dict( name = pbs.ATTR_l,
> resource = 'mem', value = str(l[2]) ) )
> 264 break
> 265 mt_file.close()
> 266 job_attrs = pbs.new_attropl( len( attrs ) +
> len( pbs_options ) )
>
> (sorry it didn't come out very well due to line breaking)
>
> The csv file contains a list of the multithreaded tools, each line
> containing:
> \t\t\n
>
> And it works fine, the jobs wait for their turn properly, but
> information is duplicated. Perhaps there would be a way to include
> something similar in galaxy's original code (if it is not already the
> case, I may not be up-to-date) without duplicating data.
>
> I hope that helps :)
>
> Best regards,
> L-A
>
> ___
> The Galaxy User list should be used for the discussion of
> Galaxy analysis and other features on the public server
> at usegalaxy.org. Please keep all replies on the list by
> using "reply all" in your mail client. For discussion of
> local Galaxy instances and the Galaxy source code, please
> use the Galaxy Development list:
>
> http://lists.bx.psu.edu/listinfo/galaxy-dev
>
> To manage your subscriptions to this and other Galaxy lists,
> please use the interface at:
>
> http://lists.bx.psu.edu/
___
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org. Please keep all replies on the list by
using "reply all" in your mail client. For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists,
please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-user] suggestion for multithreading
On Thu, 2 Jun 2011 13:31:04 -0400, Nate Coraor wrote: > Peter Cock wrote: >> On Thu, Jun 2, 2011 at 6:23 PM, Nate Coraor wrote: >> > Peter Cock wrote: >> >> On a related point, I've previously suggested a $variable could be >> >> defined >> >> for use in tool XML wrappers to set the number of threads. This number >> >> could come from a general configuration file, or perhaps via the >> >> cluster >> >> settings - the point is the tool doesn't need to know, it just gets >> >> told how >> >> many threads it is allowed. >> > >> > Yeah, that's the idea. >> > >> > job_conf.xml: >> > >> > >> > >> > >> > >> > >> > batch >> > >> > >> > >> > >> > >> > 8 >> > >> > >> > >> > >> > pbs.py then knows to translate '8' to >> > '-l nodes=1:ppn=8'. >> > >> > Your tool can access that value a bunch, like .cores. >> > >> > The same should be possible for other consumables. >> > >> >> Sounds good. Would there be a global default setting somewhere in >> universe.ini or elsewhere for when job_conf.xml didn't set a value? > > default_cluster_job_runner will remain for backwards compatibility, but > we'll ship a sample job_conf.xml that runs everything locally by > default. > > --nate Haha, and I did that before realizing I could do just what I needed by writing tool-specific pbs:// URLs at the end of the config file... I'm such an idiot. But I really like what you did of it and I have a couple of questions. Concerning the single-threaded tools, what would happen if the number of threads set in the xml file was >1 ? Could it be possible to forbid a tool to run on a given node? Thanks, L-A > >> >> Peter >> ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] suggestion for multithreading
Le 02/06/2011 21:39, Nate Coraor a écrit : Louise-Amélie Schmitt wrote: default_cluster_job_runner will remain for backwards compatibility, but we'll ship a sample job_conf.xml that runs everything locally by default. --nate Haha, and I did that before realizing I could do just what I needed by writing tool-specific pbs:// URLs at the end of the config file... I'm such an idiot. Haha, okay, I don't think i even noticed since I was distracted by your implementation being a step in the way we want to go with it. damn, I shouldn't have said anything :D But I really like what you did of it and I have a couple of questions. Concerning the single-threaded tools, what would happen if the number of threads set in the xml file was>1 ? It'd consume extra slots, but the tool itself would just run as usual. So what about an attribute in the tool tag that would notify wether the tool is actually multithreaded, so that this doesn't happen? Something like multithreaded="true/false" ? Could it be possible to forbid a tool to run on a given node? Hrm. In PBS you could do it using node properties/neednodes or resource requirements. I'd have to think a bit about how to do this in a more general way in the XML. --nate Ok thank you! :) L-A Thanks, L-A Peter ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] Flagstat crashes in non-linear workflows with TORQUE
Le 06/04/2011 13:57, Nate Coraor a écrit :
Louise-Amélie Schmitt wrote:
Hello everyone
I observed an issue when flagstat is incorporated in a workflow in which
the BAM file it works on is also used by another program (generate
pileup for instance) and is NOT the input dataset (generated by sam to
bam within the workflow).
I tested it with the local job runner and with TORQUE (with the pbs
scheduler and Maui).
- With the local job runner, it works just fine
- With TORQUE I get the following error message:
pbs_submit failed, PBS error 15033: No free connections
Hi,
This can most likely be fixed by increasing the value of NCONNECTS in
the TORQUE source, in src/include/libpbs.h, and recompiling on your
TORQUE server. I haven't seen a problem after increasing the value to
20.
--nate
Hi,
Sorry this is old but I tried recompiling torque after setting the
NCONNECTS to 20 and the issue's still there.
But there's more: It doesn't affect only flagstat but other non-linear
workflows. One of the two jobs that are submitted when their "father"
stopped running triggers the same error:
PBS error 15033: No free connections
And the best part is that it works fine sometimes. But when it crashes,
it's always the same job that crashes.
Does anyone have a clue?
Cheers,
L-A
Surprisingly, other non-linear workflows work fine. I only observed this
error with flagstat. Moreover, when flagstat is in a linear workflow, it
works fine too. Ad if it is non-linear but the input dataset is the bam
file flagstat works on, it works fine too.
Please find attached one of the test workflow where I found the error.
The input dataset is a sam file.
Any clue?
Cheers,
LA
{
"a_galaxy_workflow": "true",
"annotation": "to see if it fails if not forked",
"format-version": "0.1",
"name": "test flagstat",
"steps": {
"0": {
"annotation": "",
"id": 0,
"input_connections": {},
"inputs": [
{
"description": "",
"name": "Input Dataset"
}
],
"name": "Input dataset",
"outputs": [],
"position": {
"left": 200,
"top": 200
},
"tool_errors": null,
"tool_id": null,
"tool_state": "{\"name\": \"Input Dataset\"}",
"tool_version": null,
"type": "data_input",
"user_outputs": []
},
"1": {
"annotation": "",
"id": 1,
"input_connections": {
"source|input1": {
"id": 0,
"output_name": "output"
}
},
"inputs": [],
"name": "SAM-to-BAM",
"outputs": [
{
"name": "output1",
"type": "bam"
}
],
"position": {
"left": 274.5,
"top": 307
},
"tool_errors": null,
"tool_id": "sam_to_bam",
"tool_state": "{\"source\": \"{\\\"index_source\\\": \\\"cached\\\", \\\"input1\\\": null,
\\\"__current_case__\\\": 0}\", \"__page__\": 0}",
"tool_version": "1.1.1",
"type": "tool",
"user_outputs": []
},
"2": {
"annotation": "",
"id": 2,
"input_connections": {
"input1": {
"id": 1,
"output_name": "output1"
}
},
"inputs": [],
"name": "flagstat",
"outputs": [
{
"name": "output1",
"type": "txt"
}
],
"position": {
"left": 396.5,
"top": 445
},
"tool_errors&
Re: [galaxy-user] Most Popular Linux flavour for Galaxy
Hi, Our local instance runs on CentOS 5 and is happy with it :) We didn't have the choice though. Regards L-A Le 07/06/2011 14:00, Matloob Khushi a écrit : Hello Galaxy Users We are anticipating setting up a local instance of Galaxy and wondering what is the most popular (or best) flavour of Linux people out there using. I wonder which OS on UseGalaxy.org has been used. Thanks for your help. Regards Matloob PhD Candidate ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] New Galaxy User
Hi 1 -> You might want to read the bitbucket wiki instead: https://bitbucket.org/galaxy/galaxy-central/wiki/Home There you can find all the information you will need to understand the configuration files, but don't expect a detailed description of the whole directory structure. The information is divided in topics. It's easier to understand that way anyway. 2 -> If you mean the prerequisites, you might just need an appropriate version of python. (since you already have mercurial) If you mean the software behind each tool, it has to be in the $PATH of the user running Galaxy. If you mean the eggs, Galaxy will do it for you. Hope that helps :) (hopefully I didn't forget anything, lack of sleep is a pain) Regards, L-A Le 16/06/2011 13:14, gaurav thareja a écrit : Hi All, I am a new user of Galaxy. I am trying to configure galaxy on my local system. I took clone of repository(https://bitbucket.org/galaxy/galaxy-dist). But I have couple of issues while understanding code base: 1. Is there any help file explaining directory structure and what all configuration files used for? 2. I got list of all dependencies from website, Do I have to install all tools in /usr/local/bin/ or can be installed at any custom location? I have already gone through tutorial (http://gmod.org/wiki/Galaxy_Tutorial) which was quite helpful and interesting but doesn't quite talk about my doubts above. Regards Gaurav Thareja -- Regards Gaurav Thareja ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-user] How to delete multiple items in libraries?
Hello everyone In the admin panel > Manage data libraries section I have an issue with deleting datasets. When I click on each dataset's triangle menu and ask for deletion, everything is fine. But whenever I select datasets and ask for deletion with the drop down menu at the bottom of the page, Galaxy tells me I don't have the rights to delete them. I get the following message: "You are not authorized to delete any of the selected datasets." Did I miss anything? Thanks, L-A ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] How to delete multiple items in libraries?
Ok no problem :) Thanks a lot L-A Le 20/06/2011 17:50, Greg Von Kuster a écrit : Hello Louise-Amelie, This behavior was corrected in change set 5612:3297ec86d657, which should be available in the distribution within the next week or so. Sorry for the inconvenience. Greg Von Kuster On Jun 20, 2011, at 11:06 AM, Louise-Amélie Schmitt wrote: Hello everyone In the admin panel > Manage data libraries section I have an issue with deleting datasets. When I click on each dataset's triangle menu and ask for deletion, everything is fine. But whenever I select datasets and ask for deletion with the drop down menu at the bottom of the page, Galaxy tells me I don't have the rights to delete them. I get the following message: "You are not authorized to delete any of the selected datasets." Did I miss anything? Thanks, L-A ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org <http://usegalaxy.org>. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ Greg Von Kuster Galaxy Development Team g...@bx.psu.edu <mailto:g...@bx.psu.edu> ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-user] BAM files loading and mandatory grooming
Hello everyone Whenever we try to load BAM files without copying them, we get an error stating the files need grooming, and can't use them at all. Is it this serious? Would there be a way to bypass that? Thanks! L-A ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] BAM files loading and mandatory grooming
Hello Jennifer, and thanks for your answer Would you be able to help with a bit more detail and testing? #2 sounds like it may be the issue, but without knowing more right now, I'll provided the next troubleshooting steps. Yeah, I realise I was not very clear, sorry about that. 1 - this is in your own install and it is current with the latest -dist or are you using -central (which pull?). Did something change between the last successful BAM load and the new problem? This is indeed a local install (two of them actually). $ hg head changeset: 5585:8c11dd28a3cf tag: tip user:Nate Coraor date:Thu May 19 10:07:53 2011 -0400 summary: Add Picard and fastqc tools to Main I can't remember whether I ever could successfully load a bam file... 2 - "without copying" them means maybe that you are using a symbolic link at your own site? This "load" is into a Library (this is how to move data transferred outside of the Galaxy mechanisms into a history, i.e. around a local file system by direct copy or similar). Load into a Library first, then copy to history. Instruction in wiki here: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files No, it's just that when I add new datasets to a library from filesystem paths I choose the 'Link files without copying into Galaxy' option. 3 - or do you mean "without copying" by using the FTP method set up locally, followed by a load into a history? 4 - BAM file metadata (pencil icon -> Edit options) looks OK? Or, this is the problem? It looks ok. Well I guess. When I click on the file in the library, in the 'Miscellaneous information' section I have: "The uploaded files need grooming, so change your Copy data into Galaxy? selection to be Copy files into Galaxy instead of Link to files without copying into Galaxy so grooming can be performed." 5 - SAMTools is loaded into your instance? Tools function on other BAM files or all? Some simple SAMTools commands (that work with BAM files) function line command OK on these (to rule out problem with BAM files themselves - missing .bai index could be a problem). Samtools works like charm. I've been using it through these Galaxy instances for a long time now. 6 - if you start with the same data and convert SAM->BAM within Galaxy, does the SAM load and is the resulting BAM file OK? I can't run anything on them, since they're tagged "error" no tool want them as an input. If I could, I would just ignore the issue. 7 - If you load the BAM file at the public Galaxy web site (using FTP), the load is successful and the BAM file once imported into a history appears OK? Please share link from this test in case we need to examine. (Options -> Share or Publish, generate link, email to me and I can share with dev team if needed). If you mean when the data is actually copied in Galaxy, yes everything works fine since the grooming is perform during the upload process. I never tried the public instance though. If you need me to share one of the files with you there, I'll have to ask for permission first, since the data's not mine. Thanks for providing more info or perhaps you will find that one of these uncovers the issue, Thanks for your help :) Regards, L-A Best, Jen Galaxy team On 7/18/11 2:05 AM, Louise-Amélie Schmitt wrote: Hello everyone Whenever we try to load BAM files without copying them, we get an error stating the files need grooming, and can't use them at all. Is it this serious? Would there be a way to bypass that? Thanks! L-A ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] BAM files loading and mandatory grooming
Hello Jennifer, Aaaah so the grooming means creating the .bai file? Ok, I didn't know, thanks for the information. But that leaves me wondering why it is necessary to copy the data (unless the alignments are not sorted yet, which is possible knowing where the data comes from). I won't have much time left to look into it but if I can I'll let you know. Thanks, L-A Le 26/07/2011 19:50, Jennifer Jackson a écrit : Hello L-A, The load is necessary to create/link the index .bam index. No work-around is available, however if you develop one, we would be glad to consider it as an addition. If there are any updates on our side, we will send more information. Thanks! Jen Galaxy team On 7/18/11 2:05 AM, Louise-Amélie Schmitt wrote: Hello everyone Whenever we try to load BAM files without copying them, we get an error stating the files need grooming, and can't use them at all. Is it this serious? Would there be a way to bypass that? Thanks! L-A ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] BAM files loading and mandatory grooming
Le 27/07/2011 10:17, Peter Cock a écrit : 2011/7/27 Louise-Amélie Schmitt: Hello Jennifer, Aaaah so the grooming means creating the .bai file? Ok, I didn't know, thanks for the information. But that leaves me wondering why it is necessary to copy the data (unless the alignments are not sorted yet, which is possible knowing where the data comes from). I won't have much time left to look into it but if I can I'll let you know. Thanks, L-A Sadly samtools sort doesn't set the header information, so it is not enough to look at that to say if and how a SAM/BAM file is sorted. I think what the Galaxy code does now is attempt to index the BAM file (as a safe way to find out if is really is suitably sorted), and if that fails sorts it by co-ordinate (which will make a new BAM file) and then indexes the sorted version. Peter Ah ok, I understand! It should be fine though, since we won't need to load bam files anymore shortly. Thanks for all the information! Best, L-A ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-user] Galaxy logo: licence? (blog article project)
Hello, I want to write an article on Galaxy on our bioinformatics blog ( http://bioinfo-fr.net/), so I would like to know what materials I can use and under which conditions. What is the licence of the pictures (logo etc.)? Can I use them? Is there anything particular you want me to mention? Thanks, L-A ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] Galaxy logo: licence? (blog article project)
Hello Jen, That helped a lot, thank you! I didn't know the license also applied to the images. Best, L-A Le 11/04/2012 15:45, Jennifer Jackson a écrit : Hello L-A, Galaxy is open-source for all components, the license is here: http://bitbucket.org/galaxy/galaxy-central/src/3f12abbd2394/LICENSE.txt And in case you haven't seen this wiki, section 3 lists some preferred link URLs: http://wiki.g2.bx.psu.edu/Citing%20Galaxy We'll send more comments as the team has a chance to discuss. Please let us know if we can help with anything else. Thanks! Jen Galaxy team On 4/11/12 2:10 AM, Louise-Amélie Schmitt wrote: Hello, I want to write an article on Galaxy on our bioinformatics blog ( http://bioinfo-fr.net/), so I would like to know what materials I can use and under which conditions. What is the licence of the pictures (logo etc.)? Can I use them? Is there anything particular you want me to mention? Thanks, L-A ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
