[galaxy-dev] Need help setting up galaxy with swift object store.

[Varun Mittal]

Hello everyone,
  I have configured swift object store at my infrastructure. Next i want to
configure galaxy to be able to use swift object store. I am having trouble
with configuration.

Here's what the error is:
galaxy.objectstore DEBUG 2013-09-26 08:07:05,306 Getting a connection
object for 'swift' object store
Traceback (most recent call last):
  File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py",
line 35, in app_factory
app = UniverseApplication( global_conf = global_conf, **kwargs )
  File "/home/galaxy/galaxy-dist/lib/galaxy/app.py", line 57, in __init__
self.object_store = build_object_store_from_config(self.config,
fsmon=True)
  File "/home/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line
1031, in build_object_store_from_config
return S3ObjectStore(config=config)
  File "/home/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line
387, in __init__
self.bucket = self._get_bucket(self.config.os_bucket_name)
  File "/home/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line
476, in _get_bucket
bucket = self.s3_conn.get_bucket(bucket_name)
  File
"/home/galaxy/galaxy-dist/eggs/boto-2.5.2-py2.6.egg/boto/s3/connection.py",
line 391, in get_bucket
bucket.get_all_keys(headers, maxkeys=0)
  File
"/home/galaxy/galaxy-dist/eggs/boto-2.5.2-py2.6.egg/boto/s3/bucket.py",
line 360, in get_all_keys
'', headers, **params)
  File
"/home/galaxy/galaxy-dist/eggs/boto-2.5.2-py2.6.egg/boto/s3/bucket.py",
line 317, in _get_all
query_args=s)
  File
"/home/galaxy/galaxy-dist/eggs/boto-2.5.2-py2.6.egg/boto/s3/connection.py",
line 460, in make_request
auth_path = self.calling_format.build_auth_path(bucket, key)
  File
"/home/galaxy/galaxy-dist/eggs/boto-2.5.2-py2.6.egg/boto/s3/connection.py",
line 92, in build_auth_path
path = '/' + bucket
TypeError: cannot concatenate 'str' and 'NoneType' objects


Config in universe_wsgi.ini is:
# Object store mode (valid options are: disk, s3, swift, distributed,
hierarchical)
object_store = swift
os_access_key = 'RocksCluster:galaxy-swift-user'
os_secret_key = password'
#os_bucket_name = 
# If using 'swift' object store, you must specify the following connection
properties
os_host = 192.168.100.104
os_port = 
os_is_secure = False
os_conn_path = /v2.0/
# Reduced redundancy can be used only with the 's3' object store
#os_use_reduced_redundancy = False
# Size (in GB) that the cache used by object store should be limited to.
# If the value is not specified, the cache size will be limited only by the
# file system size. The file system location of the cache is considered the
# configuration of the ``file_path`` directive defined above.
#object_store_cache_size = 100


-- 
Varun Mittal
varunmitta...@gmail.com
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Visualisation of VCF files in trackster

[Jeremy Goecks]

Thanks for reporting this issue and workaround Ulf. I've committed a fix that 
addresses this issue by only includeing the -split option for BED/GFF/GTF 
datasets and not VCF:

https://bitbucket.org/galaxy/galaxy-central/commits/b8d8f81bed872e58b4691643de8a08fa41662e71

Best,
J.

On Sep 12, 2013, at 6:45 AM, Ulf Schaefer wrote:

> Dear Jeremy
> 
> Thank you for your reply and sorry for not being clear. In short I 
> solved the problem. Below is some info, in case this is useful for 
> someone else.
> 
> Thanks for your help
> 
> The situation was:
> 
> On Main:
> Visualisation of the SAM/BAM file -> OK
> Visualisation of the VCF file -> OK
> 
> On my local install:
> Visualisation of the SAM/BAM file -> OK
> Visualisation of the VCF file -> FAIL
> 
> The reason is that this command fails:
> 
> grep -v '^#' /data/database/files/000/dataset_596.dat | sort -k1,1 | 
> bedtools genomecov -bg -split -i stdin -g 
> /data/database/files/000/dataset_598.dat > temp.bg ; bedGraphToBigWig 
> temp.bg /data/database/files/000/dataset_598.dat 
> /data/database/files/000/dataset_609.dat
> 
> with "Input error: found interval with block-counts not matching 
> starts/sizes"
> 
> Where dataset_596.dat is my vcf and 
> /data/database/files/000/dataset_598.dat is my genome file.
> 
> This is produced by the bedtools genomecov bit of the command, which 
> appears to have some sort of problem with the vcf input in combination 
> with the -split option. The problem disappears with the installation of 
> the latest version of bedtools (v2.17.0), but if you are using the 
> version that you get from yum (v2.15.0) you run into this error.
> 
> Ulf
> 
> **
> The information contained in the EMail and any attachments is confidential 
> and intended solely and for the attention and use of the named addressee(s). 
> It may not be disclosed to any other person without the express authority of 
> Public Health England, or the intended recipient, or both. If you are not the 
> intended recipient, you must not disclose, copy, distribute or retain this 
> message or any part of it. This footnote also confirms that this EMail has 
> been swept for computer viruses by Symantec.Cloud, but please re-sweep any 
> attachments before opening or saving. http://www.gov.uk/PHE
> **

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] pbkdf2 encryption - disabling does not work

[Martin Čech]

Ahoj Honzo,

Python is case-sensitive. Use 'use_pbkdf2 = False' and make sure you put it
under the '[app:main]' section in the config file.

Enjoy Galaxy!

best

Martin Cech


On Wed, Sep 25, 2013 at 9:15 AM, Jan Hapala  wrote:

> Hi,
>
> I need to run Galaxy with ProFTPd, therefore I need to disable the PBKDF2
> encryption of the passwords (as stated here
> http://dev.list.galaxyproject.org/disable-PBKDF2-revert-to-SHA1-td4661274.html
> ).
>
> However, when I do add this line:
> use_pbkdf2 = false
> to the universe_wsgi.ini, restart Galaxy, submit the form for the password
> change (or register as a new user) I get an error "_hashlib.HASH object has
> no attribute __getitem__".
>
> I got the same result with a version as of Jul 2 and the most recent tag
> (10411:c42567f43aa7, Aug 19).
>
> I would be much grateful for your suggestions!
>
> Regards,
> Jan
>
>
> URL:
> http://127.0.0.1:8080/user/edit_info?cntrller=user&user_id=f2db41e1fa331b3e
> File
> '.../eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py',
> line 364 in respond
>   app_iter = self.application(environ, detect_start_response)
> File '.../eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in
> __call__
>   return self.application(environ, start_response)
> File '.../eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633
> in __call__
>   return self.application(environ, start_response)
> File '.../lib/galaxy/web/framework/base.py', line 132 in __call__
>   return self.handle_request( environ, start_response )
> File '.../lib/galaxy/web/framework/base.py', line 190 in handle_request
>   body = method( trans, **kwargs )
> File '.../lib/galaxy/webapps/galaxy/controllers/user.py', line 842 in
> edit_info
>   trans.sa_session.flush()
> File
> '.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py',
> line 114 in do
> File
> '.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py',
> line 1718 in flush
> File
> '.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py',
> line 1789 in _flush
> File
> '.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py',
> line 331 in execute
> File
> '.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py',
> line 475 in execute
> File
> '.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py',
> line 59 in save_obj
> File
> '.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py',
> line 485 in _emit_update_statements
> File
> '.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py',
> line 1449 in execute
> File
> '.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py',
> line 1584 in _execute_clauseelement
> File
> '.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py',
> line 1651 in _execute_context
> File
> '.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py',
> line 1647 in _execute_context
> File
> '.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py',
> line 475 in _init_compiled
> File
> '.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/types.py',
> line 626 in process
> File '.../lib/galaxy/model/custom_types.py', line 125 in process_bind_param
>   value = value[0:self.impl.length]
> StatementError: '_hashlib.HASH' object has no attribute '__getitem__'
> (original cause: TypeError: '_hashlib.HASH' object has no attribute
> '__getitem__') 'UPDATE galaxy_user SET update_time=%(update_time)s,
> password=%(password)s WHERE galaxy_user.id = %(galaxy_user_id)s'
> [{u'galaxy_user_id': 1, 'password': }]
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version

[Curtis Hendrickson (Campus)]

Jeremy,

Thanks for letting us know so quickly that we're heading down a non-viable path.
As I understand it, changing the tool id of the cuff* tools would solve our 
problem locally, but make workflows created here non-interoperable with Galaxy 
Central.

We'll focus on moving to latest tool versions and educating uses on 
non-reproducibility issues/work-arounds - probably providing the older wrappers 
under a different tool_id, as your average user will want the latest one.

If we're blessed with some extra time, we'll tool-shed-ify it, but I can't 
commit to that at the moment.

Thanks again for the super quick responses.

Regards,
Curtis


From: Jeremy Goecks [mailto:jeremy.goe...@emory.edu]
Sent: Wednesday, September 25, 2013 10:43 AM
To: Curtis Hendrickson (Campus)
Cc: galaxy-dev@lists.bx.psu.edu; Shantanu Pavgi (Campus)
Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff 
version


Jeremy,

I'm sure that would get both tools to show up, but it seems to me that would 
miss the goal of past/future reproducibility.

My goal is to have to two versions of the program available at the same time -
Cuffdiff_wrapper 0.0.5 (cuffdiff/links 1.3) the historical one that all the 
current histories/workflows reference, so those don't break,
and
Cuffdiff_wrapper 0.0.6 (cuffdiff/links 2.1) the new version for use going 
forward, which has a distinct version so that those histories/workflows are 
reproducible
and the choice of which version to use is the users.
I want to avoid someone who wants to re-run last-month's cuffdiff with slightly 
different parameters being forced (w/o their knowledge) to run a more recent 
version of cuffdiff - which would then give an apples to oranges comparison.

With 2 versions of the same ID installed, I expected to either see two listings 
in the tool pane AND/OR a version pulldown on the cuffdiff tool params pane.

Is the only way to get two versions of the same tool id installed via toolshed?

Yes.

Unfortunately, it's not possible to manually install multiple versions of the 
same tool; the development efforts of the Galaxy team are focused on multiple 
version installation via only the tool shed for now. However, we'd welcome a 
pull request that adds this functionality to Galaxy.

Given that you want to provide backward compatibility, it's probably best to 
keep the tool_id for the older Cuffdiff wrapper and change the new one; 
mercurial merges should handle this change gracefully as it's small.

Best,
J.




Sorry - I feel like I've mis-understood the tool version/id resolution 
architecture. Happy to talk on the phone if that would be a better use of your 
time.

Thanks,
Curtis



From: Jeremy Goecks [mailto:jeremy.goe...@emory.edu]
Sent: Wednesday, September 25, 2013 8:43 AM
To: Curtis Hendrickson (Campus)
Cc: galaxy-dev@lists.bx.psu.edu; Shantanu 
Pavgi (Campus)
Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff 
version

The problem is likely a collision on the tool_id; update the tool ID for one of 
the wrappers and that should solve the problem.

Best,
J.

On Sep 24, 2013, at 5:30 PM, Curtis Hendrickson (Campus) wrote:



Jeremy,

We didn't have time to factor out to toolshed just now, so we took the 
following approach
1.   Take 1.3-compatible cuffdiff wrapper (0.0.5) and move it to 
cuffdiff_1.3_wrapper.py/xml - and add a dependency on cufflinks verson=1.3.0 
(and set that up).
2.   Apply your patch, changing current cuffdiff wrapper to (0.0.6)
3.   Add a line in tool_conf.xml  so that both your modern cuffdiff v6 
wrapper and our cuffdiff v5 wrappers are included.

When we restarted, and went to the GUI, there was only ONE cuffdiff tool 
listed, and when you clicked on it, it brought up the 0.0.5 version (the 2nd 
listed in tool_conf).
There was no version pull down to get the 0.0.6 version.,

Any idea what I've missed?

Regards,
Curtis


From: Jeremy Goecks [mailto:jeremy.goe...@emory.edu]
Sent: Thursday, September 19, 2013 3:17 PM
To: Curtis Hendrickson (Campus)
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff 
version

We can hack the old wrapper back in, using a hard-coded path to the old 
cuffdiff.

You could also use Managed Tool Dependencies for multiple versions of cuffdiff:

http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies




Shantanu suggests it might be a better world if we create a tool-shed 
repository for the cuffdiff 1.3-2.1 (wrapper 0.0.5) version, then install that.
Would that be a better path, or just add confusion?

This is on the right path. The right thing to do is migrate the Cuff* tool 
wrappers out of the galaxy-central/dist repositories; this is a bit more effort 
than simply creating the tool-shed repository for the wrappers.

There are two reasons I haven't done this:

(a) time;
(b) because the tools-and consequently the wrappers-are still in a

Re: [galaxy-dev] Want faster dataset import

[Dannon Baker]

Have you tried to use data libraries for this?  There's an import mechanism
there that'll allow you to simply link to the file on disk without
copy/upload.  I believe the "example_watch_folder.py" sample script (in the
distribution) does just this via the API, if you want an example.


On Mon, Sep 23, 2013 at 5:15 PM, Ted Goldstein  wrote:

> I want to frequently import many tens of thousands of datasets. The files
> are on the same sever as Galaxy. But the upload based  mechanism is really
> really slow. It takes hours to load this many files, yet the data is not
> moving at all!
>
> What is the best strategy to go about making a faster bulk import?  I can
> imagine a tight loop that  is
>
> My datatypes are limited.
>
> foreach folder
>new LibraryFolder
>
> foreach file in each directory
> new LibraryDataset
> new LibraryDatasetDatasetAssociation
>
> flush once at the end.
>
> Thoughts?
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Local Galaxy Install - Multiple Input Files for Workflow Missing?

[Dannon Baker]

Are you using "Input Dataset" steps in your workflows?  The multiple inputs
feature uses these to know how to distribute inputs -- other than that no
other configuration steps are necessary.


On Tue, Sep 24, 2013 at 2:43 PM, Adam Brenner  wrote:

> Howdy,
>
> On our local galaxy cluster, I have gotten a report that the multiple
> input files selection on workflows is missing. How do I enable it?
>
> Looking at the past mailing list[1] this is the item that is missing
> from our setup -- the tooltip icon. We are running a version of galaxy
> that is roughly ~one month old: c42567f43aa7.
>
> [1]:
> http://dev.list.galaxyproject.org/Looking-for-recommendations-How-to-run-galaxy-workflows-in-batch-td4362836.html#a4362874
>
> Thanks!
> -Adam
>
> --
> Adam Brenner
> Computer Science, Undergraduate Student
> Donald Bren School of Information and Computer Sciences
>
> Research Computing Support
> Office of Information Technology
> http://www.oit.uci.edu/rcs/
>
> University of California, Irvine
> www.ics.uci.edu/~aebrenne/
> aebre...@uci.edu
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] pgcleanup problem

[Dannon Baker]

Eric,

What version of posgresql are you using?  The script has a comment that
indicates you need 9.1+ (and I'm using 9.1.9), and it works out of the box
for me.

-Dannon


On Fri, Sep 20, 2013 at 7:35 AM, Eric Kuyt  wrote:

> Hi All,
>
> I'm trying to do some cleanup in my test environment (galaxy-dist)
>
> and pgcleanup.py ends with
>
> Traceback (most recent call last):
>>   File "./scripts/cleanup_datasets/pgcleanup.py", line 773, in 
>> cleanup = Cleanup()
>>   File "./scripts/cleanup_datasets/pgcleanup.py", line 55, in __init__
>> self.__connect_db()
>>   File "./scripts/cleanup_datasets/pgcleanup.py", line 114, in
>> __connect_db
>> self.conn = psycopg2.connect(**args)
>> TypeError: 'username' is an invalid keyword argument for this function
>
>
> Unless I add
>
> args['user'] = args['username']
> del args['username']
>
>
> Is this a bug or a config error?
>
> Eric
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version

[Jeremy Goecks]

> Jeremy,
>  
> I’m sure that would get both tools to show up, but it seems to me that would 
> miss the goal of past/future reproducibility.
>  
> My goal is to have to two versions of the program available at the same time –
> Cuffdiff_wrapper 0.0.5 (cuffdiff/links 1.3) the historical one that all the 
> current histories/workflows reference, so those don’t break,
> and
> Cuffdiff_wrapper 0.0.6 (cuffdiff/links 2.1) the new version for use going 
> forward, which has a distinct version so that those histories/workflows are 
> reproducible
> and the choice of which version to use is the users.
> I want to avoid someone who wants to re-run last-month’s cuffdiff with 
> slightly different parameters being forced (w/o their knowledge) to run a 
> more recent version of cuffdiff – which would then give an apples to oranges 
> comparison.
>  
> With 2 versions of the same ID installed, I expected to either see two 
> listings in the tool pane AND/OR a version pulldown on the cuffdiff tool 
> params pane.
>  
> Is the only way to get two versions of the same tool id installed via 
> toolshed?

Yes.

Unfortunately, it's not possible to manually install multiple versions of the 
same tool; the development efforts of the Galaxy team are focused on multiple 
version installation via only the tool shed for now. However, we'd welcome a 
pull request that adds this functionality to Galaxy.

Given that you want to provide backward compatibility, it's probably best to 
keep the tool_id for the older Cuffdiff wrapper and change the new one; 
mercurial merges should handle this change gracefully as it's small.

Best,
J.


>  
> Sorry – I feel like I’ve mis-understood the tool version/id resolution 
> architecture. Happy to talk on the phone if that would be a better use of 
> your time.
>  
> Thanks,
> Curtis
>  
>  
>  
> From: Jeremy Goecks [mailto:jeremy.goe...@emory.edu] 
> Sent: Wednesday, September 25, 2013 8:43 AM
> To: Curtis Hendrickson (Campus)
> Cc: galaxy-dev@lists.bx.psu.edu; Shantanu Pavgi (Campus)
> Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff 
> version
>  
> The problem is likely a collision on the tool_id; update the tool ID for one 
> of the wrappers and that should solve the problem.
>  
> Best,
> J.
>  
> On Sep 24, 2013, at 5:30 PM, Curtis Hendrickson (Campus) wrote:
> 
> 
> Jeremy,
>  
> We didn’t have time to factor out to toolshed just now, so we took the 
> following approach
> 1.   Take 1.3-compatible cuffdiff wrapper (0.0.5) and move it to 
> cuffdiff_1.3_wrapper.py/xml – and add a dependency on cufflinks verson=1.3.0 
> (and set that up).
> 2.   Apply your patch, changing current cuffdiff wrapper to (0.0.6)
> 3.   Add a line in tool_conf.xml  so that both your modern cuffdiff v6 
> wrapper and our cuffdiff v5 wrappers are included.
>  
> When we restarted, and went to the GUI, there was only ONE cuffdiff tool 
> listed, and when you clicked on it, it brought up the 0.0.5 version (the 2nd 
> listed in tool_conf).
> There was no version pull down to get the 0.0.6 version.,
>  
> Any idea what I’ve missed?
>  
> Regards,
> Curtis
>  
>  
> From: Jeremy Goecks [mailto:jeremy.goe...@emory.edu] 
> Sent: Thursday, September 19, 2013 3:17 PM
> To: Curtis Hendrickson (Campus)
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff 
> version
>  
> We can hack the old wrapper back in, using a hard-coded path to the old 
> cuffdiff.
>  
> You could also use Managed Tool Dependencies for multiple versions of 
> cuffdiff:
>  
> http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies
> 
> 
> 
> Shantanu suggests it might be a better world if we create a tool-shed 
> repository for the cuffdiff 1.3-2.1 (wrapper 0.0.5) version, then install 
> that.
> Would that be a better path, or just add confusion?
>  
> This is on the right path. The right thing to do is migrate the Cuff* tool 
> wrappers out of the galaxy-central/dist repositories; this is a bit more 
> effort than simply creating the tool-shed repository for the wrappers.
>  
> There are two reasons I haven't done this:
>  
> (a) time;
> (b) because the tools—and consequently the wrappers—are still in active 
> development, there needs to be a place for managing development of the 
> wrappers. Hence, the wrappers need a home on bitbucket or github, but we (the 
> Galaxy team) haven't determined where these wrappers should live.
>  
> (a) is a much more significant barrier than (b), so if a pull request 
> migrating Cuff* tools out of the distribution were made, I'm sure (b) could 
> be worked out.
>  
> Best,
> J.
>  
>  

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproje

Re: [galaxy-dev] Dependencies for complex datatypes

[Peter Cock]

On Tue, Sep 17, 2013 at 10:34 AM, Peter Cock  wrote:
> Hello all,
>
> Bjoern and I were talking about doing some work on more
> datatype definitions, but this raises the question about how
> to handle dependencies.
>
> First of all, complex datatypes in Galaxy are defined using
> Python code, so any Python dependency must be available
> to the main Galaxy process (unlike Python dependencies
> for jobs which are run on separate processes). For example,
> if I wrote GenBank/EMBL definitions using Biopython, could
> this dependency be handled via the Tool Shed?
>
> Second, the Python code for some datatypes may call a
> binary command line tool. For example, I was thinking of
> using the blastdbcmd binary within the BLAST database
> file format definitions to provide more useful peek
> information. Again, how could this dependency be
> handled via the Tool Shed?

Another real example: Right now I am working on wrapping
MIRA v4, and defining a custom datatype 'mira' for its own
assembly output format. I would like to handle conversion
to other formats like ACE, SAM, etc (as part of the datatype
definition) but this will mean a dependency on the miraconvert
binary.

I can instead provide a miraconvert wrapper as another
tool, but that doesn't give the full potential that using a full
datatype definition could. Is this the only sensible option
at the moment?

Regards,

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version

[Curtis Hendrickson (Campus)]

Jeremy,

I'm sure that would get both tools to show up, but it seems to me that would 
miss the goal of past/future reproducibility.

My goal is to have to two versions of the program available at the same time -
Cuffdiff_wrapper 0.0.5 (cuffdiff/links 1.3) the historical one that all the 
current histories/workflows reference, so those don't break,
and
Cuffdiff_wrapper 0.0.6 (cuffdiff/links 2.1) the new version for use going 
forward, which has a distinct version so that those histories/workflows are 
reproducible
and the choice of which version to use is the users.
I want to avoid someone who wants to re-run last-month's cuffdiff with slightly 
different parameters being forced (w/o their knowledge) to run a more recent 
version of cuffdiff - which would then give an apples to oranges comparison.

With 2 versions of the same ID installed, I expected to either see two listings 
in the tool pane AND/OR a version pulldown on the cuffdiff tool params pane.

Is the only way to get two versions of the same tool id installed via toolshed?

Sorry - I feel like I've mis-understood the tool version/id resolution 
architecture. Happy to talk on the phone if that would be a better use of your 
time.

Thanks,
Curtis



From: Jeremy Goecks [mailto:jeremy.goe...@emory.edu]
Sent: Wednesday, September 25, 2013 8:43 AM
To: Curtis Hendrickson (Campus)
Cc: galaxy-dev@lists.bx.psu.edu; Shantanu Pavgi (Campus)
Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff 
version

The problem is likely a collision on the tool_id; update the tool ID for one of 
the wrappers and that should solve the problem.

Best,
J.

On Sep 24, 2013, at 5:30 PM, Curtis Hendrickson (Campus) wrote:


Jeremy,

We didn't have time to factor out to toolshed just now, so we took the 
following approach
1.   Take 1.3-compatible cuffdiff wrapper (0.0.5) and move it to 
cuffdiff_1.3_wrapper.py/xml - and add a dependency on cufflinks verson=1.3.0 
(and set that up).
2.   Apply your patch, changing current cuffdiff wrapper to (0.0.6)
3.   Add a line in tool_conf.xml  so that both your modern cuffdiff v6 
wrapper and our cuffdiff v5 wrappers are included.

When we restarted, and went to the GUI, there was only ONE cuffdiff tool 
listed, and when you clicked on it, it brought up the 0.0.5 version (the 2nd 
listed in tool_conf).
There was no version pull down to get the 0.0.6 version.,

Any idea what I've missed?

Regards,
Curtis


From: Jeremy Goecks [mailto:jeremy.goe...@emory.edu]
Sent: Thursday, September 19, 2013 3:17 PM
To: Curtis Hendrickson (Campus)
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff 
version

We can hack the old wrapper back in, using a hard-coded path to the old 
cuffdiff.

You could also use Managed Tool Dependencies for multiple versions of cuffdiff:

http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies



Shantanu suggests it might be a better world if we create a tool-shed 
repository for the cuffdiff 1.3-2.1 (wrapper 0.0.5) version, then install that.
Would that be a better path, or just add confusion?

This is on the right path. The right thing to do is migrate the Cuff* tool 
wrappers out of the galaxy-central/dist repositories; this is a bit more effort 
than simply creating the tool-shed repository for the wrappers.

There are two reasons I haven't done this:

(a) time;
(b) because the tools-and consequently the wrappers-are still in active 
development, there needs to be a place for managing development of the 
wrappers. Hence, the wrappers need a home on bitbucket or github, but we (the 
Galaxy team) haven't determined where these wrappers should live.

(a) is a much more significant barrier than (b), so if a pull request migrating 
Cuff* tools out of the distribution were made, I'm sure (b) could be worked out.

Best,
J.


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff version

[Jeremy Goecks]

The problem is likely a collision on the tool_id; update the tool ID for one of 
the wrappers and that should solve the problem.

Best,
J.

On Sep 24, 2013, at 5:30 PM, Curtis Hendrickson (Campus) wrote:

> Jeremy,
>  
> We didn’t have time to factor out to toolshed just now, so we took the 
> following approach
> 1.   Take 1.3-compatible cuffdiff wrapper (0.0.5) and move it to 
> cuffdiff_1.3_wrapper.py/xml – and add a dependency on cufflinks verson=1.3.0 
> (and set that up).
> 2.   Apply your patch, changing current cuffdiff wrapper to (0.0.6)
> 3.   Add a line in tool_conf.xml  so that both your modern cuffdiff v6 
> wrapper and our cuffdiff v5 wrappers are included.
>  
> When we restarted, and went to the GUI, there was only ONE cuffdiff tool 
> listed, and when you clicked on it, it brought up the 0.0.5 version (the 2nd 
> listed in tool_conf).
> There was no version pull down to get the 0.0.6 version.,
>  
> Any idea what I’ve missed?
>  
> Regards,
> Curtis
>  
>  
> From: Jeremy Goecks [mailto:jeremy.goe...@emory.edu] 
> Sent: Thursday, September 19, 2013 3:17 PM
> To: Curtis Hendrickson (Campus)
> Cc: galaxy-dev@lists.bx.psu.edu
> Subject: Re: [galaxy-dev] cuffdiff wrapper not synchronized with cuffdiff 
> version
>  
> We can hack the old wrapper back in, using a hard-coded path to the old 
> cuffdiff.
>  
> You could also use Managed Tool Dependencies for multiple versions of 
> cuffdiff:
>  
> http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies
> 
> 
> Shantanu suggests it might be a better world if we create a tool-shed 
> repository for the cuffdiff 1.3-2.1 (wrapper 0.0.5) version, then install 
> that.
> Would that be a better path, or just add confusion?
>  
> This is on the right path. The right thing to do is migrate the Cuff* tool 
> wrappers out of the galaxy-central/dist repositories; this is a bit more 
> effort than simply creating the tool-shed repository for the wrappers.
>  
> There are two reasons I haven't done this:
>  
> (a) time;
> (b) because the tools—and consequently the wrappers—are still in active 
> development, there needs to be a place for managing development of the 
> wrappers. Hence, the wrappers need a home on bitbucket or github, but we (the 
> Galaxy team) haven't determined where these wrappers should live.
>  
> (a) is a much more significant barrier than (b), so if a pull request 
> migrating Cuff* tools out of the distribution were made, I'm sure (b) could 
> be worked out.
>  
> Best,
> J.
>  

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] pbkdf2 encryption - disabling does not work

[Jan Hapala]

Hi,

I need to run Galaxy with ProFTPd, therefore I need to disable the PBKDF2
encryption of the passwords (as stated here
http://dev.list.galaxyproject.org/disable-PBKDF2-revert-to-SHA1-td4661274.html
).

However, when I do add this line:
use_pbkdf2 = false
to the universe_wsgi.ini, restart Galaxy, submit the form for the password
change (or register as a new user) I get an error "_hashlib.HASH object has
no attribute __getitem__".

I got the same result with a version as of Jul 2 and the most recent tag
(10411:c42567f43aa7, Aug 19).

I would be much grateful for your suggestions!

Regards,
Jan


URL:
http://127.0.0.1:8080/user/edit_info?cntrller=user&user_id=f2db41e1fa331b3e
File
'.../eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py',
line 364 in respond
  app_iter = self.application(environ, detect_start_response)
File '.../eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in
__call__
  return self.application(environ, start_response)
File '.../eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633
in __call__
  return self.application(environ, start_response)
File '.../lib/galaxy/web/framework/base.py', line 132 in __call__
  return self.handle_request( environ, start_response )
File '.../lib/galaxy/web/framework/base.py', line 190 in handle_request
  body = method( trans, **kwargs )
File '.../lib/galaxy/webapps/galaxy/controllers/user.py', line 842 in
edit_info
  trans.sa_session.flush()
File
'.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py',
line 114 in do
File
'.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py',
line 1718 in flush
File
'.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py',
line 1789 in _flush
File
'.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py',
line 331 in execute
File
'.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py',
line 475 in execute
File
'.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py',
line 59 in save_obj
File
'.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py',
line 485 in _emit_update_statements
File
'.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py',
line 1449 in execute
File
'.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py',
line 1584 in _execute_clauseelement
File
'.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py',
line 1651 in _execute_context
File
'.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py',
line 1647 in _execute_context
File
'.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py',
line 475 in _init_compiled
File
'.../eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/types.py',
line 626 in process
File '.../lib/galaxy/model/custom_types.py', line 125 in process_bind_param
  value = value[0:self.impl.length]
StatementError: '_hashlib.HASH' object has no attribute '__getitem__'
(original cause: TypeError: '_hashlib.HASH' object has no attribute
'__getitem__') 'UPDATE galaxy_user SET update_time=%(update_time)s,
password=%(password)s WHERE galaxy_user.id = %(galaxy_user_id)s'
[{u'galaxy_user_id': 1, 'password': }]
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] error in 2013_08_12 DevNewsBriefs

[Nicola Soranzo]

Il 2013-09-23 20:43 Curtis Hendrickson (Campus) ha scritto:

Dear Galaxy Team

I was upgrading our servers with the 8/12 release, and going over the
DevNotes [1].

I noted item "Pull Requets Merged #5: SAMtools indexes #188 [2]".

But the file created by the pull request didn't seem to exist.

After looking in bitbucket at the history, it looks like the pull
request was backed out [3] before the release.

Have I correctly understood the situation?


Dear Curtis,
unfortunately you're right, my pull request has been backed out before 
the latest release.
It seems that the Galaxy Team intention is to first move SAMtools and 
cufflinks wrapper to the Tool Shed and then re-apply my changes.


There is a Trello card if you would like to monitor the progresses:

https://trello.com/c/GvypU9T7/1021-tools-migrate-sam-fa-indices-tools-to-the-tool-shed-and-reimplement-nicola-soranzo-s-enhancements-to-this-data-table

Nicola


Regards,

Curtis


Links:
--
[1] http://wiki.galaxyproject.org/DevNewsBriefs/2013_08_12
[2]

http://wiki.galaxyproject.org/DevNewsBriefs/2013_08_12#Pull_Requests_Merged
[3]

https://bitbucket.org/galaxy/galaxy-central/commits/73d7a93e2f8d0bafbcb0b4dd9bf562d1fa6a88e0#general-comments


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/