Re: [galaxy-dev] Problem with cleanup.py

[Jelle Scholtalbers]

Hi George,

on our instance I have noticed that the tmp (new_file_path) was taking up
quite some space and files were only deletable after restarting the web /
uwsgi process.

FYI a while ago I wrote this script which should ease the cronjob setup a
bit (does not cleanup tmp though):
https://github.com/galaxyproject/galaxy/blob/dev/scripts/maintenance.sh

Cheers,
Jelle

On Tue, Oct 30, 2018 at 5:04 AM George Weingart 
wrote:

> Hello,
>
> We are incurring a problem with cleanup.py:
>
> We have already run two cycles of cleanup.py (1-6) with a cut off of 30
> days,  we see in the logs that many datasets are deleted,  but when we do
> df -kh  to check the file system, we see creeping up utilization of space -
> no relinquishing of any space.
>
> We are running in a docker environment and our cleanup job looks as
> follows:
> 
>
> #!/bin/bash
>
> export GALAXY_VIRTUAL_ENV=/galaxy_venv
>
> source $GALAXY_VIRTUAL_ENV/bin/activate
>
> export
> script="/galaxy-central/scripts/cleanup_datasets/cleanup_datasets.py"
>
> export config="/etc/galaxy/galaxy.ini"
>
> export logdir="/export/galaxy-central/logs"
>
> export days=30
>
> export num=$1
>
> nohup $script $config -d $days -$num -r -f >> $logdir/cleanup$num.log &
>
> 
>
>
> and we run it with ./run_cleanup.sh 1
>
> or 2, 3 etc.
>
>
> Any ideas ?
>
>
> Thanks!
>
> George Weingart PhD
>
> Huttenhower Lab
>
> Harvard School of Public Health
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] tool data confusion

[Jelle Scholtalbers]

Hi all,

Again, this may be me sticking to old pre-tool shed habits - so I'd like
> to hear how other people manage their *.loc files, especially when doing
> hand editing to add entries.
>

I **try** to stick to just editing the *.loc file in tool-data for those
tools that do not have a data manager. This way I can also track them in my
local git branch. But I do agree that it can be complicated to figure out
which files to edit for which tool (bwa/bwamem...) and add the correct
entry and in the right format.
The data managers make it on the one hand easier, but harder to tell if I
was adding entries by hand or through a data manager - especially when not
being consistent..

Part of this could be mitigated if the admin web interface would allow for
editing these "Tool Data Table Entries" and not just view them ->
https://github.com/galaxyproject/galaxy/issues/6819

Cheers,
Jelle
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] AD Intergration

[Jelle Scholtalbers]

Hi John,

this error looks familiar:
https://github.com/galaxyproject/galaxy/issues/3178
https://github.com/galaxyproject/galaxy/issues/4153
https://github.com/galaxyproject/starforge/issues/130

To remedy the 'option' error:

source /home/galaxy/galaxy/.venv/bin/activate
pip install --upgrade python-ldap


Hope this brings you a step further.

- Jelle

On Wed, Jun 14, 2017 at 6:09 PM, John Chen <jchen...@yahoo.com> wrote:

> Jelle
>
> I did all that and it looks correct.. it is retrieving the correct field.
> This is the error i am still getting..  I am using pretty much the same
> option in other apps..
>
>
>
> galaxy.webapps.galaxy.controllers.user DEBUG 2017-06-14 12:04:40,648
> trans.app.config.auth_config_file: ./config/auth_conf.xml
> galaxy.auth.providers.ldap_ad DEBUG 2017-06-14 12:04:40,648 LDAP
> authenticate: email is johnu...@example.org
> galaxy.auth.providers.ldap_ad DEBUG 2017-06-14 12:04:40,648 LDAP
> authenticate: username is None
> galaxy.auth.providers.ldap_ad DEBUG 2017-06-14 12:04:40,648 LDAP
> authenticate: options are {'bind-user': '{dn}', 'search-fields':
> 'uid,mail', 'login-use-username': 'False', 'allow-register': 'True',
> 'ldap-options': 'OPT_X_TLS_REQUIRE_CERT=OPT_X_TLS_ALLOW',
> 'auto-register-email': '{email}', 'server': 'ldap://ldap.nyumc.org',
> 'auto-register': 'True', 'search-base': 'DC=example,DC=org',
> 'search-filter': '(mail={email})', 'continue-on-failure': 'True',
> 'auto-register-username': '{sAMAccountName', 'bind-password': '{password}',
> 'allow-password-change': 'False'}
> galaxy.auth.providers.ldap_ad DEBUG 2017-06-14 12:04:40,648 LDAP
> authenticate: Valid LDAP option pair OPT_X_TLS_REQUIRE_CERT=OPT_X_TLS_ALLOW
> -> 24582=3
> galaxy.auth.providers.ldap_ad ERROR 2017-06-14 12:04:40,648 LDAP
> authenticate: search exception
> Traceback (most recent call last):
>   File "/home/galaxy/galaxy/lib/galaxy/auth/providers/ldap_ad.py", line
> 118, in authenticate
> ldap.set_option(*opt)
>   File 
> "/home/galaxy/galaxy/.venv/lib/python2.7/site-packages/ldap/functions.py",
> line 135, in set_option
> return _ldap_function_call(None,_ldap.set_option,option,invalue)
>   File 
> "/home/galaxy/galaxy/.venv/lib/python2.7/site-packages/ldap/functions.py",
> line 66, in _ldap_function_call
> result = func(*args,**kwargs)
> ValueError: option error
>
>
> Are you running MS AD ?   if so, could i take a look at your config file?
>
> Thanks
> John
>
>
> --
> *From:* Jelle Scholtalbers <j.scholtalb...@gmail.com>
> *To:* Hans-Rudolf Hotz <h...@fmi.ch>
> *Cc:* John Chen <jchen...@yahoo.com>; Galaxy Dev List <galaxy-dev@lists.
> galaxyproject.org>
> *Sent:* Monday, June 12, 2017 3:16 AM
> *Subject:* Re: [galaxy-dev] AD Intergration
>
> Hi John,
>
> as a tip, you can use the tool "ldapsearch", from e.g. the package
> "openldap-client", to figure out with which attributes you search and which
> attributes you can retrieve.
>
> Examples:
> $ ldapsearch -vv -x -H ldap://dc1.example.com -b
> cn=Users,dc=exampke,dc=org" # retrieve all AD/ldap entries
> $ ldapsearch -vv -x -H ldap://dc1.example.com -b
> cn=Users,dc=exampke,dc=org "uid=a_username"  # retrieve all attributes for
> user with uid "a_username"
> $ ldapsearch -vv -x -H ldap://dc1.example.com -b
> cn=Users,dc=exampke,dc=org "sAMAccountName=a_username" mail # only
> retrieve the mail attribute by searching for the sAMAccountName
>
>
> In addition, if you get it working, you might want to switch to the more
> secure ldap*s* if that is supported by your IT.
>
> Cheers,
> Jelle
>
>
>
> On Mon, Jun 12, 2017 at 8:32 AM, Hans-Rudolf Hotz <h...@fmi.ch> wrote:
>
>
>
> On 06/09/2017 03:29 PM, John Chen wrote:
>
> Hans-Rudolf,
>
> That got me past the error, but I i am now having issue authenticating
> with against AD, as if its not able to search for the users.  Do I need
> a binding service account to search AD object?  Does the bottow 5 lines
> look correct?
>
>
> They look right, but I can't say whether they are correct. You need to
> discuss this with the person who has set up your Active Directory
>
>
> Hans-Rudolf
>
>
>
>
> cn=galaxy,ou=Secu rity,ou=somegroup,dc=example,
> dc=org
>
> ((objectCl ass=user)(sAMAccountName={
> username}))
>  ADsearchAccount< /search-user>
>  AD_Search_Pa sswrd
>  {sAMAccountName}
>
> The logs show that it found the userID and email, but gets an invalid
> password on the webportal
>
> galaxy.webapps.galaxy.controll ers.user DEBUG 2017-06-09 09:26:34,5

Re: [galaxy-dev] AD Intergration

[Jelle Scholtalbers]

Hi John,

as a tip, you can use the tool "ldapsearch", from e.g. the package
"openldap-client", to figure out with which attributes you search and which
attributes you can retrieve.

Examples:
$ ldapsearch -vv -x -H ldap://dc1.example.com -b cn=Users,dc=exampke,dc=org"
# retrieve all AD/ldap entries
$ ldapsearch -vv -x -H ldap://dc1.example.com -b cn=Users,dc=exampke,dc=org
"uid=a_username"  # retrieve all attributes for user with uid "a_username"
$ ldapsearch -vv -x -H ldap://dc1.example.com -b cn=Users,dc=exampke,dc=org
"sAMAccountName=a_username" mail # only retrieve the mail attribute by
searching for the sAMAccountName


In addition, if you get it working, you might want to switch to the more
secure ldap*s* if that is supported by your IT.

Cheers,
Jelle



On Mon, Jun 12, 2017 at 8:32 AM, Hans-Rudolf Hotz  wrote:

>
>
> On 06/09/2017 03:29 PM, John Chen wrote:
>
>> Hans-Rudolf,
>>
>> That got me past the error, but I i am now having issue authenticating
>> with against AD, as if its not able to search for the users.  Do I need
>> a binding service account to search AD object?  Does the bottow 5 lines
>> look correct?
>>
>
> They look right, but I can't say whether they are correct. You need to
> discuss this with the person who has set up your Active Directory
>
>
> Hans-Rudolf
>
>
>
>
>> cn=galaxy,ou=Security,ou=somegroup,dc=example,
>> dc=org
>>
>> ((objectClass=user)(sAMAccountName={
>> username}))
>>  ADsearchAccount
>>  AD_Search_Passwrd
>>  {sAMAccountName}
>>
>> The logs show that it found the userID and email, but gets an invalid
>> password on the webportal
>>
>> galaxy.webapps.galaxy.controllers.user DEBUG 2017-06-09 09:26:34,592
>> trans.app.config.auth_config_file: ./config/auth_conf.xml
>> galaxy.auth.providers.ldap_ad DEBUG 2017-06-09 09:26:34,592 LDAP
>> authenticate: email is testuser.n...@example.org
>> galaxy.auth.providers.ldap_ad DEBUG 2017-06-09 09:26:34,592 LDAP
>> authenticate: username is testUser
>> galaxy.auth.providers.ldap_ad DEBUG 2017-06-09 09:26:34,592 LDAP
>> authenticate: options are {'bind-user': '{sAMAccountName}',
>> 'search-fields': 'sAMAccountName,mail', 'login-use-username': 'True',
>> 'allow-register': 'False', 'auto-register-email': '{mail}', 'server':
>> 'ldap://xxx.xxx.xx', 'auto-register': 'True', 'search-base':
>> 'cn=xxx-xx,ou=Security,ou=x xxx,dc=xxx,dc=xx', 'search-filter':
>> '(&(objectClass=user)(sAMAccountName={username}))',
>> 'auto-register-username': '{sAMAccountName}', 'search-password': '',
>> 'search-user': '', 'bind-password': '{password}'}
>> galaxy.auth.providers.ldap_ad WARNING 2017-06-09 09:26:34,596 LDAP
>> authenticate: search returned no results
>> 10.127.220.227 - - [09/Jun/2017:09:26:34 -0400] "POST
>> /user/login?use_panels=False HTTP/1.1" 200 -
>> "http://glxlcdcpvm01.nyumc.org:8080/user/login?use_panels=False;
>> "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:53.0) Gecko/20100101
>> Firefox/53.0"
>>
>>
>> 
>> *From:* Hans-Rudolf Hotz 
>> *To:* John Chen ; Galaxy Dev List
>> 
>> *Sent:* Friday, June 9, 2017 3:34 AM
>> *Subject:* Re: [galaxy-dev] AD Intergration
>>
>> always keep the mailing list in the loop! in order for others to help or
>> learn
>>
>>
>>
>> On 06/08/2017 07:27 PM, John Chen wrote:
>>  > Hans-Rudolf
>>  >
>>  > This is the error I get when I start the Galaxy server.
>>  >
>> ...
>>  > xml.etree.ElementTree.ParseError: mismatched tag: line 8, column 105
>>  >
>>
>> This is very informative. Looking at line 8 in your file:
>>
>>
>> > href="ldap://ldap.xxx.xx;>ldap://ldap.xxx.xx
>>
>>
>> The element "a" is not  terminated
>>
>>
>> What happens, if you try just
>>
>> ldap://ldap.xxx.xx
>>
>>
>>
>>
>>
>> Regards, Hans-Rudolf
>>
>>
>> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] Update Galaxy from 15.10 to 16.10

[Jelle Scholtalbers]

Hi Jochen,

generally updating the database shouldn't be an issue as this is handled by
the migration scripts that incrementally update your db schema to match the
version.
But I would do a backup just in case.

For the outdated ini files you can use the tool called Meld to spot
differences between yours and the .sample files. Björn pointed me to this
tool after opening this 
PR.

Cheers,
Jelle

On Tue, Jan 17, 2017 at 11:34 AM, Jochen Bick 
wrote:

> Hi,
>
> today we tried to update from 15.10 to 16.10 which included an update of
> your database from 129 to 131.
> Do we necessary run into problem if we update from an one year old
> instance? How is the galaxy.ini updated? Or galaxy.ini does not know any
> conda yet.
>
>
> Thanks in advance.
> Cheers Jochen
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Testing workflows

[Jelle Scholtalbers]

Hi Eric,

this is a script I have used in the past
https://github.com/scholtalbers/galaxy-test-workflows
It doesn't give you the nice feedback that you probably want, but maybe you
have some use for it anyway.

Cheers,
Jelle



On Wed, Dec 14, 2016 at 4:26 AM, Eric Rasche  wrote:

> Wondering if anyone has tooling to test a workflow as it exists in a
> specific galaxy instance? I'm looking to test some of my user's manually
> built workflows in our galaxy instance to make sure that we can detect
> issues when they arise (e.g. due to changes in the interfaces of our
> non-toolshed installed tools)
>
> Ideally something like:
>
> $ my_tool $galaxy_url $api_key $workflow_id
>   
> > report.xunit.xml
>
> If not we'll build something soon.
>
> Cheers,
> Eric
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] list collection output - format set from input

[Jelle Scholtalbers]

Hi all,

I have a related issue as I'm trying to set the format on discovered
datasets. Only setting e.g 'ext="fastqsanger"' seems to work. But the
following variations do not work to set a format, and similar to Alexander,
the history items do not show any datatype with these options.


 
 
 
 
 
 
 

Is there someway to set the format by input source without writing a
wrapper script to change the extension as suggested by John?

Tested this with 'planemo serve'

Cheers,
Jelle


On Thu, Jun 25, 2015 at 9:36 PM, Alexander Vowinkel <
vowinkel.alexan...@gmail.com> wrote:

> Hi John,
>
> yes - I created this hacky solution and it works.
>
> I now tried what you said, but no success.
> Code:
>
>
>> 
>> > directory="output" />
>> 
>
>
> It puts the samples into the collection correctly, but doesn't set a data
> type.
>
> Weird enough: In the collection, the files even don't show any format
> (in the small history view on the right). Is that normal?
>
> Addon: How can I let the files in the collection have the same database
> build
> attached? The other file (log file) gets it.
>
> Best,
> Alexander
>
>
> 2015-06-25 9:12 GMT-05:00 John Chilton :
>
>> You are giving Galaxy mixed signals :). format_source will say to use
>> the data type specified by the corresponding input - but
>>
>> 
>>
>> Is saying (with the pattern __name_and_ext__) read files of the form
>> out1.fastq and assign the collection identifier to "out1" and the
>> extension/format to "fastq".
>>
>> Two things should work - one of them will and the other might but should.
>>
>> One thing you can do is not override the extension in your pattern:
>>
>> 
>>
>> or if you don't want to include the .fq in the output identifier (and
>> you probably don't want to)
>>
>> 
>>
>> If that doesn't work - it is a bug and please let me know and I will
>> attempt to fix it.
>>
>> The hackier way to get this to work that I am more confident will work
>> - is to drop the format_source - use the same pattern:
>>
>> 
>>
>> But just use a shell command or something to rename all files of the
>> form *.fq to *.${fastq_input1.ext}.
>>
>> Hope this helps.
>>
>> -John
>>
>>
>> On Thu, Jun 25, 2015 at 9:56 AM, Alexander Vowinkel
>>  wrote:
>> > no - not a typo. The tool can process both.
>> > It's just my naming, because I use it for fastq.
>> > I'll change the help tag.
>> >
>> > 2015-06-25 3:44 GMT-05:00 Peter Cock :
>> >>
>> >> Hi Alexander,
>> >>
>> >> If this wasn't a collection, I would expect  format_source to work
>> >> (possibly also using metadata_source="fastq_input1"), so perhaps
>> >> this is a bug - John?
>> >>
>> >> Peter
>> >>
>> >> P.S. Your help caption and output label both say FASTQ, but the
>> >> input also allows FASTA input. Typo?
>> >>
>> >> On Thu, Jun 25, 2015 at 2:39 AM, Alexander Vowinkel
>> >>  wrote:
>> >> > Hi,
>> >> >
>> >> > I have an input, that can be fasta,fastqsanger,fastqillumina:
>> >> >
>> >> > > >> > format="fasta,fastqsanger,fastqillumina" label="Select the fastq
>> file"
>> >> > help="Specify fastq file with reads"/>
>> >> >
>> >> >
>> >> > I have multiple outputfiles - bundled in a list collection:
>> >> >
>> >> > > label="@OUTPUT_NAME_PREFIX@
>> >> > on
>> >> > ${on_string} (Fastq Collection)" format_source="fastq_input1">
>> >> > > />
>> >> > 
>> >> >
>> >> >
>> >> > The format_source parameter doesn't work - the files in the list
>> >> > (extension
>> >> > fq) are of format "fq"
>> >> >
>> >> > How can I make it possible that they are
>> fasta,fastqsanger,fastqillumina
>> >> > depending on fastq_input1?
>> >> >
>> >> > Thanks,
>> >> > Alexander
>> >> >
>> >> > ___
>> >> > Please keep all replies on the list by using "reply all"
>> >> > in your mail client.  To manage your subscriptions to this
>> >> > and other Galaxy lists, please use the interface at:
>> >> >   https://lists.galaxyproject.org/
>> >> >
>> >> > To search Galaxy mailing lists use the unified search at:
>> >> >   http://galaxyproject.org/search/mailinglists/
>> >
>> >
>>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Change from Bitbucket to git

[Jelle Scholtalbers]

Hi Jochen,

I recently took a probably more labour intensive approach.

# backup!
# mv prev-installation to somewhere else
# git clone to old-location
# git checkout -b master origin/master
# and copy over every config, tool xml, tool dependency directory,
tool-data loc files from the prev-installation to the new git one
# add your own code changes if you have any that you want to keep
# ./run.sh or manage_db.sh upgrade
# check log and probably copy some more missing files

The upside for me was that I could finally remove all the junk that our
installation has been collecting for the last couple of year.

Cheers,
Jelle


On Tue, Jul 21, 2015 at 12:45 PM, Jochen Bick jochen.b...@usys.ethz.ch
wrote:

 Thanks Marius,

 it does not look like it's working for me. There are to many changes in
 git diff...

 the git pull and cat local_changes.diff |patch -p1 -R gives me an
 endless list of differences.

 Cheers Jochen

 On 21.07.2015 11:41, Marius van den Beek wrote:
  Hi Jochen,
 
  I recently did this. Instructions are here:
  https://wiki.galaxyproject.org/Develop/SourceCode
 
  This has worked for me:
  (~/galaxy-dist is my production galaxy dir)
  cd ~
  git clone https://github.com/galaxyproject/galaxy.git
  cd galaxy
  git reset --hard 3f1cec451f6dadee52a70a14fc6f6899f7aacdae
  cd ~/galaxy-dist
  hg update 17115
  mv .hg .hg_backup
  cp -R ~/galaxy/.git .
  git pull (at this point I got a merge error!)
  so I did:
  git diff  local_changes.diff
  cat local_changes.diff |patch -p1 -R
  git pull
  git checkout release_15.05
 
  Best,
  Marius
 
  On 21 July 2015 at 11:36, Jochen Bick jochen.b...@usys.ethz.ch wrote:
 
  Hi,
 
  is it possible to change from bitbucket to git? So I would like
  future-update my galaxy from git instead from bitbucket.
 
  Cheers Jochen
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] Ipython behind apache proxy, firefox CORS error

[Jelle Scholtalbers]

Hi Björn, Eric, all,

I'm trying to get ipython running on our instance. Without the proxy, e.g.
http://myserver:8080 it works fine. However, with the proxy in front (e.g.
http://myserver/galaxy) I get this firefox error
11:28:37.793 Cross-Origin Request Blocked: The Same Origin Policy
disallows reading the remote resource at
http://gbcs-dev:8800/ipython/39958/login?next=%2Fipython%2F39958%2Ftree.
(Reason: CORS request failed).1 unknown
or on chrome xhr request cancelled.

Do you happen to know the correct apache settings to overcome this?
I tried a few things, but I ended up with ipython starting, but with a
message cannot connect to websocket.

- Jelle
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Migrating user account from galaxy specific to LDAP

[Jelle Scholtalbers]

Hi Teshome,

I assume you have good reasons for switching to LDAP (e.g. users only need
to remember their main password), but just as a side note, we recently
switched back from LDAP authentication to local user authentication to have
a bit more control. One of the main reasons is being able to 'impersonate a
user', a capability lost when using external authentication. We do not
regret it ;)

Cheers,

Jelle

On Thu, Apr 30, 2015 at 5:01 PM, Eric Rasche e...@tamu.edu wrote:

 Hi Teshome,

 I solved this by manually editing the user's emails in the database. I did
 this during the downtime, and just switched their old email address to
 their LDAP address, causing them to see their old histories, and no new
 accounts were generated.

 If your users have already logged in (and thus have two accounts, one old,
 one new) it's slightly more complicated.
 Your best bet is to set up some downtime immediately, and do the above,
 causing the users to lose anything in their new accounts.

 Failing that, a method of last resort:
 You can update history IDs to the new users, i.e. find a mapping from old
 email + old user ID to new email + new user ID, and then run queries like 
 UPDATE
 history set user_id=new_user_id where user_id=old_user_id but that only
 migrates histories, I don't know what else is keyed on user IDs (probably
 data libraries? Workflows? Hmm.)

 Hope that helps,

 Cheers,
 Eric


 2015-04-30 1:01 GMT-05:00 Teshome Dagne Mulugeta teshome.mulug...@nmbu.no
 :

  ​Hi,


  We are currently using galaxy specific login using email address. We
 changed the authentication to LDAP. Most users email address doesn't match
 with their registered email address in LDAP. Now, users are not able to see
 their previous works which is expected of course but I have no clue how to
 fix it. Please help.


   Cheers,
  Teshome

 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/




 --
 Eric Rasche
 Programmer II

 Center for Phage Technology
 Rm 312A, BioBio
 Texas AM University
 College Station, TX 77843
 404-692-2048
 e...@tamu.edu
 rasche.e...@yandex.ru


 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Metadata error in uploading files to data libraries

[Jelle Scholtalbers]

Hi John and others,

The OSError: [Errno 2] No such file or directory is solved when putting
the shell=True in the mentioned locations - as written in the Trello
card.

However, even after fixing that, linked BAM files where not indexed, or at
least the files/_metadata_files/000/metadata_XXX.dat is empty on job
completion. After spending too much time on this, I nailed down the problem
to this setting:
outputs_to_working_directory = True

With this option set, the BAM index file is never moved to the right
location (or is is not being created.)
Setting this option to False (I think the setting was a left over from a
failed attempt to use real user job submission) this now just works.

- Jelle


On Mon, Dec 15, 2014 at 8:31 PM, John Chilton jmchil...@gmail.com wrote:

 Hello,

   Sorry we haven't made progress on this - and thanks for creating a
 Trello card (https://trello.com/c/tw75nq1U).

   It looks like samtools is not on your path, can you use it from the
 command-line? If not you should probably install it - I would suggest
 install samtools 0.1.19 with homebrew (looks like you are a on Mac).
 If you can run samtools from the command-line - can you do a which
 `samtools` from your command-line and see where it is coming from and
 then add that directory explicitly to your Galaxy PATH - say at the
 top of run.sh in your Galaxy root (let me know if you need more
 details on that).

 -John

 On Tue, Dec 9, 2014 at 3:02 AM, Jelle Scholtalbers
 j.scholtalb...@gmail.com wrote:
  Hi all,
 
  after an update to the following changeset(14859:7ba05957588a, stable,
  05.12.14), our bam files that are uploaded(linked) to a data library,
 are no
  longer indexed. The metadata_xxx.dat is created, but it stays empty.
  The following error message appears in the log, although the state of the
  dataset is 'ok':
 
  galaxy.jobs WARNING 2014-12-05 12:47:02,218 Error accessing
 /g/K/K27.bam,
  will retry: [Errno 1] Operation not permitted: '/g/K/K27.bam'
  galaxy.jobs WARNING 2014-12-08 13:38:57,045 Error accessing /g/K/K2.bam,
  will retry: [Errno 1] Operation not permitted: '/g/K/K2.bam'
 
  All file permissions are correct (i.e. galaxy owns them). Furthermore,
  executing samtools index, just works on those files:
  samtools index /g/K/K2.bam
  /g/galaxy/galaxy_data/files/_metadata_files/006/metadata_6598.dat
 
  When uploading the file - copy files into galaxy - the samtools index
 just
  works.
 
  ==
 
  Now, on a clean local install(14874:885f940bff64, stable, 05.12.14) and
  samtools installed globally and with the bam file sorted, I get the
  following situation:
  When I try to upload this bam to a data library by linking the following
  error is shown on the dataset (note: here the dataset is set in error
 state,
  which does not happen on our server)
 
  Uploaded by: schol...@embl.de
  Date uploaded: Mon Dec 8 17:42:39 2014 (UTC)
  File size: 2.6 GB
  UUID: d23cf11a-0372-41cb-939a-7c8761d78b73
  Data type: auto
  Build: ?
  Miscellaneous information: Traceback (most recent call last): File
 
 /Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py,
  line 407, in __main__() File
 
 /Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py,
  line 396, in _
  Job Standard Error
 
  Traceback (most recent call last):
File
 
 /Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py,
  line 407, in
  __main__()
File
 
 /Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py,
  line 396, in __main__
  add_file( dataset, registry, json_file, output_path )
File
 
 /Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py,
  line 294, in add_file
  if datatype.dataset_content_needs_grooming( dataset.path ):
File
 
 /Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py,
  line 147, in dataset_content_needs_grooming
  version = self._get_samtools_version()
File
 
 /Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py,
  line 129, in _get_samtools_version
  output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE,
  stdout=subprocess.PIPE ).communicate()[1]
File
 
 /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py,
  line 711, in __init__
  errread, errwrite)
File
 
 /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py,
  line 1308, in _execute_child
  raise child_exception
  OSError: [Errno 2] No such file or directory
 
  error
  Database/Build: ?
  Number of data lines: None
  Disk file: /Users/scholtalbers/workspace/idr_data/WT1.sort.bam
 
  ===
  When uploading the bam without linking, I see the following processes:
  Upload-set meta-samtools index-'error state'
 
   Miscellaneous information: uploaded bam file Traceback (most recent call
  last): File
 
 /Users/scholtalbers/workspace/galaxy-dist-new/tools