Re: [galaxy-dev] cleanup_datasets.py with postgresql 8.1

[Malcolm Tobias]

I've managed to resolve this issue.  I'm not sure how I ended up with this 
issue, but in case anyone else get's hit by it, here's what I've learned:

-At version 0.6, SQLAlchemy changed the format for connecting to postgresql:

[postgresql] The "postgres" dialect is now named "postgresql" 

-I somehow managed to get multiple eggs pointing to different versions of 
SQLAlchemy:

[galaxy@login002 galaxy-dist]$ ls eggs/SQLAlchemy-0.
SQLAlchemy-0.5.6_dev_r6498-py2.5.egg/
SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/
SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg

-During my early stages of rolling out Galaxy, I managed to use the 0.7.9 
version of SQLAlchemy which forced me to change my database_connection syntax:

[galaxy@login002 galaxy-dist]$ ./run.sh --daemon 

  -this updates some of the "eggs"

  -it doesn't start; check the logs:

[galaxy@login002 galaxy-dist]$ tail paster.log
...
ile__', None ), self.config.database_engine_options, app=self )
  File "/home/galaxy/galaxy-dist/lib/galaxy/model/migrate/check.py", line 107, 
in create_or_verify_database
% ( db_schema.version, migrate_repository.versions.latest, config_arg ) )
Exception: Your database has version '114' but this code expects version '115'. 
 Please backup your database and then migrate the schema by running 'sh 
manage_db.sh upgrade'.
Removing PID file paster.pid

[galaxy@login002 galaxy-dist]$ sh manage_db.sh upgrade
/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/url.py:105:
 SADeprecationWarning: The SQLAlchemy PostgreSQL dialect has been renamed from 
'postgres' to 'postgresql'. The new URL format is 
postgresql[+driver]://:@/

-at that point I changed my database_connection from:

database_connection = postgres://

to:

database_connection = postgresql://

and everything appeared to be happy.  Everything that is except for the cleanup 
script.

-I've now changed everything to: database_connection = postgres:// and 
everything appears to be happy.  


I really don't understand how this came out.  I was using postgresql:// for 
quite a while without issues, so it would seem that I could *not* have been 
using 0.5.6 of SQLAlchemy unless this was modified by the Galaxy community.  

Cheers,
Malcolm

On Thursday 26 September 2013 17:16:16 Jennifer Jackson wrote:
> Posting to galaxy-...@bx.psu.edu to give the 
> question better exposure to the development community.
> Please remove the galaxy-u...@bx.psu.edu 
> mailing list from all replies - we want to avoid double posts.
> 
> 
> For discussion of
> local Galaxy instances and the Galaxy source code, please
> use the Galaxy Development list:
> 
>   http://lists.bx.psu.edu/listinfo/galaxy-dev
> 
> On 9/26/13 7:29 AM, Malcolm Tobias wrote:
> 
> All,
> 
> My understanding is that the cleanup_datasets.py under 
> scripts/cleanup_datasets should be compatible with 'older' versions of 
> postgresql. I'm running 8.1 under CentOS 5. When I attempt to run the 
> scripts, it fails to clean any data sets. From the logs, I'm noticing 
> messages like:
> 
> database_connection contains an unknown SQLAlchemy database dialect: 
> postgresql
> 
> In my universe_wsgi.ini I've defined the database_connection as:
> 
> database_connection = postgresql://galaxy:@localhost:5432/galaxydb
> 
> and am confident it's working as Galaxy works fine, other than this problem 
> with deleting older datasets.
> 
> The pgcleanup.py script isn't an option, as it requires postgresql >=9.1. I 
> noticed yet another version of cleanup_datasets.py under cron which looks 
> like it might be compatible with postgres, but am leary of trying it as I'm 
> not sure what it means by "1: database directory to clean". I'm not sure what 
> directory to point this to, nor am I confident that just deleting data from 
> the directory is the safest thing.
> 
> Any suggestions on how to proceed?
> 
> Thanks!
> 
> Malcolm
> 
> --
> 
> Malcolm Tobias
> 
> 314.362.1594
> 
> 
> 
> ___
> The Galaxy User list should be used for the discussion of
> Galaxy analysis and other features on the public server
> at usegalaxy.org.  Please keep all replies on the list by
> using "reply all" in your mail client.  For discussion of
> local Galaxy instances and the Galaxy source code, please
> use the Galaxy Development list:
> 
>   http://lists.bx.psu.edu/listinfo/galaxy-dev
> 
> To manage your subscriptions to this and other Galaxy lists,
> please use the interface at:
> 
>   http://lists.bx.psu.edu/
> 
> To search Galaxy mailing lists use the unified search at:
> 
>   http://galaxyproject.org/search/mailinglists/
> 
> 
> --
> Jennifer Hillman-Jackson
> http://galaxyproject.org
> 

-- 
Malcolm Tobias
314.362.1594

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Gal

Re: [galaxy-dev] cleanup_datasets.py with postgresql 8.1

[Malcolm Tobias]

Any feedback on this?  Our local instance of Galaxy is ready to "go live" and 
this is one of the remaining issues keeping us from more wide-spread usage.

Can anyone confirm that I should be using 
scripts/cleanup_datasets/cleanup_datasets.py when I'm using postgresql?  Are 
there any limits on the versions of postgresql?

Thanks in advance,
Malcolm

On Thursday 26 September 2013 17:16:16 Jennifer Jackson wrote:
> Posting to galaxy-...@bx.psu.edu to give the 
> question better exposure to the development community.
> Please remove the galaxy-u...@bx.psu.edu 
> mailing list from all replies - we want to avoid double posts.
> 
> 
> For discussion of
> local Galaxy instances and the Galaxy source code, please
> use the Galaxy Development list:
> 
>   http://lists.bx.psu.edu/listinfo/galaxy-dev
> 
> On 9/26/13 7:29 AM, Malcolm Tobias wrote:
> 
> All,
> 
> My understanding is that the cleanup_datasets.py under 
> scripts/cleanup_datasets should be compatible with 'older' versions of 
> postgresql. I'm running 8.1 under CentOS 5. When I attempt to run the 
> scripts, it fails to clean any data sets. From the logs, I'm noticing 
> messages like:
> 
> database_connection contains an unknown SQLAlchemy database dialect: 
> postgresql
> 
> In my universe_wsgi.ini I've defined the database_connection as:
> 
> database_connection = postgresql://galaxy:@localhost:5432/galaxydb
> 
> and am confident it's working as Galaxy works fine, other than this problem 
> with deleting older datasets.
> 
> The pgcleanup.py script isn't an option, as it requires postgresql >=9.1. I 
> noticed yet another version of cleanup_datasets.py under cron which looks 
> like it might be compatible with postgres, but am leary of trying it as I'm 
> not sure what it means by "1: database directory to clean". I'm not sure what 
> directory to point this to, nor am I confident that just deleting data from 
> the directory is the safest thing.
> 
> Any suggestions on how to proceed?
> 
> Thanks!
> 
> Malcolm
> 
> --
> 
> Malcolm Tobias
> 
> 314.362.1594
> 
> 
> 
> ___
> The Galaxy User list should be used for the discussion of
> Galaxy analysis and other features on the public server
> at usegalaxy.org.  Please keep all replies on the list by
> using "reply all" in your mail client.  For discussion of
> local Galaxy instances and the Galaxy source code, please
> use the Galaxy Development list:
> 
>   http://lists.bx.psu.edu/listinfo/galaxy-dev
> 
> To manage your subscriptions to this and other Galaxy lists,
> please use the interface at:
> 
>   http://lists.bx.psu.edu/
> 
> To search Galaxy mailing lists use the unified search at:
> 
>   http://galaxyproject.org/search/mailinglists/
> 
> 
> --
> Jennifer Hillman-Jackson
> http://galaxyproject.org
> 

-- 
Malcolm Tobias
314.362.1594

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/