Re: [galaxy-dev] Question using sftp to upload file to galaxy

2019-02-01 Thread Rui Wang
Hi Marius,

Thanks for the note. The link you pasted is how I came up with the config
in the original question. However it doesn't work...it kept saying my
password is incorrect. :-(

$ sftp  -oKexAlgorithms=diffie-hellman-group14-sha1 -oPort=
bioinfoadmin@localhost
bioinfoadmin@localhost's password:
Permission denied, please try again.
bioinfoadmin@localhost's password:
Permission denied, please try again.
bioinfoadmin@localhost's password:

Not sure why this would happen. :-( Have you seen this before?

Cheers,
Rui

On Sun, Jan 27, 2019 at 10:49 PM Marius van den Beek 
wrote:

> Hi Rui,
>
> there's a fairly complete explanation and example in in
> https://docs.galaxyproject.org/en/latest/admin/special_topics/ftp.html
>
> Hope that helps,
> Marius
>
> On Mon, 28 Jan 2019 at 07:35, Rui Wang  wrote:
>
>> Hey Folks,
>>
>> I tried a few times with different configurations, but none worked. Did
>> anyone have the successful experience that could share? :-)
>>
>> Cheers,
>> Rui
>>
>> On Sat, Jan 19, 2019 at 1:43 PM Rui Wang  wrote:
>>
>>> Hey Folks,
>>>
>>> I'm looking at the instructions of using ftp with proftpd. There is a
>>> section talking about extending it to use sftp. However, the sample config
>>> isn't comprehensive. I'm wondering if anyone has a working config for
>>> reference?
>>>
>>> What's the setting of user and group? It says it should match the one in
>>> the SQLNamedQuery, what does it mean exactly? I start proftpd as root, but
>>> start galaxy as bioinfoadmin(normal user with sudo).
>>>
>>> Just fyi, my proftpd config module and config file are pasted below. I'm
>>> working it out on a trial and error fashion, please feel free to point out
>>> if anything is wrong!
>>>
>>> Cheers,
>>> Rui
>>>
>>> modules:
>>> $ sbin/proftpd -l
>>> Compiled-in modules:
>>>   mod_core.c
>>>   mod_xfer.c
>>>   mod_rlimit.c
>>>   mod_auth_unix.c
>>>   mod_auth.c
>>>   mod_ls.c
>>>   mod_log.c
>>>   mod_site.c
>>>   mod_delay.c
>>>   mod_facts.c
>>>   mod_sql.c
>>>   mod_sql_postgres.c
>>>   mod_sql_passwd.c
>>>   mod_sftp.c
>>>   mod_cap.c
>>>
>>> etc/proftpd.conf
>>>
>>> ServerTypestandalone
>>>   # You must put this in a virtual host if you want it to listen on its
>>> own port. VHost != Apache Vhost.
>>>   
>>> Port 
>>> SFTPEngine on
>>> AuthOrder mod_auth_unix.c mod_sql.c # If you don't do this you will
>>> get weird disconnects
>>> SFTPHostKey /etc/ssh/ssh_host_rsa_key
>>> RequireValidShell no
>>> MaxLoginAttempts 6
>>> ServerName  "Galaxy SFTP"
>>> DefaultServer   on
>>> Umask   077
>>> User bioinfoadmin
>>> Group   bioinfoadmin
>>> UseFtpUsers off
>>> DefaultRoot ~
>>> AllowOverwrite  on
>>> AllowStoreRestart   on
>>> SQLEngine   on
>>> SQLGroupInfosftp_groups name id members
>>>
>>> # Do not authenticate against real (system) users
>>> 
>>> AuthPAM off
>>> 
>>>
>>> # Common SQL authentication options
>>> SQLPasswordEngine   on
>>> SQLBackend  postgres
>>> SQLConnectInfo  gal...@galaxy.my.org:5432 bioinfoadmin
>>> dbpwd
>>> SQLAuthenticate users
>>>
>>> # Configuration that handles PBKDF2 encryption
>>> # Set up mod_sql to authenticate against the Galaxy database
>>> SQLAuthTypesPBKDF2
>>> SQLPasswordPBKDF2   SHA256 1 24
>>> SQLPasswordEncoding base64
>>> SQLPasswordUserSalt sql:/GetUserSalt
>>>
>>> # Define a custom query for lookup that returns a passwd-like entry.
>>> Replace 512s with the UID and GID of the user running the Galaxy server
>>> SQLUserInfo custom:/LookupGalaxyUser
>>> SQLNamedQuery   LookupGalaxyUser SELECT "email, (CASE
>>> WHEN substring(password from 1 for 6) = 'PBKDF2' THEN subs

Re: [galaxy-dev] Question using sftp to upload file to galaxy

2019-01-27 Thread Rui Wang
Hey Folks,

I tried a few times with different configurations, but none worked. Did
anyone have the successful experience that could share? :-)

Cheers,
Rui

On Sat, Jan 19, 2019 at 1:43 PM Rui Wang  wrote:

> Hey Folks,
>
> I'm looking at the instructions of using ftp with proftpd. There is a
> section talking about extending it to use sftp. However, the sample config
> isn't comprehensive. I'm wondering if anyone has a working config for
> reference?
>
> What's the setting of user and group? It says it should match the one in
> the SQLNamedQuery, what does it mean exactly? I start proftpd as root, but
> start galaxy as bioinfoadmin(normal user with sudo).
>
> Just fyi, my proftpd config module and config file are pasted below. I'm
> working it out on a trial and error fashion, please feel free to point out
> if anything is wrong!
>
> Cheers,
> Rui
>
> modules:
> $ sbin/proftpd -l
> Compiled-in modules:
>   mod_core.c
>   mod_xfer.c
>   mod_rlimit.c
>   mod_auth_unix.c
>   mod_auth.c
>   mod_ls.c
>   mod_log.c
>   mod_site.c
>   mod_delay.c
>   mod_facts.c
>   mod_sql.c
>   mod_sql_postgres.c
>   mod_sql_passwd.c
>   mod_sftp.c
>   mod_cap.c
>
> etc/proftpd.conf
>
> ServerTypestandalone
>   # You must put this in a virtual host if you want it to listen on its
> own port. VHost != Apache Vhost.
>   
> Port 
> SFTPEngine on
> AuthOrder mod_auth_unix.c mod_sql.c # If you don't do this you will
> get weird disconnects
> SFTPHostKey /etc/ssh/ssh_host_rsa_key
> RequireValidShell no
> MaxLoginAttempts 6
> ServerName  "Galaxy SFTP"
> DefaultServer   on
> Umask   077
> User bioinfoadmin
> Group   bioinfoadmin
> UseFtpUsers off
> DefaultRoot ~
> AllowOverwrite  on
> AllowStoreRestart   on
> SQLEngine   on
> SQLGroupInfosftp_groups name id members
>
> # Do not authenticate against real (system) users
> 
> AuthPAM off
> 
>
> # Common SQL authentication options
> SQLPasswordEngine   on
> SQLBackend  postgres
> SQLConnectInfo  gal...@galaxy.my.org:5432 bioinfoadmin
> dbpwd
> SQLAuthenticate users
>
> # Configuration that handles PBKDF2 encryption
> # Set up mod_sql to authenticate against the Galaxy database
> SQLAuthTypesPBKDF2
> SQLPasswordPBKDF2   SHA256 1 24
> SQLPasswordEncoding base64
> SQLPasswordUserSalt sql:/GetUserSalt
>
> # Define a custom query for lookup that returns a passwd-like entry.
> Replace 512s with the UID and GID of the user running the Galaxy server
> SQLUserInfo custom:/LookupGalaxyUser
> SQLNamedQuery   LookupGalaxyUser SELECT "email, (CASE WHEN
> substring(password from 1 for 6) = 'PBKDF2' THEN substring(password from 38
> for 69) ELSE password END) AS
> password2,512,512,'/media/galaxy/galaxy/database/ftp/%U','/bin/bash' FROM
> galaxy_user WHERE email='%U'"
>
> # Define custom query to fetch the password salt
> SQLNamedQuery   GetUserSalt SELECT "(CASE WHEN SUBSTRING
> (password from 1 for 6) = 'PBKDF2' THEN SUBSTRING (password from 21 for 16)
> END) AS salt FROM galaxy_user WHERE email='%U'"
>   
>
> # Don't use IPv6 support by default.
> UseIPv6 off
> MaxInstances30
>
> # To cause every FTP user to be "jailed" (chrooted) into their home
> # directory, uncomment this line.
> # Bar use of SITE CHMOD by default
> 
>   DenyAll
> 
>
> # Bar use of RETR (download) since this is not a public file drop
> 
>   DenyAll
> 
> ~
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

[galaxy-dev] Question using sftp to upload file to galaxy

2019-01-19 Thread Rui Wang
Hey Folks,

I'm looking at the instructions of using ftp with proftpd. There is a
section talking about extending it to use sftp. However, the sample config
isn't comprehensive. I'm wondering if anyone has a working config for
reference?

What's the setting of user and group? It says it should match the one in
the SQLNamedQuery, what does it mean exactly? I start proftpd as root, but
start galaxy as bioinfoadmin(normal user with sudo).

Just fyi, my proftpd config module and config file are pasted below. I'm
working it out on a trial and error fashion, please feel free to point out
if anything is wrong!

Cheers,
Rui

modules:
$ sbin/proftpd -l
Compiled-in modules:
  mod_core.c
  mod_xfer.c
  mod_rlimit.c
  mod_auth_unix.c
  mod_auth.c
  mod_ls.c
  mod_log.c
  mod_site.c
  mod_delay.c
  mod_facts.c
  mod_sql.c
  mod_sql_postgres.c
  mod_sql_passwd.c
  mod_sftp.c
  mod_cap.c

etc/proftpd.conf

ServerTypestandalone
  # You must put this in a virtual host if you want it to listen on its own
port. VHost != Apache Vhost.
  
Port 
SFTPEngine on
AuthOrder mod_auth_unix.c mod_sql.c # If you don't do this you will get
weird disconnects
SFTPHostKey /etc/ssh/ssh_host_rsa_key
RequireValidShell no
MaxLoginAttempts 6
ServerName  "Galaxy SFTP"
DefaultServer   on
Umask   077
User bioinfoadmin
Group   bioinfoadmin
UseFtpUsers off
DefaultRoot ~
AllowOverwrite  on
AllowStoreRestart   on
SQLEngine   on
SQLGroupInfosftp_groups name id members

# Do not authenticate against real (system) users

AuthPAM off


# Common SQL authentication options
SQLPasswordEngine   on
SQLBackend  postgres
SQLConnectInfo  gal...@galaxy.my.org:5432 bioinfoadmin dbpwd
SQLAuthenticate users

# Configuration that handles PBKDF2 encryption
# Set up mod_sql to authenticate against the Galaxy database
SQLAuthTypesPBKDF2
SQLPasswordPBKDF2   SHA256 1 24
SQLPasswordEncoding base64
SQLPasswordUserSalt sql:/GetUserSalt

# Define a custom query for lookup that returns a passwd-like entry.
Replace 512s with the UID and GID of the user running the Galaxy server
SQLUserInfo custom:/LookupGalaxyUser
SQLNamedQuery   LookupGalaxyUser SELECT "email, (CASE WHEN
substring(password from 1 for 6) = 'PBKDF2' THEN substring(password from 38
for 69) ELSE password END) AS
password2,512,512,'/media/galaxy/galaxy/database/ftp/%U','/bin/bash' FROM
galaxy_user WHERE email='%U'"

# Define custom query to fetch the password salt
SQLNamedQuery   GetUserSalt SELECT "(CASE WHEN SUBSTRING
(password from 1 for 6) = 'PBKDF2' THEN SUBSTRING (password from 21 for 16)
END) AS salt FROM galaxy_user WHERE email='%U'"
  

# Don't use IPv6 support by default.
UseIPv6 off
MaxInstances30

# To cause every FTP user to be "jailed" (chrooted) into their home
# directory, uncomment this line.
# Bar use of SITE CHMOD by default

  DenyAll


# Bar use of RETR (download) since this is not a public file drop

  DenyAll

~
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

[galaxy-dev] galaxy kickstart failed to install tools

2018-10-31 Thread Rui Wang
Hi Folks,

I tried to install some tools into the galaxy kickstart, and had the
following issues:

1. Tons of warnings were generated:

 [WARNING]: sftp transfer mechanism failed on [localhost]. Use
ANSIBLE_DEBUG=1 to see detailed information

2. I wasn't able to install the data manager or tools. Error are like:

"stderr": "(1/3) Installing repository tophat2 from devteam to section
\"ngs-rna-tools\" at revision 5c5517d2a9e9 (TRT: 0:00:01.238735)\r\n\t*
Error installing a repository (after 0:00:00.007585 seconds)! Name:
tophat2,owner: devteam, revision: 5c5517d2a9e9, error: {\"err_msg\": \"You
are not authorized to request the latest installable revision for a
repository in this Galaxy instance.\", \"err_code\": 403006}

I tried to use different revision number, but it's the same error. I also
tried to use the tool list file that comes with kickstart, it's the same.

Not sure if anyone saw this before. Please give me a hand?

Thanks,
Rui
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

[galaxy-dev] regarding galaxy kickstart

2018-10-15 Thread Rui Wang
Hi Folks,

I spent sometime playing with the kickstart, and sort of got it working.
Still a couple things not clear:

1. Is it true that there is no start/stop? My guess is that whenever you
need to restart, just re-run the playbook? Anything you want to update(for
example add new tools), you re-run the updated playbook too, correct?
Before you re-run it, do you have to manually kill everything?

2. Seems tools could easily added by editing the tool list. How about
genome data? What if I want to add a specific repo or genome data? Is there
a role for this purpose?

Please give me a hand if you could. Any input will be greatly appreciated.
:-)

Thanks so much,
Rui
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] error and question from using galaxy kickstart

2018-10-12 Thread Rui Wang
 postgres: bioinfoadmin
galaxy 127.0.0.1(36154) idle
postgres 103700 103679  0 19:42 ?00:00:12 postgres: bioinfoadmin
galaxy 127.0.0.1(36156) idle
postgres 103701 103679  0 19:42 ?00:00:12 postgres: bioinfoadmin
galaxy 127.0.0.1(36158) idle
postgres 103702 103679  0 19:42 ?00:00:12 postgres: bioinfoadmin
galaxy 127.0.0.1(36160) idle
postgres 103703 103679  0 19:42 ?00:00:11 postgres: bioinfoadmin
galaxy 127.0.0.1(36162) idle
postgres 103704 103679  0 19:42 ?00:00:11 postgres: bioinfoadmin
galaxy 127.0.0.1(36164) idle
postgres 103705 103679  0 19:42 ?00:00:11 postgres: bioinfoadmin
galaxy 127.0.0.1(36166) idle
postgres 103706 103679  0 19:42 ?00:00:11 postgres: bioinfoadmin
galaxy 127.0.0.1(36168) idle
postgres 103707 103679  0 19:42 ?00:00:11 postgres: bioinfoadmin
galaxy 127.0.0.1(36170) idle
postgres 103708 103679  0 19:42 ?00:00:11 postgres: bioinfoadmin
galaxy 127.0.0.1(36172) idle
postgres 103709 103679  0 19:42 ?00:00:11 postgres: bioinfoadmin
galaxy 127.0.0.1(36174) idle
postgres 103710 103679  0 19:42 ?00:00:04 postgres: bioinfoadmin
galaxy 127.0.0.1(36176) idle
postgres 103711 103679  0 19:42 ?00:00:04 postgres: bioinfoadmin
galaxy 127.0.0.1(36178) idle

Is there a script to do the work?

Thanks,
Rui

On Fri, Oct 12, 2018 at 8:10 PM Rui Wang  wrote:

> Hi Folks,
>
> So I managed to ran through the ansible. What I did was to remove certain
> steps. For example, I don't need slurm-drmaa and docker, then I skipped the
> update cache. After these minor changes, it finished successfully(?) with
> an error message it ignored. Then I tried to access the UI, but nothing
> worked. I pasted it below for your reference. If anyone has seen this
> before, please help. :-)
>
> Sorry for the beginner question, so once the ansible playbook ran
> through...seems the galaxy and the postgresql etc were started. How do I
> start/stop them manually? Do I have to run the playbook every time I want
> to run galaxy or only after I modify the playbook? Also, if I don't look at
> the log, I don't even know that the UI is at 127.0.0.1:4001.  Is there
> any documentation for this?
>
> Thanks,
> Rui
>
>
> error message:
>
> galaxy.web.stack INFO 2018-10-12 19:41:53,874 [p:102477,w:1,m:0]
> [MainThread] Galaxy server instance 'main.web.1' is running
> Starting server in PID 101567.
> serving on uwsgi://127.0.0.1:4001
> galaxy.jobs.handler ERROR 2018-10-12 19:42:48,487 [p:102477,w:1,m:0]
> [JobHandlerQueue.monitor_thread] Exception in monitor_step
> Traceback (most recent call last):
>   File "lib/galaxy/jobs/handler.py", line 213, in __monitor
> self.__monitor_step()
>   File "lib/galaxy/jobs/handler.py", line 272, in __monitor_step
> .order_by(model.Job.id).all()
>   File
> "/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
> line 2737, in all
> return list(self)
>   File
> "/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
> line 2889, in __iter__
> return self._execute_and_instances(context)
>   File
> "/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
> line 2912, in _execute_and_instances
> result = conn.execute(querycontext.statement, self._params)
>   File
> "/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
> line 948, in execute
> return meth(self, multiparams, params)
>   File
> "/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py",
> line 269, in _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
>   File
> "/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
> line 1060, in _execute_clauseelement
> compiled_sql, distilled_params
>   File
> "/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
> line 1200, in _execute_context
> context)
>   File
> "/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
> line 1413, in _handle_dbapi_exception
> exc_info
>   File
> "/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py",
> line 203, in raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
>   File
> "/media/libraryfiles/bioinfoadmin/bioi

[galaxy-dev] error and question from using galaxy kickstart

2018-10-12 Thread Rui Wang
Hi Folks,

So I managed to ran through the ansible. What I did was to remove certain
steps. For example, I don't need slurm-drmaa and docker, then I skipped the
update cache. After these minor changes, it finished successfully(?) with
an error message it ignored. Then I tried to access the UI, but nothing
worked. I pasted it below for your reference. If anyone has seen this
before, please help. :-)

Sorry for the beginner question, so once the ansible playbook ran
through...seems the galaxy and the postgresql etc were started. How do I
start/stop them manually? Do I have to run the playbook every time I want
to run galaxy or only after I modify the playbook? Also, if I don't look at
the log, I don't even know that the UI is at 127.0.0.1:4001.  Is there any
documentation for this?

Thanks,
Rui


error message:

galaxy.web.stack INFO 2018-10-12 19:41:53,874 [p:102477,w:1,m:0]
[MainThread] Galaxy server instance 'main.web.1' is running
Starting server in PID 101567.
serving on uwsgi://127.0.0.1:4001
galaxy.jobs.handler ERROR 2018-10-12 19:42:48,487 [p:102477,w:1,m:0]
[JobHandlerQueue.monitor_thread] Exception in monitor_step
Traceback (most recent call last):
  File "lib/galaxy/jobs/handler.py", line 213, in __monitor
self.__monitor_step()
  File "lib/galaxy/jobs/handler.py", line 272, in __monitor_step
.order_by(model.Job.id).all()
  File
"/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
line 2737, in all
return list(self)
  File
"/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
line 2889, in __iter__
return self._execute_and_instances(context)
  File
"/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
line 2912, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
  File
"/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 948, in execute
return meth(self, multiparams, params)
  File
"/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py",
line 269, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
  File
"/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 1060, in _execute_clauseelement
compiled_sql, distilled_params
  File
"/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 1200, in _execute_context
context)
  File
"/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 1413, in _handle_dbapi_exception
exc_info
  File
"/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py",
line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File
"/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 1193, in _execute_context
context)
  File
"/media/libraryfiles/bioinfoadmin/bioinfoadmin/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py",
line 507, in do_execute
cursor.execute(statement, parameters)
OperationalError: (psycopg2.OperationalError) server closed the connection
unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
 [SQL: 'SELECT EXISTS (SELECT history_dataset_association.id,
history_dataset_association.history_id,
history_dataset_association.dataset_id,
history_dataset_association.create_time,
history_dataset_association.update_time, history_dataset_association.state,
history_dataset_association.copied_from_history_dataset_association_id,
history_dataset_association.copied_from_library_dataset_dataset_association_id,
history_dataset_association.name, history_dataset_association.info,
history_dataset_association.blurb, history_dataset_association.peek,
history_dataset_association.tool_version,
history_dataset_association.extension,
history_dataset_association.metadata,
history_dataset_association.parent_id,
history_dataset_association.designation,
history_dataset_association.deleted, history_dataset_association.visible,
history_dataset_association.extended_metadata_id,
history_dataset_association.version, history_dataset_association.hid,
history_dataset_association.purged,
history_dataset_association.hidden_beneath_collection_instance_id \nFROM
history_dataset_association, job_to_output_dataset \nWHERE job.id =
job_to_output_dataset.job_id AND history_dataset_association.id =
job_to_output_dataset.dataset_id AND history_dataset_association.deleted =
true) AS anon_1, EXISTS (SELECT history_dataset_collection_association.id
\nFROM history

Re: [galaxy-dev] can't create user on a new instance

2018-10-12 Thread Rui Wang
Hey Folks,

When I used the kickstart, I hit this always:

TASK [galaxyprojectdotorg.galaxy-os : Update APT cache]
*
 [WARNING]: sftp transfer mechanism failed on [localhost]. Use
ANSIBLE_DEBUG=1 to see detailed information

fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg":
"Failed to update apt cache."}

Have you seen this before? If you do, please give me a hand. :)

Thanks,
Rui

On Fri, Oct 12, 2018 at 2:17 PM Rui Wang  wrote:

> Hi Qiang,
>
> Let me try that. I think the postgres might be contaminated by the attempt
> to use kickstart. During that it modify the config of postgres permissions
> and it took me a while to clean those to get postgres working again(which
> may or may not really clean the change completely).
>
> I'll get back to you on this. Also I'll let you know my attempts on the
> kickstart. The problem was with the updater. :-(
>
> Thanks,
> Rui
>
> On Fri, Oct 12, 2018 at 12:23 PM Qiang Gu  wrote:
>
>> Hi Rui,
>>
>> Galaxy will use the default sqlite if you comment out the db setting the
>> config file (`config/galaxy.yml`). You mean the sqlite works the same as
>> the postgresql, which doesn't allow you to create users?
>> If that's the case, try `git clone` a new galaxy repo, in the galaxy
>> folder, `git checkout remotes/original/release_18.05`, `git checkout -b
>> release_18.05`, `./run.sh`.
>> Basically, I would like to make the sqlite working first, then switch to
>> postgresql.  The above-mentioned steps may help identify the hidden issue.
>>
>> Good luck!
>>
>> -Qiang
>>
>>
>> --
>> *From:* Rui Wang [ruiwang...@gmail.com]
>> *Sent:* Friday, October 12, 2018 1:01 AM
>> *To:* Qiang Gu
>> *Cc:* galaxy-dev@lists.galaxyproject.org
>> *Subject:* Re: [galaxy-dev] can't create user on a new instance
>>
>> Hi Qiang,
>>
>> Thanks so much for the email. The default sqlite seems to be the same. I
>> spent some time with the kickstart. That would be fantastic if it
>> works...sadly it failed on me. :-( Let me make some more effort on that.
>> Btw, do you have a better version of the instructions for the kickstart?
>>
>> Cheers,
>> Rui
>>
>> On Wed, Oct 10, 2018 at 10:12 PM Qiang Gu  wrote:
>>
>>> Hi Rui,
>>>
>>> Does the default database sqlite work for you? I guess there is certain
>>> mis-configuration in your
>>> postgresql settings or certain element in the config/galaxy.yml file is
>>> broken. Try to enter your postgresql database in terminal using the
>>> same login info and create a table to make sure you have proper privileges.
>>> Alternatively, using GalaxyKickStart (
>>> https://github.com/ARTbio/GalaxyKickStart) to deploy a server
>>> isn't that difficult and a postgresql database comes with it by default.
>>> Good luck!
>>>
>>> Thanks,
>>>
>>> -Qiang
>>>
>>> --
>>> *From:* galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on
>>> behalf of Rui Wang [ruiwang...@gmail.com]
>>> *Sent:* Wednesday, October 10, 2018 9:30 PM
>>> *To:* galaxy-dev@lists.galaxyproject.org
>>> *Subject:* Re: [galaxy-dev] can't create user on a new instance
>>>
>>> Hi Folks,
>>>
>>> I still could not create new user account with the same error message.
>>> Has anyone happened to see this before?
>>>
>>> Cheers,
>>> Rui
>>>
>>> On Tue, Oct 9, 2018 at 5:48 PM Rui Wang  wrote:
>>>
>>>> Hey Folks,
>>>>
>>>> I just installed a new 18.05 instance with a new postgresql db created.
>>>> However, the UI doesn't allow me to create any user. After I provided all
>>>> input information, it says
>>>>
>>>> galaxy.web.framework.webapp WARNING 2018-10-09 17:28:35,735
>>>> [p:32925,w:1,m:0] [uWSGIWorker1Core2] Wrong session token found, denying
>>>> request
>>>>
>>>> in the server log. Btw, I enabled 'allow to create user' and admin user
>>>> list is empty.
>>>>
>>>> I looked into it, and it's a function in
>>>> lib/galaxy/web/framework/webapp.py:
>>>>
>>>> def check_csrf_token(self):
>>>> session_csrf_token =
>>>> self.request.params.get("session_csrf_token", None)
>>>> problem = False
>>>> if not session_csrf_token:
>>>> log.warning("No session_csrf_token set, denying request.")
>>>> problem = True
>>>> elif session_csrf_token != self.session_csrf_token:
>>>> log.warning("Wrong session token found, denying request.")
>>>> problem = True
>>>>
>>>> if problem:
>>>> return self.show_warn_message("Failed to authorize action.")
>>>>
>>>> Could someone give me a hand? :-)
>>>>
>>>> Cheers,
>>>> Rui
>>>>
>>>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] can't create user on a new instance

2018-10-12 Thread Rui Wang
Hi Qiang,

I looked at the postgres, here is the ownership info:

bioinfoadmin=> \l
 List of databases
 Name |Owner | Encoding |   Collate   |Ctype|
 Access privileges
--+--+--+-+-+---
 bioinfoadmin | bioinfoadmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
 citation | bioinfoadmin | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
 postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
 template0| postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
=c/postgres  +
  |  |  | | |
postgres=CTc/postgres
 template1| postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
=c/postgres  +
  |  |  | | |
postgres=CTc/postgres
(5 rows)

Does this look okay to you? the db bioinfoadmin is the one that is set in
the galaxy config, and it's owned by
the user bioinfoadmin which is used to start galaxy.

Thanks,
Rui

On Wed, Oct 10, 2018 at 10:12 PM Qiang Gu  wrote:

> Hi Rui,
>
> Does the default database sqlite work for you? I guess there is certain
> mis-configuration in your
> postgresql settings or certain element in the config/galaxy.yml file is
> broken. Try to enter your postgresql database in terminal using the same
> login info and create a table to make sure you have proper privileges.
> Alternatively, using GalaxyKickStart (
> https://github.com/ARTbio/GalaxyKickStart) to deploy a server
> isn't that difficult and a postgresql database comes with it by default.
> Good luck!
>
> Thanks,
>
> -Qiang
>
> --
> *From:* galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf
> of Rui Wang [ruiwang...@gmail.com]
> *Sent:* Wednesday, October 10, 2018 9:30 PM
> *To:* galaxy-dev@lists.galaxyproject.org
> *Subject:* Re: [galaxy-dev] can't create user on a new instance
>
> Hi Folks,
>
> I still could not create new user account with the same error message. Has
> anyone happened to see this before?
>
> Cheers,
> Rui
>
> On Tue, Oct 9, 2018 at 5:48 PM Rui Wang  wrote:
>
>> Hey Folks,
>>
>> I just installed a new 18.05 instance with a new postgresql db created.
>> However, the UI doesn't allow me to create any user. After I provided all
>> input information, it says
>>
>> galaxy.web.framework.webapp WARNING 2018-10-09 17:28:35,735
>> [p:32925,w:1,m:0] [uWSGIWorker1Core2] Wrong session token found, denying
>> request
>>
>> in the server log. Btw, I enabled 'allow to create user' and admin user
>> list is empty.
>>
>> I looked into it, and it's a function in
>> lib/galaxy/web/framework/webapp.py:
>>
>> def check_csrf_token(self):
>> session_csrf_token =
>> self.request.params.get("session_csrf_token", None)
>> problem = False
>> if not session_csrf_token:
>> log.warning("No session_csrf_token set, denying request.")
>> problem = True
>> elif session_csrf_token != self.session_csrf_token:
>> log.warning("Wrong session token found, denying request.")
>> problem = True
>>
>> if problem:
>> return self.show_warn_message("Failed to authorize action.")
>>
>> Could someone give me a hand? :-)
>>
>> Cheers,
>> Rui
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] can't create user on a new instance

2018-10-12 Thread Rui Wang
Hi Marius,

Thanks for your suggestion! I tried it with a different browser that never
accessed galaxy, but it still gave me the same message. :-( Any thoughts?

Cheers,
Rui

On Wed, Oct 10, 2018 at 11:25 PM Marius van den Beek 
wrote:

> Hi Rui,
>
> The session token is used to prevent certain kinds of attacks. Your
> browser may still have a session token for the old instance. You can delete
> the cookies for this address in your browser and you should be issued a new
> and valid token.
>
> Hope that helps,
> Marius
>
> On Thu, Oct 11, 2018, 7:12 AM Qiang Gu  wrote:
>
>> Hi Rui,
>>
>> Does the default database sqlite work for you? I guess there is certain
>> mis-configuration in your
>> postgresql settings or certain element in the config/galaxy.yml file is
>> broken. Try to enter your postgresql database in terminal using the same
>> login info and create a table to make sure you have proper privileges.
>> Alternatively, using GalaxyKickStart (
>> https://github.com/ARTbio/GalaxyKickStart) to deploy a server
>> isn't that difficult and a postgresql database comes with it by default.
>> Good luck!
>>
>> Thanks,
>>
>> -Qiang
>>
>> --
>> *From:* galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on
>> behalf of Rui Wang [ruiwang...@gmail.com]
>> *Sent:* Wednesday, October 10, 2018 9:30 PM
>> *To:* galaxy-dev@lists.galaxyproject.org
>> *Subject:* Re: [galaxy-dev] can't create user on a new instance
>>
>> Hi Folks,
>>
>> I still could not create new user account with the same error message.
>> Has anyone happened to see this before?
>>
>> Cheers,
>> Rui
>>
>> On Tue, Oct 9, 2018 at 5:48 PM Rui Wang  wrote:
>>
>>> Hey Folks,
>>>
>>> I just installed a new 18.05 instance with a new postgresql db created.
>>> However, the UI doesn't allow me to create any user. After I provided all
>>> input information, it says
>>>
>>> galaxy.web.framework.webapp WARNING 2018-10-09 17:28:35,735
>>> [p:32925,w:1,m:0] [uWSGIWorker1Core2] Wrong session token found, denying
>>> request
>>>
>>> in the server log. Btw, I enabled 'allow to create user' and admin user
>>> list is empty.
>>>
>>> I looked into it, and it's a function in
>>> lib/galaxy/web/framework/webapp.py:
>>>
>>> def check_csrf_token(self):
>>> session_csrf_token =
>>> self.request.params.get("session_csrf_token", None)
>>> problem = False
>>> if not session_csrf_token:
>>> log.warning("No session_csrf_token set, denying request.")
>>> problem = True
>>> elif session_csrf_token != self.session_csrf_token:
>>> log.warning("Wrong session token found, denying request.")
>>> problem = True
>>>
>>> if problem:
>>> return self.show_warn_message("Failed to authorize action.")
>>>
>>> Could someone give me a hand? :-)
>>>
>>> Cheers,
>>> Rui
>>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] can't create user on a new instance

2018-10-12 Thread Rui Wang
Hi Qiang,

Thanks so much for the email. The default sqlite seems to be the same. I
spent some time with the kickstart. That would be fantastic if it
works...sadly it failed on me. :-( Let me make some more effort on that.
Btw, do you have a better version of the instructions for the kickstart?

Cheers,
Rui

On Wed, Oct 10, 2018 at 10:12 PM Qiang Gu  wrote:

> Hi Rui,
>
> Does the default database sqlite work for you? I guess there is certain
> mis-configuration in your
> postgresql settings or certain element in the config/galaxy.yml file is
> broken. Try to enter your postgresql database in terminal using the same
> login info and create a table to make sure you have proper privileges.
> Alternatively, using GalaxyKickStart (
> https://github.com/ARTbio/GalaxyKickStart) to deploy a server
> isn't that difficult and a postgresql database comes with it by default.
> Good luck!
>
> Thanks,
>
> -Qiang
>
> --
> *From:* galaxy-dev [galaxy-dev-boun...@lists.galaxyproject.org] on behalf
> of Rui Wang [ruiwang...@gmail.com]
> *Sent:* Wednesday, October 10, 2018 9:30 PM
> *To:* galaxy-dev@lists.galaxyproject.org
> *Subject:* Re: [galaxy-dev] can't create user on a new instance
>
> Hi Folks,
>
> I still could not create new user account with the same error message. Has
> anyone happened to see this before?
>
> Cheers,
> Rui
>
> On Tue, Oct 9, 2018 at 5:48 PM Rui Wang  wrote:
>
>> Hey Folks,
>>
>> I just installed a new 18.05 instance with a new postgresql db created.
>> However, the UI doesn't allow me to create any user. After I provided all
>> input information, it says
>>
>> galaxy.web.framework.webapp WARNING 2018-10-09 17:28:35,735
>> [p:32925,w:1,m:0] [uWSGIWorker1Core2] Wrong session token found, denying
>> request
>>
>> in the server log. Btw, I enabled 'allow to create user' and admin user
>> list is empty.
>>
>> I looked into it, and it's a function in
>> lib/galaxy/web/framework/webapp.py:
>>
>> def check_csrf_token(self):
>> session_csrf_token =
>> self.request.params.get("session_csrf_token", None)
>> problem = False
>> if not session_csrf_token:
>> log.warning("No session_csrf_token set, denying request.")
>> problem = True
>> elif session_csrf_token != self.session_csrf_token:
>> log.warning("Wrong session token found, denying request.")
>> problem = True
>>
>> if problem:
>> return self.show_warn_message("Failed to authorize action.")
>>
>> Could someone give me a hand? :-)
>>
>> Cheers,
>> Rui
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] can't create user on a new instance

2018-10-10 Thread Rui Wang
Hi Folks,

I still could not create new user account with the same error message. Has
anyone happened to see this before?

Cheers,
Rui

On Tue, Oct 9, 2018 at 5:48 PM Rui Wang  wrote:

> Hey Folks,
>
> I just installed a new 18.05 instance with a new postgresql db created.
> However, the UI doesn't allow me to create any user. After I provided all
> input information, it says
>
> galaxy.web.framework.webapp WARNING 2018-10-09 17:28:35,735
> [p:32925,w:1,m:0] [uWSGIWorker1Core2] Wrong session token found, denying
> request
>
> in the server log. Btw, I enabled 'allow to create user' and admin user
> list is empty.
>
> I looked into it, and it's a function in
> lib/galaxy/web/framework/webapp.py:
>
> def check_csrf_token(self):
> session_csrf_token = self.request.params.get("session_csrf_token",
> None)
> problem = False
> if not session_csrf_token:
> log.warning("No session_csrf_token set, denying request.")
> problem = True
> elif session_csrf_token != self.session_csrf_token:
> log.warning("Wrong session token found, denying request.")
> problem = True
>
> if problem:
> return self.show_warn_message("Failed to authorize action.")
>
> Could someone give me a hand? :-)
>
> Cheers,
> Rui
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

[galaxy-dev] can't create user on a new instance

2018-10-09 Thread Rui Wang
Hey Folks,

I just installed a new 18.05 instance with a new postgresql db created.
However, the UI doesn't allow me to create any user. After I provided all
input information, it says

galaxy.web.framework.webapp WARNING 2018-10-09 17:28:35,735
[p:32925,w:1,m:0] [uWSGIWorker1Core2] Wrong session token found, denying
request

in the server log. Btw, I enabled 'allow to create user' and admin user
list is empty.

I looked into it, and it's a function in lib/galaxy/web/framework/webapp.py:

def check_csrf_token(self):
session_csrf_token = self.request.params.get("session_csrf_token",
None)
problem = False
if not session_csrf_token:
log.warning("No session_csrf_token set, denying request.")
problem = True
elif session_csrf_token != self.session_csrf_token:
log.warning("Wrong session token found, denying request.")
problem = True

if problem:
return self.show_warn_message("Failed to authorize action.")

Could someone give me a hand? :-)

Cheers,
Rui
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

[galaxy-dev] Diffbind error

2017-11-05 Thread Rui Wang
Hi Folks,

I found one problem with the docker galaxy instance we have. While we are
trying to run Diffbind tool, it failed to create the env for the tool.  The
error is

Conda dependency seemingly installed but failed to build job environment.

I saw the env is defined in envs/__bioconductor-diffbind\@2.0.9, so I tried
the following:

conda list --name __bioconductor-diffbind\@2.0.9 --export > file1
conda create -y --unknown --offline  --prefix diff_env --file file1 >
/dev/null

basically I was trying to manually create the env for this tool and see if
it works. file1 was generated ok but the second command returns:

UnsatisfiableError: The following specifications were found to be in
conflict:
  - bioconductor-annotate 1.50.0 r3.3.2_1
  - krb5 1.13.2 0
Use "conda info " to see the dependencies for each package.

Have anyone seen this before?

Thanks,
Rui
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] dexseq pysam error

2017-06-21 Thread Rui Wang
Hi Folks,

I'm still having difficulty with this...just wondering if anyone has seen
this before:

For some reason, I only had one time success with installing dexseq 1.22 in
the docker galaxy, which I didn't know how it worked
out. After that, when I run biocLite("DEXSeq") again, it only installs
dexseq 1.20, or even older version. I think this has something
to do with R version and bioconductor version. However, R in the conda-env
doesn't run -- it has some dependency libraries missing
seems, libicui18n etc and I could not easily resolve that. :-(

Any input will be greatly appreciated.

Thanks,
Rui

On Tue, Jun 20, 2017 at 8:11 AM, Rui Wang  wrote:

> Hi Folks,
>
> I got the galaxy docker image and played with it. However, I keep getting
> this when I use dexseq:
>
> Fatal error: Exit code 1 ()
> Could not import pysam, which is needed to process BAM file (though
> not to process text SAM files). Please install the 'pysam' library from
> https://code.google.com/p/pysam/
>
> Looks like there is a conda env that is built on the fly and it doesn't
> have some dependencies installed. If I do the following
>
> biocLite("DEXSeq")
>
> inside the conda-env, then pysam is imported properly. Seems it installs
> the dexseq 1.22. I looked at the package in /tool_deps/_conda/__
> bioconductor-dexseq@1.20.1,
> it's 1.20.1 as the name says. I'm not sure how this could be fixed...my
> thought would be update the conda env dependency to include dexseq 1.22 not
> 1.20, but I'm not familiar with conda...Could someone give me a hand? :-)
>
> Thanks,
> Rui
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

[galaxy-dev] dexseq pysam error

2017-06-20 Thread Rui Wang
Hi Folks,

I got the galaxy docker image and played with it. However, I keep getting
this when I use dexseq:

Fatal error: Exit code 1 ()
Could not import pysam, which is needed to process BAM file (though
not to process text SAM files). Please install the 'pysam' library from
https://code.google.com/p/pysam/

Looks like there is a conda env that is built on the fly and it doesn't
have some dependencies installed. If I do the following

biocLite("DEXSeq")

inside the conda-env, then pysam is imported properly. Seems it installs
the dexseq 1.22. I looked at the package in /tool_deps/_conda/
__bioconductor-dexseq@1.20.1,
it's 1.20.1 as the name says. I'm not sure how this could be fixed...my
thought would be update the conda env dependency to include dexseq 1.22 not
1.20, but I'm not familiar with conda...Could someone give me a hand? :-)

Thanks,
Rui
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

Re: [galaxy-dev] proftpd sftp setup doesn't work

2017-03-22 Thread Rui Wang
Hi Tiago,

Thanks so much for your suggestions! Please see inline.

On Wed, Mar 22, 2017 at 9:08 AM, Tiago Macedo 
wrote:

> Hi Rui,
>
> Are you trying to set up proftpd (with sftp) to use galaxy user
> credentials read from a postgresql database? Do you have the postgresql
> part working on regular ftp?
>

Yes, that's what I'm trying to achieve. And yes, it works with regular ftp.


>
> 1- The proftpd version on ubuntu 16.04's repository works fine, you don't
> need to compile it yourself (unless you really want/have to). An apt
> install proftpd-basic proftpd-mod-pgsql would do it.
>
> 2- The user and group under which proftpd runs are defined in
> proftpd.conf. You didn't show that part of your file, but it's these two
> lines:
> Userproftpd
> Group   nogroup
>
> 3- I see you're missing:
> #Before the 
> LoadModule  mod_sql_postgres.c
> LoadModule  mod_sql_passwd.c
>

When I tried to add these two lines, it always says:

fatal: unknown configuration directive 'LoadModule'

Is there anything I'm missing?


>
> AuthPAM off
>
> #Within the 
> TLSProtocol TLSv1
>

Do we really have to specify TLS, or we don't need that?

Cheers,
Rui


>
> I suppose you can specify tls 1.1 or 1.2 here, but never really tried it.
>
> In my setup I also commented out the AuthOrder line.
>
> I guess these changes should do it.
>
>
> Best,
> Tiago
>
>
>
>
>
>
>
> On 22-03-2017 08:29, Rui Wang wrote:
>
> Hi Folks,
>
> I'm trying to set up the ftp for the local galaxy instance. I got the
> regular ftp working, but not the sftp. After a few hours attempts, I'm
> wondering what I'm missing. Here is the related info:
>
> 1. Configure the build
>
> install_user=bioinfoadmin install_group=bioinfoadmin ./configure
> --prefix=/media/libraryfiles/proftpd --enable-openssl
> --with-modules=mod_sql:mod_sql_postgres:mod_sql_passwd
> --with-modules=mod_sftp:mod_tls
> make
> make install
>
> 2. Start the server
>
> sudo sbin/proftpd --config /media/libraryfiles/proftpd/etc/proftpd.conf
> -n -d 10
>
> I notice that even though I start it using sudo, the real user that is
> running proftpd is 'nobody'.
>
> 3. Config file snippet, I inserted the following piece to the original
> config file, Add 'VirtualHost' or no doesn't change anything in result
>
> 
>
> SFTPEngine on
> Port 
> SFTPLog /var/log/proftpd/sftp.log
> AuthOrder mod_auth_unix.c mod_sql.c
>
> # Configure both the RSA and DSA host keys, using the same host key
> # files that OpenSSH uses.
> SFTPHostKey /etc/ssh/ssh_host_rsa_key
> SFTPHostKey /etc/ssh/ssh_host_dsa_key
>
> #SFTPAuthMethods publickey
> SFTPAuthMethods password
>
> # Enable compression
> SFTPCompression delayed
>
> 
>
> Now when I start the server, and then when I try to test the login, I got
> this:
>
> $ sftp -P  localhost
> packet_write_wait: Connection to 127.0.0.1 port : Broken pipe
> Couldn't read packet: Connection reset by peer
>
> Accordingly, I got the following segfault in log:
>
> 2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65: ROOT PRIVS
> at main.c:1227
> 2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65: RELINQUISH
> PRIVS at main.c:1231
> 2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65: no matching
> vhost found for 127.0.0.1#, using 'ProFTPD Default Installation'
> listening on wildcard address
> 2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
> (localhost[127.0.0.1]): ROOT PRIVS at main.c:1034
> 2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
> (localhost[127.0.0.1]): SETUP PRIVS at main.c:1039
> 2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
> (localhost[127.0.0.1]): session requested from client in unknown class
> 2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
> (localhost[127.0.0.1]): performing module session initializations
> 2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
> (localhost[127.0.0.1]): mod_cap/1.1: adding CAP_SETUID and CAP_SETGID
> capabilities
> 2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
> (localhost[127.0.0.1]): ROOT PRIVS at mod_sftp.c:1674
> 2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
> (localhost[127.0.0.1]): RELINQUISH PRIVS at mod_sftp.c:1677
> 2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
> (localhost[127.0.0.1]): ROOT PRIVS at keys.c:2001
> 2017-03-2

[galaxy-dev] proftpd sftp setup doesn't work

2017-03-22 Thread Rui Wang
Hi Folks,

I'm trying to set up the ftp for the local galaxy instance. I got the
regular ftp working, but not the sftp. After a few hours attempts, I'm
wondering what I'm missing. Here is the related info:

1. Configure the build

install_user=bioinfoadmin install_group=bioinfoadmin ./configure
--prefix=/media/libraryfiles/proftpd --enable-openssl
--with-modules=mod_sql:mod_sql_postgres:mod_sql_passwd
--with-modules=mod_sftp:mod_tls
make
make install

2. Start the server

sudo sbin/proftpd --config /media/libraryfiles/proftpd/etc/proftpd.conf -n
-d 10

I notice that even though I start it using sudo, the real user that is
running proftpd is 'nobody'.

3. Config file snippet, I inserted the following piece to the original
config file, Add 'VirtualHost' or no doesn't change anything in result



SFTPEngine on
Port 
SFTPLog /var/log/proftpd/sftp.log
AuthOrder mod_auth_unix.c mod_sql.c

# Configure both the RSA and DSA host keys, using the same host key
# files that OpenSSH uses.
SFTPHostKey /etc/ssh/ssh_host_rsa_key
SFTPHostKey /etc/ssh/ssh_host_dsa_key

#SFTPAuthMethods publickey
SFTPAuthMethods password

# Enable compression
SFTPCompression delayed



Now when I start the server, and then when I try to test the login, I got
this:

$ sftp -P  localhost
packet_write_wait: Connection to 127.0.0.1 port : Broken pipe
Couldn't read packet: Connection reset by peer

Accordingly, I got the following segfault in log:

2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65: ROOT PRIVS at
main.c:1227
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65: RELINQUISH
PRIVS at main.c:1231
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65: no matching
vhost found for 127.0.0.1#, using 'ProFTPD Default Installation'
listening on wildcard address
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): ROOT PRIVS at main.c:1034
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): SETUP PRIVS at main.c:1039
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): session requested from client in unknown class
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): performing module session initializations
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): mod_cap/1.1: adding CAP_SETUID and CAP_SETGID
capabilities
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): ROOT PRIVS at mod_sftp.c:1674
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): RELINQUISH PRIVS at mod_sftp.c:1677
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): ROOT PRIVS at keys.c:2001
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): RELINQUISH PRIVS at keys.c:2008
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): ROOT PRIVS at keys.c:703
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): RELINQUISH PRIVS at keys.c:710
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): ROOT PRIVS at keys.c:703
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): RELINQUISH PRIVS at keys.c:710
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): ROOT PRIVS at keys.c:2001
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): RELINQUISH PRIVS at keys.c:2008
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): mod_sql/4.3: defaulting to 'postgres' backend
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): mod_ident/1.0: ident lookup disabled
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): ROOT PRIVS at mod_delay.c:1756
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): RELINQUISH PRIVS at mod_delay.c:1759
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): ROOT PRIVS at mod_auth.c:140
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): opening scoreboard
'/media/libraryfiles/proftpd/var/proftpd.scoreboard'
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): RELINQUISH PRIVS at mod_auth.c:142
2017-03-22 00:51:14,786 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): AuthOrder in effect, resetting auth module order
2017-03-22 00:51:14,787 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): connected - local  : 127.0.0.1:
2017-03-22 00:51:14,787 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127.0.0.1]): connected - remote : 127.0.0.1:58014
2017-03-22 00:51:14,787 cal-hsl-65 proftpd[56922] cal-hsl-65
(localhost[127