[galaxy-dev] Defunct munge processes using Torque PBS

[Matthew Shirley]

I am fairly new to PBS management, so I can't rule out some misconfiguration, 
but I have a strange issue when running Galaxy with the PBS job runner. It 
seems that munge spawns a bunch of defunct processes after running Galaxy on my 
cluster:

`ps axjf`:

1 25992 25991 25991 ?   -1 Sl   7   8:48 python 
./scripts/paster.py serve universe_wsgi.ini --daemon
25992 26032 25991 25991 ?   -1 Z7   0:00  \_ [munge] 
25992 26034 25991 25991 ?   -1 Z7   0:00  \_ [munge] 
25992 26036 25991 25991 ?   -1 Z7   0:00  \_ [munge] 

Now, these processes are being spawned by Galaxy, and I can't figure out why. 
Can anyone provide some insight or clues about where to start debugging this? 
Thanks,

Matt

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Defunct munge processes using Torque PBS

[Matthew Shirley]

Hi Nate. I do understand that this is not a bug directly stemming from the 
Galaxy code base. Munge is really just a tool to pass user credentials between 
systems during job submission to the PBS server. Galaxy is spooling jobs 
through the PBS job runner, which presumably indirectly calls munge through 
jobs submission to the PBS server. I'm just not sure why the munge process is 
sometimes becoming corrupt. This is an issue since I rapidly reach my max 
number of threads for the Galaxy user on my head node. 

At this point I guess I'll try downloading the latest stable version of Torque 
and build RPMs. I have been using what is in EPEL for RHEL6. Thanks for the 
reply, and any other thoughts are still appreciated!

On Dec 7, 2012, at 10:51 AM, Nate Coraor 
 wrote:

> On Dec 6, 2012, at 2:34 PM, Matthew Shirley wrote:
> 
>> I am fairly new to PBS management, so I can't rule out some 
>> misconfiguration, but I have a strange issue when running Galaxy with the 
>> PBS job runner. It seems that munge spawns a bunch of defunct processes 
>> after running Galaxy on my cluster:
>> 
>> `ps axjf`:
>> 
>>1 25992 25991 25991 ?   -1 Sl   7   8:48 python 
>> ./scripts/paster.py serve universe_wsgi.ini --daemon
>> 25992 26032 25991 25991 ?   -1 Z7   0:00  \_ [munge] 
>> 
>> 25992 26034 25991 25991 ?   -1 Z7   0:00  \_ [munge] 
>> 
>> 25992 26036 25991 25991 ?   -1 Z7   0:00  \_ [munge] 
>> 
>> 
>> Now, these processes are being spawned by Galaxy, and I can't figure out 
>> why. Can anyone provide some insight or clues about where to start debugging 
>> this? Thanks,
> 
> Hi Matt,
> 
> I'm not sure what munge is, it's not something provided with Galaxy.  
> Googling suggests it might be an authentication tool used in some HPC 
> environments.  Without having any familiarity with it, I can't say what 
> process in Galaxy would be interacting with it, especially since that 
> interaction must occur implicitly somewhere down the chain of normal Galaxy 
> operations.
> 
> --nate
> 
>> 
>> Matt
>> 
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>> 
>> http://lists.bx.psu.edu/
> 
> 



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] Galaxy Main cloudlaunch stall

[Matthew Shirley]

I've searched the list to see if anyone specifically mentions not being able to 
use the "cloud launch" feature on the 
main public instance, but no one seems to address it. I have no trouble 
launching Galaxy-cloudman using an AMI and pasting in my own user data. 
However, I can't successfully use the cloud launch page to do the same. I've 
tried letting it create a new security group as well as using a pre-existing 
security group. Also, I've then deleted all security groups to see if it would 
create one, which it did not. Similarly, I tried creating a new access key 
pair, with no further success. I make it through the process of entering the 
key ID and secret key, it pulls existing security group information into the 
drop-down selector, and I give a cluster name and password and submit the form. 
Then it just stalls forever. No new security groups and no EC2 instances 
launched. What is the state of this tool? Thanks.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy Main cloudlaunch stall

[Matthew Shirley]

Thanks for the response. I'll try again today. The browser I'm using is Safari 
6.0.2 on Mac OS 10.8.2. 

On Jan 18, 2013, at 10:17 AM, Dannon Baker 
 wrote:

> Cloudlaunch on the main public instance is fully supported and should work 
> fine -- I use it regularly for launching instances without issue.
> 
> There was a brief EC2 API outage yesterday (on the Amazon end) that caused 
> intermittent errors to all users of the API (including cloudlaunch), but that 
> should be resolved now.
> 
> Creating security groups and selecting (or creating, if necessary) an access 
> key should be possible -- since it's not even getting that far I'm wondering 
> if there isn't a Javascript error or something preventing the form 
> submission.  What browser are you using, and do you see any errors? 
> 
> -Dannon
> 
> On Jan 17, 2013, at 5:11 PM, Matthew Shirley  wrote:
> 
>> I've searched the list to see if anyone specifically mentions not being able 
>> to use the "cloud launch" feature on the main public instance, but no one 
>> seems to address it. I have no trouble launching Galaxy-cloudman using an 
>> AMI and pasting in my own user data. However, I can't successfully use the 
>> cloud launch page to do the same. I've tried letting it create a new 
>> security group as well as using a pre-existing security group. Also, I've 
>> then deleted all security groups to see if it would create one, which it did 
>> not. Similarly, I tried creating a new access key pair, with no further 
>> success. I make it through the process of entering the key ID and secret 
>> key, it pulls existing security group information into the drop-down 
>> selector, and I give a cluster name and password and submit the form. Then 
>> it just stalls forever. No new security groups and no EC2 instances 
>> launched. What is the state of this tool? Thanks.
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>> 
>> http://lists.bx.psu.edu/
> 
> 



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy Main cloudlaunch stall

[Matthew Shirley]

Thanks. The requests will be coming from kennedykrieger.org.

On Jan 18, 2013, at 12:36 PM, Dannon Baker 
 wrote:

> Should definitely be stable enough to use in a classroom and that it isn't 
> working for you is definitely unexpected; it's the only way I've launched 
> instances for many months now.  I'm digging through logs to see if I can find 
> any indication of why your particular requests are having issues.
> 
> 
> 
> On Jan 18, 2013, at 12:33 PM, Matthew Shirley  wrote:
> 
>> Thanks, Quang. That's not exactly what I'm looking for. My motivation is to 
>> teach an entire class how to start a Galaxy cluster on their own, with 
>> minimal knowledge of AWS or systems administration. The "cloudlaunch" 
>> feature on the main psx site seems to fit this purpose exactly, but does not 
>> work reliably for me. I've just tried using Chrome 24.0 with no success. I'm 
>> using no browser plugins, and this is a clean installation with javascript 
>> enabled. The cloudlaunch page just sits there and tells me "Launch Pending, 
>> please be patient." and I have - it's been over an hour. Any help figuring 
>> this out is appreciated. If this feature is just not mature enough to use in 
>> a classroom, that would be an acceptable answer as well.
>> 
>> 
>> On Jan 18, 2013, at 11:40 AM, Quang Trinh  
>> wrote:
>> 
>>> Hi Matthew,
>>> Would you like to try out what we put together for modENCODE DCC?
>>> Please see the README file in the docs folder at
>>> 
>>> https://github.com/modENCODE-DCC/Galaxy
>>> 
>>> Thanks,
>>> 
>>> Q
>>> 
>>> On Fri, Jan 18, 2013 at 10:17 AM, Dannon Baker  wrote:
>>>> Cloudlaunch on the main public instance is fully supported and should work 
>>>> fine -- I use it regularly for launching instances without issue.
>>>> 
>>>> There was a brief EC2 API outage yesterday (on the Amazon end) that caused 
>>>> intermittent errors to all users of the API (including cloudlaunch), but 
>>>> that should be resolved now.
>>>> 
>>>> Creating security groups and selecting (or creating, if necessary) an 
>>>> access key should be possible -- since it's not even getting that far I'm 
>>>> wondering if there isn't a Javascript error or something preventing the 
>>>> form submission.  What browser are you using, and do you see any errors?
>>>> 
>>>> -Dannon
>>>> 
>>>> On Jan 17, 2013, at 5:11 PM, Matthew Shirley  wrote:
>>>> 
>>>>> I've searched the list to see if anyone specifically mentions not being 
>>>>> able to use the "cloud launch" feature on the main public instance, but 
>>>>> no one seems to address it. I have no trouble launching Galaxy-cloudman 
>>>>> using an AMI and pasting in my own user data. However, I can't 
>>>>> successfully use the cloud launch page to do the same. I've tried letting 
>>>>> it create a new security group as well as using a pre-existing security 
>>>>> group. Also, I've then deleted all security groups to see if it would 
>>>>> create one, which it did not. Similarly, I tried creating a new access 
>>>>> key pair, with no further success. I make it through the process of 
>>>>> entering the key ID and secret key, it pulls existing security group 
>>>>> information into the drop-down selector, and I give a cluster name and 
>>>>> password and submit the form. Then it just stalls forever. No new 
>>>>> security groups and no EC2 instances launched. What is the state of this 
>>>>> tool? Thanks.
>>>>> ___
>>>>> Please keep all replies on the list by using "reply all"
>>>>> in your mail client.  To manage your subscriptions to this
>>>>> and other Galaxy lists, please use the interface at:
>>>>> 
>>>>> http://lists.bx.psu.edu/
>>>> 
>>>> ___
>>>> Please keep all replies on the list by using "reply all"
>>>> in your mail client.  To manage your subscriptions to this
>>>> and other Galaxy lists, please use the interface at:
>>>> 
>>>> http://lists.bx.psu.edu/
>>> 
>> 
> 
> 



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy Main cloudlaunch stall

[Matthew Shirley]

Thanks, Quang. That's not exactly what I'm looking for. My motivation is to 
teach an entire class how to start a Galaxy cluster on their own, with minimal 
knowledge of AWS or systems administration. The "cloudlaunch" feature on the 
main psx site seems to fit this purpose exactly, but does not work reliably for 
me. I've just tried using Chrome 24.0 with no success. I'm using no browser 
plugins, and this is a clean installation with javascript enabled. The 
cloudlaunch page just sits there and tells me "Launch Pending, please be 
patient." and I have - it's been over an hour. Any help figuring this out is 
appreciated. If this feature is just not mature enough to use in a classroom, 
that would be an acceptable answer as well.


On Jan 18, 2013, at 11:40 AM, Quang Trinh 
mailto:quang.tr...@gmail.com>>
 wrote:

Hi Matthew,
 Would you like to try out what we put together for modENCODE DCC?
Please see the README file in the docs folder at

https://github.com/modENCODE-DCC/Galaxy

Thanks,

Q

On Fri, Jan 18, 2013 at 10:17 AM, Dannon Baker  wrote:
Cloudlaunch on the main public instance is fully supported and should work fine 
-- I use it regularly for launching instances without issue.

There was a brief EC2 API outage yesterday (on the Amazon end) that caused 
intermittent errors to all users of the API (including cloudlaunch), but that 
should be resolved now.

Creating security groups and selecting (or creating, if necessary) an access 
key should be possible -- since it's not even getting that far I'm wondering if 
there isn't a Javascript error or something preventing the form submission.  
What browser are you using, and do you see any errors?

-Dannon

On Jan 17, 2013, at 5:11 PM, Matthew Shirley  wrote:

I've searched the list to see if anyone specifically mentions not being able to 
use the "cloud launch" feature on the main public instance, but no one seems to 
address it. I have no trouble launching Galaxy-cloudman using an AMI and 
pasting in my own user data. However, I can't successfully use the cloud launch 
page to do the same. I've tried letting it create a new security group as well 
as using a pre-existing security group. Also, I've then deleted all security 
groups to see if it would create one, which it did not. Similarly, I tried 
creating a new access key pair, with no further success. I make it through the 
process of entering the key ID and secret key, it pulls existing security group 
information into the drop-down selector, and I give a cluster name and password 
and submit the form. Then it just stalls forever. No new security groups and no 
EC2 instances launched. What is the state of this tool? Thanks.
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/