Re: [galaxy-dev] CloudMan Error
HI - did you find anything with the share string? Thanks, Deniz On Fri, Jul 26, 2013 at 3:32 PM, Dannon Baker wrote: > Ahh, ok, I misunderstood and thought you were saying there was a different > error for the workaround. Are you able to send me the share string > (directly if you want, feel free to drop the list) so that I might take a > look? There should be no change at all in your instances from before the > new release when using the workaround. > > > On Fri, Jul 26, 2013 at 8:17 AM, Deniz Erezyilmaz wrote: > >> Dannon- the problems that I saw were with the workaround (see last post). >> The instance launches, but the program stops when it can't find something >> (module/dependencies). >> >> There were also some different steps in setting up the instance. I had to >> create my own key pair, whereas in the past I never had to do that. I guess >> that simply using the share string that Greg at Janelia created made this >> step automatic. >> >> The cluster was working well at the end of June. The problem arose >> suddenly Saturday June 29, when the share string would not load. Thanks >> again, Deniz >> >> >> >> On Thu, Jul 25, 2013 at 5:19 PM, Dannon Baker wrote: >> >>> On Thu, Jul 25, 2013 at 4:47 PM, Deniz Erezyilmaz wrote: >>> >>> The workaround doesn't seem to work >>> >>> Can you tell me what happens when you launch with the workaround >>> mentioned previously ( >>> https://main.g2.bx.psu.edu/cloudlaunch?ami=ami-da58aab3&bucket_default=gxy-workshop)? >>> >>> >>> This should launch the pre-migration AMI and Cloudman and shouldn't have >>> any issues at all unless your cluster was already wonky. >>> >>> -Dannon >>> >> >> > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
Yes- I know that it loads in the workaround that you created, but it doesn't run. The non-workaround version running fine until the last Saturday of June. I know that Greg, who installed it, hasn't touched it in months. Could this be related to the update that was recently completed? That was supposed to be fixed last week. Is there any news? Thanks, Deniz On Tue, Jul 30, 2013 at 11:35 AM, Dannon Baker wrote: > Yes, I tried it and everything appeared to load fine using the url > indicated above. The data volume was mounted correctly, and it appears the > custom installation is in place though I would have no idea what it is or > does. > > Did you install all the custom stuff originally? If not, it may be useful > to talk to the person that did to find out if they're able to poke at it. > That file that's indicated as missing (csamtools.so) is actually there if > you take a look, and I'd expect it should all work if initialized correctly. > > > On Tue, Jul 30, 2013 at 10:10 AM, Deniz Erezyilmaz wrote: > >> HI - did you find anything with the share string? Thanks, Deniz >> >> >> On Fri, Jul 26, 2013 at 3:32 PM, Dannon Baker wrote: >> >>> Ahh, ok, I misunderstood and thought you were saying there was a >>> different error for the workaround. Are you able to send me the share >>> string (directly if you want, feel free to drop the list) so that I might >>> take a look? There should be no change at all in your instances from >>> before the new release when using the workaround. >>> >>> >>> On Fri, Jul 26, 2013 at 8:17 AM, Deniz Erezyilmaz wrote: >>> Dannon- the problems that I saw were with the workaround (see last post). The instance launches, but the program stops when it can't find something (module/dependencies). There were also some different steps in setting up the instance. I had to create my own key pair, whereas in the past I never had to do that. I guess that simply using the share string that Greg at Janelia created made this step automatic. The cluster was working well at the end of June. The problem arose suddenly Saturday June 29, when the share string would not load. Thanks again, Deniz On Thu, Jul 25, 2013 at 5:19 PM, Dannon Baker wrote: > On Thu, Jul 25, 2013 at 4:47 PM, Deniz Erezyilmaz > wrote: > > The workaround doesn't seem to work >> > > Can you tell me what happens when you launch with the workaround > mentioned previously ( > https://main.g2.bx.psu.edu/cloudlaunch?ami=ami-da58aab3&bucket_default=gxy-workshop)? > > > This should launch the pre-migration AMI and Cloudman and shouldn't > have any issues at all unless your cluster was already wonky. > > -Dannon > >>> >> > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
Yes, I tried it and everything appeared to load fine using the url indicated above. The data volume was mounted correctly, and it appears the custom installation is in place though I would have no idea what it is or does. Did you install all the custom stuff originally? If not, it may be useful to talk to the person that did to find out if they're able to poke at it. That file that's indicated as missing (csamtools.so) is actually there if you take a look, and I'd expect it should all work if initialized correctly. On Tue, Jul 30, 2013 at 10:10 AM, Deniz Erezyilmaz wrote: > HI - did you find anything with the share string? Thanks, Deniz > > > On Fri, Jul 26, 2013 at 3:32 PM, Dannon Baker wrote: > >> Ahh, ok, I misunderstood and thought you were saying there was a >> different error for the workaround. Are you able to send me the share >> string (directly if you want, feel free to drop the list) so that I might >> take a look? There should be no change at all in your instances from >> before the new release when using the workaround. >> >> >> On Fri, Jul 26, 2013 at 8:17 AM, Deniz Erezyilmaz wrote: >> >>> Dannon- the problems that I saw were with the workaround (see last >>> post). The instance launches, but the program stops when it can't find >>> something (module/dependencies). >>> >>> There were also some different steps in setting up the instance. I had >>> to create my own key pair, whereas in the past I never had to do that. I >>> guess that simply using the share string that Greg at Janelia created made >>> this step automatic. >>> >>> The cluster was working well at the end of June. The problem arose >>> suddenly Saturday June 29, when the share string would not load. Thanks >>> again, Deniz >>> >>> >>> >>> On Thu, Jul 25, 2013 at 5:19 PM, Dannon Baker wrote: >>> On Thu, Jul 25, 2013 at 4:47 PM, Deniz Erezyilmaz wrote: The workaround doesn't seem to work > Can you tell me what happens when you launch with the workaround mentioned previously ( https://main.g2.bx.psu.edu/cloudlaunch?ami=ami-da58aab3&bucket_default=gxy-workshop)? This should launch the pre-migration AMI and Cloudman and shouldn't have any issues at all unless your cluster was already wonky. -Dannon >>> >>> >> > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
Ahh, ok, I misunderstood and thought you were saying there was a different error for the workaround. Are you able to send me the share string (directly if you want, feel free to drop the list) so that I might take a look? There should be no change at all in your instances from before the new release when using the workaround. On Fri, Jul 26, 2013 at 8:17 AM, Deniz Erezyilmaz wrote: > Dannon- the problems that I saw were with the workaround (see last post). > The instance launches, but the program stops when it can't find something > (module/dependencies). > > There were also some different steps in setting up the instance. I had to > create my own key pair, whereas in the past I never had to do that. I guess > that simply using the share string that Greg at Janelia created made this > step automatic. > > The cluster was working well at the end of June. The problem arose > suddenly Saturday June 29, when the share string would not load. Thanks > again, Deniz > > > > On Thu, Jul 25, 2013 at 5:19 PM, Dannon Baker wrote: > >> On Thu, Jul 25, 2013 at 4:47 PM, Deniz Erezyilmaz wrote: >> >> The workaround doesn't seem to work >>> >> >> Can you tell me what happens when you launch with the workaround >> mentioned previously ( >> https://main.g2.bx.psu.edu/cloudlaunch?ami=ami-da58aab3&bucket_default=gxy-workshop)? >> >> >> This should launch the pre-migration AMI and Cloudman and shouldn't have >> any issues at all unless your cluster was already wonky. >> >> -Dannon >> > > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
Dannon- the problems that I saw were with the workaround (see last post). The instance launches, but the program stops when it can't find something (module/dependencies). There were also some different steps in setting up the instance. I had to create my own key pair, whereas in the past I never had to do that. I guess that simply using the share string that Greg at Janelia created made this step automatic. The cluster was working well at the end of June. The problem arose suddenly Saturday June 29, when the share string would not load. Thanks again, Deniz On Thu, Jul 25, 2013 at 5:19 PM, Dannon Baker wrote: > On Thu, Jul 25, 2013 at 4:47 PM, Deniz Erezyilmaz wrote: > > The workaround doesn't seem to work >> > > Can you tell me what happens when you launch with the workaround mentioned > previously ( > https://main.g2.bx.psu.edu/cloudlaunch?ami=ami-da58aab3&bucket_default=gxy-workshop)? > > > This should launch the pre-migration AMI and Cloudman and shouldn't have > any issues at all unless your cluster was already wonky. > > -Dannon > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
Hi- I'm wondering if the share string problem will be fixed this week. The workaround doesn't seem to work and I have needed to analyze my data for almost a month. When I try to start the run: ubuntu@ip-10-218-95-180:/mnt/galaxyData/custom/MY_MSG_RUN$ perl msg/msgCluster.pl I get: started 07/24/2013 21:16:27 ./test_dependencies.sh All required executables found in PATH All required R packages are installed Required module not found: /mnt/galaxyData/custom/bin/python_libs/lib/python/pysam-0.6-py2.7-linux-x86_64.egg/csamtools.so: undefined symbol: PyCapsule_New Error in ./test_dependencies.sh: 256 at msg/Utils.pm line 25. I am guessing that the workaround is missing something python. Thanks, Deniz On Mon, Jul 22, 2013 at 5:04 PM, Dannon Baker wrote: > The warning message printed by the first ssh attempt indicates that you > have no cloudman_key_pair.pem in the directory you executed the ssh command > from. Find this file (or create a new one) and you'll be able to ssh. > > Good luck, and please keep threads on the list instead of emailing > directly. Thanks! > > -Dannon > > On Mon, Jul 22, 2013 at 1:36 PM, wrote: > >> Hi- the share string appears to be loading, but I am unable to ssh into >> my instance using the provided command, or anything like it. Instead I get >> the following: >> >> denizerezyilmaz$ ssh -i cloudman_key_pair.pem >> ubu...@ec2-54-242-61-164.compute-1.amazonaws.com >> Warning: Identity file cloudman_key_pair.pem not accessible: No such file >> or directory. >> >> OR: >> >> denizerezyilmaz$ ssh ec2-54-242-61-164.compute-1.amazonaws.com >> Permission denied (publickey). >> >> What am I missing? Thank you, Deniz >> > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
On Thu, Jul 25, 2013 at 4:47 PM, Deniz Erezyilmaz wrote: The workaround doesn't seem to work > Can you tell me what happens when you launch with the workaround mentioned previously ( https://main.g2.bx.psu.edu/cloudlaunch?ami=ami-da58aab3&bucket_default=gxy-workshop)? This should launch the pre-migration AMI and Cloudman and shouldn't have any issues at all unless your cluster was already wonky. -Dannon ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
Hi- there is a new problem: started 07/24/2013 21:16:27 ./test_dependencies.sh All required executables found in PATH All required R packages are installed Required module not found: /mnt/galaxyData/custom/bin/python_libs/lib/python/pysam-0.6-py2.7-linux-x86_64.egg/csamtools.so: undefined symbol: PyCapsule_New Error in ./test_dependencies.sh: 256 at msg/Utils.pm line 25. On Mon, Jul 22, 2013 at 5:04 PM, Dannon Baker wrote: > The warning message printed by the first ssh attempt indicates that you > have no cloudman_key_pair.pem in the directory you executed the ssh command > from. Find this file (or create a new one) and you'll be able to ssh. > > Good luck, and please keep threads on the list instead of emailing > directly. Thanks! > > -Dannon > > On Mon, Jul 22, 2013 at 1:36 PM, wrote: > >> Hi- the share string appears to be loading, but I am unable to ssh into >> my instance using the provided command, or anything like it. Instead I get >> the following: >> >> denizerezyilmaz$ ssh -i cloudman_key_pair.pem >> ubu...@ec2-54-242-61-164.compute-1.amazonaws.com >> Warning: Identity file cloudman_key_pair.pem not accessible: No such file >> or directory. >> >> OR: >> >> denizerezyilmaz$ ssh ec2-54-242-61-164.compute-1.amazonaws.com >> Permission denied (publickey). >> >> What am I missing? Thank you, Deniz >> > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
Hi- while I was able to solve the share string problem, the share string has stopped working again. In addition, I am unable to increase disk size either through the CloudMan console or a dialog box that appears before the cloudman console. Below are the cluster status logs. The Disk status remains at 0/0 (0%) no matter what I do. So far I've waited half an hour for the string to load. Has the workaround been taken apart? - 15:14:12 - Master starting - 15:14:14 - Completed initial cluster configuration. This seems to be a new cluster; waiting to configure the type - 15:14:18 - SGE prerequisites OK; starting the service - 15:14:22 - Configuring SGE... - 15:14:30 - Successfully setup SGE; configuring SGE - 15:14:38 - Saved file 'persistent_data.yaml' to bucket 'cm-c8e9b0bd6c2cb077aa1640d64187f582' - 15:14:39 - Saved file 'cm_boot.py' to bucket 'cm-c8e9b0bd6c2cb077aa1640d64187f582' - 15:14:40 - Saved file 'cm.tar.gz' to bucket 'cm-c8e9b0bd6c2cb077aa1640d64187f582' - 15:14:40 - Saved file 'Kenchoice.clusterName' to bucket 'cm-c8e9b0bd6c2cb077aa1640d64187f582' - 15:17:05 - Initializing a 'Galaxy' cluster. - 15:17:06 - Retrieved file 'snaps.yaml' from bucket 'gxy-workshop' to 'cm_snaps.yaml'. - 15:17:09 - Saved file 'persistent_data.yaml' to bucket 'cm-c8e9b0bd6c2cb077aa1640d64187f582' On Mon, Jul 22, 2013 at 5:04 PM, Dannon Baker wrote: > The warning message printed by the first ssh attempt indicates that you > have no cloudman_key_pair.pem in the directory you executed the ssh command > from. Find this file (or create a new one) and you'll be able to ssh. > > Good luck, and please keep threads on the list instead of emailing > directly. Thanks! > > -Dannon > > On Mon, Jul 22, 2013 at 1:36 PM, wrote: > >> Hi- the share string appears to be loading, but I am unable to ssh into >> my instance using the provided command, or anything like it. Instead I get >> the following: >> >> denizerezyilmaz$ ssh -i cloudman_key_pair.pem >> ubu...@ec2-54-242-61-164.compute-1.amazonaws.com >> Warning: Identity file cloudman_key_pair.pem not accessible: No such file >> or directory. >> >> OR: >> >> denizerezyilmaz$ ssh ec2-54-242-61-164.compute-1.amazonaws.com >> Permission denied (publickey). >> >> What am I missing? Thank you, Deniz >> > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
The warning message printed by the first ssh attempt indicates that you have no cloudman_key_pair.pem in the directory you executed the ssh command from. Find this file (or create a new one) and you'll be able to ssh. Good luck, and please keep threads on the list instead of emailing directly. Thanks! -Dannon On Mon, Jul 22, 2013 at 1:36 PM, wrote: > Hi- the share string appears to be loading, but I am unable to ssh into my > instance using the provided command, or anything like it. Instead I get the > following: > > denizerezyilmaz$ ssh -i cloudman_key_pair.pem > ubu...@ec2-54-242-61-164.compute-1.amazonaws.com > Warning: Identity file cloudman_key_pair.pem not accessible: No such file > or directory. > > OR: > > denizerezyilmaz$ ssh ec2-54-242-61-164.compute-1.amazonaws.com > Permission denied (publickey). > > What am I missing? Thank you, Deniz > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
Thanks guys! I'm going to try it out now.
Would you mind letting me know when the issues are fixed next week?
(updating this thread would be fine)
Thanks again,
Greg
On Tue, Jul 16, 2013 at 12:42 PM, Dannon Baker wrote:
> Hey Greg,
>
> We put together a quick workaround until we're able to resolve the
> underlying issues. You can launch the previous incarnation of galaxy
> (pre-update AMI and Cloudman versions) using
> https://main.g2.bx.psu.edu/cloudlaunch?ami=ami-da58aab3&bucket_default=gxy-workshop
>
> Once your instance is up, enter your share string like usual and it'll work
> fine. We expect to fix these issues next week, but this should get your
> instance back up and running for now.
>
>
> On Mon, Jul 15, 2013 at 10:19 AM, greg wrote:
>>
>> Thanks for getting back to me, Enis.
>>
>> I went ahead and started a new cluster instance. I'll try to leave it
>> running today in case there's anything we want to check.
>>
>> Here's my whole process:
>>
>> Start Screen Setup:
>> --
>> http://snag.gy/DMIeC.jpg
>>
>> Entering my share string:
>>
>> http://snag.gy/wKFLy.jpg
>>
>>
>> Main Page Text and log:
>>
>> Cluster name:
>>
>> MSGGREG
>>
>> Disk status:
>>
>> 0 / 0 (0%)
>>
>> Worker status:
>>
>> Idle: 0 Available: 0 Requested: 0
>>
>> Service status:
>>
>> Applications
>>
>> Data
>>
>>
>> Cluster status log
>>
>> 14:08:32 - Master starting
>> 14:08:34 - Completed the initial cluster startup process. This is a
>> new cluster; waiting to configure the type.
>> 14:08:50 - Migration service prerequisites OK; starting the service
>> 14:08:50 - SGE service prerequisites OK; starting the service
>> 14:08:58 - Setting up SGE...
>> 14:09:13 - HTCondor service prerequisites OK; starting the service
>> 14:09:21 - Hadoop service prerequisites OK; starting the service
>> 14:09:38 - Done adding Hadoop service; service running.
>> 14:11:53 - Error creating volume from shared cluster's snapshot
>> '['snap-cfa775ba']': 'filesystems'
>>
>>
>>
>> Admin Page CloudMan Log: (unfortunately nothing is jumping out at me?)
>> ---
>>
>> CloudMan from Galaxy
>> Admin | Report bugs | Wiki | Screencast
>> The entire log file (paster.log) is shown. Show latest | Back to admin
>> view
>> Python version: (2, 7)
>> Image configuration suports: {'apps': ['cloudman', 'galaxy']}
>> 2013-07-15 14:08:32,406 DEBUGapp:68 Initializing app
>> 2013-07-15 14:08:32,407 DEBUGec2:121 Gathering instance
>> zone, attempt 0
>> 2013-07-15 14:08:32,410 DEBUGec2:127 Instance zone is
>> 'us-east-1d'
>> 2013-07-15 14:08:32,410 DEBUGec2:45 Gathering instance
>> ami, attempt 0
>> 2013-07-15 14:08:32,412 DEBUGapp:71 Running on 'ec2'
>> type of cloud in zone 'us-east-1d' using image 'ami-118bfc78'.
>> 2013-07-15 14:08:32,412 DEBUGapp:89 Getting pd.yaml
>> 2013-07-15 14:08:32,412 DEBUGec2:338 No S3 Connection,
>> creating a new one.
>> 2013-07-15 14:08:32,413 DEBUGec2:342 Got boto S3 connection.
>> 2013-07-15 14:08:32,452 DEBUG misc:212 Checking if bucket
>> 'cm-0479bd75a331acc874033e98b2e1e03e' exists... it does not.
>> 2013-07-15 14:08:32,452 DEBUG misc:583 Bucket
>> 'cm-0479bd75a331acc874033e98b2e1e03e' does not exist, did not get
>> remote file 'persistent_data.yaml'
>> 2013-07-15 14:08:32,452 DEBUGapp:96 Setting
>> deployment_version to 2
>> 2013-07-15 14:08:32,453 INFO app:103 Master starting
>> 2013-07-15 14:08:32,453 DEBUG master:55 Initializing console
>> manager - cluster start time: 2013-07-15 14:08:32.453182
>> 2013-07-15 14:08:32,453 DEBUG comm:42 AMQP Connection
>> Failure: [Errno 111] Connection refused
>> 2013-07-15 14:08:32,453 DEBUG master:791 Trying to discover
>> any worker instances associated with this cluster...
>> 2013-07-15 14:08:32,454 DEBUGec2:317 Establishing boto
>> EC2 connection
>> 2013-07-15 14:08:32,535 DEBUGec2:305 Got region as
>> 'RegionInfo:us-east-1'
>> 2013-07-15 14:08:32,777 DEBUGec2:326 Got boto EC2
>> connection for region 'us-east-1'
>> 2013-07-15 14:08:33,022 DEBUG misc:574 Retrieved file
>> 'snaps.yaml' from bucket 'cloudman' on host 's3.amazonaws.com' to
>> 'cm_snaps.yaml'.
>> 2013-07-15 14:08:33,035 DEBUGec2:286 Got region name as
>> 'us-east-1'
>> 2013-07-15 14:08:33,035 DEBUG master:226 Loaded default
>> snapshot data: [{'snap_id': 'snap-adad90fc', 'name': 'galaxy',
>> 'roles': 'galaxyTools,galaxyData'}, {'snap_id': 'snap-5b030634',
>> 'name': 'galaxyIndices', 'roles': 'galaxyIndices'}]
>> 2013-07-15 14:08:33,035 DEBUGec2:81 Gathering instance
>> id, attempt 0
>> 2013-07-15 14:08:33,037 DEBUGec2:87 Instance ID is
>> 'i-5346d733'
>> 2013-07-15 14:08:33,125 DEBUG
Re: [galaxy-dev] CloudMan Error
Thanks for getting back to me, Enis.
I went ahead and started a new cluster instance. I'll try to leave it
running today in case there's anything we want to check.
Here's my whole process:
Start Screen Setup:
--
http://snag.gy/DMIeC.jpg
Entering my share string:
http://snag.gy/wKFLy.jpg
Main Page Text and log:
Cluster name:
MSGGREG
Disk status:
0 / 0 (0%)
Worker status:
Idle: 0 Available: 0 Requested: 0
Service status:
Applications
Data
Cluster status log
14:08:32 - Master starting
14:08:34 - Completed the initial cluster startup process. This is a
new cluster; waiting to configure the type.
14:08:50 - Migration service prerequisites OK; starting the service
14:08:50 - SGE service prerequisites OK; starting the service
14:08:58 - Setting up SGE...
14:09:13 - HTCondor service prerequisites OK; starting the service
14:09:21 - Hadoop service prerequisites OK; starting the service
14:09:38 - Done adding Hadoop service; service running.
14:11:53 - Error creating volume from shared cluster's snapshot
'['snap-cfa775ba']': 'filesystems'
Admin Page CloudMan Log: (unfortunately nothing is jumping out at me?)
---
CloudMan from Galaxy
Admin | Report bugs | Wiki | Screencast
The entire log file (paster.log) is shown. Show latest | Back to admin view
Python version: (2, 7)
Image configuration suports: {'apps': ['cloudman', 'galaxy']}
2013-07-15 14:08:32,406 DEBUGapp:68 Initializing app
2013-07-15 14:08:32,407 DEBUGec2:121 Gathering instance
zone, attempt 0
2013-07-15 14:08:32,410 DEBUGec2:127 Instance zone is 'us-east-1d'
2013-07-15 14:08:32,410 DEBUGec2:45 Gathering instance
ami, attempt 0
2013-07-15 14:08:32,412 DEBUGapp:71 Running on 'ec2'
type of cloud in zone 'us-east-1d' using image 'ami-118bfc78'.
2013-07-15 14:08:32,412 DEBUGapp:89 Getting pd.yaml
2013-07-15 14:08:32,412 DEBUGec2:338 No S3 Connection,
creating a new one.
2013-07-15 14:08:32,413 DEBUGec2:342 Got boto S3 connection.
2013-07-15 14:08:32,452 DEBUG misc:212 Checking if bucket
'cm-0479bd75a331acc874033e98b2e1e03e' exists... it does not.
2013-07-15 14:08:32,452 DEBUG misc:583 Bucket
'cm-0479bd75a331acc874033e98b2e1e03e' does not exist, did not get
remote file 'persistent_data.yaml'
2013-07-15 14:08:32,452 DEBUGapp:96 Setting
deployment_version to 2
2013-07-15 14:08:32,453 INFO app:103 Master starting
2013-07-15 14:08:32,453 DEBUG master:55 Initializing console
manager - cluster start time: 2013-07-15 14:08:32.453182
2013-07-15 14:08:32,453 DEBUG comm:42 AMQP Connection
Failure: [Errno 111] Connection refused
2013-07-15 14:08:32,453 DEBUG master:791 Trying to discover
any worker instances associated with this cluster...
2013-07-15 14:08:32,454 DEBUGec2:317 Establishing boto
EC2 connection
2013-07-15 14:08:32,535 DEBUGec2:305 Got region as
'RegionInfo:us-east-1'
2013-07-15 14:08:32,777 DEBUGec2:326 Got boto EC2
connection for region 'us-east-1'
2013-07-15 14:08:33,022 DEBUG misc:574 Retrieved file
'snaps.yaml' from bucket 'cloudman' on host 's3.amazonaws.com' to
'cm_snaps.yaml'.
2013-07-15 14:08:33,035 DEBUGec2:286 Got region name as 'us-east-1'
2013-07-15 14:08:33,035 DEBUG master:226 Loaded default
snapshot data: [{'snap_id': 'snap-adad90fc', 'name': 'galaxy',
'roles': 'galaxyTools,galaxyData'}, {'snap_id': 'snap-5b030634',
'name': 'galaxyIndices', 'roles': 'galaxyIndices'}]
2013-07-15 14:08:33,035 DEBUGec2:81 Gathering instance
id, attempt 0
2013-07-15 14:08:33,037 DEBUGec2:87 Instance ID is 'i-5346d733'
2013-07-15 14:08:33,125 DEBUGec2:360 Adding tag
'clusterName:MSGGREG' to resource 'i-5346d733'
2013-07-15 14:08:33,307 DEBUGec2:360 Adding tag
'role:master' to resource 'i-5346d733'
2013-07-15 14:08:33,554 DEBUGec2:360 Adding tag
'Name:master: MSGGREG' to resource 'i-5346d733'
2013-07-15 14:08:33,744 DEBUG master:246 ud at manager start:
{'region_name': u'us-east-1', 'region_endpoint': u'ec2.amazonaws.com',
'ec2_port': None, 'deployment_version': 2, 'cloud_name': u'Amazon',
'boot_script_name': 'cm_boot.py', 'is_secure': True, 'password':
'', 'access_key': 'redacted!', 's3_port': None, 'cloud_type':
u'ec2', 'cloudman_home': '/mnt/cm', 'cluster_name': u'MSGGREG',
'freenxpass': u'', 'bucket_default': 'cloudman', 'role': 'master',
'bucket_cluster': 'cm-0479bd75a331acc874033e98b2e1e03e',
'boot_script_path': '/tmp/cm', 'secret_key': u'redacted!',
's3_conn_path': u'/', 's3_host': u's3.amazonaws.com', 'ec2_conn_path':
u'/'}
2013-07-15 14:08:33,744 DEBUG master:1858 Generating root
user's public key...
2013-07-15 14:08:33,763 DEBUG
Re: [galaxy-dev] CloudMan Error
Hi Greg, Sorry for replying really late. So, I'm guessing this was an old cluster that was shared and is now being derived on a new cluster? There was a large number of paths we explored while getting ready for the upgrade and I was of the opinion we covered that path but it seems things are not working as expected. Can look at the more detailed log on the Admin page (under CloudMan log) and see if there are more details about what's going on and why it's failing? On Thu, Jul 11, 2013 at 3:14 PM, greg wrote: > Hi guys, > > I just thought I'd check in again. None of the researches that want > to run out genotyping program can do so until I figure this out. Any > help or advice at all would be greatly appreciated. > > Thanks, > > Greg > > On Mon, Jul 8, 2013 at 8:43 AM, greg wrote: > > Hi guys, > > > > Any thoughts on this? I'm kind of stuck. > > > > (Even some pointers on where to look for more clues would be extremely > helpful.) > > > > Thanks, > > > > Greg > > > > On Fri, Jul 5, 2013 at 11:10 AM, greg wrote: > >> Hi guys, > >> > >> I'm hitting an error using CloudMan using the Share-an-Instance > >> option. It says: > >> > >> Error creating volume from shared cluster's snapshot > >> '['snap-cfa775ba']': 'filesystems'. > >> > >> Also disk stats says 0 /0 and the Applications light is yellow while > >> the data light is green. > >> > >> I'm using the share string > >> cm-808d863548acae7c2328c39a90f52e29/shared/2012-09-17--19-47 > >> > >> It's always worked in the past. > >> > >> Thanks, > >> > >> Greg > >> > >> Here's the full log: > >> > >> 14:58:18 - Master starting > >> 14:58:20 - Completed the initial cluster startup process. This is a > >> new cluster; waiting to configure the type. > >> 14:58:24 - Migration service prerequisites OK; starting the service > >> 14:58:24 - SGE service prerequisites OK; starting the service > >> 14:58:31 - Setting up SGE... > >> 14:58:51 - HTCondor service prerequisites OK; starting the service > >> 14:58:51 - HTCondor config file /etc/condor/condor_config not found! > >> 14:58:59 - Hadoop service prerequisites OK; starting the service > >> 14:59:48 - Done adding Hadoop service; service running. > >> 15:01:45 - Error creating volume from shared cluster's snapshot > >> '['snap-cfa775ba']': 'filesystems' > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
Hi guys, I just thought I'd check in again. None of the researches that want to run out genotyping program can do so until I figure this out. Any help or advice at all would be greatly appreciated. Thanks, Greg On Mon, Jul 8, 2013 at 8:43 AM, greg wrote: > Hi guys, > > Any thoughts on this? I'm kind of stuck. > > (Even some pointers on where to look for more clues would be extremely > helpful.) > > Thanks, > > Greg > > On Fri, Jul 5, 2013 at 11:10 AM, greg wrote: >> Hi guys, >> >> I'm hitting an error using CloudMan using the Share-an-Instance >> option. It says: >> >> Error creating volume from shared cluster's snapshot >> '['snap-cfa775ba']': 'filesystems'. >> >> Also disk stats says 0 /0 and the Applications light is yellow while >> the data light is green. >> >> I'm using the share string >> cm-808d863548acae7c2328c39a90f52e29/shared/2012-09-17--19-47 >> >> It's always worked in the past. >> >> Thanks, >> >> Greg >> >> Here's the full log: >> >> 14:58:18 - Master starting >> 14:58:20 - Completed the initial cluster startup process. This is a >> new cluster; waiting to configure the type. >> 14:58:24 - Migration service prerequisites OK; starting the service >> 14:58:24 - SGE service prerequisites OK; starting the service >> 14:58:31 - Setting up SGE... >> 14:58:51 - HTCondor service prerequisites OK; starting the service >> 14:58:51 - HTCondor config file /etc/condor/condor_config not found! >> 14:58:59 - Hadoop service prerequisites OK; starting the service >> 14:59:48 - Done adding Hadoop service; service running. >> 15:01:45 - Error creating volume from shared cluster's snapshot >> '['snap-cfa775ba']': 'filesystems' ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] CloudMan Error
Hi guys, Any thoughts on this? I'm kind of stuck. (Even some pointers on where to look for more clues would be extremely helpful.) Thanks, Greg On Fri, Jul 5, 2013 at 11:10 AM, greg wrote: > Hi guys, > > I'm hitting an error using CloudMan using the Share-an-Instance > option. It says: > > Error creating volume from shared cluster's snapshot > '['snap-cfa775ba']': 'filesystems'. > > Also disk stats says 0 /0 and the Applications light is yellow while > the data light is green. > > I'm using the share string > cm-808d863548acae7c2328c39a90f52e29/shared/2012-09-17--19-47 > > It's always worked in the past. > > Thanks, > > Greg > > Here's the full log: > > 14:58:18 - Master starting > 14:58:20 - Completed the initial cluster startup process. This is a > new cluster; waiting to configure the type. > 14:58:24 - Migration service prerequisites OK; starting the service > 14:58:24 - SGE service prerequisites OK; starting the service > 14:58:31 - Setting up SGE... > 14:58:51 - HTCondor service prerequisites OK; starting the service > 14:58:51 - HTCondor config file /etc/condor/condor_config not found! > 14:58:59 - Hadoop service prerequisites OK; starting the service > 14:59:48 - Done adding Hadoop service; service running. > 15:01:45 - Error creating volume from shared cluster's snapshot > '['snap-cfa775ba']': 'filesystems' ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] CloudMan Error
Hi guys, I'm hitting an error using CloudMan using the Share-an-Instance option. It says: Error creating volume from shared cluster's snapshot '['snap-cfa775ba']': 'filesystems'. Also disk stats says 0 /0 and the Applications light is yellow while the data light is green. I'm using the share string cm-808d863548acae7c2328c39a90f52e29/shared/2012-09-17--19-47 It's always worked in the past. Thanks, Greg Here's the full log: 14:58:18 - Master starting 14:58:20 - Completed the initial cluster startup process. This is a new cluster; waiting to configure the type. 14:58:24 - Migration service prerequisites OK; starting the service 14:58:24 - SGE service prerequisites OK; starting the service 14:58:31 - Setting up SGE... 14:58:51 - HTCondor service prerequisites OK; starting the service 14:58:51 - HTCondor config file /etc/condor/condor_config not found! 14:58:59 - Hadoop service prerequisites OK; starting the service 14:59:48 - Done adding Hadoop service; service running. 15:01:45 - Error creating volume from shared cluster's snapshot '['snap-cfa775ba']': 'filesystems' ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
