I'm using Whirr to create two different clusters, one hadoop cluster and the
other a cassandra cluster, that need to communicate with each other (on
EC2). I cant figure out how to configure the clusters to allow this. I
looked at the python contrib scripts, by it looks like they are hadoop
specif
ev/blog/setting-up-a-cassandra-cluster-with-the-datastax-ami
>
>
> On Mon, May 16, 2011 at 3:01 PM, John Conwell wrote:
>
>> I'm using Whirr to create two different clusters, one hadoop cluster and
>> the other a cassandra cluster, that need to communicate with ea
I have customized the install and config functions for cassandra, and put
these two files in the functions folder, and it works great from the cmd
line whirr utility.
But when launching a cluster via the Serice.launchCluster() method how do
you specify a custom function file for either install or
e application classpath (like
> > the bin/whirr script does).
> >
> > Hope that helps.
> >
> > Cheers,
> > Tom
> >
> > On Thu, May 26, 2011 at 3:29 PM, John Conwell wrote:
> >> I have customized the install and config functions for cass
In looking at the ClusterActionHandlerSupport, I notice the before/after
OtherAction event. Is this functional? How can I trigger this event?
--
Thanks,
John C
integrate APIs into what I'm
working on
Thanks,
John
On Thu, Jun 2, 2011 at 2:48 PM, Andrei Savu wrote:
> What's you use case? Maybe there is another way.
>
> As far as I know no service is using OtherAction event.
>
> -- Andrei Savu / andreisavu.ro
>
> On Fri, J
;
> -- Andrei Savu
>
> On Fri, Jun 3, 2011 at 12:59 AM, John Conwell wrote:
> > Well, the reason I want to know is two fold. First, I'm using the whirr
> > core API to spin up and provision multiple clusters. But on one of my
> > clusters, I'd need to push
w EXECUTE_ACTION, and extending
> ClusterActionHandlerSupport to expose before/afterExecute methods?
>
> Tom
>
> >
> > -- Andrei Savu
> >
> > On Fri, Jun 3, 2011 at 12:59 AM, John Conwell wrote:
> >> Well, the reason I want to know is two fold. Fir
I have a similar scenario where I had to modify the cassandra.yaml on our
cassandra ring after install. I got around that by putting a copy of our
yaml file out on S3, and modifying the install_cassandra.sh script to pull
down my custom yaml file from S3 instead of using the stock one that comes
w
Is there anyway I can configure whirr to allow me to ssh into any node in my
cluster from a different machine than the one that launched the cluster? My
clusters can be fairly long lived, and I might need to access one of the
nodes via ssh from home, or customer's office, or someone else on my tea
Thats not that feasible in a team environment, where multiple people share
multiple resources.
On Tue, Jun 7, 2011 at 4:39 PM, Khanh Nguyen wrote:
> Can you copy your .ssh/id_rsa and use it at the other machines?
>
> -k
>
> On Tue, Jun 7, 2011 at 6:52 PM, John Conwell wrote:
>
In 0.4.0, is Service.launchCluster(ClusterSpec) thread safe? Through the
API I'm spinning up 3 clusters, one after another, and I'd like to change
the code to launch them in parallel, but wanted to check if this is
threadsafe.
--
Thanks,
John C
I've got a cluster that gets started by Whirr. After its running, I need to
create a config file and copy it up to a specific folder on the VM. I'm
using the class.method SshClient.put() to copy the file up to the /tmp
directory on my VM with no security issues. But then I need to copy the
file
hould be able to do sudo without requesting a
> password.
> On Jun 13, 2011 8:55 PM, "John Conwell" wrote:
> > I've got a cluster that gets started by Whirr. After its running, I need
> to
> > create a config file and copy it up to a specific folder on the VM.
the AMI is us-east-1/ami-da0cf8b3
and OS is the default that whirr installs, Ubuntu 10.4 my thinks.
On Mon, Jun 13, 2011 at 1:13 PM, Andrei Savu wrote:
> That should work. What ami, OS are you using?
> On Jun 13, 2011 10:22 PM, "John Conwell" wrote:
> > which user. Whirr
I get the whole "security is a good thing" thing, but could someone give me
a description as to why when whirr configures hadoop it sets up the ssh
proxy to disallow all coms to the data / task nodes except via the name node
over the proxy? If I'm running on EC2, wont correctly setting up security
some performance improvements that are relevant to you.
>
>
> Sent from my phone.
>
> Cheers,
>
> -- Andrei
> On Jun 10, 2011 8:38 PM, "John Conwell" wrote:
> > In 0.4.0, is Service.launchCluster(ClusterSpec) thread safe? Through the
> > API I'
ically).
>
> I agree that it would be better if we could replace the proxy with
> something better, such as
> https://issues.apache.org/jira/browse/WHIRR-81.
>
> On Tue, Jun 14, 2011 at 9:26 AM, John Conwell wrote:
> > I get the whole "security is a good thing" thin
oh cool. Thanks for the pointer
On Wed, Jun 15, 2011 at 10:28 AM, Tom White wrote:
> On Wed, Jun 15, 2011 at 10:18 AM, John Conwell wrote:
> > Ok, that makes sense. Thanks for the clarification. It
> > is definitely unwieldy when trying to integrate whirr's API into
Yea, I can do that. What section would you want it in? Maybe a new section
on using whirr via its API?
On Tue, Jun 14, 2011 at 5:20 PM, Tom White wrote:
> On Tue, Jun 14, 2011 at 3:46 PM, John Conwell wrote:
> > So as an FYI, I just tested using the whirr API to start multiple
&
n, Jun 13, 2011 at 11:16 PM, John Conwell wrote:
> > the AMI is us-east-1/ami-da0cf8b3
> > and OS is the default that whirr installs, Ubuntu 10.4 my thinks.
> >
> > On Mon, Jun 13, 2011 at 1:13 PM, Andrei Savu
> wrote:
> >>
> >> That should work. What
gt; Thanks!
> Tom
>
> On Wed, Jun 15, 2011 at 10:34 AM, John Conwell wrote:
> > Yea, I can do that. What section would you want it in? Maybe a new
> section
> > on using whirr via its API?
> >
> > On Tue, Jun 14, 2011 at 5:20 PM, Tom White
> wrote:
> &
is mostly
> extracted from tests.
> On Jun 15, 2011 8:32 PM, "John Conwell" wrote:
> > oh cool. Thanks for the pointer
> >
> > On Wed, Jun 15, 2011 at 10:28 AM, Tom White
> wrote:
> >
> >> On Wed, Jun 15, 2011 at 10:18 AM, John Conwell wrote:
&g
/my.cfg", lines),
> exec(getFileContent(scriptPath))
> );
> controller.runScriptOnNodesMatching(
> spec, condition, builder)
>
> -- Andrei
>
> On Wed, Jun 15, 2011 at 8:38 PM, John Conwell wrote:
> > I looked at computeService.runScriptOnNodesMatching a while ago and
> couldnt
> &g
oh wait...ha, ha. Never mind. I see that I have to split the returned
instance id and create a predicate on the part after the back slash
On Thu, Jun 16, 2011 at 9:47 AM, John Conwell wrote:
> Pulled the code from RunScriptCommand as an example, and I think I'm good
> in that respe
Does whirr support secondarynamenode instances for a hadoop cluster? If so,
what are the configuration parameters in the whirr config file?
--
Thanks,
John C
u / andreisavu.ro
>
> On Wed, Jul 13, 2011 at 2:05 PM, John Conwell wrote:
> > Does whirr support secondarynamenode instances for a hadoop cluster? If
> so,
> > what are the configuration parameters in the whirr config file?
> >
> > --
> >
> > Thanks,
> > John C
> >
>
--
Thanks,
John C
ere is no hadoop-secondarynamenode role
> defined.
>
> So I believe it's not possible to start a Hadoop cluster that has a
> secondarynamenode.
>
> Tom have you used this in the past?
>
> -- Andrei
>
> On Wed, Jul 13, 2011 at 2:26 PM, John Conwell wrote:
>
ould consider refactoring that code so that the namenode & the
> > jobtracker can be started on different machine and also add support
> > for secondary namenode as discussed.
>
> +1 to both. It might be easier to do them in separate JIRAs.
>
> Tom
>
> >
> &g
29 matches
Mail list logo
