Re: jclouds support for CloudStack

[Aled Sage]

Hi all,

First apologies for letting this conversation go cold!

As Sebastian notes, the jclouds dev mailing list is the best place to 
continue discussions.


I've created some jclouds issues to capture some CloudStack related 
work. In particular, "CloudStack guide missing" would be a quick win and 
a good way into getting involved with jclouds.


   JCLOUDS-722 - Gaps in CloudStack 4.x API coverage
   <https://issues.apache.org/jira/browse/JCLOUDS-722>
   JCLOUDS-723 - CloudStack createNodesInGroup fails for service
   providers with locked down APIs
   <https://issues.apache.org/jira/browse/JCLOUDS-723>
   JCLOUDS-724 - CloudStack live tests: skip instead of fail if feature
   not available <https://issues.apache.org/jira/browse/JCLOUDS-724>
   JCLOUDS-725 - CloudStack guide missing
   <https://issues.apache.org/jira/browse/JCLOUDS-725>

There are a number of other CloudStack related issues which one can find 
by searching https://issues.apache.org/jira/browse/JCLOUDS


Aled


On 24/07/2014 20:00, Sebastien Goasguen wrote:

Great,

then you should probably join the jclouds dev list and start testing and 
contributing there.

On Jul 24, 2014, at 6:11 AM, Nguyen Anh Tu  wrote:


+1. I'm willing to contribute.

--Tuna


On Thu, Jul 24, 2014 at 1:05 PM, Priyanka Deepala <
priyanka.deepal...@gmail.com> wrote:


count me jclouds integration for more recent API features

Regards
Priyanka


On Wed, Jul 23, 2014 at 9:07 PM, Sebastien Goasguen 
wrote:


On Jul 23, 2014, at 11:32 AM, Aled Sage  wrote:


Hi all,

We are keen users of and contributors to jclouds [1], including for the

CloudStack integration.

For those who don't know it, Apache jclouds is the leading java cloud

portability library, used by a lot of companies and several Apache

projects

including Brooklyn (for which I'm a lead) and Stratos.

Would anyone in the CloudStack community be interested in helping out

with updating and improving the jclouds integration? Unfortunately the
jclouds CloudStack code hasn't had as much attention as it deserves over
the past couple of releases.

I just targeted Exoscale [2] (i.e. CloudStack 4.3) for the jclouds live

test suite, and got a range of failures. There are some concrete tasks

that

can come out of this:

* Ability to automatically skip inappropriate tests - e.g. if only
  basic networking is available, then report the advanced networking
  tests as skipped rather than failed.
  (This could use the API Discovery [3])
* Investigate other failures, to make the test suite more robust (or
  potentially to find regressions in CloudStack).
* Extend the jclouds integration for more recent API features (e.g.
  for API Discovery [3])
* Improve the publicity and documentation for jclouds CloudStack -
  e.g. it's not even mentioned in [4]


+1 from me of course. I take care of libcloud, but jclouds also needs
attention.


It's also worth noting that the jclouds live test suite is a great

regression test suite that could be run at least once on every CloudStack
release. For example, it would most likely spot regressions such as
CLOUDSTACK-6508 [5,6].
I applied the patch for 6508 in the 4.3 branch, so you will see it in

4.3.1

Aled

[1] jclouds.apache.org <http://jclouds.apache.org>
[2] www.exoscale.ch <http://www.exoscale.ch>
[3]

http://cloudstack.apache.org/docs/api/apidocs-4.3/user/listApis.html

[4] https://jclouds.apache.org/guides/
[5] https://issues.apache.org/jira/browse/CLOUDSTACK-6508
[6] https://issues.apache.org/jira/browse/JCLOUDS-557

Re: jclouds support for CloudStack

[Sebastien Goasguen]

Great,

then you should probably join the jclouds dev list and start testing and 
contributing there.

On Jul 24, 2014, at 6:11 AM, Nguyen Anh Tu  wrote:

> +1. I'm willing to contribute.
> 
> --Tuna
> 
> 
> On Thu, Jul 24, 2014 at 1:05 PM, Priyanka Deepala <
> priyanka.deepal...@gmail.com> wrote:
> 
>> count me jclouds integration for more recent API features
>> 
>> Regards
>> Priyanka
>> 
>> 
>> On Wed, Jul 23, 2014 at 9:07 PM, Sebastien Goasguen 
>> wrote:
>> 
>>> 
>>> On Jul 23, 2014, at 11:32 AM, Aled Sage  wrote:
>>> 
>>>> Hi all,
>>>> 
>>>> We are keen users of and contributors to jclouds [1], including for the
>>> CloudStack integration.
>>>> 
>>>> For those who don't know it, Apache jclouds is the leading java cloud
>>> portability library, used by a lot of companies and several Apache
>> projects
>>> including Brooklyn (for which I'm a lead) and Stratos.
>>>> 
>>>> Would anyone in the CloudStack community be interested in helping out
>>> with updating and improving the jclouds integration? Unfortunately the
>>> jclouds CloudStack code hasn't had as much attention as it deserves over
>>> the past couple of releases.
>>>> 
>>>> I just targeted Exoscale [2] (i.e. CloudStack 4.3) for the jclouds live
>>> test suite, and got a range of failures. There are some concrete tasks
>> that
>>> can come out of this:
>>>> 
>>>> * Ability to automatically skip inappropriate tests - e.g. if only
>>>>  basic networking is available, then report the advanced networking
>>>>  tests as skipped rather than failed.
>>>>  (This could use the API Discovery [3])
>>>> * Investigate other failures, to make the test suite more robust (or
>>>>  potentially to find regressions in CloudStack).
>>>> * Extend the jclouds integration for more recent API features (e.g.
>>>>  for API Discovery [3])
>>>> * Improve the publicity and documentation for jclouds CloudStack -
>>>>  e.g. it's not even mentioned in [4]
>>>> 
>>> 
>>> +1 from me of course. I take care of libcloud, but jclouds also needs
>>> attention.
>>> 
>>>> It's also worth noting that the jclouds live test suite is a great
>>> regression test suite that could be run at least once on every CloudStack
>>> release. For example, it would most likely spot regressions such as
>>> CLOUDSTACK-6508 [5,6].
>>>> 
>>> 
>>> I applied the patch for 6508 in the 4.3 branch, so you will see it in
>> 4.3.1
>>> 
>>>> Aled
>>>> 
>>>> [1] jclouds.apache.org <http://jclouds.apache.org>
>>>> [2] www.exoscale.ch <http://www.exoscale.ch>
>>>> [3]
>> http://cloudstack.apache.org/docs/api/apidocs-4.3/user/listApis.html
>>>> [4] https://jclouds.apache.org/guides/
>>>> [5] https://issues.apache.org/jira/browse/CLOUDSTACK-6508
>>>> [6] https://issues.apache.org/jira/browse/JCLOUDS-557
>>> 
>>> 
>> 

Re: jclouds support for CloudStack

[Nguyen Anh Tu]

+1. I'm willing to contribute.

--Tuna


On Thu, Jul 24, 2014 at 1:05 PM, Priyanka Deepala <
priyanka.deepal...@gmail.com> wrote:

> count me jclouds integration for more recent API features
>
> Regards
> Priyanka
>
>
> On Wed, Jul 23, 2014 at 9:07 PM, Sebastien Goasguen 
> wrote:
>
> >
> > On Jul 23, 2014, at 11:32 AM, Aled Sage  wrote:
> >
> > > Hi all,
> > >
> > > We are keen users of and contributors to jclouds [1], including for the
> > CloudStack integration.
> > >
> > > For those who don't know it, Apache jclouds is the leading java cloud
> > portability library, used by a lot of companies and several Apache
> projects
> > including Brooklyn (for which I'm a lead) and Stratos.
> > >
> > > Would anyone in the CloudStack community be interested in helping out
> > with updating and improving the jclouds integration? Unfortunately the
> > jclouds CloudStack code hasn't had as much attention as it deserves over
> > the past couple of releases.
> > >
> > > I just targeted Exoscale [2] (i.e. CloudStack 4.3) for the jclouds live
> > test suite, and got a range of failures. There are some concrete tasks
> that
> > can come out of this:
> > >
> > > * Ability to automatically skip inappropriate tests - e.g. if only
> > >   basic networking is available, then report the advanced networking
> > >   tests as skipped rather than failed.
> > >   (This could use the API Discovery [3])
> > > * Investigate other failures, to make the test suite more robust (or
> > >   potentially to find regressions in CloudStack).
> > > * Extend the jclouds integration for more recent API features (e.g.
> > >   for API Discovery [3])
> > > * Improve the publicity and documentation for jclouds CloudStack -
> > >   e.g. it's not even mentioned in [4]
> > >
> >
> > +1 from me of course. I take care of libcloud, but jclouds also needs
> > attention.
> >
> > > It's also worth noting that the jclouds live test suite is a great
> > regression test suite that could be run at least once on every CloudStack
> > release. For example, it would most likely spot regressions such as
> > CLOUDSTACK-6508 [5,6].
> > >
> >
> > I applied the patch for 6508 in the 4.3 branch, so you will see it in
> 4.3.1
> >
> > > Aled
> > >
> > > [1] jclouds.apache.org <http://jclouds.apache.org>
> > > [2] www.exoscale.ch <http://www.exoscale.ch>
> > > [3]
> http://cloudstack.apache.org/docs/api/apidocs-4.3/user/listApis.html
> > > [4] https://jclouds.apache.org/guides/
> > > [5] https://issues.apache.org/jira/browse/CLOUDSTACK-6508
> > > [6] https://issues.apache.org/jira/browse/JCLOUDS-557
> >
> >
>

Re: jclouds support for CloudStack

[Priyanka Deepala]

count me jclouds integration for more recent API features

Regards
Priyanka


On Wed, Jul 23, 2014 at 9:07 PM, Sebastien Goasguen 
wrote:

>
> On Jul 23, 2014, at 11:32 AM, Aled Sage  wrote:
>
> > Hi all,
> >
> > We are keen users of and contributors to jclouds [1], including for the
> CloudStack integration.
> >
> > For those who don't know it, Apache jclouds is the leading java cloud
> portability library, used by a lot of companies and several Apache projects
> including Brooklyn (for which I'm a lead) and Stratos.
> >
> > Would anyone in the CloudStack community be interested in helping out
> with updating and improving the jclouds integration? Unfortunately the
> jclouds CloudStack code hasn't had as much attention as it deserves over
> the past couple of releases.
> >
> > I just targeted Exoscale [2] (i.e. CloudStack 4.3) for the jclouds live
> test suite, and got a range of failures. There are some concrete tasks that
> can come out of this:
> >
> > * Ability to automatically skip inappropriate tests - e.g. if only
> >   basic networking is available, then report the advanced networking
> >   tests as skipped rather than failed.
> >   (This could use the API Discovery [3])
> > * Investigate other failures, to make the test suite more robust (or
> >   potentially to find regressions in CloudStack).
> > * Extend the jclouds integration for more recent API features (e.g.
> >   for API Discovery [3])
> > * Improve the publicity and documentation for jclouds CloudStack -
> >   e.g. it's not even mentioned in [4]
> >
>
> +1 from me of course. I take care of libcloud, but jclouds also needs
> attention.
>
> > It's also worth noting that the jclouds live test suite is a great
> regression test suite that could be run at least once on every CloudStack
> release. For example, it would most likely spot regressions such as
> CLOUDSTACK-6508 [5,6].
> >
>
> I applied the patch for 6508 in the 4.3 branch, so you will see it in 4.3.1
>
> > Aled
> >
> > [1] jclouds.apache.org <http://jclouds.apache.org>
> > [2] www.exoscale.ch <http://www.exoscale.ch>
> > [3] http://cloudstack.apache.org/docs/api/apidocs-4.3/user/listApis.html
> > [4] https://jclouds.apache.org/guides/
> > [5] https://issues.apache.org/jira/browse/CLOUDSTACK-6508
> > [6] https://issues.apache.org/jira/browse/JCLOUDS-557
>
>

RE: jclouds support for CloudStack

[Suresh Sadhu]

+1 
Count me on testing this integration.

-Original Message-
From: Aled Sage [mailto:aled.s...@gmail.com] 
Sent: 23 July 2014 21:02
To: dev@cloudstack.apache.org
Subject: jclouds support for CloudStack

Hi all,

We are keen users of and contributors to jclouds [1], including for the 
CloudStack integration.

For those who don't know it, Apache jclouds is the leading java cloud 
portability library, used by a lot of companies and several Apache projects 
including Brooklyn (for which I'm a lead) and Stratos.

Would anyone in the CloudStack community be interested in helping out with 
updating and improving the jclouds integration? Unfortunately the jclouds 
CloudStack code hasn't had as much attention as it deserves over the past 
couple of releases.

I just targeted Exoscale [2] (i.e. CloudStack 4.3) for the jclouds live test 
suite, and got a range of failures. There are some concrete tasks that can come 
out of this:

  * Ability to automatically skip inappropriate tests - e.g. if only
basic networking is available, then report the advanced networking
tests as skipped rather than failed.
(This could use the API Discovery [3])
  * Investigate other failures, to make the test suite more robust (or
potentially to find regressions in CloudStack).
  * Extend the jclouds integration for more recent API features (e.g.
for API Discovery [3])
  * Improve the publicity and documentation for jclouds CloudStack -
e.g. it's not even mentioned in [4]

It's also worth noting that the jclouds live test suite is a great regression 
test suite that could be run at least once on every CloudStack release. For 
example, it would most likely spot regressions such as CLOUDSTACK-6508 [5,6].

Aled

[1] jclouds.apache.org <http://jclouds.apache.org> [2] www.exoscale.ch 
<http://www.exoscale.ch> [3] 
http://cloudstack.apache.org/docs/api/apidocs-4.3/user/listApis.html
[4] https://jclouds.apache.org/guides/
[5] https://issues.apache.org/jira/browse/CLOUDSTACK-6508
[6] https://issues.apache.org/jira/browse/JCLOUDS-557

jclouds support for CloudStack

[Aled Sage]

Hi all,

We are keen users of and contributors to jclouds [1], including for the 
CloudStack integration.


For those who don't know it, Apache jclouds is the leading java cloud 
portability library, used by a lot of companies and several Apache 
projects including Brooklyn (for which I'm a lead) and Stratos.


Would anyone in the CloudStack community be interested in helping out 
with updating and improving the jclouds integration? Unfortunately the 
jclouds CloudStack code hasn't had as much attention as it deserves over 
the past couple of releases.


I just targeted Exoscale [2] (i.e. CloudStack 4.3) for the jclouds live 
test suite, and got a range of failures. There are some concrete tasks 
that can come out of this:


 * Ability to automatically skip inappropriate tests - e.g. if only
   basic networking is available, then report the advanced networking
   tests as skipped rather than failed.
   (This could use the API Discovery [3])
 * Investigate other failures, to make the test suite more robust (or
   potentially to find regressions in CloudStack).
 * Extend the jclouds integration for more recent API features (e.g.
   for API Discovery [3])
 * Improve the publicity and documentation for jclouds CloudStack -
   e.g. it's not even mentioned in [4]

It's also worth noting that the jclouds live test suite is a great 
regression test suite that could be run at least once on every 
CloudStack release. For example, it would most likely spot regressions 
such as CLOUDSTACK-6508 [5,6].


Aled

[1] jclouds.apache.org <http://jclouds.apache.org>
[2] www.exoscale.ch <http://www.exoscale.ch>
[3] http://cloudstack.apache.org/docs/api/apidocs-4.3/user/listApis.html
[4] https://jclouds.apache.org/guides/
[5] https://issues.apache.org/jira/browse/CLOUDSTACK-6508
[6] https://issues.apache.org/jira/browse/JCLOUDS-557

Re: jclouds support for CloudStack

[Sebastien Goasguen]

On Jul 23, 2014, at 11:32 AM, Aled Sage  wrote:

> Hi all,
> 
> We are keen users of and contributors to jclouds [1], including for the 
> CloudStack integration.
> 
> For those who don't know it, Apache jclouds is the leading java cloud 
> portability library, used by a lot of companies and several Apache projects 
> including Brooklyn (for which I'm a lead) and Stratos.
> 
> Would anyone in the CloudStack community be interested in helping out with 
> updating and improving the jclouds integration? Unfortunately the jclouds 
> CloudStack code hasn't had as much attention as it deserves over the past 
> couple of releases.
> 
> I just targeted Exoscale [2] (i.e. CloudStack 4.3) for the jclouds live test 
> suite, and got a range of failures. There are some concrete tasks that can 
> come out of this:
> 
> * Ability to automatically skip inappropriate tests - e.g. if only
>   basic networking is available, then report the advanced networking
>   tests as skipped rather than failed.
>   (This could use the API Discovery [3])
> * Investigate other failures, to make the test suite more robust (or
>   potentially to find regressions in CloudStack).
> * Extend the jclouds integration for more recent API features (e.g.
>   for API Discovery [3])
> * Improve the publicity and documentation for jclouds CloudStack -
>   e.g. it's not even mentioned in [4]
> 

+1 from me of course. I take care of libcloud, but jclouds also needs attention.

> It's also worth noting that the jclouds live test suite is a great regression 
> test suite that could be run at least once on every CloudStack release. For 
> example, it would most likely spot regressions such as CLOUDSTACK-6508 [5,6].
> 

I applied the patch for 6508 in the 4.3 branch, so you will see it in 4.3.1

> Aled
> 
> [1] jclouds.apache.org <http://jclouds.apache.org>
> [2] www.exoscale.ch <http://www.exoscale.ch>
> [3] http://cloudstack.apache.org/docs/api/apidocs-4.3/user/listApis.html
> [4] https://jclouds.apache.org/guides/
> [5] https://issues.apache.org/jira/browse/CLOUDSTACK-6508
> [6] https://issues.apache.org/jira/browse/JCLOUDS-557

Re: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Andrew Bayer]

Once I get it a bit more refined, yeah - probably gonna talk about it a bit
at the ACS meetup in Santa Clara next month too.

A.

On Tue, Jul 23, 2013 at 11:47 AM, Sebastien Goasguen wrote:

>
> On Jul 22, 2013, at 10:58 AM, Andrew Bayer  wrote:
>
> > Woot, making progress - unsurprisingly, all of the jclouds live tests
> that
> > expect to be able to create a VM and then SSH into it fail, but that's
> > entirely reasonable. =) I'm making progress through the other failing
> tests
> > - catching some issues in the jclouds CloudStack implementation along the
> > way. Thanks, everybody!
> >
>
> any chance you can write a quick blog post or wiki entry ?
>
> > A.
> >
> > On Sun, Jul 21, 2013 at 9:25 PM, Prasanna Santhanam 
> wrote:
> >
> >> Hi - the advanced zone setup should work without troubles. The current
> >> master is broken and a couple of bugs are pending for this after the
> >> baremetal merge done recently.
> >>
> >> CLOUDSTACK-3481 and CLOUDSTACK-1812.
> >>
> >> On Sun, Jul 21, 2013 at 08:32:42PM -0700, Andrew Bayer wrote:
> >>> Finally got a chance to work on this - trying to get the basic
> networking
> >>> setup with Marvin seems to be barfing - I get the following exception
> in
> >>> the CS logs when I run mvn -Pdeveloper,marvin.setup
> >>> -Dmarvin.config=setup/dev/basic.cfg -pl :cloud-marvin integration-test:
> >>>
> >>> WARN  [cloud.network.NetworkServiceImpl] (1384506397@qtp-455967038-0:)
> >>> Exception:
> >>> com.cloud.exception.InvalidParameterValueException: Invalid Network
> >> Service
> >>> Provider=BaremetalDhcpProvider
> >>> at
> >>>
> >>
> com.cloud.network.NetworkServiceImpl.addProviderToPhysicalNetwork(NetworkServiceImpl.java:3239)
> >>> at
> >>>
> >>
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> >>> at
> >>>
> >>
> com.cloud.network.NetworkServiceImpl.addDefaultBaremetalProvidersToPhysicalNetwork(NetworkServiceImpl.java:3747)
> >>> at
> >>>
> >>
> com.cloud.network.NetworkServiceImpl.createPhysicalNetwork(NetworkServiceImpl.java:2517)
> >>> at
> >>>
> >>
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> >>> at
> >>>
> >>
> org.apache.cloudstack.api.command.admin.network.CreatePhysicalNetworkCmd.create(CreatePhysicalNetworkCmd.java:160)
> >>> at
> com.cloud.api.ApiDispatcher.dispatchCreateCmd(ApiDispatcher.java:104)
> >>> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:476)
> >>> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:372)
> >>> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:305)
> >>> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66)
> >>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> >>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> >>> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> >>> at
> >> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
> >>> at
> >>>
> >>
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> >>> at
> >> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> >>> at
> >> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> >>> at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> >>> at
> >>>
> >>
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> >>> at
> >>>
> >>
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> >>> at
> >> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> >>> at org.mortbay.jetty.Server.handle(Server.java:326)
> >>> at
> >> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> >>> at
> >>>
> >>
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> >>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> >>> at org.mortbay.jetty.HttpParser.parseAvailable(Ht

Re: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Sebastien Goasguen]

On Jul 22, 2013, at 10:58 AM, Andrew Bayer  wrote:

> Woot, making progress - unsurprisingly, all of the jclouds live tests that
> expect to be able to create a VM and then SSH into it fail, but that's
> entirely reasonable. =) I'm making progress through the other failing tests
> - catching some issues in the jclouds CloudStack implementation along the
> way. Thanks, everybody!
> 

any chance you can write a quick blog post or wiki entry ?

> A.
> 
> On Sun, Jul 21, 2013 at 9:25 PM, Prasanna Santhanam  wrote:
> 
>> Hi - the advanced zone setup should work without troubles. The current
>> master is broken and a couple of bugs are pending for this after the
>> baremetal merge done recently.
>> 
>> CLOUDSTACK-3481 and CLOUDSTACK-1812.
>> 
>> On Sun, Jul 21, 2013 at 08:32:42PM -0700, Andrew Bayer wrote:
>>> Finally got a chance to work on this - trying to get the basic networking
>>> setup with Marvin seems to be barfing - I get the following exception in
>>> the CS logs when I run mvn -Pdeveloper,marvin.setup
>>> -Dmarvin.config=setup/dev/basic.cfg -pl :cloud-marvin integration-test:
>>> 
>>> WARN  [cloud.network.NetworkServiceImpl] (1384506397@qtp-455967038-0:)
>>> Exception:
>>> com.cloud.exception.InvalidParameterValueException: Invalid Network
>> Service
>>> Provider=BaremetalDhcpProvider
>>> at
>>> 
>> com.cloud.network.NetworkServiceImpl.addProviderToPhysicalNetwork(NetworkServiceImpl.java:3239)
>>> at
>>> 
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>> at
>>> 
>> com.cloud.network.NetworkServiceImpl.addDefaultBaremetalProvidersToPhysicalNetwork(NetworkServiceImpl.java:3747)
>>> at
>>> 
>> com.cloud.network.NetworkServiceImpl.createPhysicalNetwork(NetworkServiceImpl.java:2517)
>>> at
>>> 
>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>> at
>>> 
>> org.apache.cloudstack.api.command.admin.network.CreatePhysicalNetworkCmd.create(CreatePhysicalNetworkCmd.java:160)
>>> at com.cloud.api.ApiDispatcher.dispatchCreateCmd(ApiDispatcher.java:104)
>>> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:476)
>>> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:372)
>>> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:305)
>>> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>> at
>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
>>> at
>>> 
>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>> at
>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>> at
>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
>>> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>> at
>>> 
>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>> at
>>> 
>> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
>>> at
>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>> at org.mortbay.jetty.Server.handle(Server.java:326)
>>> at
>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>> at
>>> 
>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>> at
>>> 
>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>>> at
>>> 
>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>> ERROR [cloud.api.ApiServer] (1384506397@qtp-455967038-0:) unhandled
>>> exception executing api command: createPhysicalNetwork
>>> com.cloud.utils.exception.CloudRuntimeException: Fail to create a
>> physical
>>> network
>>> at
>>> 
>> com.cloud.network.NetworkServiceImpl.createPhysicalNetwor

Re: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Andrew Bayer]

Woot, making progress - unsurprisingly, all of the jclouds live tests that
expect to be able to create a VM and then SSH into it fail, but that's
entirely reasonable. =) I'm making progress through the other failing tests
- catching some issues in the jclouds CloudStack implementation along the
way. Thanks, everybody!

A.

On Sun, Jul 21, 2013 at 9:25 PM, Prasanna Santhanam  wrote:

> Hi - the advanced zone setup should work without troubles. The current
> master is broken and a couple of bugs are pending for this after the
> baremetal merge done recently.
>
> CLOUDSTACK-3481 and CLOUDSTACK-1812.
>
> On Sun, Jul 21, 2013 at 08:32:42PM -0700, Andrew Bayer wrote:
> > Finally got a chance to work on this - trying to get the basic networking
> > setup with Marvin seems to be barfing - I get the following exception in
> > the CS logs when I run mvn -Pdeveloper,marvin.setup
> > -Dmarvin.config=setup/dev/basic.cfg -pl :cloud-marvin integration-test:
> >
> > WARN  [cloud.network.NetworkServiceImpl] (1384506397@qtp-455967038-0:)
> > Exception:
> > com.cloud.exception.InvalidParameterValueException: Invalid Network
> Service
> > Provider=BaremetalDhcpProvider
> >  at
> >
> com.cloud.network.NetworkServiceImpl.addProviderToPhysicalNetwork(NetworkServiceImpl.java:3239)
> > at
> >
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> >  at
> >
> com.cloud.network.NetworkServiceImpl.addDefaultBaremetalProvidersToPhysicalNetwork(NetworkServiceImpl.java:3747)
> > at
> >
> com.cloud.network.NetworkServiceImpl.createPhysicalNetwork(NetworkServiceImpl.java:2517)
> >  at
> >
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> > at
> >
> org.apache.cloudstack.api.command.admin.network.CreatePhysicalNetworkCmd.create(CreatePhysicalNetworkCmd.java:160)
> >  at com.cloud.api.ApiDispatcher.dispatchCreateCmd(ApiDispatcher.java:104)
> > at com.cloud.api.ApiServer.queueCommand(ApiServer.java:476)
> >  at com.cloud.api.ApiServer.handleRequest(ApiServer.java:372)
> > at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:305)
> >  at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> >  at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> > at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> >  at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
> > at
> >
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> >  at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> > at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> >  at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> > at
> >
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> >  at
> >
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> > at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> >  at org.mortbay.jetty.Server.handle(Server.java:326)
> > at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> >  at
> >
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> >  at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> >  at
> >
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> > at
> >
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> > ERROR [cloud.api.ApiServer] (1384506397@qtp-455967038-0:) unhandled
> > exception executing api command: createPhysicalNetwork
> > com.cloud.utils.exception.CloudRuntimeException: Fail to create a
> physical
> > network
> >  at
> >
> com.cloud.network.NetworkServiceImpl.createPhysicalNetwork(NetworkServiceImpl.java:2526)
> > at
> >
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> >  at
> >
> org.apache.cloudstack.api.command.admin.network.CreatePhysicalNetworkCmd.create(CreatePhysicalNetworkCmd.java:160)
> > at com.cloud.api.ApiDispatcher.dispatchCreateCmd(ApiDispatcher.java:104)
> >  a

Re: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Prasanna Santhanam]

Hi - the advanced zone setup should work without troubles. The current
master is broken and a couple of bugs are pending for this after the
baremetal merge done recently. 

CLOUDSTACK-3481 and CLOUDSTACK-1812.

On Sun, Jul 21, 2013 at 08:32:42PM -0700, Andrew Bayer wrote:
> Finally got a chance to work on this - trying to get the basic networking
> setup with Marvin seems to be barfing - I get the following exception in
> the CS logs when I run mvn -Pdeveloper,marvin.setup
> -Dmarvin.config=setup/dev/basic.cfg -pl :cloud-marvin integration-test:
> 
> WARN  [cloud.network.NetworkServiceImpl] (1384506397@qtp-455967038-0:)
> Exception:
> com.cloud.exception.InvalidParameterValueException: Invalid Network Service
> Provider=BaremetalDhcpProvider
>  at
> com.cloud.network.NetworkServiceImpl.addProviderToPhysicalNetwork(NetworkServiceImpl.java:3239)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>  at
> com.cloud.network.NetworkServiceImpl.addDefaultBaremetalProvidersToPhysicalNetwork(NetworkServiceImpl.java:3747)
> at
> com.cloud.network.NetworkServiceImpl.createPhysicalNetwork(NetworkServiceImpl.java:2517)
>  at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> org.apache.cloudstack.api.command.admin.network.CreatePhysicalNetworkCmd.create(CreatePhysicalNetworkCmd.java:160)
>  at com.cloud.api.ApiDispatcher.dispatchCreateCmd(ApiDispatcher.java:104)
> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:476)
>  at com.cloud.api.ApiServer.handleRequest(ApiServer.java:372)
> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:305)
>  at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
>  at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>  at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>  at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
>  at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>  at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>  at org.mortbay.jetty.Server.handle(Server.java:326)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>  at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>  at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>  at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> ERROR [cloud.api.ApiServer] (1384506397@qtp-455967038-0:) unhandled
> exception executing api command: createPhysicalNetwork
> com.cloud.utils.exception.CloudRuntimeException: Fail to create a physical
> network
>  at
> com.cloud.network.NetworkServiceImpl.createPhysicalNetwork(NetworkServiceImpl.java:2526)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>  at
> org.apache.cloudstack.api.command.admin.network.CreatePhysicalNetworkCmd.create(CreatePhysicalNetworkCmd.java:160)
> at com.cloud.api.ApiDispatcher.dispatchCreateCmd(ApiDispatcher.java:104)
>  at com.cloud.api.ApiServer.queueCommand(ApiServer.java:476)
> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:372)
>  at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:305)
> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66)
>  at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>  at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
>  at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>  at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>  at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
>  at org.mortb

Re: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Andrew Bayer]

Finally got a chance to work on this - trying to get the basic networking
setup with Marvin seems to be barfing - I get the following exception in
the CS logs when I run mvn -Pdeveloper,marvin.setup
-Dmarvin.config=setup/dev/basic.cfg -pl :cloud-marvin integration-test:

WARN  [cloud.network.NetworkServiceImpl] (1384506397@qtp-455967038-0:)
Exception:
com.cloud.exception.InvalidParameterValueException: Invalid Network Service
Provider=BaremetalDhcpProvider
 at
com.cloud.network.NetworkServiceImpl.addProviderToPhysicalNetwork(NetworkServiceImpl.java:3239)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
 at
com.cloud.network.NetworkServiceImpl.addDefaultBaremetalProvidersToPhysicalNetwork(NetworkServiceImpl.java:3747)
at
com.cloud.network.NetworkServiceImpl.createPhysicalNetwork(NetworkServiceImpl.java:2517)
 at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
org.apache.cloudstack.api.command.admin.network.CreatePhysicalNetworkCmd.create(CreatePhysicalNetworkCmd.java:160)
 at com.cloud.api.ApiDispatcher.dispatchCreateCmd(ApiDispatcher.java:104)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:476)
 at com.cloud.api.ApiServer.handleRequest(ApiServer.java:372)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:305)
 at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
 at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
 at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
 at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
 at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
 at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
 at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
 at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
ERROR [cloud.api.ApiServer] (1384506397@qtp-455967038-0:) unhandled
exception executing api command: createPhysicalNetwork
com.cloud.utils.exception.CloudRuntimeException: Fail to create a physical
network
 at
com.cloud.network.NetworkServiceImpl.createPhysicalNetwork(NetworkServiceImpl.java:2526)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
 at
org.apache.cloudstack.api.command.admin.network.CreatePhysicalNetworkCmd.create(CreatePhysicalNetworkCmd.java:160)
at com.cloud.api.ApiDispatcher.dispatchCreateCmd(ApiDispatcher.java:104)
 at com.cloud.api.ApiServer.queueCommand(ApiServer.java:476)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:372)
 at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:305)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
 at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
 at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
 at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
 at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
 at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
 at org.mortbay.je

Re: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Chip Childers]

On Tue, Jul 09, 2013 at 05:24:53PM +, Edison Su wrote:
> I think it's the issue https://issues.apache.org/jira/browse/CLOUDSTACK-3139, 
> that your mgt server can't access the url of template.

Yeah, it appears so.  Thanks for searching for me.  Apparently I was too
lazy to actually check myself.  ;-)

RE: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Edison Su]

> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Tuesday, July 09, 2013 7:52 AM
> To: dev@cloudstack.apache.org
> Subject: Re: Using the ACS simulator for jclouds "live" tests - some 
> questions.
> =)
> 
> On Tue, Jul 09, 2013 at 01:44:26AM +, Edison Su wrote:
> > The simulator issue should be fixed on 4.2 branch.
> 
> Do you mind pulling those changes into master?

Done.

RE: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Edison Su]

I think it's the issue https://issues.apache.org/jira/browse/CLOUDSTACK-3139, 
that your mgt server can't access the url of template.

> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Tuesday, July 09, 2013 9:41 AM
> To: 
> Subject: Re: Using the ACS simulator for jclouds "live" tests - some 
> questions.
> =)
> 
> During the setup process, I now get some NumberFormatExceptions.
> 
> I call:  mvn -Pdeveloper,marvin.setup
> -Dmarvin.config=setup/dev/advanced.cfg -pl :cloud-marvin integration-test
> 
> I see in the console:
> 
> ERROR [agent.manager.AgentManagerImpl] (Simulator-Agent-Mgr-1:)
> Monitor DownloadListener says there is an error in the connect process for 4
> due to null
> java.lang.NumberFormatException: null
> at java.lang.Long.parseLong(Long.java:404)
> at java.lang.Long.parseLong(Long.java:483)
> at com.cloud.utils.UriUtils.getRemoteSize(UriUtils.java:113)
> at
> org.apache.cloudstack.storage.image.TemplateServiceImpl.handleTemplate
> Sync(TemplateServiceImpl.java:325)
> at
> com.cloud.storage.download.DownloadListener.processConnect(DownloadL
> istener.java:307)
> at
> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(
> AgentManagerImpl.java:586)
> at
> com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(A
> gentManagerImpl.java:1480)
> at
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(Resource
> ManagerImpl.java:1798)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceMan
> agerImpl.java:824)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManager
> Impl.java:621)
> at
> com.cloud.agent.manager.MockAgentManagerImpl$SystemVMHandler.run(
> MockAgentManagerImpl.java:354)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
> a:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
> va:615)
> at java.lang.Thread.run(Thread.java:722)
> INFO  [agent.manager.AgentManagerImpl] (Simulator-Agent-Mgr-1:) Host 4
> is disconnecting with event AgentDisconnected WARN
> [cloud.resource.ResourceManagerImpl] (Simulator-Agent-Mgr-1:) Unable to
> connect due to
> com.cloud.utils.exception.CloudRuntimeException: Unable to connect 4 at
> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(
> AgentManagerImpl.java:605)
> at
> com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(A
> gentManagerImpl.java:1480)
> at
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(Resource
> ManagerImpl.java:1798)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceMan
> agerImpl.java:824)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManager
> Impl.java:621)
> at
> com.cloud.agent.manager.MockAgentManagerImpl$SystemVMHandler.run(
> MockAgentManagerImpl.java:354)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
> a:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
> va:615)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.NumberFormatException: null at
> java.lang.Long.parseLong(Long.java:404)
> at java.lang.Long.parseLong(Long.java:483)
> at com.cloud.utils.UriUtils.getRemoteSize(UriUtils.java:113)
> at
> org.apache.cloudstack.storage.image.TemplateServiceImpl.handleTemplate
> Sync(TemplateServiceImpl.java:325)
> at
> com.cloud.storage.download.DownloadListener.processConnect(DownloadL
> istener.java:307)
> at
> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(
> AgentManagerImpl.java:586)
> ... 8 more
> Exception in thread "Simulator-Agent-Mgr-1"
> com.cloud.utils.exception.CloudRuntimeException: Cannot transit agent
> status with event AgentDisconnected for host 4, mangement server id is
> 2200358551563,Unable to transition to a new state from Creating via
> AgentDisconnected at
> com.cloud.agent.manager.AgentManagerImpl.agentStatusTransitTo(Agent
> ManagerImpl.java:1411)
> at
> com.cloud.resource.ResourceManagerImpl.markHostAsDisconnected(Resou
> rceManagerImpl.java:1748)
> at
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(Resource
> ManagerImpl.java:1814)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceMan
> agerImpl.java:824)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManager
> Impl.java:621)
> at
> com.cloud.agent.manager.MockAgentManagerImpl$SystemVMHandler.run(
> MockAgentManagerImpl.java:354)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
> a:1145)
> 

Re: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Chip Childers]

During the setup process, I now get some NumberFormatExceptions.

I call:  mvn -Pdeveloper,marvin.setup
-Dmarvin.config=setup/dev/advanced.cfg -pl :cloud-marvin
integration-test

I see in the console:

ERROR [agent.manager.AgentManagerImpl] (Simulator-Agent-Mgr-1:)
Monitor DownloadListener says there is an error in the connect process
for 4 due to null
java.lang.NumberFormatException: null
at java.lang.Long.parseLong(Long.java:404)
at java.lang.Long.parseLong(Long.java:483)
at com.cloud.utils.UriUtils.getRemoteSize(UriUtils.java:113)
at 
org.apache.cloudstack.storage.image.TemplateServiceImpl.handleTemplateSync(TemplateServiceImpl.java:325)
at 
com.cloud.storage.download.DownloadListener.processConnect(DownloadListener.java:307)
at 
com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:586)
at 
com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(AgentManagerImpl.java:1480)
at 
com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1798)
at 
com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:824)
at 
com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:621)
at 
com.cloud.agent.manager.MockAgentManagerImpl$SystemVMHandler.run(MockAgentManagerImpl.java:354)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
INFO  [agent.manager.AgentManagerImpl] (Simulator-Agent-Mgr-1:) Host 4
is disconnecting with event AgentDisconnected
WARN  [cloud.resource.ResourceManagerImpl] (Simulator-Agent-Mgr-1:)
Unable to connect due to
com.cloud.utils.exception.CloudRuntimeException: Unable to connect 4
at 
com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:605)
at 
com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(AgentManagerImpl.java:1480)
at 
com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1798)
at 
com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:824)
at 
com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:621)
at 
com.cloud.agent.manager.MockAgentManagerImpl$SystemVMHandler.run(MockAgentManagerImpl.java:354)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.NumberFormatException: null
at java.lang.Long.parseLong(Long.java:404)
at java.lang.Long.parseLong(Long.java:483)
at com.cloud.utils.UriUtils.getRemoteSize(UriUtils.java:113)
at 
org.apache.cloudstack.storage.image.TemplateServiceImpl.handleTemplateSync(TemplateServiceImpl.java:325)
at 
com.cloud.storage.download.DownloadListener.processConnect(DownloadListener.java:307)
at 
com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:586)
... 8 more
Exception in thread "Simulator-Agent-Mgr-1"
com.cloud.utils.exception.CloudRuntimeException: Cannot transit agent
status with event AgentDisconnected for host 4, mangement server id is
2200358551563,Unable to transition to a new state from Creating via
AgentDisconnected
at 
com.cloud.agent.manager.AgentManagerImpl.agentStatusTransitTo(AgentManagerImpl.java:1411)
at 
com.cloud.resource.ResourceManagerImpl.markHostAsDisconnected(ResourceManagerImpl.java:1748)
at 
com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1814)
at 
com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:824)
at 
com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:621)
at 
com.cloud.agent.manager.MockAgentManagerImpl$SystemVMHandler.run(MockAgentManagerImpl.java:354)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)



On Mon, Jul 8, 2013 at 9:44 PM, Edison Su  wrote:
> The simulator issue should be fixed on 4.2 branch.
>
>> -Original Message-
>> From: Chip Childers [mailto:chip.child...@sungard.com]
>> Sent: Monday, July 08, 2013 1:56 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: Using the ACS simulator for jclouds "live" tests - some 
>> questions.
>> =)
>>
>> On Thu, Jul 04, 2013 at 08:49:52PM +0530, Prasanna Santhanam wrote:
>> > On Mon, Jun 24, 2013 at 11:08:20AM -0700, Andrew Bayer wrote:
>> > > - At least one available simulator template (best-case would be
>> > > Ubuntu
>> > > 10.04 or 12.04) that can be "booted up" out of the box - for
>> > > whatever reason (I honestly can't tell, from the brief

Re: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Chip Childers]

On Tue, Jul 09, 2013 at 01:44:26AM +, Edison Su wrote:
> The simulator issue should be fixed on 4.2 branch.

Do you mind pulling those changes into master?

RE: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Edison Su]

The simulator issue should be fixed on 4.2 branch.

> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Monday, July 08, 2013 1:56 PM
> To: dev@cloudstack.apache.org
> Subject: Re: Using the ACS simulator for jclouds "live" tests - some 
> questions.
> =)
> 
> On Thu, Jul 04, 2013 at 08:49:52PM +0530, Prasanna Santhanam wrote:
> > On Mon, Jun 24, 2013 at 11:08:20AM -0700, Andrew Bayer wrote:
> > > - At least one available simulator template (best-case would be
> > > Ubuntu
> > > 10.04 or 12.04) that can be "booted up" out of the box - for
> > > whatever reason (I honestly can't tell, from the brief dive I did
> > > through the code), all the builtin templates come up with
> > > isready=false in the listTemplates output, which makes jclouds think
> > > there are no templates, etc, etc... I thought this is because
> > > they're all in Allocated state, not Ready, but even changing them to
> Ready in the DB didn't do the trick.
> >
> > The guest OS is something that doesn't matter to the simulator.
> > Whatever you ask for will be deployed. One limitation of the simulator
> > is that it keeps CloudStack 'happy' by telling it it's got whatever is
> > requested. So any negative scenarios will be harder to test without a
> > change in the simulator.
> >
> > The templates issue still exists but with cloudmonkey you can register
> > your template and it will come to ready state:
> >
> > cloudmonkey>> register template format=RAW hypervisor=Simulator
> > cloudmonkey>> isfeatured=true ispublic=true name=tiny displaytext=tiny
> > cloudmonkey>> url=http://test.url.com/test.raw
> > cloudmonkey>> ostypeid=bb62589e-e3cd-11e2-ad39-ea6434daa8ba
> > cloudmonkey>> zoneid=55318568-8d4b-412c-8c10-82f47651eb51
> 
> I've been using [1], and I keep hitting an issue with finding the template 
> prior
> to creating a VM.
> 
> What's the simulator trick to convincing the server that the templates are
> "Ready"?
> 
> Snipped results:
> 
> 
> [INFO] --- exec-maven-plugin:1.2.1:exec (integration-test) @ cloud-marvin --
> - ERROR Test Deploy Virtual Machine ... ERROR ERROR ERROR Test to create
> disk offering ... ok Test to update existing disk offering ... ok Test to 
> delete
> disk offering ... ok test update configuration setting at zone level scope 
> ... ok
> Test guest vlan range dedication ... ok Test to update a physical network and
> extend its vlan ... ERROR ERROR Test to acquire a provisioned public ip
> range ... ok Test to create a portable public ip range ... ok Test public IP 
> range
> dedication ... ok Test for create region ... ok ERROR
> 
> ==
> 
> ERROR: test suite for  'integration.smoke.test_affinity_groups.TestDeployVmWithAffinityGroup'>
> --
> Traceback (most recent call last):
>   File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 208, in 
> run
> self.setUp()
>   File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 291, in
> setUp
> self.setupContext(ancestor)
>   File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 314, in
> setupContext
> try_run(context, names)
>   File "/usr/local/lib/python2.7/dist-packages/nose/util.py", line 469, in
> try_run
> return func()
>   File "/home/sg-
> user/cloudstack/test/integration/smoke/test_affinity_groups.py", line 81, in
> setUpClass
> cls.services["ostype"]
>   File "/usr/local/lib/python2.7/dist-
> packages/marvin/integration/lib/common.py", line 141, in get_template
> ostypeid)
> Exception: Exception: Failed to find template with OSTypeID: f7691800-e7e9-
> 11e2-8591-02004f97000b

RE: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Edison Su]

> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Monday, July 08, 2013 1:56 PM
> To: dev@cloudstack.apache.org
> Subject: Re: Using the ACS simulator for jclouds "live" tests - some 
> questions.
> =)
> 
> On Thu, Jul 04, 2013 at 08:49:52PM +0530, Prasanna Santhanam wrote:
> > On Mon, Jun 24, 2013 at 11:08:20AM -0700, Andrew Bayer wrote:
> > > - At least one available simulator template (best-case would be
> > > Ubuntu
> > > 10.04 or 12.04) that can be "booted up" out of the box - for
> > > whatever reason (I honestly can't tell, from the brief dive I did
> > > through the code), all the builtin templates come up with
> > > isready=false in the listTemplates output, which makes jclouds think
> > > there are no templates, etc, etc... I thought this is because
> > > they're all in Allocated state, not Ready, but even changing them to
> Ready in the DB didn't do the trick.
> >
> > The guest OS is something that doesn't matter to the simulator.
> > Whatever you ask for will be deployed. One limitation of the simulator
> > is that it keeps CloudStack 'happy' by telling it it's got whatever is
> > requested. So any negative scenarios will be harder to test without a
> > change in the simulator.
> >
> > The templates issue still exists but with cloudmonkey you can register
> > your template and it will come to ready state:
> >
> > cloudmonkey>> register template format=RAW hypervisor=Simulator
> > cloudmonkey>> isfeatured=true ispublic=true name=tiny displaytext=tiny
> > cloudmonkey>> url=http://test.url.com/test.raw
> > cloudmonkey>> ostypeid=bb62589e-e3cd-11e2-ad39-ea6434daa8ba
> > cloudmonkey>> zoneid=55318568-8d4b-412c-8c10-82f47651eb51
> 
> I've been using [1], and I keep hitting an issue with finding the template 
> prior
> to creating a VM.
> 
> What's the simulator trick to convincing the server that the templates are
> "Ready"?

Seems there are bugs related to simulator after object store branch is merged, 
I am fixing the simulator to work with latest storage code.

> 
> Snipped results:
> 
> 
> [INFO] --- exec-maven-plugin:1.2.1:exec (integration-test) @ cloud-marvin --
> - ERROR Test Deploy Virtual Machine ... ERROR ERROR ERROR Test to create
> disk offering ... ok Test to update existing disk offering ... ok Test to 
> delete
> disk offering ... ok test update configuration setting at zone level scope 
> ... ok
> Test guest vlan range dedication ... ok Test to update a physical network and
> extend its vlan ... ERROR ERROR Test to acquire a provisioned public ip
> range ... ok Test to create a portable public ip range ... ok Test public IP 
> range
> dedication ... ok Test for create region ... ok ERROR
> 
> ==
> 
> ERROR: test suite for  'integration.smoke.test_affinity_groups.TestDeployVmWithAffinityGroup'>
> --
> Traceback (most recent call last):
>   File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 208, in 
> run
> self.setUp()
>   File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 291, in
> setUp
> self.setupContext(ancestor)
>   File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 314, in
> setupContext
> try_run(context, names)
>   File "/usr/local/lib/python2.7/dist-packages/nose/util.py", line 469, in
> try_run
> return func()
>   File "/home/sg-
> user/cloudstack/test/integration/smoke/test_affinity_groups.py", line 81, in
> setUpClass
> cls.services["ostype"]
>   File "/usr/local/lib/python2.7/dist-
> packages/marvin/integration/lib/common.py", line 141, in get_template
> ostypeid)
> Exception: Exception: Failed to find template with OSTypeID: f7691800-e7e9-
> 11e2-8591-02004f97000b

Re: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Chip Childers]

On Thu, Jul 04, 2013 at 08:49:52PM +0530, Prasanna Santhanam wrote:
> On Mon, Jun 24, 2013 at 11:08:20AM -0700, Andrew Bayer wrote:
> > - At least one available simulator template (best-case would be Ubuntu
> > 10.04 or 12.04) that can be "booted up" out of the box - for whatever
> > reason (I honestly can't tell, from the brief dive I did through the code),
> > all the builtin templates come up with isready=false in the listTemplates
> > output, which makes jclouds think there are no templates, etc, etc... I
> > thought this is because they're all in Allocated state, not Ready, but even
> > changing them to Ready in the DB didn't do the trick.
> 
> The guest OS is something that doesn't matter to the simulator.
> Whatever you ask for will be deployed. One limitation of the simulator
> is that it keeps CloudStack 'happy' by telling it it's got whatever is
> requested. So any negative scenarios will be harder to test without a
> change in the simulator.
> 
> The templates issue still exists but with cloudmonkey you can register
> your template and it will come to ready state:
> 
> cloudmonkey>> register template format=RAW hypervisor=Simulator 
> isfeatured=true ispublic=true name=tiny displaytext=tiny 
> url=http://test.url.com/test.raw 
> ostypeid=bb62589e-e3cd-11e2-ad39-ea6434daa8ba 
> zoneid=55318568-8d4b-412c-8c10-82f47651eb51

I've been using [1], and I keep hitting an issue with finding the
template prior to creating a VM.

What's the simulator trick to convincing the server that the templates
are "Ready"?

Snipped results:


[INFO] --- exec-maven-plugin:1.2.1:exec (integration-test) @ cloud-marvin ---
ERROR
Test Deploy Virtual Machine ... ERROR
ERROR
ERROR
Test to create disk offering ... ok
Test to update existing disk offering ... ok
Test to delete disk offering ... ok
test update configuration setting at zone level scope ... ok
Test guest vlan range dedication ... ok
Test to update a physical network and extend its vlan ... ERROR
ERROR
Test to acquire a provisioned public ip range ... ok
Test to create a portable public ip range ... ok
Test public IP range dedication ... ok
Test for create region ... ok
ERROR

==
ERROR: test suite for 
--
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 208, in run
self.setUp()
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 291, in 
setUp
self.setupContext(ancestor)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 314, in 
setupContext
try_run(context, names)
  File "/usr/local/lib/python2.7/dist-packages/nose/util.py", line 469, in 
try_run
return func()
  File 
"/home/sg-user/cloudstack/test/integration/smoke/test_affinity_groups.py", line 
81, in setUpClass
cls.services["ostype"]
  File 
"/usr/local/lib/python2.7/dist-packages/marvin/integration/lib/common.py", line 
141, in get_template
ostypeid)
Exception: Exception: Failed to find template with OSTypeID: 
f7691800-e7e9-11e2-8591-02004f97000b

Re: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Andrew Bayer]

Awesome! I'll try to play with Marvin and cloudmonkey to do prep for
jclouds tests with the simulator this weekend. Thanks!

A.

On Thu, Jul 4, 2013 at 8:19 AM, Prasanna Santhanam  wrote:

> Andrew - Firstly, apologize for the delayed response. You caught me at
> a bad time when I was attempting to fix the simulator which was broken
> when you sent this email. It took longer than expected to fix and get
> it back in shape.
>
> My comments are inline.
>
> On Mon, Jun 24, 2013 at 11:08:20AM -0700, Andrew Bayer wrote:
> > Hey all -
> >
> > So, at David Nalley's suggestion, I'm trying to run jclouds' live tests
> for
> > ACS against a simulator instance, so that I can actually run the full
> > battery of API tests (including global admin, advanced and basic
> > networking, etc) without having to set up a ton of CloudStack instances.
> In
> > a perfect world, it'd be great if there was a standalone war that used h2
> > for the DB and had different flags to pass to it for different starting
> > settings, but I'd be immensely happy to be able to get the following in
> > place on startup (or an easy script to run to do this on top of the base
> > setup):
> >
> Indeed, this is something David discussed with me and it's a great way
> to run the live tests of jclouds. Right now the simulator jar is
> packaged as a plugin. But there's a few spring related connections to
> be made to get it to work with CloudStack installs. I'll figure out
> how we can package (may be a simulator cloudstack rpm) so you can have
> this running.
>
> The simulator itself only provides the ability to mock hypervisor
> commands but the setup of the network model, zone type etc needs to be
> done from the outside. This can be done with the help of marvin's json
> configuration commands. It's just an API driver that can set things up
> for you. There are samples in our code base under
> setup/dev/advanced.cfg and setup/dev/basic.cfg. The tutorial on marvin
> also has this information on how to generate your own style of
> deployment. [1]
>
> [1] https://cwiki.apache.org/confluence/x/QQzMAQ
>
> > - A hardcoded API/secret key for admin, so that I don't have to go click
> > through to create/get it, and can instead just use the same thing every
> > time.
>
> The regular setup of the simulator can be followed by a couple of
> cloudmonkey (ACS CLI) commands to achieve this.
>
> >
> > - At least one available simulator template (best-case would be Ubuntu
> > 10.04 or 12.04) that can be "booted up" out of the box - for whatever
> > reason (I honestly can't tell, from the brief dive I did through the
> code),
> > all the builtin templates come up with isready=false in the listTemplates
> > output, which makes jclouds think there are no templates, etc, etc... I
> > thought this is because they're all in Allocated state, not Ready, but
> even
> > changing them to Ready in the DB didn't do the trick.
>
> The guest OS is something that doesn't matter to the simulator.
> Whatever you ask for will be deployed. One limitation of the simulator
> is that it keeps CloudStack 'happy' by telling it it's got whatever is
> requested. So any negative scenarios will be harder to test without a
> change in the simulator.
>
> The templates issue still exists but with cloudmonkey you can register
> your template and it will come to ready state:
>
> cloudmonkey>> register template format=RAW hypervisor=Simulator
> isfeatured=true ispublic=true name=tiny displaytext=tiny url=
> http://test.url.com/test.rawostypeid=bb62589e-e3cd-11e2-ad39-ea6434daa8ba
> zoneid=55318568-8d4b-412c-8c10-82f47651eb51
>
>
> >
> > - A fake host for starting fake instances on - this may already be there,
> > but I never got far enough to be sure.
>
> Which tutorial are you following to get through this?
> Here's the right link:
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+with+Python#Marvin-TestingwithPython-
>
> >
> > The above would at least get me able to run the most recent set of live
> > tests I've been working on (testing the new CloudStack implementation of
> > jclouds' cross-compute abstraction for image creation/destruction), but
> in
> > the longer term, I'd absolutely love to be able to run our whole suite
> > against the simulator - obviously, it'd just be an approximation of the
> > real thing, and I'd still try to find real existing ACS setups to finagle
> > access to in order to run the tests against real, live setups, but
> anything
> > I can automate is a huge win for jclouds, and I imagine it'd be very
> useful
> > for libcloud, deltacloud, fog, etc as well!
> >
>
> thanks - indeed it will be. I hope to hear more from your tests.
>
> --
> Prasanna.,
>
> 
> Powered by BigRock.com
>
>

Re: Using the ACS simulator for jclouds "live" tests - some questions. =)

[Prasanna Santhanam]

Andrew - Firstly, apologize for the delayed response. You caught me at
a bad time when I was attempting to fix the simulator which was broken
when you sent this email. It took longer than expected to fix and get
it back in shape.

My comments are inline.

On Mon, Jun 24, 2013 at 11:08:20AM -0700, Andrew Bayer wrote:
> Hey all -
> 
> So, at David Nalley's suggestion, I'm trying to run jclouds' live tests for
> ACS against a simulator instance, so that I can actually run the full
> battery of API tests (including global admin, advanced and basic
> networking, etc) without having to set up a ton of CloudStack instances. In
> a perfect world, it'd be great if there was a standalone war that used h2
> for the DB and had different flags to pass to it for different starting
> settings, but I'd be immensely happy to be able to get the following in
> place on startup (or an easy script to run to do this on top of the base
> setup):
> 
Indeed, this is something David discussed with me and it's a great way
to run the live tests of jclouds. Right now the simulator jar is
packaged as a plugin. But there's a few spring related connections to
be made to get it to work with CloudStack installs. I'll figure out
how we can package (may be a simulator cloudstack rpm) so you can have
this running.

The simulator itself only provides the ability to mock hypervisor
commands but the setup of the network model, zone type etc needs to be
done from the outside. This can be done with the help of marvin's json
configuration commands. It's just an API driver that can set things up
for you. There are samples in our code base under
setup/dev/advanced.cfg and setup/dev/basic.cfg. The tutorial on marvin
also has this information on how to generate your own style of
deployment. [1] 

[1] https://cwiki.apache.org/confluence/x/QQzMAQ 

> - A hardcoded API/secret key for admin, so that I don't have to go click
> through to create/get it, and can instead just use the same thing every
> time.

The regular setup of the simulator can be followed by a couple of
cloudmonkey (ACS CLI) commands to achieve this.

> 
> - At least one available simulator template (best-case would be Ubuntu
> 10.04 or 12.04) that can be "booted up" out of the box - for whatever
> reason (I honestly can't tell, from the brief dive I did through the code),
> all the builtin templates come up with isready=false in the listTemplates
> output, which makes jclouds think there are no templates, etc, etc... I
> thought this is because they're all in Allocated state, not Ready, but even
> changing them to Ready in the DB didn't do the trick.

The guest OS is something that doesn't matter to the simulator.
Whatever you ask for will be deployed. One limitation of the simulator
is that it keeps CloudStack 'happy' by telling it it's got whatever is
requested. So any negative scenarios will be harder to test without a
change in the simulator.

The templates issue still exists but with cloudmonkey you can register
your template and it will come to ready state:

cloudmonkey>> register template format=RAW hypervisor=Simulator isfeatured=true 
ispublic=true name=tiny displaytext=tiny url=http://test.url.com/test.raw 
ostypeid=bb62589e-e3cd-11e2-ad39-ea6434daa8ba 
zoneid=55318568-8d4b-412c-8c10-82f47651eb51


> 
> - A fake host for starting fake instances on - this may already be there,
> but I never got far enough to be sure.

Which tutorial are you following to get through this?
Here's the right link:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+with+Python#Marvin-TestingwithPython-

> 
> The above would at least get me able to run the most recent set of live
> tests I've been working on (testing the new CloudStack implementation of
> jclouds' cross-compute abstraction for image creation/destruction), but in
> the longer term, I'd absolutely love to be able to run our whole suite
> against the simulator - obviously, it'd just be an approximation of the
> real thing, and I'd still try to find real existing ACS setups to finagle
> access to in order to run the tests against real, live setups, but anything
> I can automate is a huge win for jclouds, and I imagine it'd be very useful
> for libcloud, deltacloud, fog, etc as well!
> 

thanks - indeed it will be. I hope to hear more from your tests.

-- 
Prasanna.,


Powered by BigRock.com

Using the ACS simulator for jclouds "live" tests - some questions. =)

[Andrew Bayer]

Hey all -

So, at David Nalley's suggestion, I'm trying to run jclouds' live tests for
ACS against a simulator instance, so that I can actually run the full
battery of API tests (including global admin, advanced and basic
networking, etc) without having to set up a ton of CloudStack instances. In
a perfect world, it'd be great if there was a standalone war that used h2
for the DB and had different flags to pass to it for different starting
settings, but I'd be immensely happy to be able to get the following in
place on startup (or an easy script to run to do this on top of the base
setup):

- A hardcoded API/secret key for admin, so that I don't have to go click
through to create/get it, and can instead just use the same thing every
time.

- At least one available simulator template (best-case would be Ubuntu
10.04 or 12.04) that can be "booted up" out of the box - for whatever
reason (I honestly can't tell, from the brief dive I did through the code),
all the builtin templates come up with isready=false in the listTemplates
output, which makes jclouds think there are no templates, etc, etc... I
thought this is because they're all in Allocated state, not Ready, but even
changing them to Ready in the DB didn't do the trick.

- A fake host for starting fake instances on - this may already be there,
but I never got far enough to be sure.

The above would at least get me able to run the most recent set of live
tests I've been working on (testing the new CloudStack implementation of
jclouds' cross-compute abstraction for image creation/destruction), but in
the longer term, I'd absolutely love to be able to run our whole suite
against the simulator - obviously, it'd just be an approximation of the
real thing, and I'd still try to find real existing ACS setups to finagle
access to in order to run the tests against real, live setups, but anything
I can automate is a huge win for jclouds, and I imagine it'd be very useful
for libcloud, deltacloud, fog, etc as well!

A.

Re: jclouds code

[Mike Tutkowski]

Yeah, that information is part of the Compute Offering.

So, when you kick off a VM in CloudStack, you have to specify the ID of the
Compute Offering you are using.


On Wed, May 1, 2013 at 5:42 PM, Seif Eddine Jemli  wrote:

> i couldn't see the parameters of the VM in the method, for example the CPU
> and the RAM for each VM to deploy.
> does it exist in the method parameters?
>
>
>
> 2013/5/2 Mike Tutkowski 
>
>> Hi,
>>
>> Yes, the code to deploy a VM should be related to a method called
>> getDeployVirtualMachineCommand.  This will build up the command you want to
>> send to the executeCloudStackCommand method.
>>
>> If you want it to run on XenServer, you will of course need to make sure
>> you have a XenServer host added into CloudStack.  In the cluster this host
>> resides, you'll need Primary Storage with a Storage Tag that you define.
>>
>> If you reference this Storage Tag from a Compute Offering that you
>> execute, the VM should be placed on the Primary Storage that's being used
>> by your XenServer host.
>>
>> I will CC the CloudStack e-mail list so other people can feel free to
>> chime in with suggestions.
>>
>> The reason I sent my code directly to you was because the CloudStack
>> e-mail list will filter attachments out.
>>
>> Having the entire group in on the conversation might be useful regarding
>> comments they might make.
>>
>> Talk to you later!
>>
>>
>> On Wed, May 1, 2013 at 5:15 PM, Seif Eddine Jemli <
>> seifeddineje...@gmail.com> wrote:
>>
>>> thanks Mike.
>>>
>>> I am a beginner. i am doing this for a school project (i am a computer
>>> science student, i graduate the next year)
>>> does this code contain part that enable me to deploy virtual machines on
>>> xenserver hosts?
>>>
>>>
>>>
>>>
>>> 2013/5/2 Mike Tutkowski 
>>>
 Here you go.

 So, just a couple of things to note:

 This little program demos how to issue CloudStack API commands, but it
 also demos how to interact with XenServer and ESX as well as my company's
 SAN (SolidFire).  Of course just ignore what is not of interest to you. :)
  You will most likely find the executeCloudStackCommand method to be of
 interest.

 The name of the program (CreateVolume) is completely legacy as this
 program does a lot more than create a volume on our SAN these days. :)

 You'll see a lot of code is commented out.  I tend to comment code out
 and in within this program depending on what I want test with it.

 Let me know if you have any questions.


 On Wed, May 1, 2013 at 4:14 PM, Seif Eddine Jemli <
 seifeddineje...@gmail.com> wrote:

> thanks a lot .
>
>
> 2013/5/1 Mike Tutkowski 
>
>> Hi,
>>
>> No problem.
>>
>> At the moment, I do not have access to the file from where I'm
>> located, but I should be able to send it to you within a few hours.
>>
>> Talk to you later,
>>
>>
>> On Wed, May 1, 2013 at 4:07 PM, Seif Eddine Jemli <
>> seifeddineje...@gmail.com> wrote:
>>
>>> Hi Mike,
>>>
>>> thanks for you reply on the mailing list.
>>>
>>> i do have interest in your code.
>>>
>>> can you please send it to me to use it as a tuto?
>>>
>>> thanks in advance
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>>  *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the 
>> cloud
>> *™*
>>
>
>


 --
 *Mike Tutkowski*
 *Senior CloudStack Developer, SolidFire Inc.*
 e: mike.tutkow...@solidfire.com
 o: 303.746.7302
 Advancing the way the world uses the 
 cloud
 *™*

>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the 
>> cloud
>> *™*
>>
>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: jclouds code

[Mike Tutkowski]

Hi,

Yes, the code to deploy a VM should be related to a method called
getDeployVirtualMachineCommand.  This will build up the command you want to
send to the executeCloudStackCommand method.

If you want it to run on XenServer, you will of course need to make sure
you have a XenServer host added into CloudStack.  In the cluster this host
resides, you'll need Primary Storage with a Storage Tag that you define.

If you reference this Storage Tag from a Compute Offering that you execute,
the VM should be placed on the Primary Storage that's being used by your
XenServer host.

I will CC the CloudStack e-mail list so other people can feel free to chime
in with suggestions.

The reason I sent my code directly to you was because the CloudStack e-mail
list will filter attachments out.

Having the entire group in on the conversation might be useful regarding
comments they might make.

Talk to you later!


On Wed, May 1, 2013 at 5:15 PM, Seif Eddine Jemli  wrote:

> thanks Mike.
>
> I am a beginner. i am doing this for a school project (i am a computer
> science student, i graduate the next year)
> does this code contain part that enable me to deploy virtual machines on
> xenserver hosts?
>
>
>
>
> 2013/5/2 Mike Tutkowski 
>
>> Here you go.
>>
>> So, just a couple of things to note:
>>
>> This little program demos how to issue CloudStack API commands, but it
>> also demos how to interact with XenServer and ESX as well as my company's
>> SAN (SolidFire).  Of course just ignore what is not of interest to you. :)
>>  You will most likely find the executeCloudStackCommand method to be of
>> interest.
>>
>> The name of the program (CreateVolume) is completely legacy as this
>> program does a lot more than create a volume on our SAN these days. :)
>>
>> You'll see a lot of code is commented out.  I tend to comment code out
>> and in within this program depending on what I want test with it.
>>
>> Let me know if you have any questions.
>>
>>
>> On Wed, May 1, 2013 at 4:14 PM, Seif Eddine Jemli <
>> seifeddineje...@gmail.com> wrote:
>>
>>> thanks a lot .
>>>
>>>
>>> 2013/5/1 Mike Tutkowski 
>>>
 Hi,

 No problem.

 At the moment, I do not have access to the file from where I'm located,
 but I should be able to send it to you within a few hours.

 Talk to you later,


 On Wed, May 1, 2013 at 4:07 PM, Seif Eddine Jemli <
 seifeddineje...@gmail.com> wrote:

> Hi Mike,
>
> thanks for you reply on the mailing list.
>
> i do have interest in your code.
>
> can you please send it to me to use it as a tuto?
>
> thanks in advance
>



 --
 *Mike Tutkowski*
  *Senior CloudStack Developer, SolidFire Inc.*
 e: mike.tutkow...@solidfire.com
 o: 303.746.7302
 Advancing the way the world uses the 
 cloud
 *™*

>>>
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the 
>> cloud
>> *™*
>>
>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: Jclouds

[Mike Tutkowski]

I have been using the CloudStack API from a test Java client I wrote.

If the code for this is of interest to you, just let me know.


On Wed, May 1, 2013 at 11:46 AM, David Nalley  wrote:

> On Wed, May 1, 2013 at 1:14 PM, Seif Eddine Jemli
>  wrote:
> > Hi,
> >
> >
> >
> > i am trying to implement a java code that should enable me to do simple
> > things: deploy Virtual machines for example.
> >
> >
> > I was advised to use Jclouds API, the problem is that i didn't find
> > "enough"  documentation on how to use Jclouds on a cloudstack private
> cloud
> > .
> >
> > Where can i find a jclouds code on cloudstack that i can use as a start
> > (like a tuto)?
> >
> > thanks
>
>
> This isn't the first time this has come up.
>
> Let me paste an IRC conversation that should help you make progress.
>
> 13:27 < adrianc> ke4qqq I think using the raw cloudstack api isn't
> something there's a very nice guide on
> 13:28 < adrianc> however, they can get started quite easily using
> jclouds-examples/compute-basics
> 13:28 < adrianc> and pass in the url to the cloudstack instance
> 13:28 < adrianc> abayer also has cloudcat
> 13:28 < adrianc> which is a cloudstack app
> 13:28 < adrianc> yp
> 13:28 < adrianc> ke4qqq the other thing is that there's tons of
> examples, if one is ok looking at live tests
> 13:29 < adrianc> ex. apis/cloudstack/src/test/java/...*LiveTest.java
> 13:29 < adrianc> contextBuilder.overrides
> 13:30 < adrianc> ke4qqq anyway we can do a lot better on asf in
> cloudstack doc stuff
>
> The cloudcat referenced is: https://github.com/abayer/cloudcat  <--
> which uses jclouds for CloudStack, so should give you an example.
>
> --David
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*

Re: Jclouds

[David Nalley]

On Wed, May 1, 2013 at 1:14 PM, Seif Eddine Jemli
 wrote:
> Hi,
>
>
>
> i am trying to implement a java code that should enable me to do simple
> things: deploy Virtual machines for example.
>
>
> I was advised to use Jclouds API, the problem is that i didn't find
> "enough"  documentation on how to use Jclouds on a cloudstack private cloud
> .
>
> Where can i find a jclouds code on cloudstack that i can use as a start
> (like a tuto)?
>
> thanks


This isn't the first time this has come up.

Let me paste an IRC conversation that should help you make progress.

13:27 < adrianc> ke4qqq I think using the raw cloudstack api isn't
something there's a very nice guide on
13:28 < adrianc> however, they can get started quite easily using
jclouds-examples/compute-basics
13:28 < adrianc> and pass in the url to the cloudstack instance
13:28 < adrianc> abayer also has cloudcat
13:28 < adrianc> which is a cloudstack app
13:28 < adrianc> yp
13:28 < adrianc> ke4qqq the other thing is that there's tons of
examples, if one is ok looking at live tests
13:29 < adrianc> ex. apis/cloudstack/src/test/java/...*LiveTest.java
13:29 < adrianc> contextBuilder.overrides
13:30 < adrianc> ke4qqq anyway we can do a lot better on asf in
cloudstack doc stuff

The cloudcat referenced is: https://github.com/abayer/cloudcat  <--
which uses jclouds for CloudStack, so should give you an example.

--David

Jclouds

[Seif Eddine Jemli]

Hi,



i am trying to implement a java code that should enable me to do simple
things: deploy Virtual machines for example.


I was advised to use Jclouds API, the problem is that i didn't find
"enough"  documentation on how to use Jclouds on a cloudstack private cloud
.

Where can i find a jclouds code on cloudstack that i can use as a start
(like a tuto)?

thanks