On Jun 23, 2014, at 3:45 PM, sebgoa <run...@gmail.com> wrote:
>
> On Jun 23, 2014, at 3:33 PM, Prasanna Santhanam <t...@apache.org> wrote:
>
>> Be advised that advanced.cfg might be adding the NFS store as
>> secondary storage as opposed to cacheStorage that the S3 image store
>> needs it to be. At least that's how it was the last time I ran it.
>
> Thanks.
>
> That's exactly what I am trying to figure out. Can we migrate from a NFS
> secondary storage to an object store.
>
> If I move the NFS to be a cache store then I can add a new ImageStore. But
> the updatecloudstoobjectstore which is supposed to do everything at once,
> does not work for me.
>
Ok, problem solved. When using the API, the provider key is case sensitive. so
you need to pass S3 and not s3 :(
>>
>> On Mon, Jun 23, 2014 at 02:45:55PM +0200, sebgoa wrote:
>>>
>>> On Jun 23, 2014, at 2:30 PM, Koushik Das <koushik....@citrix.com> wrote:
>>>
>>>> This has nothing to do with the zoneId. If you search the source code for
>>>> the error message "Failed to add data store: DataCenter id is null, and
>>>> simulator image store has to be associated with a data center", it points
>>>> toSimulatorImageStoreLifeCycleImpl.initialize() method. This indicates
>>>> that somehow the image store provider for Simulator is getting picked up.
>>>> For S3 provider type, the S3ImageStoreLifeCycleImpl should get invoked and
>>>> in that the zoneId is not used so there won't be issues.
>>>
>>> Happy to discuss it on IRC, if you could join.
>>>
>>> in SimulatorImageStoreLifeCycleImpl.initialize() on 4.4-forward I see:
>>>
>>> Long dcId = (Long)dsInfos.get("zoneId");
>>> String url = (String)dsInfos.get("url");
>>> String name = (String)dsInfos.get("name");
>>> if (name == null) {
>>> name = url;
>>> }
>>> ...
>>>
>>> if (dcId == null) {
>>> throw new InvalidParameterValueException("DataCenter id is null,
>>> and simulator image store has to be associated with a data center");
>>> }
>>>
>>> So my non Java expertise tells me that dcId is null because zoneId is not
>>> set ???
>>>
>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Sebastien Goasguen [mailto:run...@gmail.com]
>>>> Sent: Monday, 23 June 2014 5:47 PM
>>>> To: dev@cloudstack.apache.org
>>>> Subject: Re: S3 use with simulator
>>>>
>>>>
>>>> On Jun 23, 2014, at 7:14 AM, Koushik Das <koushik....@citrix.com> wrote:
>>>>
>>>>> This is the request
>>>>>
>>>>> http://localhost:8096/api?command=updateCloudToUseObjectStore&url=http
>>>>> ://localhost:1234/test&name=test&provider=S3&details[0].key=accesskey&
>>>>> details[0].value=dummy&details[1].key=secretkey&details[1].value=dummy
>>>>
>>>> Any chance you can try with a signed api request ?
>>>>
>>>> It looks like it's not finding a zoneId ...hence dcId is null....
>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: sebgoa [mailto:run...@gmail.com]
>>>>> Sent: Monday, 23 June 2014 4:40 PM
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: Re: S3 use with simulator
>>>>>
>>>>>
>>>>> On Jun 23, 2014, at 12:51 PM, Koushik Das <koushik....@citrix.com> wrote:
>>>>>
>>>>>> inline
>>>>>>
>>>>>> On 23-Jun-2014, at 4:05 PM, Sebastien Goasguen <run...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Koushik, thanks for the hint, I checked out 4.4-forward and made
>>>>>>> that small change.
>>>>>>>
>>>>>>> I still add the same error:
>>>>>>>
>>>>>>>> update cloudtouseobjectstore url=http://localhost:9081/riak-cs
>>>>>>>> name=riakcs provider=s3 details[0].key=accesskey
>>>>>>>> details[0].value=ZMJD6-90S2MST4NZMK1Z details[1].key=secretkey
>>>>>>>> details[1].value=ecJlaZebrYKj_qYaIfzlRR_1izojGVWjBRFx0Q==
>>>>>>> 530: Failed to add data store: DataCenter id is null, and simulator
>>>>>>> image store has to be associated with a data center
>>>>>>>
>>>>>>> However this actually *removed* the NFS store (from the simulated basic
>>>>>>> zone).
>>>>>>>
>>>>>>
>>>>>> I verified using advanced zone, not sure if there are any issues with
>>>>>> basic zone. The API response I get is the following:
>>>>>> <updatecloudtouseobjectstoreresponse
>>>>>> cloud-stack-version="4.4.0-SNAPSHOT"><imagestore><id>55b7e6f7-ec27-4d
>>>>>> a
>>>>>> 8-b4a0-b209a3e3d825</id><name>test</name><url>http://localhost:1234/t
>>>>>> e
>>>>>> st</url><protocol>http</protocol><providername>S3</providername><scop
>>>>>> e
>>>>>>> REGION</scope><details><name>secretkey</name><value>dummy</value></d
>>>>>>> e
>>>>>> tails><details><name>accesskey</name><value>dummy</value></details></
>>>>>> tails>i
>>>>>> magestore></updatecloudtouseobjectstoreresponse>
>>>>>>
>>>>>
>>>>> Hum, strange. I just checked with an advanced zone from
>>>>> setup/dev/advanced.cfg and I get the same error 'datacenter id is null'.
>>>>>
>>>>> Can you send the exact request that you sent ?
>>>>>
>>>>>> Also in the image_store table the entries look like below, note that the
>>>>>> 'role' of NFS store changed to 'ImageCache'.
>>>>>>
>>>>>> '1', 'nfs://10.147.28.6:/export/home/sandbox/secondary', 'NFS',
>>>>>> 'nfs', 'nfs://10.147.28.6:/export/home/sandbox/secondary', '1',
>>>>>> 'ZONE', 'ImageCache', '8eb3e701-7e2b-4925-87f8-f8ea5f97b863',
>>>>>> '/mnt/05b33ef7-da6a-33eb-9981-b4efb90ed99b/', '2014-06-23 06:27:58',
>>>>>> NULL, NULL, NULL '2', 'test', 'S3', 'http',
>>>>>> 'http://localhost:1234/test', NULL, 'REGION', 'Image',
>>>>>> '55b7e6f7-ec27-4da8-b4a0-b209a3e3d825', NULL, '2014-06-23 07:37:55',
>>>>>> NULL, NULL, NULL
>>>>>>
>>>>>
>>>>> Yes I can confirm this. It gets listed in the UI under 'secondary staging
>>>>> store', and it's in the db.
>>>>>
>>>>>>
>>>>>>> I was then able to add:
>>>>>>>
>>>>>>>> add imagestore name=riakcs provider=S3
>>>>>>>> url=http://localhost:9081/riak-cs details[0].key=accesskey
>>>>>>>> details[0].value=ZMJD6-90S2MST4NZMK1Z details[1].key=secretkey
>>>>>>>> details[1].value=ecJlaZebrYKj_qYaIfzlRR_1izojGVWjBRFx0Q==
>>>>>>> imagestore:
>>>>>>> name = riakcs
>>>>>>> id = 1a60d62a-c1e9-4d1c-8b35-d5cd687f6de4
>>>>>>> details:
>>>>>>> +-----------+------------------------------------------+
>>>>>>> | name | value |
>>>>>>> +-----------+------------------------------------------+
>>>>>>> | secretkey | ecJlaZebrYKj_qYaIfzlRR_1izojGVWjBRFx0Q== |
>>>>>>> | accesskey | ZMJD6-90S2MST4NZMK1Z |
>>>>>>> +-----------+------------------------------------------+
>>>>>>> protocol = http
>>>>>>> providername = S3
>>>>>>> scope = REGION
>>>>>>> url = http://localhost:9081/riak-cs
>>>>>>>
>>>>>>> Apparently successfully, even though the removal of the NFS store
>>>>>>> entirely is a bit worrisome.
>>>>>>>
>>>>>>> On Jun 23, 2014, at 4:32 AM, Koushik Das <koushik....@citrix.com> wrote:
>>>>>>>
>>>>>>>> The error message is from the simulator storage plugin, not sure why
>>>>>>>> you are getting it even after setting provider types as S3.
>>>>>>>> I tried the same on a 4.4-forward setup and got a different error (as
>>>>>>>> below).
>>>>>>>> "You can only add new image stores from the same provider NFS already
>>>>>>>> added"
>>>>>>>>
>>>>>>>> This error is there as the simulator storage provider doesn't support
>>>>>>>> migration to object store. Once I fixed it, the image store got added
>>>>>>>> successfully and the NFS store was converted to a cache store.
>>>>>>>> This is the change I made
>>>>>>>>
>>>>>>>> diff --git
>>>>>>>> a/plugins/hypervisors/simulator/src/org/apache/cloudstack/storage/d
>>>>>>>> a tastore/lifecycle/SimulatorImageStoreLifeCycleImpl.java
>>>>>>>> b/plugins/hypervisors/simulator/src/org/apache/cloudstack/storage/d
>>>>>>>> a tastore/lifecycle/SimulatorImageStoreLifeCycleImpl.java
>>>>>>>> index edf1e28..537c7bd 100644
>>>>>>>> ---
>>>>>>>> a/plugins/hypervisors/simulator/src/org/apache/cloudstack/storage/d
>>>>>>>> a tastore/lifecycle/SimulatorImageStoreLifeCycleImpl.java
>>>>>>>> +++ b/plugins/hypervisors/simulator/src/org/apache/cloudstack/stora
>>>>>>>> +++ g e/datastore/lifecycle/SimulatorImageStoreLifeCycleImpl.java
>>>>>>>> @@ -133,7 +133,7 @@ public class SimulatorImageStoreLifeCycleImpl
>>>>>>>> implements ImageStoreLifeCycle {
>>>>>>>> */
>>>>>>>> @Override
>>>>>>>> public boolean migrateToObjectStore(DataStore store) {
>>>>>>>> - return false;
>>>>>>>> + return imageStoreHelper.convertToStagingStore(store);
>>>>>>>> }
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Min Chen [mailto:min.c...@citrix.com]
>>>>>>>> Sent: Saturday, 21 June 2014 3:44 AM
>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>> Subject: Re: S3 use with simulator
>>>>>>>>
>>>>>>>> Got it. The error seems from simulator, maybe Koushik has better idea
>>>>>>>> on that. I didn't use that on simulator env though.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> -min
>>>>>>>>
>>>>>>>> On 6/20/14 11:05 AM, "Sebastien Goasguen" <run...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I already have an Nsf secondary storage. I understood that i could
>>>>>>>>> not add a second secondary storage that is s3 based, and that the
>>>>>>>>> only way was to use the updatecloud.... Api
>>>>>>>>>
>>>>>>>>> -Sebastien
>>>>>>>>>
>>>>>>>>>> On 20 Jun 2014, at 18:49, Min Chen <min.c...@citrix.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Sebastien,
>>>>>>>>>>
>>>>>>>>>> I am curious why you are using updateCloudToUseObjectStore api to
>>>>>>>>>> add S3 as a secondary store in your data center? To set up a
>>>>>>>>>> data center using S3, you need to use addImageStore api.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> -min
>>>>>>>>>>
>>>>>>>>>>> On 6/20/14 7:47 AM, "Sebastien Goasguen" <run...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> I switch to provider=S3 but I get:
>>>>>>>>>>>
>>>>>>>>>>>> 530: Failed to add data store: DataCenter id is null, and
>>>>>>>>>>>> simulator image store has to be associated with a data center
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> On Jun 20, 2014, at 7:07 AM, Koushik Das
>>>>>>>>>>>> <koushik....@citrix.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> As per the code the following are the supported image store
>>>>>>>>>>>> provider types
>>>>>>>>>>>>
>>>>>>>>>>>> // constants for provider names static final String NFS_IMAGE =
>>>>>>>>>>>> "NFS"; static final String S3_IMAGE = "S3"; static final String
>>>>>>>>>>>> SWIFT_IMAGE = "Swift"; static final String SAMPLE_IMAGE =
>>>>>>>>>>>> "Sample"; static final String SMB = "NFS"; static final String
>>>>>>>>>>>> DEFAULT_PRIMARY = "DefaultPrimary";
>>>>>>>>>>>>
>>>>>>>>>>>> Try passing S3 in the provider.
>>>>>>>>>>>> BTW when you setup simulator the image provider is NFS.
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Sebastien Goasguen [mailto:run...@gmail.com]
>>>>>>>>>>>> Sent: Thursday, 19 June 2014 5:08 PM
>>>>>>>>>>>> To: dev@cloudstack.apache.org
>>>>>>>>>>>> Subject: S3 use with simulator
>>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I am using the simulator and started a basic zone.
>>>>>>>>>>>> I have an S3 object store locally (riakCS), and I am trying to
>>>>>>>>>>>> add it to the infra using the 'cloudtouseobjectstore' api with
>>>>>>>>>>>> cloudmonkey.
>>>>>>>>>>>>
>>>>>>>>>>>> I tried with:
>>>>>>>>>>>>
>>>>>>>>>>>>> update cloudtouseobjectstore url=http://localhost:9081/riak-cs
>>>>>>>>>>>>> name=riak provider=riakcs
>>>>>>>>>>>>>
>>>>>>>>>>>>> details[0].key=accesskey&details[0].value=STU6Z-ZMK1TPMDAXL9I1
>>>>>>>>>>>>> &
>>>>>>>>>>>>> deta
>>>>>>>>>>>>> ils
>>>>>>>>>>>>> [1
>>>>>>>>>>>>>
>>>>>>>>>>>>> ].key=secretkey&details[1].value=8OuY3mHDXihu0Tdb2aVJ4vuYZLBAl
>>>>>>>>>>>>> 5
>>>>>>>>>>>>> Z7Ni
>>>>>>>>>>>>> WKs
>>>>>>>>>>>>> g=
>>>>>>>>>>>>> =
>>>>>>>>>>>> 530: Failed to add data store: DataCenter id is null, and
>>>>>>>>>>>> simulator image store has to be associated with a data center
>>>>>>>>>>>>
>>>>>>>>>>>> I am not sure if the arguments are right, has anyone done this
>>>>>>>>>>>> before ?
>>>>>>>>>>>>
>>>>>>>>>>>> thanks,
>>>>>>>>>>>>
>>>>>>>>>>>> -sebastien
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>
>> --
>> Prasanna.,
>>
>> ------------------------
>> Powered by BigRock.com
>>
>
