Hi Udara, Did try with a fresh BAM pack? Are you pointing to external
Cassandra?
On Mon, May 5, 2014 at 1:44 PM, Udara Rathnayake <uda...@wso2.com> wrote:
> I will try to debug
> org.wso2.carbon.databridge.persistence.cassandra.datastore.CassandraConnector
> and see.
>
>
> On Mon, May 5, 2014 at 5:05 PM, Udara Rathnayake <uda...@wso2.com> wrote:
>
>> Hi Inosh,
>>
>> I tried the given toolbox, inside a fresh BAM 2.4.1 pack with the default
>> sample publisher. I still get the same warning[1].
>> I think this behavior has nothing to with my environment?
>>
>> [1].
>>
>> [2014-05-05 16:52:22,450] INFO
>> {org.wso2.carbon.bam.toolbox.deployer.core.BAMToolBoxDeployer} - Deploying
>> file:/home/udara/wso2/playground/wso2bam-2.4.1/repository/deployment/server/bam-toolbox/KPI_Phone_Retail_Store.tbox
>> [2014-05-05 16:52:22,944] INFO
>> {org.wso2.carbon.databridge.core.DataBridge} - admin connected
>> [2014-05-05 16:52:22,950] INFO
>> {org.wso2.carbon.databridge.streamdefn.registry.datastore.RegistryStreamDefinitionStore}
>> - Stream definition added to registry successfully :
>> org.wso2.bam.phone.retail.store.kpi:1.0.0
>> [2014-05-05 16:52:23,005] INFO
>> {org.wso2.carbon.ntask.core.impl.AbstractQuartzTaskManager} - Task
>> scheduled: [-1234][HIVE_TASK][phone_retail_store_script]
>> [2014-05-05 16:52:23,393] INFO
>> {org.wso2.carbon.bam.toolbox.deployer.core.BAMToolBoxDeployer} - Deployed
>> successfully file:
>> /home/udara/wso2/playground/wso2bam-2.4.1/repository/deployment/server/bam-toolbox/KPI_Phone_Retail_Store.tbox
>> [2014-05-05 16:52:31,154] INFO
>> {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil} -
>> 'admin@carbon.super [-1234]' logged in at [2014-05-05 16:52:31,154+0530]
>> [2014-05-05 16:54:28,562] INFO
>> {org.wso2.carbon.databridge.core.DataBridge} - admin connected
>> [2014-05-05 16:54:28,607] INFO
>> {org.wso2.carbon.databridge.persistence.cassandra.datastore.ClusterFactory}
>> - Initializing cluster
>> [2014-05-05 16:54:30,593] INFO
>> {org.wso2.carbon.databridge.core.DataBridge} -
>> f95ff042-07db-4a28-8035-fc5189edfef6 disconnected
>> [2014-05-05 16:55:12,440] WARN
>> {org.wso2.carbon.databridge.persistence.cassandra.datastore.CassandraConnector}
>> - Secondary Index creation is not successful...
>> me.prettyprint.hector.api.exceptions.HInvalidRequestException:
>> InvalidRequestException(why:Illegal index name payload_brand_index)
>>
>> Regards,
>> UdaraR
>>
>>
>> On Mon, May 5, 2014 at 4:38 PM, Inosh Goonewardena <in...@wso2.com>wrote:
>>
>>> Hi Udara,
>>>
>>> I couldn't try with your toolbox since I was unable to get your sample
>>> data publisher to work in the first go. Since I am on site I didn't spend
>>> time on fixing data publisher issue. However, I have tested secondary index
>>> creation with kpi-definition toolbox we ship with BAM by adding same
>>> attribute('brand') as secondary index and it is working properly. Please
>>> find the attached kpi-definition toolbox. Deploy it and publish data by
>>> using the standard samples. So until I further investigate what is wrong in
>>> your scenario, please use the working toolbox that I have attached and try
>>> to make your sample working.
>>>
>>>
>>> On Mon, May 5, 2014 at 11:29 AM, Udara Rathnayake <uda...@wso2.com>wrote:
>>>
>>>> Hi Inosh,
>>>>
>>>> Please find the modified kpi-definition sample which I have used to
>>>> publish data.
>>>>
>>>> Regards,
>>>> UdaraR
>>>>
>>>>
>>>> On Mon, May 5, 2014 at 1:32 PM, Udara Rathnayake <uda...@wso2.com>wrote:
>>>>
>>>>> Hi Inosh,
>>>>>
>>>>> I have attached the toolbox, Please note that I have modified HL7.tbox
>>>>> to suits with the kpi-definition BAM sample[1].
>>>>> In addition I have introduced 'brand' secondary index and modified
>>>>> kpi-definition sample[1] accordingly.
>>>>> Thanks.
>>>>>
>>>>> [1] <BAM_2.4.1_HOME>/samples/kpi-definition
>>>>>
>>>>> Regards,
>>>>> UdaraR
>>>>>
>>>>>
>>>>> On Mon, May 5, 2014 at 1:18 PM, Inosh Goonewardena <in...@wso2.com>wrote:
>>>>>
>>>>>> Hi Udara,
>>>>>>
>>>>>> Please attach your toolbox
>>>>>>
>>>>>>
>>>>>> On Mon, May 5, 2014 at 9:02 AM, Udara Rathnayake <uda...@wso2.com>wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I tried the same toolbox in released BAM 2.4.1[1]. But still I get
>>>>>>> the same secondary index creation failure,
>>>>>>>
>>>>>>> [2014-05-05 12:06:50,248] WARN
>>>>>>> {org.wso2.carbon.databridge.persistence.cassandra.datastore.CassandraConnector}
>>>>>>> - Secondary Index creation is not successful...
>>>>>>> me.prettyprint.hector.api.exceptions.HInvalidRequestException:
>>>>>>> InvalidRequestException(why:Illegal index name payload_brand_index)
>>>>>>>
>>>>>>> Everything(toolbox deployment,Stream definition creation in
>>>>>>> registry,event publish to Cassandra) works as expected, only secondary
>>>>>>> index creation fails with the above warning.
>>>>>>> Please note that I'm not using customindexes here, only secondary
>>>>>>> indexes.
>>>>>>>
>>>>>>> @Bhathiya,
>>>>>>>
>>>>>>> In my toolbox I have following,
>>>>>>>
>>>>>>> streams.definitions=defn1
>>>>>>> streams.definitions.defn1.filename=org.wso2.social.activity
>>>>>>> streams.definitions.defn1.username=admin
>>>>>>> streams.definitions.defn1.password=admin
>>>>>>> streams.definitions.defn1.description=This is the datastream for
>>>>>>> social framework
>>>>>>> streams.definitions.defn1.secondaryindexes=brand
>>>>>>>
>>>>>>> I don't see any space character in this definition(This is the same
>>>>>>> which I have used early as well).
>>>>>>> I guess you made the above comment based on the warning
>>>>>>> message(why:Illegal index name payload_brand_index)
>>>>>>>
>>>>>>> [1] http://ec2-54-198-211-48.compute-1.amazonaws.com/chunk09/wso2bam
>>>>>>> -2.4.1.zip
>>>>>>>
>>>>>>> Regards,
>>>>>>> UdaraR
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Apr 22, 2014 at 2:55 AM, Bhathiya Jayasekara <
>>>>>>> bhath...@wso2.com> wrote:
>>>>>>>
>>>>>>>> Hi Udara,
>>>>>>>>
>>>>>>>> On Mon, Mar 17, 2014 at 10:56 PM, Udara Rathnayake <uda...@wso2.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Mar 17, 2014 at 11:42 PM, Inosh Goonewardena <
>>>>>>>>> in...@wso2.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Udara,
>>>>>>>>>>
>>>>>>>>>> I have deployed your toolbox and check. Actually, custom index
>>>>>>>>>> are getting created properly. Following are the corresponding
>>>>>>>>>> IndexCF for
>>>>>>>>>> the custom index properties you have defined.
>>>>>>>>>>
>>>>>>>>>> 867513878
>>>>>>>>>> 489450871
>>>>>>>>>>
>>>>>>>>>> But there is a problem with secondary index creation. It's
>>>>>>>>>> because there are dots('.') in attribute names. Following logs
>>>>>>>>>> prints in
>>>>>>>>>> console when deploying the toolbox.
>>>>>>>>>>
>>>>>>>>>> [2014-03-17 18:52:27,935] WARN
>>>>>>>>>> {org.wso2.carbon.databridge.persistence.cassandra.datastore.CassandraConnector}
>>>>>>>>>> - Secondary Index creation is not successful...
>>>>>>>>>> me.prettyprint.hector.api.exceptions.HInvalidRequestException:
>>>>>>>>>> InvalidRequestException(why:Illegal index name
>>>>>>>>>> payload_actor.id_index).
>>>>>>>>>>
>>>>>>>>>> Actually, if you want to create secondary indexes, you can
>>>>>>>>>> directly use 'Cassandra Keyspace' UI in management console too. But
>>>>>>>>>> because
>>>>>>>>>> you have dots in your column keys you are not allowed to do that in
>>>>>>>>>> there
>>>>>>>>>> too. I think it's better to use underscore instead of dot since those
>>>>>>>>>> attributes are used as column keys.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Even after I change attribute names it gives,
>>>>>>>>>
>>>>>>>>> [2014-03-18 11:20:05,512] WARN
>>>>>>>>> {org.wso2.carbon.databridge.persistence.cassandra.datastore.CassandraConnector}
>>>>>>>>> - Secondary Index creation is not successful...
>>>>>>>>> me.prettyprint.hector.api.exceptions.HInvalidRequestException:
>>>>>>>>> InvalidRequestException(why:Illegal index name
>>>>>>>>> payload_context_id_index)
>>>>>>>>>
>>>>>>>>
>>>>>>>> I know this is an old email. :) Just read it. Hope you've already
>>>>>>>> got solved the issue. Seems your index name started with a space
>>>>>>>> character.
>>>>>>>> That must be the reason for this error.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Bhathiya
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> But even with (.) character within attribute name you can run
>>>>>>>>> "CREATE INDEX ON" query on cqlsh without any trouble.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Mar 17, 2014 at 6:47 PM, Udara Rathnayake <
>>>>>>>>>> uda...@wso2.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Mar 17, 2014 at 11:09 PM, Inosh Goonewardena <
>>>>>>>>>>> in...@wso2.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Mar 17, 2014 at 6:15 PM, Udara Rathnayake <
>>>>>>>>>>>> uda...@wso2.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Inosh,
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Mar 17, 2014 at 9:04 PM, Inosh Goonewardena <
>>>>>>>>>>>>> in...@wso2.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Mar 17, 2014 at 4:13 PM, Udara Rathnayake <
>>>>>>>>>>>>>> uda...@wso2.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Inosh/Maninda,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As per the chat I had with Maninda I have deployed a toolbox
>>>>>>>>>>>>>>> with the following content.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *wso2_social_data_publisher_def*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *{ 'streamId':'org.wso2.social.activity:4.0.0',
>>>>>>>>>>>>>>> 'name':'org.wso2.social.activity', 'version':'4.0.0',
>>>>>>>>>>>>>>> 'description':
>>>>>>>>>>>>>>> 'store json object and mete-data describing each activity',
>>>>>>>>>>>>>>> 'tags':['social', 'activity'], 'metaData':[ ],
>>>>>>>>>>>>>>> 'correlationData':[ ], 'payloadData':[
>>>>>>>>>>>>>>> {'name':'id','type':'STRING'}, {'name':'context.id
>>>>>>>>>>>>>>> <http://context.id>','type':'STRING'}, {'name':'actor.id
>>>>>>>>>>>>>>> <http://actor.id>','type':'STRING'}, {'name':'tenant.id
>>>>>>>>>>>>>>> <http://tenant.id>','type':'STRING'},
>>>>>>>>>>>>>>> {'name':'body','type':'STRING'} ]}*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *streams.properties*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *streams.definitions=defn1streams.definitions.defn1.filename=wso2_social_data_publisher_def
>>>>>>>>>>>>>>> streams.definitions.defn1.username=adminstreams.definitions.defn1.password=adminstreams.definitions.defn1.description=This
>>>>>>>>>>>>>>> is the datastream published from wso2
>>>>>>>>>>>>>>> socialstreams.definitions.defn1.secondaryindexes=context.id
>>>>>>>>>>>>>>> <http://context.id>,actor.id <http://actor.id>,tenant.id
>>>>>>>>>>>>>>> <http://tenant.id>
>>>>>>>>>>>>>>> streams.definitions.defn1.customindexes=context.id
>>>>>>>>>>>>>>> <http://context.id>,id*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Then I have published few activities using a simple client.
>>>>>>>>>>>>>>> Everything(key-space creation, publishing activities) work
>>>>>>>>>>>>>>> perfectly but I don't see any indexes when I browse the column
>>>>>>>>>>>>>>> family
>>>>>>>>>>>>>>> though admin console.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In which keyspace did u check? IndexCF name is a random
>>>>>>>>>>>>>> generated number and getting created in EVENT_INDEX_KEYSPACE.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I checked column family information ( EVENT_KS >
>>>>>>>>>>>>> org_wso2_social_activity) where we display Column Name, Validator
>>>>>>>>>>>>> Type,
>>>>>>>>>>>>> Index Name, Actions details. For every colum index name value is
>>>>>>>>>>>>> "--NOT
>>>>>>>>>>>>> INDEXED--".
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes. This is the correct place to check whether secondary
>>>>>>>>>>>> indexes have been created. In your case it seems they are not
>>>>>>>>>>>> created.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Now I checked key-space information EVENT_INDEX_KS, still I
>>>>>>>>>>>>> don't see any index related information there.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Here you have the Custom Index CFs. According to the current
>>>>>>>>>>>> implementation separate CF created per index property.
>>>>>>>>>>>>
>>>>>>>>>>>> Can you share your toolbox with me so that I can have a look.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Find the attached toolbox.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for looking into this.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have used WSO2 BAM 2.4.1 alpha pack for this.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What am I doing wrong here?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> UdaraR
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> *Udara Rathnayake*
>>>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Mobile : +94 772207239
>>>>>>>>>>>>>>> Twitter : http://twitter.com/udarakr
>>>>>>>>>>>>>>> Blog : http://udarakr.blogspot.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Inosh Goonewardena
>>>>>>>>>>>>>> Associate Technical Lead- WSO2 Inc.
>>>>>>>>>>>>>> Mobile: +94779966317
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> *Udara Rathnayake*
>>>>>>>>>>>>> Software Engineer
>>>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mobile : +94 772207239
>>>>>>>>>>>>> Twitter : http://twitter.com/udarakr
>>>>>>>>>>>>> Blog : http://udarakr.blogspot.com
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Inosh Goonewardena
>>>>>>>>>>>> Associate Technical Lead- WSO2 Inc.
>>>>>>>>>>>> Mobile: +94779966317
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> *Udara Rathnayake*
>>>>>>>>>>> Software Engineer
>>>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>>>
>>>>>>>>>>> Mobile : +94 772207239
>>>>>>>>>>> Twitter : http://twitter.com/udarakr
>>>>>>>>>>> Blog : http://udarakr.blogspot.com
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Regards,
>>>>>>>>>>
>>>>>>>>>> Inosh Goonewardena
>>>>>>>>>> Associate Technical Lead- WSO2 Inc.
>>>>>>>>>> Mobile: +94779966317
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Udara Rathnayake*
>>>>>>>>> Software Engineer
>>>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>>>
>>>>>>>>> Mobile : +94 772207239
>>>>>>>>> Twitter : http://twitter.com/udarakr
>>>>>>>>> Blog : http://udarakr.blogspot.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Dev mailing list
>>>>>>>>> Dev@wso2.org
>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Bhathiya Jayasekara*
>>>>>>>> *Software Engineer,*
>>>>>>>> *WSO2 inc., http://wso2.com <http://wso2.com>*
>>>>>>>>
>>>>>>>> *Phone: +94715478185 <%2B94715478185>*
>>>>>>>> *LinkedIn: http://www.linkedin.com/in/bhathiyaj
>>>>>>>> <http://www.linkedin.com/in/bhathiyaj>*
>>>>>>>> *Twitter: https://twitter.com/bhathiyax
>>>>>>>> <https://twitter.com/bhathiyax>*
>>>>>>>> *Blog: http://movingaheadblog.blogspot.com
>>>>>>>> <http://movingaheadblog.blogspot.com/>*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Udara Rathnayake*
>>>>>>> Software Engineer
>>>>>>> WSO2 Inc. : http://wso2.com
>>>>>>>
>>>>>>> Mobile : +94 772207239
>>>>>>> Twitter : http://twitter.com/udarakr
>>>>>>> Blog : http://udarakr.blogspot.com
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>>
>>>>>> Inosh Goonewardena
>>>>>> Associate Technical Lead- WSO2 Inc.
>>>>>> Mobile: +94779966317
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Udara Rathnayake*
>>>>> Software Engineer
>>>>> WSO2 Inc. : http://wso2.com
>>>>>
>>>>> Mobile : +94 772207239
>>>>> Twitter : http://twitter.com/udarakr
>>>>> Blog : http://udarakr.blogspot.com
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Udara Rathnayake*
>>>> Software Engineer
>>>> WSO2 Inc. : http://wso2.com
>>>>
>>>> Mobile : +94 772207239
>>>> Twitter : http://twitter.com/udarakr
>>>> Blog : http://udarakr.blogspot.com
>>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>>
>>> Inosh Goonewardena
>>> Associate Technical Lead- WSO2 Inc.
>>> Mobile: +94779966317
>>>
>>
>>
>>
>> --
>> *Udara Rathnayake*
>> Software Engineer
>> WSO2 Inc. : http://wso2.com
>>
>> Mobile : +94 772207239
>> Twitter : http://twitter.com/udarakr
>> Blog : http://udarakr.blogspot.com
>>
>>
>
>
> --
> *Udara Rathnayake*
> Software Engineer
> WSO2 Inc. : http://wso2.com
>
> Mobile : +94 772207239
> Twitter : http://twitter.com/udarakr
> Blog : http://udarakr.blogspot.com
>
>
--
Regards,
Inosh Goonewardena
Associate Technical Lead- WSO2 Inc.
Mobile: +94779966317
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
