[Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi All, I am working on https://wso2.org/jira/browse/APIMANAGER-2064. By looking at the Cassandra database on BAM, I could figure out some requests publish to Cassandra database and some do not when I invoke the api. That is the reason for the missing requests in statistics dashboard. I tried debugging the APIMgtUsageDataBridgeDataPublisher class since it has publishEvent method to publish data. APIMgtUsageDataBridgeDataPublisher class uses LoadBalancingDataPublisher as the dataPublisher in this case. I found that every time when I invoke the api via rest client, this method will execute. but randomly some requests are not publishing in Cassandra database. I used the latest org.wso2.carbon.databridge.agent.thrift jar. but the problem is still there. if requests come to publishEvent method, should they publish in Cassandra database?and is there any known reason for data get not published in cassendra database. if not where I can find the data publisher to cassendra in BAM from API Manager. Thanks -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0711241005* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Lakshman, It should be in Cassandra if publishEvent method executes. How do you validate the data loss in Cassandra side? BAM embedded Cassandra explorer is a not good tool for that. Can you please verify that with tool like CQL or cassandra-cli? Regards, Gihan On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha wrote: > Hi All, > > I am working on https://wso2.org/jira/browse/APIMANAGER-2064. > > By looking at the Cassandra database on BAM, I could figure out some > requests publish to Cassandra database and some do not when I invoke the > api. That is the reason for the missing requests in statistics dashboard. > > I tried debugging the APIMgtUsageDataBridgeDataPublisher class since it > has publishEvent method to publish data. APIMgtUsageDataBridgeDataPublisher > class uses LoadBalancingDataPublisher as the dataPublisher in this case. > > I found that every time when I invoke the api via rest client, this method > will execute. but randomly some requests are not publishing in Cassandra > database. I used the latest org.wso2.carbon.databridge.agent.thrift jar. > but the problem is still there. > > if requests come to publishEvent method, should they publish in Cassandra > database?and is there any known reason for data get not published in > cassendra database. if not where I can find the data publisher to cassendra > in BAM from API Manager. > > Thanks > -- > Lakshman Udayakantha > WSO2 Inc. www.wso2.com > lean.enterprise.middleware > Mobile: *0711241005 <0711241005>* > > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- W.G. Gihan Anuruddha Senior Software Engineer | WSO2, Inc. M: +94772272595 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi, This can be an issue for other products as well. ES, AS and EMM also publishing data to BAM. In ES we are using load balancing data publisher. I also notice if we call stop method, sometimes data not get publish to BAM. We changed this to initiate data publisher at the first event and call data publishers stop method in the bundle activator's stop method. Need to test with load. Thanks On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha wrote: > Hi Lakshman, > > It should be in Cassandra if publishEvent method executes. How do you > validate the data loss in Cassandra side? BAM embedded Cassandra explorer > is a not good tool for that. Can you please verify that with tool like CQL > or cassandra-cli? > > Regards, > Gihan > > On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha > wrote: > >> Hi All, >> >> I am working on https://wso2.org/jira/browse/APIMANAGER-2064. >> >> By looking at the Cassandra database on BAM, I could figure out some >> requests publish to Cassandra database and some do not when I invoke the >> api. That is the reason for the missing requests in statistics dashboard. >> >> I tried debugging the APIMgtUsageDataBridgeDataPublisher class since it >> has publishEvent method to publish data. APIMgtUsageDataBridgeDataPublisher >> class uses LoadBalancingDataPublisher as the dataPublisher in this case. >> >> I found that every time when I invoke the api via rest client, this >> method will execute. but randomly some requests are not publishing in >> Cassandra database. I used the >> latest org.wso2.carbon.databridge.agent.thrift jar. but the problem is >> still there. >> >> if requests come to publishEvent method, should they publish in Cassandra >> database?and is there any known reason for data get not published in >> cassendra database. if not where I can find the data publisher to cassendra >> in BAM from API Manager. >> >> Thanks >> -- >> Lakshman Udayakantha >> WSO2 Inc. www.wso2.com >> lean.enterprise.middleware >> Mobile: *0711241005 <0711241005>* >> >> >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > W.G. Gihan Anuruddha > Senior Software Engineer | WSO2, Inc. > M: +94772272595 > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Manoj Gunawardena Tech Lead WSO2, Inc.: http://wso2.com lean.enterprise.middleware Mobile : +94 77 2291643 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Gihan, These data in cassandra database are summarised by a hive script and insert into a relational database. I checked that relational database table also. Number of records in cassandra database and relational database table are same. Then it should not be a problem with cassandra explorer. Thanks On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena wrote: > Hi, > > This can be an issue for other products as well. ES, AS and EMM also > publishing data to BAM. In ES we are using load balancing data publisher. I > also notice if we call stop method, sometimes data not get publish to BAM. > We changed this to initiate data publisher at the first event and call data > publishers stop method in the bundle activator's stop method. > Need to test with load. > > Thanks > > > On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha wrote: > >> Hi Lakshman, >> >> It should be in Cassandra if publishEvent method executes. How do you >> validate the data loss in Cassandra side? BAM embedded Cassandra explorer >> is a not good tool for that. Can you please verify that with tool like CQL >> or cassandra-cli? >> >> Regards, >> Gihan >> >> On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha > > wrote: >> >>> Hi All, >>> >>> I am working on https://wso2.org/jira/browse/APIMANAGER-2064. >>> >>> By looking at the Cassandra database on BAM, I could figure out some >>> requests publish to Cassandra database and some do not when I invoke the >>> api. That is the reason for the missing requests in statistics dashboard. >>> >>> I tried debugging the APIMgtUsageDataBridgeDataPublisher class since it >>> has publishEvent method to publish data. APIMgtUsageDataBridgeDataPublisher >>> class uses LoadBalancingDataPublisher as the dataPublisher in this case. >>> >>> I found that every time when I invoke the api via rest client, this >>> method will execute. but randomly some requests are not publishing in >>> Cassandra database. I used the >>> latest org.wso2.carbon.databridge.agent.thrift jar. but the problem is >>> still there. >>> >>> if requests come to publishEvent method, should they publish in >>> Cassandra database?and is there any known reason for data get not published >>> in cassendra database. if not where I can find the data publisher to >>> cassendra in BAM from API Manager. >>> >>> Thanks >>> -- >>> Lakshman Udayakantha >>> WSO2 Inc. www.wso2.com >>> lean.enterprise.middleware >>> Mobile: *0711241005 <0711241005>* >>> >>> >>> ___ >>> Dev mailing list >>> Dev@wso2.org >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> >> -- >> W.G. Gihan Anuruddha >> Senior Software Engineer | WSO2, Inc. >> M: +94772272595 >> >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > Manoj Gunawardena > Tech Lead > WSO2, Inc.: http://wso2.com > lean.enterprise.middleware > Mobile : +94 77 2291643 > -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0711241005* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Lakshman, How many records did you send to BAM and how many events are lost? Regards, Gihan On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha wrote: > Hi Gihan, > > These data in cassandra database are summarised by a hive script and > insert into a relational database. I checked that relational database table > also. Number of records in cassandra database and relational database table > are same. Then it should not be a problem with cassandra explorer. > > Thanks > > On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena > wrote: > >> Hi, >> >> This can be an issue for other products as well. ES, AS and EMM also >> publishing data to BAM. In ES we are using load balancing data publisher. I >> also notice if we call stop method, sometimes data not get publish to BAM. >> We changed this to initiate data publisher at the first event and call data >> publishers stop method in the bundle activator's stop method. >> Need to test with load. >> >> Thanks >> >> >> On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha wrote: >> >>> Hi Lakshman, >>> >>> It should be in Cassandra if publishEvent method executes. How do you >>> validate the data loss in Cassandra side? BAM embedded Cassandra >>> explorer is a not good tool for that. Can you please verify that with >>> tool like CQL or cassandra-cli? >>> >>> Regards, >>> Gihan >>> >>> On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> Hi All, I am working on https://wso2.org/jira/browse/APIMANAGER-2064. By looking at the Cassandra database on BAM, I could figure out some requests publish to Cassandra database and some do not when I invoke the api. That is the reason for the missing requests in statistics dashboard. I tried debugging the APIMgtUsageDataBridgeDataPublisher class since it has publishEvent method to publish data. APIMgtUsageDataBridgeDataPublisher class uses LoadBalancingDataPublisher as the dataPublisher in this case. I found that every time when I invoke the api via rest client, this method will execute. but randomly some requests are not publishing in Cassandra database. I used the latest org.wso2.carbon.databridge.agent.thrift jar. but the problem is still there. if requests come to publishEvent method, should they publish in Cassandra database?and is there any known reason for data get not published in cassendra database. if not where I can find the data publisher to cassendra in BAM from API Manager. Thanks -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0711241005 <0711241005>* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >>> -- >>> W.G. Gihan Anuruddha >>> Senior Software Engineer | WSO2, Inc. >>> M: +94772272595 >>> >>> ___ >>> Dev mailing list >>> Dev@wso2.org >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> >> -- >> Manoj Gunawardena >> Tech Lead >> WSO2, Inc.: http://wso2.com >> lean.enterprise.middleware >> Mobile : +94 77 2291643 >> > > > > -- > Lakshman Udayakantha > WSO2 Inc. www.wso2.com > lean.enterprise.middleware > Mobile: *0711241005 <0711241005>* > > -- W.G. Gihan Anuruddha Senior Software Engineer | WSO2, Inc. M: +94772272595 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
HI Gihan , They are lost randomly. Thanks On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha wrote: > Hi Lakshman, > > How many records did you send to BAM and how many events are lost? > > Regards, > Gihan > > On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha > wrote: > >> Hi Gihan, >> >> These data in cassandra database are summarised by a hive script and >> insert into a relational database. I checked that relational database table >> also. Number of records in cassandra database and relational database table >> are same. Then it should not be a problem with cassandra explorer. >> >> Thanks >> >> On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena >> wrote: >> >>> Hi, >>> >>> This can be an issue for other products as well. ES, AS and EMM also >>> publishing data to BAM. In ES we are using load balancing data publisher. I >>> also notice if we call stop method, sometimes data not get publish to BAM. >>> We changed this to initiate data publisher at the first event and call data >>> publishers stop method in the bundle activator's stop method. >>> Need to test with load. >>> >>> Thanks >>> >>> >>> On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha >>> wrote: >>> Hi Lakshman, It should be in Cassandra if publishEvent method executes. How do you validate the data loss in Cassandra side? BAM embedded Cassandra explorer is a not good tool for that. Can you please verify that with tool like CQL or cassandra-cli? Regards, Gihan On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < lakshm...@wso2.com> wrote: > Hi All, > > I am working on https://wso2.org/jira/browse/APIMANAGER-2064. > > By looking at the Cassandra database on BAM, I could figure out some > requests publish to Cassandra database and some do not when I invoke the > api. That is the reason for the missing requests in statistics dashboard. > > I tried debugging the APIMgtUsageDataBridgeDataPublisher class since > it has publishEvent method to publish data. > APIMgtUsageDataBridgeDataPublisher class uses LoadBalancingDataPublisher > as > the dataPublisher in this case. > > I found that every time when I invoke the api via rest client, this > method will execute. but randomly some requests are not publishing in > Cassandra database. I used the > latest org.wso2.carbon.databridge.agent.thrift jar. but the problem is > still there. > > if requests come to publishEvent method, should they publish in > Cassandra database?and is there any known reason for data get not > published > in cassendra database. if not where I can find the data publisher to > cassendra in BAM from API Manager. > > Thanks > -- > Lakshman Udayakantha > WSO2 Inc. www.wso2.com > lean.enterprise.middleware > Mobile: *0711241005 <0711241005>* > > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- W.G. Gihan Anuruddha Senior Software Engineer | WSO2, Inc. M: +94772272595 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >>> -- >>> Manoj Gunawardena >>> Tech Lead >>> WSO2, Inc.: http://wso2.com >>> lean.enterprise.middleware >>> Mobile : +94 77 2291643 >>> >> >> >> >> -- >> Lakshman Udayakantha >> WSO2 Inc. www.wso2.com >> lean.enterprise.middleware >> Mobile: *0711241005 <0711241005>* >> >> > > > -- > W.G. Gihan Anuruddha > Senior Software Engineer | WSO2, Inc. > M: +94772272595 > -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0711241005* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Lakshman, You mean data is therw in cassandra and cassandra explorer doesn't show these data? Did you compare by using cassandra CLI tool? On Wednesday, October 22, 2014, Lakshman Udayakantha wrote: > HI Gihan , > > They are lost randomly. > > Thanks > > On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha > wrote: > >> Hi Lakshman, >> >> How many records did you send to BAM and how many events are lost? >> >> Regards, >> Gihan >> >> On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha > > wrote: >> >>> Hi Gihan, >>> >>> These data in cassandra database are summarised by a hive script and >>> insert into a relational database. I checked that relational database table >>> also. Number of records in cassandra database and relational database table >>> are same. Then it should not be a problem with cassandra explorer. >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena >> > wrote: >>> Hi, This can be an issue for other products as well. ES, AS and EMM also publishing data to BAM. In ES we are using load balancing data publisher. I also notice if we call stop method, sometimes data not get publish to BAM. We changed this to initiate data publisher at the first event and call data publishers stop method in the bundle activator's stop method. Need to test with load. Thanks On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha >>> > wrote: > Hi Lakshman, > > It should be in Cassandra if publishEvent method executes. How do you > validate the data loss in Cassandra side? BAM embedded Cassandra > explorer is a not good tool for that. Can you please verify that with > tool like CQL or cassandra-cli? > > Regards, > Gihan > > On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < > lakshm...@wso2.com > > wrote: > >> Hi All, >> >> I am working on https://wso2.org/jira/browse/APIMANAGER-2064. >> >> By looking at the Cassandra database on BAM, I could figure out some >> requests publish to Cassandra database and some do not when I invoke the >> api. That is the reason for the missing requests in statistics dashboard. >> >> I tried debugging the APIMgtUsageDataBridgeDataPublisher class since >> it has publishEvent method to publish data. >> APIMgtUsageDataBridgeDataPublisher class uses LoadBalancingDataPublisher >> as >> the dataPublisher in this case. >> >> I found that every time when I invoke the api via rest client, this >> method will execute. but randomly some requests are not publishing in >> Cassandra database. I used the >> latest org.wso2.carbon.databridge.agent.thrift jar. but the problem is >> still there. >> >> if requests come to publishEvent method, should they publish in >> Cassandra database?and is there any known reason for data get not >> published >> in cassendra database. if not where I can find the data publisher to >> cassendra in BAM from API Manager. >> >> Thanks >> -- >> Lakshman Udayakantha >> WSO2 Inc. www.wso2.com >> lean.enterprise.middleware >> Mobile: *0711241005 <0711241005>* >> >> >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > W.G. Gihan Anuruddha > Senior Software Engineer | WSO2, Inc. > M: +94772272595 > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Manoj Gunawardena Tech Lead WSO2, Inc.: http://wso2.com lean.enterprise.middleware Mobile : +94 77 2291643 >>> >>> >>> >>> -- >>> Lakshman Udayakantha >>> WSO2 Inc. www.wso2.com >>> lean.enterprise.middleware >>> Mobile: *0711241005 <0711241005>* >>> >>> >> >> >> -- >> W.G. Gihan Anuruddha >> Senior Software Engineer | WSO2, Inc. >> M: +94772272595 >> > > > > -- > Lakshman Udayakantha > WSO2 Inc. www.wso2.com > lean.enterprise.middleware > Mobile: *0711241005* > > -- Regards, Inosh Goonewardena Associate Technical Lead- WSO2 Inc. Mobile: +94779966317 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Inosh, data in cassandra database are summized to a relational database. records in cassandra database and this relational table is same. so I think there is no problem with cassandra explorer. Thanks On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena wrote: > Hi Lakshman, > > You mean data is therw in cassandra and cassandra explorer doesn't show > these data? Did you compare by using cassandra CLI tool? > > On Wednesday, October 22, 2014, Lakshman Udayakantha > wrote: > >> HI Gihan , >> >> They are lost randomly. >> >> Thanks >> >> On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha wrote: >> >>> Hi Lakshman, >>> >>> How many records did you send to BAM and how many events are lost? >>> >>> Regards, >>> Gihan >>> >>> On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> Hi Gihan, These data in cassandra database are summarised by a hive script and insert into a relational database. I checked that relational database table also. Number of records in cassandra database and relational database table are same. Then it should not be a problem with cassandra explorer. Thanks On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena wrote: > Hi, > > This can be an issue for other products as well. ES, AS and EMM also > publishing data to BAM. In ES we are using load balancing data publisher. > I > also notice if we call stop method, sometimes data not get publish to BAM. > We changed this to initiate data publisher at the first event and call > data > publishers stop method in the bundle activator's stop method. > Need to test with load. > > Thanks > > > On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha > wrote: > >> Hi Lakshman, >> >> It should be in Cassandra if publishEvent method executes. How do >> you validate the data loss in Cassandra side? BAM embedded Cassandra >> explorer is a not good tool for that. Can you please verify that >> with tool like CQL or cassandra-cli? >> >> Regards, >> Gihan >> >> On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < >> lakshm...@wso2.com> wrote: >> >>> Hi All, >>> >>> I am working on https://wso2.org/jira/browse/APIMANAGER-2064. >>> >>> By looking at the Cassandra database on BAM, I could figure out some >>> requests publish to Cassandra database and some do not when I invoke the >>> api. That is the reason for the missing requests in statistics >>> dashboard. >>> >>> I tried debugging the APIMgtUsageDataBridgeDataPublisher class since >>> it has publishEvent method to publish data. >>> APIMgtUsageDataBridgeDataPublisher class uses >>> LoadBalancingDataPublisher as >>> the dataPublisher in this case. >>> >>> I found that every time when I invoke the api via rest client, this >>> method will execute. but randomly some requests are not publishing in >>> Cassandra database. I used the >>> latest org.wso2.carbon.databridge.agent.thrift jar. but the problem is >>> still there. >>> >>> if requests come to publishEvent method, should they publish in >>> Cassandra database?and is there any known reason for data get not >>> published >>> in cassendra database. if not where I can find the data publisher to >>> cassendra in BAM from API Manager. >>> >>> Thanks >>> -- >>> Lakshman Udayakantha >>> WSO2 Inc. www.wso2.com >>> lean.enterprise.middleware >>> Mobile: *0711241005 <0711241005>* >>> >>> >>> ___ >>> Dev mailing list >>> Dev@wso2.org >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> >> -- >> W.G. Gihan Anuruddha >> Senior Software Engineer | WSO2, Inc. >> M: +94772272595 >> >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > Manoj Gunawardena > Tech Lead > WSO2, Inc.: http://wso2.com > lean.enterprise.middleware > Mobile : +94 77 2291643 > -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0711241005 <0711241005>* >>> >>> >>> -- >>> W.G. Gihan Anuruddha >>> Senior Software Engineer | WSO2, Inc. >>> M: +94772272595 >>> >> >> >> >> -- >> Lakshman Udayakantha >> WSO2 Inc. www.wso2.com >> lean.enterprise.middleware >> Mobile: *0711241005* >> >> > > -- > Regards, > > Inosh Goonewardena > Associate Technical Lead- WSO2 Inc. > Mobile: +94779966317 > > -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0711241005* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi, On Wednesday, October 22, 2014, Lakshman Udayakantha wrote: > Hi Inosh, > > data in cassandra database are summized to a relational database. records > in cassandra database and this relational table is same. so I think there > is no problem with cassandra explorer. > Ok understood. I have misread your previous email. > > Thanks > > On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena > wrote: > >> Hi Lakshman, >> >> You mean data is therw in cassandra and cassandra explorer doesn't show >> these data? Did you compare by using cassandra CLI tool? >> >> On Wednesday, October 22, 2014, Lakshman Udayakantha > > wrote: >> >>> HI Gihan , >>> >>> They are lost randomly. >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha wrote: >>> Hi Lakshman, How many records did you send to BAM and how many events are lost? Regards, Gihan On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < lakshm...@wso2.com> wrote: > Hi Gihan, > > These data in cassandra database are summarised by a hive script and > insert into a relational database. I checked that relational database > table > also. Number of records in cassandra database and relational database > table > are same. Then it should not be a problem with cassandra explorer. > > Thanks > > On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena > wrote: > >> Hi, >> >> This can be an issue for other products as well. ES, AS and EMM also >> publishing data to BAM. In ES we are using load balancing data >> publisher. I >> also notice if we call stop method, sometimes data not get publish to >> BAM. >> We changed this to initiate data publisher at the first event and call >> data >> publishers stop method in the bundle activator's stop method. >> Need to test with load. >> >> Thanks >> >> >> On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha >> wrote: >> >>> Hi Lakshman, >>> >>> It should be in Cassandra if publishEvent method executes. How do >>> you validate the data loss in Cassandra side? BAM embedded >>> Cassandra explorer is a not good tool for that. Can you please >>> verify that with tool like CQL or cassandra-cli? >>> >>> Regards, >>> Gihan >>> >>> On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> Hi All, I am working on https://wso2.org/jira/browse/APIMANAGER-2064. By looking at the Cassandra database on BAM, I could figure out some requests publish to Cassandra database and some do not when I invoke the api. That is the reason for the missing requests in statistics dashboard. I tried debugging the APIMgtUsageDataBridgeDataPublisher class since it has publishEvent method to publish data. APIMgtUsageDataBridgeDataPublisher class uses LoadBalancingDataPublisher as the dataPublisher in this case. I found that every time when I invoke the api via rest client, this method will execute. but randomly some requests are not publishing in Cassandra database. I used the latest org.wso2.carbon.databridge.agent.thrift jar. but the problem is still there. if requests come to publishEvent method, should they publish in Cassandra database?and is there any known reason for data get not published in cassendra database. if not where I can find the data publisher to cassendra in BAM from API Manager. Thanks -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0711241005 <0711241005>* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >>> -- >>> W.G. Gihan Anuruddha >>> Senior Software Engineer | WSO2, Inc. >>> M: +94772272595 >>> >>> ___ >>> Dev mailing list >>> Dev@wso2.org >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> >> -- >> Manoj Gunawardena >> Tech Lead >> WSO2, Inc.: http://wso2.com >> lean.enterprise.middleware >> Mobile : +94 77 2291643 >> > > > > -- > Lakshman Udayakantha > WSO2 Inc. www.wso2.com > lean.enterprise.middleware > Mobile: *0711241005 <0711241005>* > > -- W.G. Gihan Anuruddha Senior Software Engineer | WSO2, Inc. M: +94772272595 >>> >>> >>> >>> -- >>> Lakshman Udayakantha >>> WSO2 Inc. www.wso2.com >>> lean.enterprise.middleware >>> Mobile: *0711241005* >>> >>> >> >>
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Lakshman, Can you please do this test. I believe currently you are using BAM 2.4.1. Please you this 2.5.0 RC [1]. Clean your RDBMS as well. Then using a tool like JMeter send 10,000 requests to APIM and see you are getting 10,000 events in BAM. Please run these tests without any debugging enabled. [1] - https://svn.wso2.org/repos/wso2/people/maninda/BAM_packs/2014-10-10/wso2bam-2.5.0.zip On Wed, Oct 22, 2014 at 9:19 AM, Lakshman Udayakantha wrote: > Hi Inosh, > > data in cassandra database are summized to a relational database. records > in cassandra database and this relational table is same. so I think there > is no problem with cassandra explorer. > > Thanks > > On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena > wrote: > >> Hi Lakshman, >> >> You mean data is therw in cassandra and cassandra explorer doesn't show >> these data? Did you compare by using cassandra CLI tool? >> >> On Wednesday, October 22, 2014, Lakshman Udayakantha >> wrote: >> >>> HI Gihan , >>> >>> They are lost randomly. >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha wrote: >>> Hi Lakshman, How many records did you send to BAM and how many events are lost? Regards, Gihan On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < lakshm...@wso2.com> wrote: > Hi Gihan, > > These data in cassandra database are summarised by a hive script and > insert into a relational database. I checked that relational database > table > also. Number of records in cassandra database and relational database > table > are same. Then it should not be a problem with cassandra explorer. > > Thanks > > On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena > wrote: > >> Hi, >> >> This can be an issue for other products as well. ES, AS and EMM also >> publishing data to BAM. In ES we are using load balancing data >> publisher. I >> also notice if we call stop method, sometimes data not get publish to >> BAM. >> We changed this to initiate data publisher at the first event and call >> data >> publishers stop method in the bundle activator's stop method. >> Need to test with load. >> >> Thanks >> >> >> On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha >> wrote: >> >>> Hi Lakshman, >>> >>> It should be in Cassandra if publishEvent method executes. How do >>> you validate the data loss in Cassandra side? BAM embedded >>> Cassandra explorer is a not good tool for that. Can you please >>> verify that with tool like CQL or cassandra-cli? >>> >>> Regards, >>> Gihan >>> >>> On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> Hi All, I am working on https://wso2.org/jira/browse/APIMANAGER-2064. By looking at the Cassandra database on BAM, I could figure out some requests publish to Cassandra database and some do not when I invoke the api. That is the reason for the missing requests in statistics dashboard. I tried debugging the APIMgtUsageDataBridgeDataPublisher class since it has publishEvent method to publish data. APIMgtUsageDataBridgeDataPublisher class uses LoadBalancingDataPublisher as the dataPublisher in this case. I found that every time when I invoke the api via rest client, this method will execute. but randomly some requests are not publishing in Cassandra database. I used the latest org.wso2.carbon.databridge.agent.thrift jar. but the problem is still there. if requests come to publishEvent method, should they publish in Cassandra database?and is there any known reason for data get not published in cassendra database. if not where I can find the data publisher to cassendra in BAM from API Manager. Thanks -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0711241005 <0711241005>* ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >>> -- >>> W.G. Gihan Anuruddha >>> Senior Software Engineer | WSO2, Inc. >>> M: +94772272595 >>> >>> ___ >>> Dev mailing list >>> Dev@wso2.org >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> >> -- >> Manoj Gunawardena >> Tech Lead >> WSO2, Inc.: http://wso2.com >> lean.enterprise.middleware >> Mobile : +94 77 2291643 >> > > > > -- > Lakshman Udayakantha > WSO2 Inc. www.wso2.com >
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Gihan, I will do that test and get back to you. Thanks On Wed, Oct 22, 2014 at 9:23 AM, Inosh Goonewardena wrote: > Hi, > > On Wednesday, October 22, 2014, Lakshman Udayakantha > wrote: > >> Hi Inosh, >> >> data in cassandra database are summized to a relational database. records >> in cassandra database and this relational table is same. so I think there >> is no problem with cassandra explorer. >> > > Ok understood. I have misread your previous email. > >> >> Thanks >> >> On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena >> wrote: >> >>> Hi Lakshman, >>> >>> You mean data is therw in cassandra and cassandra explorer doesn't show >>> these data? Did you compare by using cassandra CLI tool? >>> >>> On Wednesday, October 22, 2014, Lakshman Udayakantha >>> wrote: >>> HI Gihan , They are lost randomly. Thanks On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha wrote: > Hi Lakshman, > > How many records did you send to BAM and how many events are lost? > > Regards, > Gihan > > On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < > lakshm...@wso2.com> wrote: > >> Hi Gihan, >> >> These data in cassandra database are summarised by a hive script and >> insert into a relational database. I checked that relational database >> table >> also. Number of records in cassandra database and relational database >> table >> are same. Then it should not be a problem with cassandra explorer. >> >> Thanks >> >> On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena >> wrote: >> >>> Hi, >>> >>> This can be an issue for other products as well. ES, AS and EMM also >>> publishing data to BAM. In ES we are using load balancing data >>> publisher. I >>> also notice if we call stop method, sometimes data not get publish to >>> BAM. >>> We changed this to initiate data publisher at the first event and call >>> data >>> publishers stop method in the bundle activator's stop method. >>> Need to test with load. >>> >>> Thanks >>> >>> >>> On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha >>> wrote: >>> Hi Lakshman, It should be in Cassandra if publishEvent method executes. How do you validate the data loss in Cassandra side? BAM embedded Cassandra explorer is a not good tool for that. Can you please verify that with tool like CQL or cassandra-cli? Regards, Gihan On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < lakshm...@wso2.com> wrote: > Hi All, > > I am working on https://wso2.org/jira/browse/APIMANAGER-2064. > > By looking at the Cassandra database on BAM, I could figure out > some requests publish to Cassandra database and some do not when I > invoke > the api. That is the reason for the missing requests in statistics > dashboard. > > I tried debugging the APIMgtUsageDataBridgeDataPublisher class > since it has publishEvent method to publish data. > APIMgtUsageDataBridgeDataPublisher class uses > LoadBalancingDataPublisher as > the dataPublisher in this case. > > I found that every time when I invoke the api via rest client, > this method will execute. but randomly some requests are not > publishing in > Cassandra database. I used the > latest org.wso2.carbon.databridge.agent.thrift jar. but the problem is > still there. > > if requests come to publishEvent method, should they publish in > Cassandra database?and is there any known reason for data get not > published > in cassendra database. if not where I can find the data publisher to > cassendra in BAM from API Manager. > > Thanks > -- > Lakshman Udayakantha > WSO2 Inc. www.wso2.com > lean.enterprise.middleware > Mobile: *0711241005 <0711241005>* > > > ___ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- W.G. Gihan Anuruddha Senior Software Engineer | WSO2, Inc. M: +94772272595 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >>> -- >>> Manoj Gunawardena >>> Tech Lead >>> WSO2, Inc.: http://wso2.com >>> lean.enterprise.middleware >>> Mobile : +94 77 2291643 >>> >> >> >> >> -- >> Lakshman Udayakantha >> WSO2 Inc. www.wso2.com >> lean.enterprise.middleware >>
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Lakshman, First you should make sure all the required statistics are published to BAM with the data publisher. In order to do it you can, 1. Add a debug log to the data publisher publishing method to log all the messages passed through it. 2. Then you can use Cassandra Explorer or another tool like cassandra cli to see whether all the events sent to BAM are stored in Cassandra. 3. ONLY AFTER the above two was confirmed working properly we have to look into next stage, summarised data. 4. Clean H2/MySQL tables and run the hive script again with drop table commands for all Hive tables at the start of the script. 5. Then if the Hive query runs without any issue look at the summarised data in H2/MySQL tables. Therefore let the publisher publish several events to BAM and then verify the above given steps. There were several situations I have seen earlier that had problems with not publishing data due to an other publisher side error in logic. Thanks. *Maninda Edirisooriya* Senior Software Engineer *WSO2, Inc.*lean.enterprise.middleware. *Blog* : http://maninda.blogspot.com/ *E-mail* : mani...@wso2.com *Skype* : @manindae *Twitter* : @maninda On Wed, Oct 22, 2014 at 9:28 AM, Lakshman Udayakantha wrote: > Hi Gihan, > > I will do that test and get back to you. > > Thanks > > On Wed, Oct 22, 2014 at 9:23 AM, Inosh Goonewardena > wrote: > >> Hi, >> >> On Wednesday, October 22, 2014, Lakshman Udayakantha >> wrote: >> >>> Hi Inosh, >>> >>> data in cassandra database are summized to a relational database. >>> records in cassandra database and this relational table is same. so I think >>> there is no problem with cassandra explorer. >>> >> >> Ok understood. I have misread your previous email. >> >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena >>> wrote: >>> Hi Lakshman, You mean data is therw in cassandra and cassandra explorer doesn't show these data? Did you compare by using cassandra CLI tool? On Wednesday, October 22, 2014, Lakshman Udayakantha < lakshm...@wso2.com> wrote: > HI Gihan , > > They are lost randomly. > > Thanks > > On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha > wrote: > >> Hi Lakshman, >> >> How many records did you send to BAM and how many events are lost? >> >> Regards, >> Gihan >> >> On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < >> lakshm...@wso2.com> wrote: >> >>> Hi Gihan, >>> >>> These data in cassandra database are summarised by a hive script and >>> insert into a relational database. I checked that relational database >>> table >>> also. Number of records in cassandra database and relational database >>> table >>> are same. Then it should not be a problem with cassandra explorer. >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena >>> wrote: >>> Hi, This can be an issue for other products as well. ES, AS and EMM also publishing data to BAM. In ES we are using load balancing data publisher. I also notice if we call stop method, sometimes data not get publish to BAM. We changed this to initiate data publisher at the first event and call data publishers stop method in the bundle activator's stop method. Need to test with load. Thanks On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha wrote: > Hi Lakshman, > > It should be in Cassandra if publishEvent method executes. How do > you validate the data loss in Cassandra side? BAM embedded > Cassandra explorer is a not good tool for that. Can you please > verify that with tool like CQL or cassandra-cli? > > Regards, > Gihan > > On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < > lakshm...@wso2.com> wrote: > >> Hi All, >> >> I am working on https://wso2.org/jira/browse/APIMANAGER-2064. >> >> By looking at the Cassandra database on BAM, I could figure out >> some requests publish to Cassandra database and some do not when I >> invoke >> the api. That is the reason for the missing requests in statistics >> dashboard. >> >> I tried debugging the APIMgtUsageDataBridgeDataPublisher class >> since it has publishEvent method to publish data. >> APIMgtUsageDataBridgeDataPublisher class uses >> LoadBalancingDataPublisher as >> the dataPublisher in this case. >> >> I found that every time when I invoke the api via rest client, >> this method will execute. but randomly some requests are not >> publishing in >> Cassandra database. I used the >> latest org.wso2.carbon.databridge.agent.thrif
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Gihan, I sent 1 requests with the preconditions you have mentioned. only one record is missed in cassendra database for every time I sent 1 requests via APIM. Thanks On Wed, Oct 22, 2014 at 9:28 AM, Lakshman Udayakantha wrote: > Hi Gihan, > > I will do that test and get back to you. > > Thanks > > On Wed, Oct 22, 2014 at 9:23 AM, Inosh Goonewardena > wrote: > >> Hi, >> >> On Wednesday, October 22, 2014, Lakshman Udayakantha >> wrote: >> >>> Hi Inosh, >>> >>> data in cassandra database are summized to a relational database. >>> records in cassandra database and this relational table is same. so I think >>> there is no problem with cassandra explorer. >>> >> >> Ok understood. I have misread your previous email. >> >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena >>> wrote: >>> Hi Lakshman, You mean data is therw in cassandra and cassandra explorer doesn't show these data? Did you compare by using cassandra CLI tool? On Wednesday, October 22, 2014, Lakshman Udayakantha < lakshm...@wso2.com> wrote: > HI Gihan , > > They are lost randomly. > > Thanks > > On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha > wrote: > >> Hi Lakshman, >> >> How many records did you send to BAM and how many events are lost? >> >> Regards, >> Gihan >> >> On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < >> lakshm...@wso2.com> wrote: >> >>> Hi Gihan, >>> >>> These data in cassandra database are summarised by a hive script and >>> insert into a relational database. I checked that relational database >>> table >>> also. Number of records in cassandra database and relational database >>> table >>> are same. Then it should not be a problem with cassandra explorer. >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena >>> wrote: >>> Hi, This can be an issue for other products as well. ES, AS and EMM also publishing data to BAM. In ES we are using load balancing data publisher. I also notice if we call stop method, sometimes data not get publish to BAM. We changed this to initiate data publisher at the first event and call data publishers stop method in the bundle activator's stop method. Need to test with load. Thanks On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha wrote: > Hi Lakshman, > > It should be in Cassandra if publishEvent method executes. How do > you validate the data loss in Cassandra side? BAM embedded > Cassandra explorer is a not good tool for that. Can you please > verify that with tool like CQL or cassandra-cli? > > Regards, > Gihan > > On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < > lakshm...@wso2.com> wrote: > >> Hi All, >> >> I am working on https://wso2.org/jira/browse/APIMANAGER-2064. >> >> By looking at the Cassandra database on BAM, I could figure out >> some requests publish to Cassandra database and some do not when I >> invoke >> the api. That is the reason for the missing requests in statistics >> dashboard. >> >> I tried debugging the APIMgtUsageDataBridgeDataPublisher class >> since it has publishEvent method to publish data. >> APIMgtUsageDataBridgeDataPublisher class uses >> LoadBalancingDataPublisher as >> the dataPublisher in this case. >> >> I found that every time when I invoke the api via rest client, >> this method will execute. but randomly some requests are not >> publishing in >> Cassandra database. I used the >> latest org.wso2.carbon.databridge.agent.thrift jar. but the problem >> is >> still there. >> >> if requests come to publishEvent method, should they publish in >> Cassandra database?and is there any known reason for data get not >> published >> in cassendra database. if not where I can find the data publisher to >> cassendra in BAM from API Manager. >> >> Thanks >> -- >> Lakshman Udayakantha >> WSO2 Inc. www.wso2.com >> lean.enterprise.middleware >> Mobile: *0711241005 <0711241005>* >> >> >> ___ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > W.G. Gihan Anuruddha > Senior Software Engineer | WSO2, Inc. > M: +94772272595 > > ___ >
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Lakshman, That's is a good observation. So I believe this event should be either first one or the last one. Can you please do a another small test? Shutdown the BAM server. Delete the Cassandra data folder completely (You can find that in repository/databases/cassandra) Truncate the APIM related RDBMS tables. Run your script with 100 requests. Please try to identify the message that going to miss. I think we need to check data sending part in APIM side? Also add a debug log message to publishEvent method with static counter and print that value. So it will be easy to identify the issue. Regards, Gihan On Wed, Oct 22, 2014 at 6:37 PM, Lakshman Udayakantha wrote: > Hi Gihan, > > I sent 1 requests with the preconditions you have mentioned. only one > record is missed in cassendra database for every time I sent 1 requests > via APIM. > > Thanks > > On Wed, Oct 22, 2014 at 9:28 AM, Lakshman Udayakantha > wrote: > >> Hi Gihan, >> >> I will do that test and get back to you. >> >> Thanks >> >> On Wed, Oct 22, 2014 at 9:23 AM, Inosh Goonewardena >> wrote: >> >>> Hi, >>> >>> On Wednesday, October 22, 2014, Lakshman Udayakantha >>> wrote: >>> Hi Inosh, data in cassandra database are summized to a relational database. records in cassandra database and this relational table is same. so I think there is no problem with cassandra explorer. >>> >>> Ok understood. I have misread your previous email. >>> Thanks On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena wrote: > Hi Lakshman, > > You mean data is therw in cassandra and cassandra explorer doesn't > show these data? Did you compare by using cassandra CLI tool? > > On Wednesday, October 22, 2014, Lakshman Udayakantha < > lakshm...@wso2.com> wrote: > >> HI Gihan , >> >> They are lost randomly. >> >> Thanks >> >> On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha >> wrote: >> >>> Hi Lakshman, >>> >>> How many records did you send to BAM and how many events are lost? >>> >>> Regards, >>> Gihan >>> >>> On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> Hi Gihan, These data in cassandra database are summarised by a hive script and insert into a relational database. I checked that relational database table also. Number of records in cassandra database and relational database table are same. Then it should not be a problem with cassandra explorer. Thanks On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena >>> > wrote: > Hi, > > This can be an issue for other products as well. ES, AS and EMM > also publishing data to BAM. In ES we are using load balancing data > publisher. I also notice if we call stop method, sometimes data not > get > publish to BAM. We changed this to initiate data publisher at the > first > event and call data publishers stop method in the bundle activator's > stop > method. > Need to test with load. > > Thanks > > > On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha > wrote: > >> Hi Lakshman, >> >> It should be in Cassandra if publishEvent method executes. How >> do you validate the data loss in Cassandra side? BAM embedded >> Cassandra explorer is a not good tool for that. Can you please >> verify that with tool like CQL or cassandra-cli? >> >> Regards, >> Gihan >> >> On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < >> lakshm...@wso2.com> wrote: >> >>> Hi All, >>> >>> I am working on https://wso2.org/jira/browse/APIMANAGER-2064. >>> >>> By looking at the Cassandra database on BAM, I could figure out >>> some requests publish to Cassandra database and some do not when I >>> invoke >>> the api. That is the reason for the missing requests in statistics >>> dashboard. >>> >>> I tried debugging the APIMgtUsageDataBridgeDataPublisher class >>> since it has publishEvent method to publish data. >>> APIMgtUsageDataBridgeDataPublisher class uses >>> LoadBalancingDataPublisher as >>> the dataPublisher in this case. >>> >>> I found that every time when I invoke the api via rest client, >>> this method will execute. but randomly some requests are not >>> publishing in >>> Cassandra database. I used the >>> latest org.wso2.carbon.databridge.agent.thrift jar. but the problem >>> is >>> still there. >>> >>> if requests come to publishEvent method, should
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Gihan, I will try that way. thank you so much for the support you have given so far Thanks On Wed, Oct 22, 2014 at 6:49 PM, Gihan Anuruddha wrote: > Hi Lakshman, > > That's is a good observation. So I believe this event should be either > first one or the last one. Can you please do a another small test? > Shutdown the BAM server. > Delete the Cassandra data folder completely (You can find that in > repository/databases/cassandra) > Truncate the APIM related RDBMS tables. > Run your script with 100 requests. > > Please try to identify the message that going to miss. > > I think we need to check data sending part in APIM side? Also add a debug > log message to publishEvent method with static counter and print that > value. So it will be easy to identify the issue. > > Regards, > Gihan > > On Wed, Oct 22, 2014 at 6:37 PM, Lakshman Udayakantha > wrote: > >> Hi Gihan, >> >> I sent 1 requests with the preconditions you have mentioned. only one >> record is missed in cassendra database for every time I sent 1 requests >> via APIM. >> >> Thanks >> >> On Wed, Oct 22, 2014 at 9:28 AM, Lakshman Udayakantha > > wrote: >> >>> Hi Gihan, >>> >>> I will do that test and get back to you. >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 9:23 AM, Inosh Goonewardena >>> wrote: >>> Hi, On Wednesday, October 22, 2014, Lakshman Udayakantha < lakshm...@wso2.com> wrote: > Hi Inosh, > > data in cassandra database are summized to a relational database. > records in cassandra database and this relational table is same. so I > think > there is no problem with cassandra explorer. > Ok understood. I have misread your previous email. > > Thanks > > On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena > wrote: > >> Hi Lakshman, >> >> You mean data is therw in cassandra and cassandra explorer doesn't >> show these data? Did you compare by using cassandra CLI tool? >> >> On Wednesday, October 22, 2014, Lakshman Udayakantha < >> lakshm...@wso2.com> wrote: >> >>> HI Gihan , >>> >>> They are lost randomly. >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha >>> wrote: >>> Hi Lakshman, How many records did you send to BAM and how many events are lost? Regards, Gihan On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < lakshm...@wso2.com> wrote: > Hi Gihan, > > These data in cassandra database are summarised by a hive script > and insert into a relational database. I checked that relational > database > table also. Number of records in cassandra database and relational > database > table are same. Then it should not be a problem with cassandra > explorer. > > Thanks > > On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena < > man...@wso2.com> wrote: > >> Hi, >> >> This can be an issue for other products as well. ES, AS and EMM >> also publishing data to BAM. In ES we are using load balancing data >> publisher. I also notice if we call stop method, sometimes data not >> get >> publish to BAM. We changed this to initiate data publisher at the >> first >> event and call data publishers stop method in the bundle activator's >> stop >> method. >> Need to test with load. >> >> Thanks >> >> >> On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha > > wrote: >> >>> Hi Lakshman, >>> >>> It should be in Cassandra if publishEvent method executes. How >>> do you validate the data loss in Cassandra side? BAM embedded >>> Cassandra explorer is a not good tool for that. Can you please >>> verify that with tool like CQL or cassandra-cli? >>> >>> Regards, >>> Gihan >>> >>> On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> Hi All, I am working on https://wso2.org/jira/browse/APIMANAGER-2064. By looking at the Cassandra database on BAM, I could figure out some requests publish to Cassandra database and some do not when I invoke the api. That is the reason for the missing requests in statistics dashboard. I tried debugging the APIMgtUsageDataBridgeDataPublisher class since it has publishEvent method to publish data. APIMgtUsageDataBridgeDataPublisher class uses LoadBalancingDataPublisher as the dataPublisher in this case. I found that every time when
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Lakshman, This is very useful results, Thanks for sharing. We will do the same test for ES as well. Is this test run for single BAM instance, or multiple? (load balanced environment) Thanks On Wed, Oct 22, 2014 at 6:37 PM, Lakshman Udayakantha wrote: > Hi Gihan, > > I sent 1 requests with the preconditions you have mentioned. only one > record is missed in cassendra database for every time I sent 1 requests > via APIM. > > Thanks > > On Wed, Oct 22, 2014 at 9:28 AM, Lakshman Udayakantha > wrote: > >> Hi Gihan, >> >> I will do that test and get back to you. >> >> Thanks >> >> On Wed, Oct 22, 2014 at 9:23 AM, Inosh Goonewardena >> wrote: >> >>> Hi, >>> >>> On Wednesday, October 22, 2014, Lakshman Udayakantha >>> wrote: >>> Hi Inosh, data in cassandra database are summized to a relational database. records in cassandra database and this relational table is same. so I think there is no problem with cassandra explorer. >>> >>> Ok understood. I have misread your previous email. >>> Thanks On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena wrote: > Hi Lakshman, > > You mean data is therw in cassandra and cassandra explorer doesn't > show these data? Did you compare by using cassandra CLI tool? > > On Wednesday, October 22, 2014, Lakshman Udayakantha < > lakshm...@wso2.com> wrote: > >> HI Gihan , >> >> They are lost randomly. >> >> Thanks >> >> On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha >> wrote: >> >>> Hi Lakshman, >>> >>> How many records did you send to BAM and how many events are lost? >>> >>> Regards, >>> Gihan >>> >>> On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> Hi Gihan, These data in cassandra database are summarised by a hive script and insert into a relational database. I checked that relational database table also. Number of records in cassandra database and relational database table are same. Then it should not be a problem with cassandra explorer. Thanks On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena >>> > wrote: > Hi, > > This can be an issue for other products as well. ES, AS and EMM > also publishing data to BAM. In ES we are using load balancing data > publisher. I also notice if we call stop method, sometimes data not > get > publish to BAM. We changed this to initiate data publisher at the > first > event and call data publishers stop method in the bundle activator's > stop > method. > Need to test with load. > > Thanks > > > On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha > wrote: > >> Hi Lakshman, >> >> It should be in Cassandra if publishEvent method executes. How >> do you validate the data loss in Cassandra side? BAM embedded >> Cassandra explorer is a not good tool for that. Can you please >> verify that with tool like CQL or cassandra-cli? >> >> Regards, >> Gihan >> >> On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < >> lakshm...@wso2.com> wrote: >> >>> Hi All, >>> >>> I am working on https://wso2.org/jira/browse/APIMANAGER-2064. >>> >>> By looking at the Cassandra database on BAM, I could figure out >>> some requests publish to Cassandra database and some do not when I >>> invoke >>> the api. That is the reason for the missing requests in statistics >>> dashboard. >>> >>> I tried debugging the APIMgtUsageDataBridgeDataPublisher class >>> since it has publishEvent method to publish data. >>> APIMgtUsageDataBridgeDataPublisher class uses >>> LoadBalancingDataPublisher as >>> the dataPublisher in this case. >>> >>> I found that every time when I invoke the api via rest client, >>> this method will execute. but randomly some requests are not >>> publishing in >>> Cassandra database. I used the >>> latest org.wso2.carbon.databridge.agent.thrift jar. but the problem >>> is >>> still there. >>> >>> if requests come to publishEvent method, should they publish in >>> Cassandra database?and is there any known reason for data get not >>> published >>> in cassendra database. if not where I can find the data publisher to >>> cassendra in BAM from API Manager. >>> >>> Thanks >>> -- >>> Lakshman Udayakantha >>> WSO2 Inc. www.wso2.com >>> lean.enterprise.middleware >>> Mobile: *07
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Manoj, I tested this with single BAM instance. As Gihan mentioned ,Still need to check there is a problem in APIM side. I will update this thread if problem is in APIM Thanks On Wed, Oct 22, 2014 at 7:49 PM, Manoj Gunawardena wrote: > Hi Lakshman, > > This is very useful results, Thanks for sharing. We will do the same test > for ES as well. Is this test run for single BAM instance, or multiple? > (load balanced environment) > > Thanks > > On Wed, Oct 22, 2014 at 6:37 PM, Lakshman Udayakantha > wrote: > >> Hi Gihan, >> >> I sent 1 requests with the preconditions you have mentioned. only one >> record is missed in cassendra database for every time I sent 1 requests >> via APIM. >> >> Thanks >> >> On Wed, Oct 22, 2014 at 9:28 AM, Lakshman Udayakantha > > wrote: >> >>> Hi Gihan, >>> >>> I will do that test and get back to you. >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 9:23 AM, Inosh Goonewardena >>> wrote: >>> Hi, On Wednesday, October 22, 2014, Lakshman Udayakantha < lakshm...@wso2.com> wrote: > Hi Inosh, > > data in cassandra database are summized to a relational database. > records in cassandra database and this relational table is same. so I > think > there is no problem with cassandra explorer. > Ok understood. I have misread your previous email. > > Thanks > > On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena > wrote: > >> Hi Lakshman, >> >> You mean data is therw in cassandra and cassandra explorer doesn't >> show these data? Did you compare by using cassandra CLI tool? >> >> On Wednesday, October 22, 2014, Lakshman Udayakantha < >> lakshm...@wso2.com> wrote: >> >>> HI Gihan , >>> >>> They are lost randomly. >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha >>> wrote: >>> Hi Lakshman, How many records did you send to BAM and how many events are lost? Regards, Gihan On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < lakshm...@wso2.com> wrote: > Hi Gihan, > > These data in cassandra database are summarised by a hive script > and insert into a relational database. I checked that relational > database > table also. Number of records in cassandra database and relational > database > table are same. Then it should not be a problem with cassandra > explorer. > > Thanks > > On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena < > man...@wso2.com> wrote: > >> Hi, >> >> This can be an issue for other products as well. ES, AS and EMM >> also publishing data to BAM. In ES we are using load balancing data >> publisher. I also notice if we call stop method, sometimes data not >> get >> publish to BAM. We changed this to initiate data publisher at the >> first >> event and call data publishers stop method in the bundle activator's >> stop >> method. >> Need to test with load. >> >> Thanks >> >> >> On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha > > wrote: >> >>> Hi Lakshman, >>> >>> It should be in Cassandra if publishEvent method executes. How >>> do you validate the data loss in Cassandra side? BAM embedded >>> Cassandra explorer is a not good tool for that. Can you please >>> verify that with tool like CQL or cassandra-cli? >>> >>> Regards, >>> Gihan >>> >>> On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> Hi All, I am working on https://wso2.org/jira/browse/APIMANAGER-2064. By looking at the Cassandra database on BAM, I could figure out some requests publish to Cassandra database and some do not when I invoke the api. That is the reason for the missing requests in statistics dashboard. I tried debugging the APIMgtUsageDataBridgeDataPublisher class since it has publishEvent method to publish data. APIMgtUsageDataBridgeDataPublisher class uses LoadBalancingDataPublisher as the dataPublisher in this case. I found that every time when I invoke the api via rest client, this method will execute. but randomly some requests are not publishing in Cassandra database. I used the latest org.wso2.carbon.databridge.agent.thrift jar. but the problem is still there. if requests come to publi
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi All, As per the offline discussion with Lakmali, The reason for this issue is when one end point is down, It will fail to go to the bam mediator(which is responsible for publishing event to destination statistics table in BAM cassandra database)which is added after send mediator in synapse configuration(that is generated when new api is created) for every 30 seconds. I just checked that by adding a log mediator between send mediator and bam mediator. Seems like no issue with bam mediator. It publishes all the request it receives. need to find why call will not through bam mediator for every 30 seconds. I added the sample inSequence generated here. kindly expect your feedback on this. http://10.100.5.161:8310/services"/> http://10.100.5.161:8310/services"; /> http://10.100.5.161:8290/services"/> http://10.100.5.161:8290/services"; /> http://10.100.5.161:8300/services"/> http://10.100.5.161:8300/services"; /> Thanks On Wed, Oct 22, 2014 at 9:00 PM, Lakshman Udayakantha wrote: > Hi Manoj, > > I tested this with single BAM instance. As Gihan mentioned ,Still need to > check there is a problem in APIM side. I will update this thread if problem > is in APIM > > Thanks > > On Wed, Oct 22, 2014 at 7:49 PM, Manoj Gunawardena > wrote: > >> Hi Lakshman, >> >> This is very useful results, Thanks for sharing. We will do the same test >> for ES as well. Is this test run for single BAM instance, or multiple? >> (load balanced environment) >> >> Thanks >> >> On Wed, Oct 22, 2014 at 6:37 PM, Lakshman Udayakantha > > wrote: >> >>> Hi Gihan, >>> >>> I sent 1 requests with the preconditions you have mentioned. only >>> one record is missed in cassendra database for every time I sent 1 >>> requests via APIM. >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 9:28 AM, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> Hi Gihan, I will do that test and get back to you. Thanks On Wed, Oct 22, 2014 at 9:23 AM, Inosh Goonewardena wrote: > Hi, > > On Wednesday, October 22, 2014, Lakshman Udayakantha < > lakshm...@wso2.com> wrote: > >> Hi Inosh, >> >> data in cassandra database are summized to a relational database. >> records in cassandra database and this relational table is same. so I >> think >> there is no problem with cassandra explorer. >> > > Ok understood. I have misread your previous email. > >> >> Thanks >> >> On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena >> wrote: >> >>> Hi Lakshman, >>> >>> You mean data is therw in cassandra and cassandra explorer doesn't >>> show these data? Did you compare by using cassandra CLI tool? >>> >>> On Wednesday, October 22, 2014, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> HI Gihan , They are lost randomly. Thanks On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha wrote: > Hi Lakshman, > > How many records did you send to BAM and how many events are lost? > > Regards, > Gihan > > On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < > lakshm...@wso2.com> wrote: > >> Hi Gihan, >> >> These data in cassandra database are summarised by a hive script >> and insert into a relational database. I checked that relational >> database >> table also. Number of records in cassandra database and relational >> database >> table are same. Then it should not be a problem with cassandra >> explorer. >> >> Thanks >> >> On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena < >> man...@wso2.com> wrote: >> >>> Hi, >>> >>> This can be an issue for other products as well. ES, AS and EMM >>> also publishing data to BAM. In ES we are using load balancing data >>> publisher. I also notice if we call stop method, sometimes data not >>> get >>> publish to BAM. We changed this to initiate data publisher at the >>> first >>> event and call data publishers stop method in the bundle >>> activator's stop >>> method. >>> Need to test with load. >>> >>> Thanks >>> >>> >>> On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha < >>> gi...@wso2.com> wrote: >>> Hi Lakshman, It should be in Cassandra if publishEvent method executes. How do you validate the data loss in Cassandra side? BAM embedded Cassandra explorer is a not good tool for that. Can you please verify that with tool like CQL or cassandra-cli? Regards, Gihan On Tue, Oct 21, 2
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Lakshman, AFAIK this looks like a configuration issue. You have given a name to endpoints which were defined inline. Can you please change the configuration as given in article [1] and simplify this. Then try that out again. [1] https://docs.wso2.com/display/ESB481/Sample+54%3A+Session+Affinity+Load+Balancing+between+Three+Endpoints Thanks & Regards, On Mon, Oct 27, 2014 at 10:31 AM, Lakshman Udayakantha wrote: > Hi All, > > As per the offline discussion with Lakmali, The reason for this issue is > when one end point is down, It will fail to go to the bam mediator(which > is responsible for publishing event to destination statistics table in BAM > cassandra database)which is added after send mediator in synapse > configuration(that is generated when new api is created) for every 30 > seconds. I just checked that by adding a log mediator between send mediator > and bam mediator. Seems like no issue with bam mediator. It publishes all > the request it receives. need to find why call will not through bam > mediator for every 30 seconds. I added the sample inSequence generated > here. kindly expect your feedback on this. > > > > > > > > http://10.100.5.161:8310/services"/> > http://10.100.5.161:8310/services > "/> > > > http://10.100.5.161:8290/services"/> > http://10.100.5.161:8290/services > "/> > > > http://10.100.5.161:8300/services"/> > http://10.100.5.161:8300/services > "/> > > > > > > > > > > > > > > > > > Thanks > > On Wed, Oct 22, 2014 at 9:00 PM, Lakshman Udayakantha > wrote: > >> Hi Manoj, >> >> I tested this with single BAM instance. As Gihan mentioned ,Still need to >> check there is a problem in APIM side. I will update this thread if problem >> is in APIM >> >> Thanks >> >> On Wed, Oct 22, 2014 at 7:49 PM, Manoj Gunawardena >> wrote: >> >>> Hi Lakshman, >>> >>> This is very useful results, Thanks for sharing. We will do the same >>> test for ES as well. Is this test run for single BAM instance, or multiple? >>> (load balanced environment) >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 6:37 PM, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> Hi Gihan, I sent 1 requests with the preconditions you have mentioned. only one record is missed in cassendra database for every time I sent 1 requests via APIM. Thanks On Wed, Oct 22, 2014 at 9:28 AM, Lakshman Udayakantha < lakshm...@wso2.com> wrote: > Hi Gihan, > > I will do that test and get back to you. > > Thanks > > On Wed, Oct 22, 2014 at 9:23 AM, Inosh Goonewardena > wrote: > >> Hi, >> >> On Wednesday, October 22, 2014, Lakshman Udayakantha < >> lakshm...@wso2.com> wrote: >> >>> Hi Inosh, >>> >>> data in cassandra database are summized to a relational database. >>> records in cassandra database and this relational table is same. so I >>> think >>> there is no problem with cassandra explorer. >>> >> >> Ok understood. I have misread your previous email. >> >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena >>> wrote: >>> Hi Lakshman, You mean data is therw in cassandra and cassandra explorer doesn't show these data? Did you compare by using cassandra CLI tool? On Wednesday, October 22, 2014, Lakshman Udayakantha < lakshm...@wso2.com> wrote: > HI Gihan , > > They are lost randomly. > > Thanks > > On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha > wrote: > >> Hi Lakshman, >> >> How many records did you send to BAM and how many events are lost? >> >> Regards, >> Gihan >> >> On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < >> lakshm...@wso2.com> wrote: >> >>> Hi Gihan, >>> >>> These data in cassandra database are summarised by a hive script >>> and insert into a relational database. I checked that relational >>> database >>> table also. Number of records in cassandra database and relational >>> database >>> table are same. Then it should not be a problem with cassandra >>> explorer. >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena < >>> man...@wso2.com> wrote: >>> Hi, This can be an issue for other products as well. ES, AS and EMM also publishing data to BAM. In ES we are using load balancing data publisher. I also notice if we call stop method, sometimes data not get publish to BAM. We changed this to initiate data publisher at the first event and call data publishers stop method in the bundle act
Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM
Hi Ravindra, Thanks for your suggestion. Hi all, Seems like the following is the root cause for this issue: When an endpoint is unavailable, we noticed that, whatever is given after the send mediator (in the insequence) doesn't get executed. As a result, the BAM mediator, which is placed after the send mediator, doesn't get executed; and it will not receive the event that the API was invoked. Hence the stats go wrong. I created an API with a load-balanced endpoint in ESB 4.8.1 and noticed the same behaviour. Refer to mail thread in dev list with subject "[ESB] Issue in Load Balanced Endpoint when one end point is unavailable". Thanks, Dilini On Tue, Oct 28, 2014 at 9:11 AM, Ravindra Ranwala wrote: > Hi Lakshman, > > AFAIK this looks like a configuration issue. You have given a name to > endpoints which were defined inline. Can you please change the > configuration as given in article [1] and simplify this. Then try that out > again. > > > [1] > https://docs.wso2.com/display/ESB481/Sample+54%3A+Session+Affinity+Load+Balancing+between+Three+Endpoints > > > Thanks & Regards, > > On Mon, Oct 27, 2014 at 10:31 AM, Lakshman Udayakantha > wrote: > >> Hi All, >> >> As per the offline discussion with Lakmali, The reason for this issue is >> when one end point is down, It will fail to go to the bam mediator(which >> is responsible for publishing event to destination statistics table in BAM >> cassandra database)which is added after send mediator in synapse >> configuration(that is generated when new api is created) for every 30 >> seconds. I just checked that by adding a log mediator between send mediator >> and bam mediator. Seems like no issue with bam mediator. It publishes all >> the request it receives. need to find why call will not through bam >> mediator for every 30 seconds. I added the sample inSequence generated >> here. kindly expect your feedback on this. >> >> >> >> >> >> >> >> http://10.100.5.161:8310/services"/> >> http://10.100.5.161:8310/services"/> >> >> >> http://10.100.5.161:8290/services"/> >> http://10.100.5.161:8290/services"/> >> >> >> http://10.100.5.161:8300/services"/> >> http://10.100.5.161:8300/services"/> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> Thanks >> >> On Wed, Oct 22, 2014 at 9:00 PM, Lakshman Udayakantha > > wrote: >> >>> Hi Manoj, >>> >>> I tested this with single BAM instance. As Gihan mentioned ,Still need >>> to check there is a problem in APIM side. I will update this thread if >>> problem is in APIM >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 7:49 PM, Manoj Gunawardena >>> wrote: >>> Hi Lakshman, This is very useful results, Thanks for sharing. We will do the same test for ES as well. Is this test run for single BAM instance, or multiple? (load balanced environment) Thanks On Wed, Oct 22, 2014 at 6:37 PM, Lakshman Udayakantha < lakshm...@wso2.com> wrote: > Hi Gihan, > > I sent 1 requests with the preconditions you have mentioned. only > one record is missed in cassendra database for every time I sent 1 > requests via APIM. > > Thanks > > On Wed, Oct 22, 2014 at 9:28 AM, Lakshman Udayakantha < > lakshm...@wso2.com> wrote: > >> Hi Gihan, >> >> I will do that test and get back to you. >> >> Thanks >> >> On Wed, Oct 22, 2014 at 9:23 AM, Inosh Goonewardena >> wrote: >> >>> Hi, >>> >>> On Wednesday, October 22, 2014, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> Hi Inosh, data in cassandra database are summized to a relational database. records in cassandra database and this relational table is same. so I think there is no problem with cassandra explorer. >>> >>> Ok understood. I have misread your previous email. >>> Thanks On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena >>> > wrote: > Hi Lakshman, > > You mean data is therw in cassandra and cassandra explorer doesn't > show these data? Did you compare by using cassandra CLI tool? > > On Wednesday, October 22, 2014, Lakshman Udayakantha < > lakshm...@wso2.com> wrote: > >> HI Gihan , >> >> They are lost randomly. >> >> Thanks >> >> On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha >> wrote: >> >>> Hi Lakshman, >>> >>> How many records did you send to BAM and how many events are >>> lost? >>> >>> Regards, >>> Gihan >>> >>> On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> Hi Gihan, These data in cassandra database are summarised by a hive script and insert into a relationa
