Hello,
It is running fine now. Got expected results. Thank you
----- Original Message -----
From: "Jaikit Jilka" <jji...@leadferret.com>
To: "users" <users@apex.apache.org>
Sent: Thursday, July 7, 2016 12:55:58 PM
Subject: Re: Jdbcoutputoperator implementation
Hello,
I am not able to open application master log.
----- Original Message -----
From: "Pradeep A. Dalvi" <p...@apache.org>
To: "users" <users@apex.apache.org>
Sent: Thursday, July 7, 2016 12:35:13 PM
Subject: Re: Jdbcoutputoperator implementation
Please share Application Master logs.
On Thu, Jul 7, 2016 at 11:43 AM, Jaikit Jilka <jji...@leadferret.com> wrote:
> Hello,
>
> I completed my application now. Till now I was launching this application in
> Sandbox and it works perfectly fine. I am getting expected results. But now I
> tried running it on live cluster and it is not working. It is not giving me
> any error but all my operators are killed by application master. So do I need
> to do anything for launching it on live cluster?
>
> ----- Original Message -----
> From: "Jaikit Jilka" <jji...@leadferret.com>
> To: "users" <users@apex.apache.org>
> Sent: Thursday, June 30, 2016 11:29:25 AM
> Subject: Re: Jdbcoutputoperator implementation
>
> Hello,
>
> Now my every operator is working fine. But counter operator does not emit
> anthing. It is just processing the data. Here is my counter.java operator
> file.
>
> ----- Original Message -----
> From: "Jaikit Jilka" <jji...@leadferret.com>
> To: "users" <users@apex.apache.org>
> Sent: Thursday, June 30, 2016 10:25:35 AM
> Subject: Re: Jdbcoutputoperator implementation
>
> Hello,
>
> Got it. I have to create a outport of type pojo. Working on it.
>
> ----- Original Message -----
> From: "Jaikit Jilka" <jji...@leadferret.com>
> To: "users" <users@apex.apache.org>
> Sent: Thursday, June 30, 2016 10:11:10 AM
> Subject: Re: Jdbcoutputoperator implementation
>
> Hello,
>
> I can not emit output.emit(pojo) only emits string. It it giving me error as
> not method found for emit(pojoEvent).
>
> ----- Original Message -----
> From: "Yunhan Wang" <yun...@datatorrent.com>
> To: "users" <users@apex.apache.org>
> Sent: Wednesday, June 29, 2016 2:59:41 PM
> Subject: Re: Jdbcoutputoperator implementation
>
> Hi Jaikit,
>
> I think the main problem here is: your Counter Operator takes String as
> input and emits String as output. However, your JdbcOutputOperator needs
> POJO as input. So you need to modify Counter Operator to emit the POJO
> instead of String. You could do that by writing separate function e.g.
> emitTuples as below:
>
> public void emitTuples(String word)
>
> {
> PojoEvent pojo = new PojoEvent();
> pojo.setName(word);
> output.emit(pojo);
> }
>
> Also, you need to add this to your properties.xml:
>
> <property>
>
> <name>dt.application.test.operator.JdbcOutput.port.input.attr.TUPLE_CLASS</name>
> <value>com.mycompany.test.PojoEvent</value>
> </property>
>
> Hope this can help.
>
> Thanks,
> Yunhan
>
>
>
>
>
> On Wed, Jun 29, 2016 at 1:14 PM, Jaikit Jilka <jji...@leadferret.com> wrote:
>
>> This is my POJO class. Can you please tell me what changes should I make
>> into this.
>>
>> Thank you,
>>
>> Jaikit Jilka
>>
>>
>> ----- Original Message -----
>> From: "Pradeep A. Dalvi" <p...@apache.org>
>> To: "users" <users@apex.apache.org>
>> Sent: Tuesday, June 28, 2016 4:56:57 PM
>> Subject: Re: Jdbcoutputoperator implementation
>>
>> By adding a string member variable to POJO class. Use that member
>> variable to send the data to JdbcPOJOOutputOperator.
>>
>> On Tue, Jun 28, 2016 at 4:34 PM, Jaikit Jilka <jji...@leadferret.com>
>> wrote:
>> > So how to convert a java string into POJO?
>> >
>> > ----- Original Message -----
>> > From: "Pradeep A. Dalvi" <p...@apache.org>
>> > To: "users" <users@apex.apache.org>
>> > Sent: Tuesday, June 28, 2016 4:30:09 PM
>> > Subject: Re: Jdbcoutputoperator implementation
>> >
>> > JdbcPOJOOutputOperator expects data as POJO, whose class name is
>> > specified using Context.PortContext.TUPLE_CLASS i.e.
>> > com.mycompany.test.PojoEvent here. Previous operator is probably
>> > sending String to JdbcPOJOOutputOperator.
>> >
>> > On Tue, Jun 28, 2016 at 4:16 PM, Jaikit Jilka <jji...@leadferret.com>
>> wrote:
>> >> Hello,
>> >>
>> >> I am taking data from a .txt file and spliting words and inserting it
>> into database.
>> >>
>> >> ----- Original Message -----
>> >> From: "Pradeep A. Dalvi" <p...@apache.org>
>> >> To: "users" <users@apex.apache.org>
>> >> Sent: Tuesday, June 28, 2016 4:12:31 PM
>> >> Subject: Re: Jdbcoutputoperator implementation
>> >>
>> >> What is the type of data emitted by previous operator to
>> JdbcPOJOOutputOperator?
>> >>
>> >> On Tue, Jun 28, 2016 at 3:46 PM, Jaikit Jilka <jji...@leadferret.com>
>> wrote:
>> >>> Hello,
>> >>>
>> >>> I am getting an error as follow:
>> >>>
>> >>> Stopped running due to an exception. java.lang.ClassCastException:
>> java.lang.String cannot be cast to com.mycompany.test.PojoEvent
>> >>> at SC.get(Unknown Source)
>> >>> at
>> com.datatorrent.lib.db.jdbc.JdbcPOJOOutputOperator.setStatementParameters(JdbcPOJOOutputOperator.java:175)
>> >>> at
>> com.datatorrent.lib.db.jdbc.AbstractJdbcTransactionableOutputOperator.processBatch(AbstractJdbcTransactionableOutputOperator.java:114)
>> >>> at
>> com.datatorrent.lib.db.jdbc.AbstractJdbcTransactionableOutputOperator.endWindow(AbstractJdbcTransactionableOutputOperator.java:93)
>> >>> at
>> com.datatorrent.stram.engine.GenericNode.processEndWindow(GenericNode.java:145)
>> >>> at
>> com.datatorrent.stram.engine.GenericNode.run(GenericNode.java:351)
>> >>> at
>> com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1388)
>> >>>
>> >>> I am not sure where to check the cast mismatch. I am inserting a value
>> in the database.
>> >>>
>> >>> Thank You,
>> >>>
>> >>> Jaikit Jilka
>> >>>
>> >>> ----- Original Message -----
>> >>> From: "Pradeep A. Dalvi" <p...@apache.org>
>> >>> To: "users" <users@apex.apache.org>
>> >>> Sent: Friday, June 24, 2016 4:45:18 PM
>> >>> Subject: Re: Jdbcoutputoperator implementation
>> >>>
>> >>> What are errors/exceptions seen in JdbcOutput container logs?
>> >>> Are you setting Context.PortContext.TUPLE_CLASS to POJO class being
>> used?
>> >>>
>> >>> --
>> >>> prad
>> >>>
>> >>> On Fri, Jun 24, 2016 at 2:13 PM, Jaikit Jilka <jji...@leadferret.com>
>> wrote:
>> >>>> Hello,
>> >>>>
>> >>>> I resolved that error and now I can launch the application, But now
>> the container which has jdbcoperator is getting killed. I am attaching the
>> dt.log file of container which is running but does not have any operator.
>> >>>>
>> >>>> Thank You,
>> >>>>
>> >>>> Jaikit Jilka
>> >>>>
>> >>>> ----- Original Message -----
>> >>>> From: "Pradeep A. Dalvi" <p...@apache.org>
>> >>>> To: "users" <users@apex.apache.org>
>> >>>> Sent: Friday, June 24, 2016 11:41:47 AM
>> >>>> Subject: Re: Jdbcoutputoperator implementation
>> >>>>
>> >>>> Yes, because it is not yet released. Compile & install latest malhar
>> >>>> in local ~/.m2 directory.
>> >>>> For that, git clone Malhar library repository. And do 'mvn clean
>> >>>> install' in that malhar clone.
>> >>>>
>> >>>> On Fri, Jun 24, 2016 at 11:31 AM, Jaikit Jilka <jji...@leadferret.com>
>> wrote:
>> >>>>> Hello,
>> >>>>>
>> >>>>> When I change my malher version in POM.xml to the latest i.e
>> 3.5.0-SNAPSHOT it gives me an build time error : Failure to find
>> org.apache.apex:malhar-library:jar:3.5.0-SNAPSHOT in
>> https://www.datatorrent.com/maven/content/repositories/snapshots/.
>> >>>>>
>> >>>>> Thank You,
>> >>>>>
>> >>>>> Jaikit Jilka
>> >>>>>
>> >>>>> ----- Original Message -----
>> >>>>> From: "Pradeep A. Dalvi" <p...@apache.org>
>> >>>>> To: "users" <users@apex.apache.org>
>> >>>>> Sent: Friday, June 24, 2016 11:13:42 AM
>> >>>>> Subject: Re: Jdbcoutputoperator implementation
>> >>>>>
>> >>>>> This issue was in 3.4 malhar and fixed later on. Fix is present in
>> >>>>> master branch. You can use latest malhar.
>> >>>>> For reference:
>> https://github.com/apache/apex-malhar/blob/master/library/src/main/java/com/datatorrent/lib/db/jdbc/AbstractJdbcTransactionableOutputOperator.java#L142
>> >>>>>
>> >>>>> On Fri, Jun 24, 2016 at 10:59 AM, Jaikit Jilka <
>> jji...@leadferret.com> wrote:
>> >>>>>> Hello,
>> >>>>>>
>> >>>>>> I am setting UpdateCommand to empty string from properties.xml file
>> still I am getting the same error
>> >>>>>>
>> >>>>>> <property>
>> >>>>>>
>>
>> <name>dt.application.MyWordCountApplication.operator.JdbcOutput.prop.setupdateCommand</name>
>> >>>>>> <value> </value>
>> >>>>>> </property>
>> >>>>>>
>> >>>>>> error:
>> >>>>>>
>> >>>>>> An error occurred trying to launch the application. Server message:
>> javax.validation.ConstraintViolationException: Operator JdbcOutput violates
>> constraints
>> [ConstraintViolationImpl{rootBean=JdbcPOJOOutputOperator{name=null},
>> propertyPath='updateCommand', message='may not be null',
>> leafBean=JdbcPOJOOutputOperator{name=null}, value=null}] at
>> com.datatorrent.stram.plan.logical.LogicalPlan.validate(LogicalPlan.java:1680)
>> at com.datatorrent.stram.StramClient.<init>(StramClient.java:161) at
>> com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.java:509)
>> at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2050)
>> at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3456) at
>> com.datatorrent.stram.cli.DTCli.access$7100(DTCli.java:106) at
>> com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1895) at
>> com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
>> >>>>>>
>> >>>>>> Thank You,
>> >>>>>>
>> >>>>>> Jaikit Jilka
>> >>>>>>
>> >>>>>> ----- Original Message -----
>> >>>>>> From: "Munagala Ramanath" <r...@datatorrent.com>
>> >>>>>> To: "users" <users@apex.apache.org>
>> >>>>>> Sent: Thursday, June 23, 2016 3:32:42 PM
>> >>>>>> Subject: Re: Jdbcoutputoperator implementation
>> >>>>>>
>> >>>>>> Please look at:
>> >>>>>>
>> >>>>>> http://docs.datatorrent.com/beginner/ and
>> >>>>>> http://docs.datatorrent.com/application_packages/
>> >>>>>>
>> >>>>>> for examples on how to set properties from XML files.
>> >>>>>>
>> >>>>>> Ram
>> >>>>>>
>> >>>>>> On Thu, Jun 23, 2016 at 3:25 PM, Jaikit Jilka <
>> jji...@leadferret.com> wrote:
>> >>>>>>
>> >>>>>>> Hello,
>> >>>>>>>
>> >>>>>>> How to pass an empty string for updatecommand in properties.xml.
>> >>>>>>> When I change my malhar version to 3.5.0-SNAPSHOT it gives me a
>> complie
>> >>>>>>> time error -Failure to find
>> >>>>>>> org.apache.apex:malhar-library:jar:3.5.0-SNAPSHOT in
>> >>>>>>> https://www.datatorrent.com/maven/content/repositories/snapshots/
>> >>>>>>>
>> >>>>>>> Thank You,
>> >>>>>>>
>> >>>>>>> Jaikit Jilka
>> >>>>>>>
>> >>>>>>> ----- Original Message -----
>> >>>>>>> From: "Pradeep A. Dalvi" <p...@apache.org>
>> >>>>>>> To: "users" <users@apex.apache.org>
>> >>>>>>> Sent: Thursday, June 23, 2016 12:43:40 PM
>> >>>>>>> Subject: Re: Jdbcoutputoperator implementation
>> >>>>>>>
>> >>>>>>> Hi Jaikit,
>> >>>>>>>
>> >>>>>>> This constraint violation exception was enforced from
>> >>>>>>> AbstractJdbcTransactionableOutputOperator. In 3.3/3.4 malhar
>> library,
>> >>>>>>> updateCommand parameter has constraint set to @NotNull on getter
>> >>>>>>> method.
>> >>>>>>>
>> >>>>>>> You could do either of following to solve this:
>> >>>>>>> 1. Set updateCommand from properties.xml to empty string
>> >>>>>>> 2. Use master branch of Malhar
>> >>>>>>> 3. Remove @NotNull on getUpdateCommand in
>> >>>>>>> AbstractJdbcTransactionableOutputOperator
>> >>>>>>>
>> >>>>>>> Cheers,
>> >>>>>>> Pradeep A. Dalvi
>> >>>>>>>
>> >>>>>>> On Thu, Jun 23, 2016 at 11:36 AM, Jaikit Jilka <
>> jji...@leadferret.com>
>> >>>>>>> wrote:
>> >>>>>>> > Hi Devendra,
>> >>>>>>> >
>> >>>>>>> > I tried to run your application. I followed all steps mentioned
>> but I am
>> >>>>>>> getting following error.
>> >>>>>>> >
>> >>>>>>> > An error occurred trying to launch the application. Server
>> message:
>> >>>>>>> javax.validation.ConstraintViolationException: Operator JdbcOutput
>> violates
>> >>>>>>> constraints
>> >>>>>>>
>> [ConstraintViolationImpl{rootBean=JdbcPOJOOutputOperator{name=null},
>> >>>>>>> propertyPath='updateCommand', message='may not be null',
>> >>>>>>> leafBean=JdbcPOJOOutputOperator{name=null}, value=null}] at
>> >>>>>>>
>> com.datatorrent.stram.plan.logical.LogicalPlan.validate(LogicalPlan.java:1680)
>> >>>>>>> at com.datatorrent.stram.StramClient.<init>(StramClient.java:161)
>> at
>> >>>>>>>
>> com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.java:509)
>> >>>>>>> at
>> com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2050)
>> >>>>>>> at
>> com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3456) at
>> >>>>>>> com.datatorrent.stram.cli.DTCli.access$7100(DTCli.java:106) at
>> >>>>>>>
>> com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1895) at
>> >>>>>>> com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
>> >>>>>>> >
>> >>>>>>> > Thank you,
>> >>>>>>> >
>> >>>>>>> > Jaikit Jilka
>> >>>>>>> >
>> >>>>>>> > ----- Original Message -----
>> >>>>>>> > From: "Devendra Tagare" <devend...@datatorrent.com>
>> >>>>>>> > To: "users" <users@apex.apache.org>
>> >>>>>>> > Sent: Thursday, June 23, 2016 10:32:17 AM
>> >>>>>>> > Subject: Re: Jdbcoutputoperator implementation
>> >>>>>>> >
>> >>>>>>> > Hi Jalkit,
>> >>>>>>> >
>> >>>>>>> > The JdbcPOJOOutputOperator is being configured with the
>> >>>>>>> > JdbcTransactionalStore which requires an additional dt_meta
>> table.
>> >>>>>>> >
>> >>>>>>> > The table schema is here -
>> >>>>>>> >
>> >>>>>>>
>> https://github.com/devtagare/examples/blob/SPOI-8251/tutorials/jdbcToJdbc/src/test/resources/example.sql
>> >>>>>>> >
>> >>>>>>> > You can find an example application which uses the
>> JdbcPOJOOutputOperator
>> >>>>>>> > here -
>> >>>>>>> >
>> >>>>>>>
>> https://github.com/devtagare/examples/tree/SPOI-8251/tutorials/jdbcToJdbc
>> >>>>>>> >
>> >>>>>>> > Thanks,
>> >>>>>>> > Dev
>> >>>>>>> >
>> >>>>>>> > On Thu, Jun 23, 2016 at 10:17 AM, Jaikit Jilka <
>> jji...@leadferret.com>
>> >>>>>>> > wrote:
>> >>>>>>> >
>> >>>>>>> >> Hello,
>> >>>>>>> >>
>> >>>>>>> >> I am currently working on an application which simply read data
>> from
>> >>>>>>> .txt
>> >>>>>>> >> file and slpits the words and inserts them into mysql. But I am
>> getting
>> >>>>>>> >> error. I am using jdbcpojooutputoperator from malhar library.
>> But I am
>> >>>>>>> >> getting an error on ActiveFieldInfos which says it is protected
>> in
>> >>>>>>> >> jdbcpojoinputoperator and also I do not know what to do in
>> >>>>>>> Application.java
>> >>>>>>> >> file. I am attaching both the file here. Kindly help me resolve
>> the
>> >>>>>>> issue.
>> >>>>>>> >>
>> >>>>>>> >> Thank You,
>> >>>>>>> >>
>> >>>>>>> >> Jaikit Jilka
>> >>>>>>>
>>
