Re: JDBCtoJDBC Example

178)) > - Catching up from 1482184295500 to 1482184311524 > > 2016-12-19 13:51:53,548 INFO engine.StreamingContainer > (StreamingContainer.java:processHeartbeatResponse(799)) - Undeploy > request: [1, 2] > > 2016-12-19 13:51:53,549 INFO engine.StreamingContainer > (Stream

Re: JDBCtoJDBC Example

Hi, properties.xml seems to indicate that the databaseUrl is of the form jdbc:oracle:thin:@//host:port/db Can you try setting the databaseUrl to the below format which OracleDriver recommends - here jdbc:oracl

Re: kryo Serealization Exception

Hi, You can set the Locality of the parser and the writer to Container local. This will ensure that Generic Record from the parser does not get serialized between containers. Thanks, Dev On Fri, Aug 19, 2016 at 2:21 PM, Mukkamula, Suryavamshivardhan (CWM-NR) < suryavamshivardhan.mukkam...@rbc.c

Re: supported file formats today ?

Hi Raja, We have a proposal for Parquet writer backed by a WAL in Malhar.I believe Avro write support is also planned. As of now there is nothing out of the box to support ORC/RC support. You can propose it to the community and someone would likely take it up or you can pick it up and contribute

Re: Force Fail Application

Hi, Please see a sample here, https://github.com/apache/apex-malhar/blob/3ce83708f795b081d564be357a8333928154398e/library/src/test/java/com/datatorrent/lib/testbench/SeedEventGeneratorTest.java Thanks, Dev On Mon, Jul 18, 2016 at 10:08 AM, hsy...@gmail.com wrote: > He Michael, > > You can thr

Re: DAG is failing due to memory issues

You can increase the maximum allocation MB but it will require a resource manager restart. Thanks, Dev On Jul 12, 2016 9:01 AM, "Raja.Aravapalli" wrote: > > Thanks for the response Sandesh. > > Since our yarn-site is configured with value *32768* for the property * > yarn.scheduler.maximum-all

Re: Questions about dag validation

Hi Siyuan, Can you bump up the version to 3.5.0-SNAPSHOT.There was a fix for the error mentioned below. Thanks, Dev On Jul 6, 2016 12:17 PM, "hsy...@gmail.com" wrote: > Hey guys, > > I've a problem using jdbc output operator. It is same as > > http://stackoverflow.com/questions/37887390/datato

Affinity support in Apex

Hi All, I was checking out affinity support in a DAG that has 3 operators, Operator 1 -> Operator 2 -> Operator 3 I am trying to ensure that Operator 1 and Operator 3 are on the same node always and Operator 2 could be anywhere. Here's the relevant code snippet, List operatorList = new Lin

Re: Jdbcoutputoperator implementation

Hi Jalkit, The JdbcPOJOOutputOperator is being configured with the JdbcTransactionalStore which requires an additional dt_meta table. The table schema is here - https://github.com/devtagare/examples/blob/SPOI-8251/tutorials/jdbcToJdbc/src/test/resources/example.sql You can find an example applic

Re: kafka input is processing records in a jumbled order

anks a lot Devendra Tagare for the response. > > What you said is very clear and understandable. But, wondering, I am NOT > getting that partition level order!! My operator is processing the records > in jumbled order rather than in sequence! > And, I am saying this because, I am gener

Re: kafka input is processing records in a jumbled order

Hi Raja, When you apply ONE_TO_MANY partitioning scheme, one instance of the operator consumes from many partitions of a kafka topic. When you look at the consumed data, all the events coming from a given partition would be ordered but there are no ordering guarantees across partitions since kafk

Re: kafka offset commit

Hi, I had started work on an offset manager for kafka 0.8x sometime back which got left mid-way.This implementation was using kafka topics to store offsets (similar to 0.9 implementation) https://github.com/apache/apex-malhar/pull/156 If the community is using it, I can incorporate the comments

Re: Time controlled ingestion operator

Hi Sunil, The current implementation of JDBC input operator does not support a time-controlled behavior. As a reference on how to do polling reads there is a PR opened - https://github.com/apache/incubator-apex-malhar/pull/282 One of the solutions for the use-case using Apex features would be as