Re: Flink on AWS - ActiveMQ connector

2020-03-02 Thread Tzu-Li (Gordon) Tai
the respective Flink connector documentation pages (for example [1] for Flink's Apache Kafka connector) on how to use those connectors in your jobs. As for the ActiveMQ connector provided by Apache Bahir, there's also a Maven artifact for that shipped by Apache Bahir [2]. Cheers, Gordon [1] https

Flink on AWS - ActiveMQ connector

2020-03-02 Thread KristoffSC
Hi all, In AWS documentation [1] we can see that AWS provides some set of connectors for Flink. I would need to use an ActiveMQ one provided by [2]. Currently I'm using Docker based stand alone Job Cluster and not AWS one. Whats up with those connectors provided by AWS? Will I be able to use my

Re: ActiveMQ connector

2020-01-30 Thread Piotr Nowojski
f your > `HashMap` to Flink managed state during `snapshotState` call." > > From [1] we can read > "Each parallel instance of the Kafka consumer maintains a map of topic > partitions and offsets as its Operator State." > > Oskar was asking about ActiveMq and not

Re: ActiveMQ connector

2020-01-30 Thread KristoffSC
call." >From [1] we can read "Each parallel instance of the Kafka consumer maintains a map of topic partitions and offsets as its Operator State." Oskar was asking about ActiveMq and not Kafka but I guess the rule applies here also. The ActiveMq connector he is using is this

Re: ActiveMQ connector

2020-01-30 Thread Piotr Nowojski
; [2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface <https://cwiki.apache.org/confluence/display/FLINK/FLIP-27:+Refactor+Source+Interface> > On 29 Jan 2020, at 13:08, OskarM wrote: > > Hi all, > > I am using Flink with Bahir's Apache

ActiveMQ connector

2020-01-29 Thread OskarM
Hi all, I am using Flink with Bahir's Apache ActiveMQ connector. However it's quite dated and poses many limitations, most notably the source supports only ByteMessages, does not support parallelism and has a bug that is only fixed in a snapshot version. So I started implementing my own

Re: activemq connector not working..

2018-03-15 Thread Puneet Kinra
I tried getting this in logs.. 2018-03-15 20:59:38,154 INFO org.apache.flink.streaming.runtime.tasks.StreamTask - No state backend has been configured, using default state backend (Memory / JobManager) 2018-03-15 20:59:38,296 INFO

Re: activemq connector not working..

2018-03-15 Thread Puneet Kinra
I tried in cluster as well . On Wed, Mar 14, 2018 at 10:01 PM, Timo Walther wrote: > Hi Puneet, > > are you running this job on the cluster or locally in your IDE? > > Regards, > Timo > > > Am 14.03.18 um 13:49 schrieb Puneet Kinra: > > Hi > > I used apache bahir connector

Re: activemq connector not working..

2018-03-14 Thread Timo Walther
Hi Puneet, are you running this job on the cluster or locally in your IDE? Regards, Timo Am 14.03.18 um 13:49 schrieb Puneet Kinra: Hi I used apache bahir connector  below is the code.the job is getting finished and not generated the output as well ,ideal it should keep on running below

activemq connector not working..

2018-03-14 Thread Puneet Kinra
Hi I used apache bahir connector below is the code.the job is getting finished and not generated the output as well ,ideal it should keep on running below the code. import org.apache.activemq.ActiveMQConnectionFactory; import org.apache.flink.streaming.api.datastream.DataStream; import