Flink 1.12.0 docker image MISSING

2021-01-05 Thread Alexandru Vasiu
Hi, Docker image for flink 1.12 is missing from Docker Hub. Thank you, Alex Vasiu -- ComplyAdvantage is a trading name of IVXS TECHNOLOGY ROMANIA. This message, including any attachments, is intended only for the use of the individual(s) to whom it is addressed and may contain information

Manage multiple jobs in Flink

2020-11-02 Thread Alexandru Vasiu
Hi, I have some questions: 1. How can you manage multiple jars (jobs) easily using Flink? 2. All jobs should run on the same task manager or do we need to use one for each job? 3. Can we store the jars in some persistent storage (such as S3) and start a job for each jar from that storage? 4.

Re: Async Datastream Checkpointing

2020-02-28 Thread Alexandru Vasiu
Hi, That's how we used the executor. I think the problem is that the web requests took too long time to complete (3-4 seconds) because the requests are using a proxy server. I also transformed the asyncDataStream using a flatMap and same issue (no successfull checkpoint). If I used a simple web

Re: Async Datastream Checkpointing

2020-02-27 Thread Alexandru Vasiu
? Thank you, Alex On Thu, Feb 27, 2020 at 4:24 PM Alexandru Vasiu < alexandru.va...@complyadvantage.com> wrote: > Hi, > > We tried to > use ExecutionContext.fromExecutor(Executors.newFixedThreadPool(1)) as > executor and we still have this problem. We even tried even other &

Async Datastream Checkpointing

2020-02-26 Thread Alexandru Vasiu
Hi, We have a pipeline which has somewhere a step of *AsyncDataStream.unorderedWait *where some web requests are executed. The pipeline works, but when it tries to make checkpoints it fails always with a timeout error (and it stops at the component containing this async data stream). We are using

Re: POJO ERROR

2019-12-19 Thread Alexandru Vasiu
ards, > Timo > > > On 19.12.19 14:50, Alexandru Vasiu wrote: > > Nope, because scalaBuildVersion is the scala version including minor > > version so in this case: 2.12.10 and we used it just where we need. > > We used scalaVersion to specify for each library what scala is used,

Re: POJO ERROR

2019-12-19 Thread Alexandru Vasiu
Walther wrote: > I see a mismatch between scalaBuildVersion and scalaVersion could this > be the issue? > > Regards, > Timo > > > On 19.12.19 14:33, Alexandru Vasiu wrote: > > This is a part of my Gradle config: > > > > ext { > > scal

Re: POJO ERROR

2019-12-19 Thread Alexandru Vasiu
19, 2019 at 3:14 PM Timo Walther wrote: > That's sounds like a classloading or most likely dependency issue. > > Are all dependencies including Flink use the same Scala version? Could > you maybe share reproducible some code with us? > > Regards, > Timo > > > On 19.12.1

Re: POJO ERROR

2019-12-19 Thread Alexandru Vasiu
a/math/Ordering$$anon$9" when I'm trying to apply the map/flatMap function over the stream (which is from a Kafka consumer). Alex On Thu, Dec 19, 2019 at 2:24 PM Alexandru Vasiu wrote: > I used `case class` for example case class A(a: Map[String, String]) so it > should work > &g

Re: POJO ERROR

2019-12-19 Thread Alexandru Vasiu
ractor and treat it as a generic type > because it is not a POJO. > > I hope this helps. > > Regards, > Timo > > > On 19.12.19 12:41, Alexandru Vasiu wrote: > > Hi, > > > > I use flink-scala version 1.9.1 and scala 2.12.10, and I defined a data > >

POJO ERROR

2019-12-19 Thread Alexandru Vasiu
Hi, I use flink-scala version 1.9.1 and scala 2.12.10, and I defined a data type which is a bit more complex: it has a list in it and even a dictionary. When I try to use a custom map I got this error: INFO org.apache.flink.api.java.typeutils.TypeExtractor - class A does not contain a setter