Re: Flink Serialization and case class fields limit

gt; Andrey > > [1] https://underscore.io/blog/posts/2016/10/11/twenty-two.html > > > On 16 Nov 2018, at 13:10, Andrea Sella wrote: > > Hey squirrels, > > I've started to study more in-depth Flink Serialization and its "type > system". > > I have a genera

Flink Serialization and case class fields limit

Hey squirrels, I've started to study more in-depth Flink Serialization and its "type system". I have a generated case class using scalapb that has more than 30 fields; I've seen that Flink still uses the CaseClassSerializer, the TypeInformation is CaseClassTypeInfo, even if in the docs[1] is

Re: OutputFormat in streaming job

would rather go for the SinkFunction, which is better suited for > streaming jobs. > > Cheers, Fabian > > 2016-05-06 14:10 GMT+02:00 Andrea Sella <andrea.se...@radicalbit.io>: > >> Hi Fabian, >> >> ATM I am not interesting to guarantee exactly-once processing,

Re: OutputFormat in streaming job

s not guarantee exactly-once processing. In case > of a failure, it might emit some records a second time. Hence the results > will be written at least once. > > Hope this helps, > Fabian > > 2016-05-06 12:45 GMT+02:00 Andrea Sella <andrea.se...@radicalbit.io>: > >&g

OutputFormat in streaming job

Hi, I created a custom OutputFormat to send data to a remote actor, there are issues to use an OutputFormat into a stream job? Or it will treat like a Sink? I prefer to use it in order to create a custom ActorSystem per TM in the configure method. Cheers, Andrea

Re: Flink and YARN ship folder

` with -yt params it will work fine. is it a bug? Cheers, Andrea 2016-03-14 15:00 GMT+01:00 Andrea Sella <andrea.se...@radicalbit.io>: > Hi Robert, > > Ok, thank you. > > 2016-03-14 11:13 GMT+01:00 Robert Metzger <rmetz...@apache.org>: > >> Hi Andrea, >> &

Re: Integration Alluxio and Flink

into > the lib folder. > > On Tue, Mar 15, 2016 at 5:40 PM, Andrea Sella <andrea.se...@radicalbit.io> > wrote: > >> Hi Till, >> >> I put the jar as dependency of my job on build.sbt. I need to do >> somenthing else? >> >> val flinkDep

Re: JobManager Dashboard and YARN execution

ys access JobManager logs through the UI. >> >> Stephan >> >> >> On Wed, Mar 9, 2016 at 12:47 PM, Andrea Sella <andrea.se...@radicalbit.io >> > wrote: >> >>> Hi, >>> I am experimenting the integration between Apache Flink and YARN. &g

Re: Integration Alluxio and Flink

ricsoftware.minlog", "minlog"). exclude("org.apache.hadoop","hadoop-yarn-common") ) `show compile:dependencyClasspath` shows alluxio-core-client as dependency. https://github.com/alkagin/alluxio-wordcount/ Thanks, Andrea 2016-03-15 17:09 GMT+01:00 Til

Re: Integration Alluxio and Flink

nk is using > version 3.8 whereas alluxio-core-client uses version 3.2.2. I think you > have to exclude or shade this dependency away. > > Cheers, > Till > ​ > > On Mon, Mar 14, 2016 at 5:12 PM, Andrea Sella <andrea.se...@radicalbit.io> > wrote: > >> Hi Till,

Re: Integration Alluxio and Flink

since the Alluxio > documentation indicates that it works with Flink (but I cannot tell for > which version). > > Cheers, > Till > > On Mon, Mar 14, 2016 at 3:18 PM, Andrea Sella <andrea.se...@radicalbit.io> > wrote: > >> Hi to all, >> >> I'm tryi

Integration Alluxio and Flink

Hi to all, I'm trying to integrate Alluxio and Apache Flink, I followed Running Flink on Alluxio to setup Flink. I tested in local mode executing: bin/flink run ./examples/batch/WordCount.jar --input

Re: Flink and YARN ship folder

> > > On Sat, Mar 12, 2016 at 3:25 PM, Andrea Sella <andrea.se...@radicalbit.io> > wrote: > >> Hi Ufuk, >> >> I'm trying to execute the WordCount batch example with input and output >> on Alluxio, i followed Running Flink on Alluxio >> <http://w

Re: Flink and YARN ship folder

hermore, > you can check the classpath of the JVM in the YARN logs of the > JobManager/TaskManager processes. > > – Ufuk > > > On Fri, Mar 11, 2016 at 5:33 PM, Andrea Sella > <andrea.se...@radicalbit.io> wrote: > > Hi, > > > > There is a way to add ext

Flink and YARN ship folder

Hi, There is a way to add external dependencies to Flink Job, running on YARN, not using HADOOP_CLASSPATH? I am looking for a similar idea to standalone mode using lib folder. BR, Andrea

JobManager Dashboard and YARN execution

Hi, I am experimenting the integration between Apache Flink and YARN. When I am running a Flink job using yarn-cluster, is the Dashboard available to monitor the execution? Is it also available for long running session? Is it possibile retrieve logs from the dashboard or I have to pass via yarn

Re: Webclient script misses building from source

art of the JobManager dashboard by > setting: > > jobmanager.web.submit.enable: false > > in flink-conf.yaml. > > Cheers, > Aljoscha > > On 08 Mar 2016, at 14:21, Andrea Sella <andrea.se...@radicalbit.io> > wrote: > > > > Hi, > > > > I've

Webclient script misses building from source

Hi, I've built Flink from source but I was not able to find in build-target/bin the script start-webclient.sh to launch the WebUI. The script is available just in the binaries or I have to add an argument to trigger its generation? Thanks in advance, Andrea

Underlying TaskManager's Actor System

Hi, There is a way to access to the underlying TaskManager's Actor System? Thank you in advance, Andrea

Re: Unexpected behavior with Scala App trait.

.x/src/library/scala/App.scala#L31 > > > On Jan 19, 2016, at 12:08 AM, Andrea Sella <andrea.se...@radicalbit.io> > wrote: > > > > Hi, > > > > I was implementing TF-IDF example of flink-training when I faced a > problem with NPE during the deploy of my Job. >