Re: Flink DataSet Iterate updating additional variable

ojects/flink/flink-docs-stable/dev/batch/#broadcast-variables > > Regards, > Roman > > > On Mon, Jul 13, 2020 at 3:49 PM Antonio Martínez Carratalá < > amarti...@alto-analytics.com> wrote: > >> Hello >> >> I'm trying to implement the ForceAtlas2

Flink DataSet Iterate updating additional variable

Hello I'm trying to implement the ForceAtlas2 (graph layout) algorithm in Flink using datasets, it is an iterative algorithm and I have most of it ready, but there is something I don't know how to do. Apart from the dataset with the coordinates (x,y) of each node I need an additional variable to r

Re: Flink in EMR configuration problem

ing to allocate 2 > TaskManagers AND 1 JobManager with 6GB heap size each? > > Piotrek > > > On 31 Mar 2020, at 17:01, Antonio Martínez Carratalá < > amarti...@alto-analytics.com> wrote: > > > > Hi, I'm trying to run a job in a Flink cluster in Amazon EMR

Flink in EMR configuration problem

Hi, I'm trying to run a job in a Flink cluster in Amazon EMR from java code but I'm having some problems This is how I create the cluster: StepConfig copyJarStep = new StepConfig() .wit

Re: Run several jobs in parallel in same EMR cluster?

askManagerOptions.java#L197-L199 > > On Mon, Mar 30, 2020 at 1:22 PM Antonio Martínez Carratalá < > amarti...@alto-analytics.com> wrote: > >> Hello >> >> I'm running Flink over Amazon EMR and I'm trying to send several >> different batc

Run several jobs in parallel in same EMR cluster?

Hello I'm running Flink over Amazon EMR and I'm trying to send several different batch jobs to the cluster after creating it. This is my cluster creation code: StepConfig copyJarStep = new StepConfig() .withName("copy-jar-step")

Re: Flink gelly dependency in transient EMR cluster

arStep(new HadoopJarStepConfig("command-runner.jar") .withArgs("bash", "-c", "sudo cp /usr/lib/flink/opt/flink-gelly_2.11-1.8.0.jar /usr/lib/flink/lib")); On Thu, Mar 12, 2020 at 9:43 AM Antonio Martínez Carratalá < amarti...@alto-analytics.co

Flink gelly dependency in transient EMR cluster

Hello, I'm trying to run a flink job that works with graphs in a transient cluster in EMR, here is my code: -- HadoopJarStepConfig copyJarStepConf = new HadoopJarStepConfig() .withJar("command-runner.jar") .withArgs("bash", "-c", "aws s3 cp s3://" + BUCKET_NAME

Re: Flink remote batch execution in dynamic cluster

tandalone, Yarn, Kubernetes and Docker do. > > Piotrek > > On 28 Feb 2020, at 10:25, Antonio Martínez Carratalá < > amarti...@alto-analytics.com> wrote: > > Hello > > I'm working on a project with Flink 1.8. I'm running my code from Java in > a remot

Flink remote batch execution in dynamic cluster

Hello I'm working on a project with Flink 1.8. I'm running my code from Java in a remote Flink as described here https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/cluster_execution.html . That part is working, but I want to configure a dynamic Flink cluster to execute the jobs Imag