Re: spark streaming failing to replicate blocks

If you can reproduce, then i think you can open up a jira for this. Thanks Best Regards On Fri, Oct 23, 2015 at 1:37 PM, Eugen Cepoi wrote: > When fixing the port to the same values as in the stack trace it works > too. The network config of the slaves seems correct. > >

Re: spark streaming failing to replicate blocks

Mostly a network issue, you need to check your network configuration from the aws console and make sure the ports are accessible within the cluster. Thanks Best Regards On Thu, Oct 22, 2015 at 8:53 PM, Eugen Cepoi wrote: > Huh indeed this worked, thanks. Do you know why

Re: spark streaming failing to replicate blocks

When fixing the port to the same values as in the stack trace it works too. The network config of the slaves seems correct. Thanks, Eugen 2015-10-23 8:30 GMT+02:00 Akhil Das : > Mostly a network issue, you need to check your network configuration from > the aws

Re: spark streaming failing to replicate blocks

Can you try fixing spark.blockManager.port to specific port and see if the issue exists? Thanks Best Regards On Mon, Oct 19, 2015 at 6:21 PM, Eugen Cepoi wrote: > Hi, > > I am running spark streaming 1.4.1 on EMR (AMI 3.9) over YARN. > The job is reading data from

Re: spark streaming failing to replicate blocks

Huh indeed this worked, thanks. Do you know why this happens, is that some known issue? Thanks, Eugen 2015-10-22 19:08 GMT+07:00 Akhil Das : > Can you try fixing spark.blockManager.port to specific port and see if the > issue exists? > > Thanks > Best Regards > > On

spark streaming failing to replicate blocks

Hi, I am running spark streaming 1.4.1 on EMR (AMI 3.9) over YARN. The job is reading data from Kinesis and the batch size is of 30s (I used the same value for the kinesis checkpointing). In the executor logs I can see every 5 seconds a sequence of stacktraces indicating that the block