Unsubscribe
ures)
spark.streaming.kafka.consumer.poll.ms=1
Have tried increasing driver memory to 4GB and also increased driver.cores to 4.
If anybody has faced similar issues please provide some pointers to how to
address this issue.
Thanks a lot for your time.
Regards,
Edwin
Hi,
You can enable backpressure to handle this.
spark.streaming.backpressure.enabled
spark.streaming.receiver.maxRate
Thanks,
Edwin
On Mar 18, 2017, 12:53 AM -0400, sagarcasual . <sagarcas...@gmail.com>, wrote:
> Hi, we have spark 1.6.1 streaming from Kafka (0.10.1) topic usi
to man Yarn and impala co-exist.
Thanks,
Edwin
On Mar 16, 2017, 5:44 AM -0400, yohann jardin <yohannjar...@hotmail.com>, wrote:
> Hello everyone,
>
> I'm also really interested in the answers as I will be facing the same issue
> soon.
> Muthu, if you evaluate again Apache
I'm using Apache Spark 1.1.0 and I'm currently having issue with broadcast
method. So when I call broadcast function on a small dataset to a 5 nodes
cluster, I experiencing the Error sending message as driverActor is null
after broadcast the variables several times (apps running under jboss).
Any
Hi,
Is there a way to check the status of the SparkContext regarding whether
it's alive or not through the code, not through UI or anything else?
Thanks
Edwin
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Proper-way-to-check-SparkContext-s-status
). So does anyone have any idea on this?
Thanks
Edwin
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/cache-function-is-not-working-on-RDD-from-parallelize-tp18219.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
be a problem (can tell from the UI). what worth mentioning is that one rdd
is significant bigger than the other one (much bigger), does anyone have any
idea why?
Thanks
Edwin
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/apache-spark-union-function-cause
does union function cause any data shuffling?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/apache-spark-union-function-cause-executors-disassociate-Lost-executor-1-on-172-32-1-12-remote-Akka--tp15442p15444.html
Sent from the Apache Spark User List
19:02:45,963 INFO [org.apache.spark.MapOutputTrackerMaster]
(spark-akka.actor.default-dispatcher-14) Size of output statuses for shuffle
1 is 216 bytes
19:02:45,964 INFO [org.apache.spark.scheduler.DAGScheduler]
(spark-akka.actor.default-dispatcher-14) Got job 5 (getCallSite at null:-1)
with
bytes.
Thanks
Edwin
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Transient-association-error-on-a-3-nodes-cluster-tp14914.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
11 matches
Mail list logo