Unsubscribe
ures)
spark.streaming.kafka.consumer.poll.ms=1
Have tried increasing driver memory to 4GB and also increased driver.cores to 4.
If anybody has faced similar issues please provide some pointers to how to
address this issue.
Thanks a lot for your time.
Regards,
Edwin
Hi,
You can enable backpressure to handle this.
spark.streaming.backpressure.enabled
spark.streaming.receiver.maxRate
Thanks,
Edwin
On Mar 18, 2017, 12:53 AM -0400, sagarcasual . , wrote:
> Hi, we have spark 1.6.1 streaming from Kafka (0.10.1) topic using direct
> approach. The streamin
to man Yarn and impala co-exist.
Thanks,
Edwin
On Mar 16, 2017, 5:44 AM -0400, yohann jardin , wrote:
> Hello everyone,
>
> I'm also really interested in the answers as I will be facing the same issue
> soon.
> Muthu, if you evaluate again Apache Ignite, can you share your
I'm using Apache Spark 1.1.0 and I'm currently having issue with broadcast
method. So when I call broadcast function on a small dataset to a 5 nodes
cluster, I experiencing the "Error sending message as driverActor is null"
after broadcast the variables several times (apps running under jboss).
An
Hi,
Is there a way to check the status of the SparkContext regarding whether
it's alive or not through the code, not through UI or anything else?
Thanks
Edwin
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Proper-way-to-check-SparkContext-s-s
lized). So does anyone have any idea on this?
Thanks
Edwin
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/cache-function-is-not-working-on-RDD-from-parallelize-tp18219.html
Sent from the Apache Spark User List mailing list archive at Nabbl
Are you using hive 0.13?
Switching back to HadoopDefaultAuthenticator in your hive-site.xml worth a
shot
hive.security.authenticator.manager
org.apache.hadoop.hive.ql.security.HadoopDefaultAuthenticator
- Edwin
On Fri, Oct 3, 2014 at 4:25 PM, Li HM wrote:
> I
19:02:45,963 INFO [org.apache.spark.MapOutputTrackerMaster]
(spark-akka.actor.default-dispatcher-14) Size of output statuses for shuffle
1 is 216 bytes
19:02:45,964 INFO [org.apache.spark.scheduler.DAGScheduler]
(spark-akka.actor.default-dispatcher-14) Got job 5 (getCallSite at null:-1)
with 24
does union function cause any data shuffling?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/apache-spark-union-function-cause-executors-disassociate-Lost-executor-1-on-172-32-1-12-remote-Akka--tp15442p15444.html
Sent from the Apache Spark User List mailing
x27;t
be a problem (can tell from the UI). what worth mentioning is that one rdd
is significant bigger than the other one (much bigger), does anyone have any
idea why?
Thanks
Edwin
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/apache-spark-union-function-
53539 bytes.
Thanks
Edwin
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Transient-association-error-on-a-3-nodes-cluster-tp14914.html
Sent from the Apache Spark User List mailing list archive at Nabbl
12 matches
Mail list logo