contradiction with the abstract/high level (functional
programming) approach when I have to know/consider how Spark doest it.
Van: Rishabh Wadhawan [mailto:rishabh...@gmail.com]
Verzonden: donderdag 2 juni 2016 06:06
Aan: Yash Sharma
CC: Ted Yu ; Matthew Young ; Michel
Hubert ; user
Hi,
My Spark application throws stackoverflow exceptions after a while.
The DAGScheduler function submitMissingTasks tries to serialize a Tuple
(MapPartitionsRDD, EsSpark..saveToEs) which is handled with a recursive
algorithm.
The recursive algorithm is too deep and results in a stackoverflow ex
Hi,
I have an Spark application which generates StackOverflowError exceptions after
30+ min.
Anyone any ideas?
Seems like problems with deserialization of checkpoint data?
16/05/25 10:48:51 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 55449.0
(TID 5584, host81440-cld.opentsp.co
Hi,
I'm thinking of upgdrading our kafka cluster to 0.9.
Will this be a problem for the Spark Streaming + Kafka Direct Approach
Integration using artifact spark-streaming-kafka_2.10 (1.6.1)?
groupId = org.apache.spark
artifactId = spark-streaming-kafka_2.10
version = 1.6.1
Because the docume
We're running Kafka 0.8.2.2
Is that the problem, why?
-Oorspronkelijk bericht-
Van: Sean Owen [mailto:so...@cloudera.com]
Verzonden: woensdag 4 mei 2016 10:41
Aan: Michel Hubert
CC: user@spark.apache.org
Onderwerp: Re: run-example streaming.KafkaWordCount fails on CDH 5.7.0
Pleas
This is production.
Van: Mich Talebzadeh [mailto:mich.talebza...@gmail.com]
Verzonden: dinsdag 26 april 2016 12:01
Aan: Michel Hubert
CC: user@spark.apache.org
Onderwerp: Re: Kafka exception in Apache Spark
Hi Michael,
Is this production or test?
Dr Mich Talebzadeh
LinkedIn
https
Hi,
I use a Kafka direct stream approach.
My Spark application was running ok.
This morning we upgraded to CDH 5.7.0
And when I re-started my Spark application I get exceptions.
It seems a problem with the direct stream approach.
Any ideas how to fix this?
User class threw exception: org.apac
foreachRDD(new VoidFunction>() {
public void call(JavaRDD rdd) throws Exception {
for (TopData t: rdd.take(top)) {
jedis …
}
May this resulted in a memory leak?
Van: Ted Yu [mailto:yuzhih...@gmail.com]
Verzonden: donderdag 24 maart 2016 15:15
Aan: Michel Hub
Yu [mailto:yuzhih...@gmail.com]
Verzonden: donderdag 24 maart 2016 14:33
Aan: Michel Hubert
CC: user@spark.apache.org
Onderwerp: Re: apache spark errors
Which release of Spark are you using ?
Have you looked the tasks whose Ids were printed to see if there was more clue ?
Thanks
On Thu, Mar 24
HI,
I constantly get these errors:
0[Executor task launch worker-15] ERROR org.apache.spark.executor.Executor
- Managed memory leak detected; size = 6564500 bytes, TID = 38969
310002 [Executor task launch worker-12] ERROR
org.apache.spark.executor.Executor - Managed memory leak detected;
Hi,
I'm trying to run a Spark 1.6.0 application on a CDH 5.6.0 cluster.
How do I submit the uber-jar so it's totally self-reliant?
With kind regards,
Mitchel
spark-submit --class TEST --master yarn-cluster ./uber-TEST-1.0-SNAPSHOT.jar
Spark 1.6.1
Version: Cloudera Express 5.6.0
16/03/22 09
not be the time of the time the batch was scheduled.
I want to retrieve the job/task schedule time of the batch for which my
updateStateByKey(..) routine is called.
Is this possible?
With kind regards,
Michel Hubert
Hi,
I was just wondering how you generated to second image with the charts.
What product?
From: Anand Nalya [mailto:anand.na...@gmail.com]
Sent: donderdag 9 juli 2015 11:48
To: spark users
Subject: Breaking lineage and reducing stages in Spark Streaming
Hi,
I've an application in which an rdd i
Hi,
I've developed a POC Spark Streaming application.
But it seems to perform better on my development machine than on our cluster.
I submit it to yarn on our cloudera cluster.
But my first question is more detailed:
In de application UI (:4040) I see in the streaming section that the batch
p
14 matches
Mail list logo