Yes,the exception occured sometimes,but at the end the final result rised.
2015-03-26 11:08 GMT+08:00 Saisai Shao :
> Would you mind running again to see if this exception can be reproduced
> again, since exception in MapOutputTracker seldom occurs, maybe some other
> exceptions which lead to t
Would you mind running again to see if this exception can be reproduced
again, since exception in MapOutputTracker seldom occurs, maybe some other
exceptions which lead to this error.
Thanks
Jerry
2015-03-26 10:55 GMT+08:00 李铖 :
> One more exception.How to fix it .Anybody help me ,please.
>
>
>
One more exception.How to fix it .Anybody help me ,please.
org.apache.spark.shuffle.MetadataFetchFailedException: Missing an output
location for shuffle 0
at
org.apache.spark.MapOutputTracker$$anonfun$org$apache$spark$MapOutputTracker$$convertMapStatuses$1.apply(MapOutputTracker.scala:386)
at
org
Yes, it works after I append the two properties in spark-defaults.conf.
As I use python programing on spark platform,the python api does not have
SparkConf api.
Thanks.
2015-03-25 21:07 GMT+08:00 Cheng Lian :
> Oh, just noticed that you were calling sc.setSystemProperty. Actually
> you need t
Oh, just noticed that you were calling |sc.setSystemProperty|. Actually
you need to set this property in SparkConf or in spark-defaults.conf.
And there are two configurations related to Kryo buffer size,
* spark.kryoserializer.buffer.mb, which is the initial size, and
* spark.kryoserializer.b
Here is the full track
15/03/25 17:48:34 WARN TaskSetManager: Lost task 0.0 in stage 1.0 (TID 1,
cloud1): com.esotericsoftware.kryo.KryoException: Buffer overflow.
Available: 0, required: 39135
at com.esotericsoftware.kryo.io.Output.require(Output.java:138)
at com.esotericsoftware.kryo.io.Output.w
Could you please provide the full stack trace?
On 3/25/15 6:26 PM, 李铖 wrote:
It is ok when I do query data from a small hdfs file.
But if the hdfs file is 152m,I got this exception.
I try this code
.'sc.setSystemProperty("spark.kryoserializer.buffer.mb",'256')'.error
still.
```
com.esoterics
It is ok when I do query data from a small hdfs file.
But if the hdfs file is 152m,I got this exception.
I try this code
.'sc.setSystemProperty("spark.kryoserializer.buffer.mb",'256')'.error
still.
```
com.esotericsoftware.kryo.KryoException: Buffer overflow. Available: 0,
required: 39135
at com.e
