Thank you ,
I have tried to resolve this issue by making changes in the spark
configuration and use two fields as DICTIONARY_INCLUDE.
test data(30G) load 8 times, each time about 1.5 minutes to complete
Is currently testing another larger data, hope to be successful, thank you
very much for
Hi Feng,
You can also refer the below links wherein the spark users have tried to
resolve this issue by making changes in the configuration. This might help
you.
https://stackoverflow.com/questions/28901123/why-do-spark-jobs-fail-with-org-apache-spark-shuffle-metadatafetchfailedexceptio
Hello,
I don't get much from the logs but the error seems related to memory issue
from Spark. From your old emails I get that you are using 3 node cluster. Is
that all 3 node has nodemanager and datanodes?
So better give only less number of executors and provide more memory to it
like below.
Hi
1.Source code can be compiled successfully with script "mvn clean
-DskipTests -Pspark-2.1 -Pbuild-with-format package"
2.Can query carbondata file properly in Spark-shell.
3.License file looks good.
4.Signature file looks good
5.Hash checksum files look good
6.NOTICE file looks good
My vote :
