Hi,
We would like to control how spark generates its application id. Currently,
it changes everytime we restart the job and also hard to correlate. For e.g.
it looks like this - app-20161129054045-0096.
I would like to control how this id gets generated to its easier to track
when jmx metrics
Hi All,
I am a newbie to spark and want to know if there is any performance
difference between map vs mapPartitions if I am doing strictly a per item
transformation?
For e.g.
reversedWords = words.map(w => w.reverse());
vs.
reversedWords = words.mapPartitions(pwordsIterator => {
List
Hi Colzer,
Thanks for the response. My main question was about writing one file per
"key" i.e. have a file with all values for a given key. So in the pseudo
code that I have above, am I opening/creating the file in the right place?.
Once the file is created and closed, I cannot append to it.
Any recommendations? comments?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Writing-all-values-for-same-key-to-one-file-tp27455p27480.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
