I should add that I tried using a waiter on the _SUCCESS file but it did not prove successful as due to its small size compared to the part-0 file it seems to be appearing before the part-0 file in s3, even though it was written afterwards.
-- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
