Hey Panos, our solution allows us to analyze the full path and modify the file name. so for multiple partitions, we can extract the values of the partitions and then inject them into the file name.
for example, for the following file: s3://some_bucket/some_folder/partition1=value1/partition2=value2/part-123.c001.csv we will store it like that: s3://some_bucket/some_folder/value1-value2-part-123.c001.csv and since we can modify the file name as we want, then theoretically two files can get the same name. which will either throw an exception or override of one of the files, according to the configurations. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
