Re: Cached Parquet file paths problem

2016-03-30 Thread psmolinski
bumping up the topic. For the moment I stay with 1.5.2, but I would like to switch to 1.6.x and this issue is a blocker. Thanks, Piotr -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Cached-Parquet-file-paths-problem-tp26576p26637.html Sent from the

Cached Parquet file paths problem

2016-03-23 Thread psmolinski
Hi, After migration from Spark 1.5.2 to 1.6.1 I faced strange issue. I have a Parquet directory with partitions. Each partition (month) is a subject of incremental ETL that takes current Avro files and replaces the corresponding Parquet files. Now there is a problem that appeared in 1.6.x: I