Solved it. The anonymous RDDs can be cached in the cacheManager in SQLContext.
In order to remove all the cached content use: sqlContext.clearCache() The warning symptom about failed data frame registration is the following entry in the log: 16/04/16 20:18:39 [tp439928219-110] WARN CacheManager: Asked to cache already cached data. Piotr On Wed, Mar 30, 2016 at 3:10 PM, psmolinski <piotr.smolinski...@gmail.com> wrote: > bumping up the topic. > > For the moment I stay with 1.5.2, but I would like to switch to 1.6.x and > this issue is a blocker. > > Thanks, > Piotr > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Cached-Parquet-file-paths-problem-tp26576p26637.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >