Spark + Parquet, parquet dictionary

2020-09-05 Thread Mania Abdi
Hello everyone, I have two questions about Parquet File format: 1. Where is the parquet dictionary is stored in ParquetFile? Is it stored in the Footer of the file? Or is it stored in each page? 2. When Spark reads a Parquet File, how is an RDD partitioned to read a ParquetFile? Does it allocate

[jira] [Commented] (PARQUET-1822) Parquet without Hadoop dependencies

2020-09-05 Thread Ben Watson (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191022#comment-17191022 ] Ben Watson commented on PARQUET-1822: - I also had this problem and I hope I can help you. I