Re: Concatenation of parquet files

2021-09-15 Thread Pau Tallada
umn > metadata (which is in the footer). So if you relocate the footer then > these offsets will need to be updated. > > On Tue, Sep 14, 2021 at 3:01 AM Pau Tallada wrote: > > > > Dear Gabor, > > > > Thanks a lot for the clarification! ☺ > > I understa

Re: Concatenation of parquet files

2021-09-14 Thread Pau Tallada
e > small files problem we need to merge the row groups and for that we need to > decode the original data so we can re-create the statistics (at least for > bloom filters). > > Long story short, theoretically it is solvable but it is a feature we > haven't implemente

Concatenation of parquet files

2021-09-14 Thread Pau Tallada
:* PAR1 ROW GROUP A1 ROW GROUP A2 ROW GROUP B1 ROW GROUP B2 NEW FOOTER A+B Thanks a lot in advance for the help understanding this, Best regards, Pau. -- ---------- Pau Tallada Crespí Departament de Serveis Port d'Informació Científica (PIC) Tel: +34 93 170 2729 --