Hi I have made my changes for api here, does it look good and is this what you were seeking from me? The writer- api is still in the works and I need to make the reader more generic to support all class data types.
https://github.com/a2un/arrow/blob/PARQUET-1404-Add-index-pages-to-the-format-to-support-efficient-page-skipping-to-parquet-cpp/cpp/examples/parquet/low-level-api/reader-writer-with-index.cc Regards, Arun Balajiee From: Wes McKinney<mailto:wesmck...@gmail.com> Sent: Tuesday, February 4, 2020 11:24 PM To: Parquet Dev<mailto:dev@parquet.apache.org> Cc: Deepak Majeti<mailto:deepak.maj...@microfocus.com>; Anatoli Shein<mailto:sh...@microfocus.com> Subject: Re: Arrow 1404: Adding index for Page-level Skipping hi Arun, We can keep the discussion going on here and on GitHub when you have a pull request to discuss. There are a number of different people who can give advice. Thanks On Tue, Feb 4, 2020 at 10:11 PM Lekshmi Narayanan, Arun Balajiee <arl...@pitt.edu> wrote: > > Actually I made some changes after the date on the pull request ( even in > this year), which are not getting reflected on this compare link > > Regards, > Arun Balajiee > > From: Wes McKinney<mailto:wesmck...@gmail.com> > Sent: Tuesday, February 4, 2020 6:43 PM > To: Parquet Dev<mailto:dev@parquet.apache.org> > Cc: Deepak Majeti<mailto:deepak.maj...@microfocus.com>; Anatoli > Shein<mailto:sh...@microfocus.com> > Subject: Re: Arrow 1404: Adding index for Page-level Skipping > > Here's a compare link in case others want to have a look > > https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Farrow%2Fcompare%2Fmaster...a2un%3APARQUET-1404-Add-index-pages-to-the-format-to-support-efficient-page-skipping-to-parquet-cpp&data=02%7C01%7CARL122%40pitt.edu%7C81d483c7190248e9b6d908d7a9f35550%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637164734890368849&sdata=uN6KpqxuoRrTuhoysKHkN8N9XVF8dMQTa2BfBupVCpE%3D&reserved=0 > > On Tue, Feb 4, 2020 at 5:41 PM Wes McKinney <wesmck...@gmail.com> wrote: > > > > hi Arun, > > > > I took a brief look at your branch. One thing that is missing is the > > proposed public APIs that use the index pages -- that would be very > > helpful for this discussion. > > > > I don't think we have any code for doing random access of a particular > > data page in a column chunk, so having as an initial matter would also > > be helpful. > > > > - Wes > > > > On Tue, Feb 4, 2020 at 2:28 PM Lekshmi Narayanan, Arun Balajiee > > <arl...@pitt.edu> wrote: > > > > > > Hi Parquet dev > > > > > > Deepak Majeti was my dev lead during my summer internship, from when I am > > > trying to add a few changes in the Arrow Parquet Project for the ticket > > > below > > > > > > https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FPARQUET-1404&data=02%7C01%7CARL122%40pitt.edu%7C81d483c7190248e9b6d908d7a9f35550%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637164734890368849&sdata=6ae98Gu1roe4pGw5moc8D4nwdKNNJ4HC058Ktdo8%2F8I%3D&reserved=0 > > > (Assigned to Deepak) > > > > > > With this regard, I am making a few changes to src/parquet/file_reader.cc > > > ( in a fork on my repository) > > > > > > https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fa2un%2Farrow%2Ftree%2FPARQUET-1404-Add-index-pages-to-the-format-to-support-efficient-page-skipping-to-parquet-cpp%2Fcpp&data=02%7C01%7CARL122%40pitt.edu%7C81d483c7190248e9b6d908d7a9f35550%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637164734890378845&sdata=gefWxwn8DMq7LnCLQZLpWmml%2FeNcy2XvDR2iL%2BfteKw%3D&reserved=0 > > > > > > I am stuck at trying to read a particular row using the index that I get > > > in the page_location array struct of offset index. Could you help me with > > > this ? and if there have been discussions on the forums for this as well, > > > could you direct me to that link? > > > > > > Regards, > > > Arun Balajiee > > > >
