Re: Discrepancy in parquet format documentation

Hi All - Sorry I missed this email chain. I've been mostly responsible for building the infrastructure around the new parquet-site website, but have mostly left the existing content alone. I'm happy to just link to the parquet-format repo, but that would mean the content is no longer searchable fro

Re: Discrepancy in parquet format documentation

t; version of the format they are looking at. Therefore linking to the > format repo (and maybe add different versions as well) sounds much > better to me. > > Best, > Gang > > On Tue, Mar 5, 2024 at 3:18 AM Vinoo Ganesh > wrote: > > > Hi All - Sorry I missed t

Delete branch on parquet-mr

we're officially good to go. Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com

Re: Delete branch on parquet-mr

This is done - thanks for deleting, Gang! On Mon, Mar 4, 2024 at 11:09 PM Vinoo Ganesh wrote: > Hi Xinli / Gang, > As the last part of fixing > https://issues.apache.org/jira/browse/PARQUET-2442, can one of you delete > this branch https://github.com/apache/parquet-mr/tree/gh

parquet-format status

Hi Parquet Dev - There have been some conversations about content stored on the parquet-format github repo vs. the website. Doing a cursory pass of the parquet-format repo, it looks like, other than the markdown documentation stored in the repo, most of t

Re: parquet-format status

ld me the code may still be used by > legacy projects. So it would not be easy to do such a move. > > Best, > Gang > > On Wed, Mar 6, 2024 at 10:31 AM Vinoo Ganesh > wrote: > >> Hi Parquet Dev - >> >> There have been some conversations about content stored on t

Re: parquet-format status

building procedure isn't properly > > setup to deal with it. > > > > ISTM the "right" solution would be for the Parquet website to > > automatically update its contents based on the latest released version > > of parquet-format. Perhaps using a git

Re: parquet-format status

the docs. I don't know > how could we represent the thrift file properly for the site. > > Cheers, > Gabor > > Vinoo Ganesh ezt írta (időpont: 2024. márc. 7., > Cs, 14:05): > > > Hi Antoine - Perhaps my thoughts weren't clear - but I'm mostly pointing >

Removal of deprecated code in parquet-format

Hi All - As discussed at the Parquet Meeting on March 26 2024, I'm starting an email chain to discuss the removal of the deprecated code ( https://github.com/apache/parquet-format/tree/master/src/main/java/org/apache/parquet/format) in the parquet-format repo. Gábor marked the code as deprecated ~6

Re: Removal of deprecated code in parquet-format

parquet-format at [1]. It > seems > the risk is low for the removal. > > [1] > https://mvnrepository.com/artifact/org.apache.parquet/parquet-format/usages > > Best, > Gang > > On Thu, Mar 28, 2024 at 7:12 AM Vinoo Ganesh > wrote: > > > Hi All - As discus

Re: INFO :: which version of Parquet jar supports Parquet V2 encoding

Hi Prem - Maybe I can help clarify to the best of my knowledge. Parquet V2 as a standard isn't finalized just yet. Meaning there is no formal, *finalized* "contract" that specifies what it means to write data in the V2 version. The discussions/conversations about what the final V2 standard may be a

Re: INFO :: which version of Parquet jar supports Parquet V2 encoding

am, > Do you have any clue in which version of parquet-mr jar Parquet V2 > encoding code is available ? > > On Sun, Apr 21, 2024 at 6:21 PM Prem Sahoo wrote: > >> Thanks Vinoo for the valuable information . >> >> On Sat, Apr 20, 2024 at 5:07 PM Vinoo Ganesh &g

Re: INFO :: which version of Parquet jar supports Parquet V2 encoding

not official yet :( > Sent from my iPhone > > On Apr 22, 2024, at 8:12 AM, Prem Sahoo wrote: > > Thank you Vinoo , will check internally do we really need this atm with > so many caution. > Sent from my iPhone > > On Apr 21, 2024, at 9:24 PM, Vinoo Ganesh wrote: > &g

Re: INFO :: which version of Parquet jar supports Parquet V2 encoding

Hi Prem, Wes' comment on the thread you posted on the arrow dev list should clear up your confusion: https://lists.apache.org/thread/72qwr66wf3xyrl5cozgojz88ct23qzxx. There is a difference between the "standard" itself (parquet-format) and the implementation (parquet-mr, etc...). Parquet-format (h

Re: INFO :: which version of Parquet jar supports Parquet V2 encoding

hare a link where it says Parquet V2 is not official or > not stable for use by third parties ? > > > On Wed, Apr 24, 2024 at 11:28 AM Vinoo Ganesh > wrote: > >> Hi Prem, Wes' comment on the thread you posted on the arrow dev list >> should clear up your confusi

Re: INFO :: which version of Parquet jar supports Parquet V2 encoding

ed to have the related native libraries installed for some > codecs. > > Cheers, > Gabor > > Prem Sahoo ezt írta (időpont: 2024. ápr. 24., Sze, > 20:10): > > > Hello Vinoo, > > Thanks for your assistance . Pyarrow folks are using Parquet V2 though it > > is no

Re: [VOTE] Release Apache Parquet 1.14.0 RC0

+1 (non-binding) Bumped to 1.14.0-SNAPSHOT in Spark and ran a few tests too On Tue, Apr 30, 2024 at 10:20 AM Xinli shang wrote: > +1 (binding) > > Validated the KEY > > On Tue, Apr 30, 2024 at 1:18 AM Gang Wu wrote: > > > Thank you! > > > > On Tue, Apr 30, 2024 at 4:16 PM Gábor Szádovszky

Re: Parquet feature matrix

I’d love to have this on the website. Thanks, Ed! > On May 6, 2024, at 19:27, Ed Seidl wrote: > > Hi all, > Given the recent confusion on this list concerning Parquet V1 vs V2, I was > wondering if there was any interest in the community to create a feature > matrix that users could consult t

Re: [DISCUSS] Propose changing the default branch of the parquet-site repo

+1, this would be great. It's something Xinli and I discussed when we first made the website updates, but it ended up falling off of the list. It would be great to have this updated. On Sat, May 11, 2024 at 8:52 PM Andrew Lamb wrote: > Hello, > > I would like to propose changing the default b

Re: [ANNOUNCE] New Parquet PMC Member: Gang Wu

Congrats, Gang!! On Sat, May 11, 2024 at 8:45 PM Claire McGinty wrote: > Congrats Gang!! Well deserved! > > - Claire > > On Sat, May 11, 2024 at 6:22 PM Fokko Driesprong wrote: > > > Congrats Gang, well deserved! > > > > Op za 11 mei 2024 om 20:00 schreef Jason Z > > > > > Congrats Gang! >

Re: Interest in Parquet V3

I don't have strong feelings about this one way or the other, but would gladly put my hand up to help collaborate on proposals/implementation as we figure this out. On Sun, May 12, 2024 at 5:31 AM Andrew Lamb wrote: > My opinion is that most (if not all) of the proposed benefits from these >

Re: Updates to Apache Parquet Twitter account

We looked into this about a year ago and I think @Julien Le Dem may be the person with access to the Parquet twitter. On Mon, May 13, 2024 at 4:44 PM Bryce Mecum wrote: > Hi all, > > Andrew Lamb's recent ticket [1] made me take a look at the > @ApacheParquet [2] Twitter account and I noticed

Re: [DISCUSS] rename parquet-mr to parquet-java?

+1, I think this will make things a lot clearer! (non-binding) On Wed, May 15, 2024 at 12:36 PM Jacques Nadeau wrote: > +1000 > > On Wed, May 15, 2024 at 6:30 AM Andrew Lamb > wrote: > > > Julien had a great suggestion[1] to rename the parquet-mr repository to > > parquet-java to reduce con

Re: [Parquet-java] Are there release instructions documented any place?

Theoretically, these are them: https://parquet.apache.org/docs/contribution-guidelines/releasing/, but https://lists.apache.org/thread/5oohcx3m16kqs8dmtl3vm1cgd8z0q10b was recently raised to discuss how to better improve it. On Fri, May 24, 2024 at 2:23 PM Micah Kornfield wrote: > I see to ha

Re: Congrats to Julien Le Dem for being next PMC Chair

Congrats, Julien! On Tue, Jul 2, 2024 at 9:01 PM wish maple wrote: > Congrats Julien > > Best, > Xuwei Fu > > Micah Kornfield 于2024年7月3日周三 08:58写道： > > > Congrats Julien > > > > On Tuesday, July 2, 2024, Andrew Lamb wrote: > > > > > Congratulations Julien! > > > > > > On Tue, Jul 2, 2024,

Re: [VOTE] Adopt proposal on new features for parquet-format and release for Parquet Java

+1 (non-binding) Thank you Micah for all of your work on this! On Thu, Jul 4, 2024 at 5:28 AM Andrew Lamb wrote: > +1 (non binding) > > Thank you Micah for all the effort you have put into gathering feedback and > building consensus > > Andrew > > On Thu, Jul 4, 2024 at 2:48 AM Alkis Evlogi

Re: [DISCUSS] Parquet sync day and time

8 am - 10 am PT works for me, as does something a bit later EST (I also have the same challenges Andrew does) On Tue, Jul 9, 2024 at 7:54 AM Andrew Lamb wrote: > Thank you for bringing this up. My company is spread through the US (East > and West) and Europe, so the current 9AM PT/Noon ET slot

Parquet-tools Replacement

just found this ticket: https://issues.apache.org/jira/browse/PARQUET-1666 too. Is there a recommended replacement for parquet-tools? If so, could someone point me to it? Thanks! Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com

Re: Parquet-tools Replacement

Hi Xinli, Great - thank you! Just to make sure, you mean this right? https://github.com/apache/parquet-mr/tree/master/parquet-cli ( https://mvnrepository.com/artifact/org.apache.parquet/parquet-cli). Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Tue, Jan 4, 2022 at 12:49 PM Xinli shang

Re: Parquet-tools Replacement

Sounds great, thank you! Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Tue, Jan 4, 2022 at 9:07 PM Xinli shang wrote: > That is correct! > > On Tue, Jan 4, 2022 at 12:29 PM Vinoo Ganesh > wrote: > > > Hi Xinli, > > Great - thank you! Just to make sure, you

Re: Get uncompressed size of parquet file via parquet-cli

Ironically, I've needed this and added it recently on my fork of my parquet. Happy to contribute it back: https://issues.apache.org/jira/browse/PARQUET-2129 https://github.com/apache/parquet-mr/pull/949 Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Sun, Feb 20, 2022 at 1:18 PM Xinli

Re: Parquet Website Questions

Sounds great, thanks Martin! Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Wed, Mar 2, 2022 at 2:42 AM Martin Grigorov wrote: > Hi Vinoo, > > According to https://infra-reports.apache.org/site-source/ Parquet's site > is currently hosted at > https://gitbox.apache.o

[Request for Feedback] New Parquet Website

love any feedback you may have. Thanks! Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com

Parquet Website Launched

roduction#website-development-and-deployment . Thanks to Xinli for his help getting this over the finish line. Please let me know if you have any feedback or feature requests. Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com

Re: Parquet Website Launched

s on migrating to this? Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Fri, Mar 25, 2022 at 1:48 PM Antoine Pitrou wrote: > > Hello, > > Just for the record, I find the introductory sentence a bit weird: > """ > Apache Parquet is a columnar storage format

Re: Parquet Website Launched

Sounds great, I'll PR a change in. Thanks for the feedback! Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Fri, Mar 25, 2022 at 2:08 PM Antoine Pitrou wrote: > On Fri, 25 Mar 2022 14:00:11 -0400 > Vinoo Ganesh > wrote: > > Hi Antoine, > > Thanks for the fee

Re: Parquet Website Launched

Change here: https://github.com/apache/parquet-site/pull/21 Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Sat, Mar 26, 2022 at 2:31 PM Vinoo Ganesh wrote: > Sounds great, I'll PR a change in. Thanks for the feedback! > > Thanks, > Vinoo Ganesh | vinoo.gan...@gmail.com

Re: Cannot find doap file: http://parquet.apache.org/doap_Parquet.rdf

I have restored and updated the file in this PR: https://github.com/apache/parquet-site/pull/22. Once it merges, it will appear again. Thank you for flagging. Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Sun, Mar 27, 2022 at 7:44 AM sebb wrote: > Please restore the file or update

Re: Parquet Website Launched

earch may look, you can look on the upper right hand side of the staged website: https://parquet.staged.apache.org/. Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Mon, Mar 28, 2022 at 6:22 AM Maya Anderson wrote: > Hi Vinoo, > > Is there an option to search inside the documentati

Re: Parquet Website Launched

Thank you for the kind words, Fokko! Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Mon, Mar 28, 2022 at 9:25 AM Driesprong, Fokko wrote: > Hey Vinoo, > > Thanks for sharing. The new website looks absolutely amazing! > > Kind regards, Fokko > > Op ma 28 mrt. 2022 o

Re: Cannot find doap file: http://parquet.apache.org/doap_Parquet.rdf

The PR has merged and the file has been restored. Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Sun, Mar 27, 2022 at 7:14 AM Vinoo Ganesh wrote: > I have restored and updated the file in this PR: > https://github.com/apache/parquet-site/pull/22. Once it merges, it will > app

Re: Parquet Website Launched

Hi Maya, The search functionality is live now: https://parquet.apache.org/. Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Mon, Mar 28, 2022 at 8:08 AM Vinoo Ganesh wrote: > Hi Maya, > Thanks for the feedback. A search feature will be released soon. We're > waiting

Re: join mailing list

Good point, Aaron. I'll add this onto the website. Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Tue, Jul 19, 2022 at 8:58 AM Aaron Niskode-Dossett wrote: > Hi Sol, > > Welcome! You can send an email to "dev-subscr...@parquet.apache.org" to > start that proc

Re: Is there a parquet users list?

Hi Sol, There isn't a users list for parquet. You can ask questions here on the dev list. I'll update the website to make this more clear. Thanks, Vinoo Ganesh | vinoo.gan...@gmail.com On Tue, Jul 19, 2022 at 12:10 PM Sol Lederman wrote: > Sorry to spam the dev list. I

[jira] [Created] (PARQUET-2096) Upgrade Thrift to 0.15.0

Vinoo Ganesh created PARQUET-2096: - Summary: Upgrade Thrift to 0.15.0 Key: PARQUET-2096 URL: https://issues.apache.org/jira/browse/PARQUET-2096 Project: Parquet Issue Type: Improvement

[jira] [Created] (PARQUET-2128) Bump Thrift to 0.16.0

Vinoo Ganesh created PARQUET-2128: - Summary: Bump Thrift to 0.16.0 Key: PARQUET-2128 URL: https://issues.apache.org/jira/browse/PARQUET-2128 Project: Parquet Issue Type: Improvement

[jira] [Created] (PARQUET-2129) Add uncompressedSize to "meta" output

Vinoo Ganesh created PARQUET-2129: - Summary: Add uncompressedSize to "meta" output Key: PARQUET-2129 URL: https://issues.apache.org/jira/browse/PARQUET-2129 Project: Parquet

[jira] [Resolved] (PARQUET-2128) Bump Thrift to 0.16.0

[ https://issues.apache.org/jira/browse/PARQUET-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoo Ganesh resolved PARQUET-2128. --- Resolution: Fixed Fixed in https://github.com/apache/parquet-mr/pull/948 > Bump Thrift

[jira] [Commented] (PARQUET-2129) Add uncompressedSize to "meta" output

[ https://issues.apache.org/jira/browse/PARQUET-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17503206#comment-17503206 ] Vinoo Ganesh commented on PARQUET-2129: --- Fixed in: https://github.com/ap

[jira] [Resolved] (PARQUET-2129) Add uncompressedSize to "meta" output

[ https://issues.apache.org/jira/browse/PARQUET-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoo Ganesh resolved PARQUET-2129. --- Resolution: Fixed https://github.com/apache/parquet-mr/pull/949 > Add uncompressedSize

[jira] [Updated] (PARQUET-2140) parquet-cli unable to read UUID values

[ https://issues.apache.org/jira/browse/PARQUET-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoo Ganesh updated PARQUET-2140: -- Description: I am finding that parquet-cli throws when trying to read UUID values