Re: [VOTE] Update the table statistics (puffin stats) spec

2025-07-28 Thread Piotr Findeisen
+1 thanks for correcting this On Tue, 29 Jul 2025 at 01:15, Ryan Blue wrote: > +1 thanks for looking into this. > > On Mon, Jul 28, 2025 at 3:39 PM Amogh Jahagirdar <2am...@gmail.com> wrote: > >> +1 to fixing this to be a long >> >> On Mon, Jul 28, 2025 at 4:38 PM Kevin Liu wrote: >> >>> +1 Gre

Re: [DISCUSS] Questionable usage of snapshot summaries and snapshot references?

2025-07-28 Thread fqossify
Hello Andrew, I think you'll find the discussion in this GitHub issue [1] very relevant to your problem. In fact, this problem has come up a few times in the Iceberg community and I think now would be a good time to revisit this discussion. > It may be useful in similar situations to update the t

Re: [VOTE] Update the table statistics (puffin stats) spec

2025-07-28 Thread Ryan Blue
+1 thanks for looking into this. On Mon, Jul 28, 2025 at 3:39 PM Amogh Jahagirdar <2am...@gmail.com> wrote: > +1 to fixing this to be a long > > On Mon, Jul 28, 2025 at 4:38 PM Kevin Liu wrote: > >> +1 Great catch. I also did a search for "snapshot-id" in >> https://iceberg.apache.org/spec/ ever

Re: [VOTE] Update the table statistics (puffin stats) spec

2025-07-28 Thread Amogh Jahagirdar
+1 to fixing this to be a long On Mon, Jul 28, 2025 at 4:38 PM Kevin Liu wrote: > +1 Great catch. I also did a search for "snapshot-id" in > https://iceberg.apache.org/spec/ every other reference is `long` :) > > Best, > Kevin Liu > > On Mon, Jul 28, 2025 at 3:05 PM Daniel Weeks wrote: > >> +1,

Re: [VOTE] Update the table statistics (puffin stats) spec

2025-07-28 Thread Kevin Liu
+1 Great catch. I also did a search for "snapshot-id" in https://iceberg.apache.org/spec/ every other reference is `long` :) Best, Kevin Liu On Mon, Jul 28, 2025 at 3:05 PM Daniel Weeks wrote: > +1, no objection from me > > On Mon, Jul 28, 2025 at 11:39 AM Russell Spitzer < > russell.spit...@gm

Re: [VOTE] Update the table statistics (puffin stats) spec

2025-07-28 Thread Daniel Weeks
+1, no objection from me On Mon, Jul 28, 2025 at 11:39 AM Russell Spitzer wrote: > I'm generally a +1 here since any implementation > not using a long would have hit a bug a long time ago when > interacting with any of the major engines > > I do want to make sure we let this vote go for at leas

Re: [EXTERNAL] Re: [DISCUSS] Iceberg Summit NA 2026 and Iceberg Summit EU 2026 ?

2025-07-28 Thread Roy Hasson
Running both shows in a single year requires a lot of effort and coordination but I think it's the biggest bang for the buck and between the large corp sponsors we can make it happen. Alternating leaves one group out every year and FOMO results in unnecessary travel, which is what we're trying t

Re: [DISCUSS] Ambiguity over 'total-position-deletes' as V2 (Legacy) or V3 (Deletion Vectors) in Scan Planning

2025-07-28 Thread Scott Cowell
For me it's a question of whether engines should be expected to use some of these higher order abstractions in the core library, and the extent to which the core library exposes lower level building blocks for those engines choosing to go in a different direction. In Dremio, while we started with

Re: [EXTERNAL] Re: [DISCUSS] Iceberg Summit NA 2026 and Iceberg Summit EU 2026 ?

2025-07-28 Thread Alex Stephen
+1 to both events (NA + EU) Even with virtual events, time zones are a large barrier for attendees. Having both events or alternating is the best way to accommodate vastly different time zones. On Mon, Jul 28, 2025 at 12:31 PM Russell Spitzer wrote: > I would love to have an event that's easier

Re: [EXTERNAL] Re: [DISCUSS] Iceberg Summit NA 2026 and Iceberg Summit EU 2026 ?

2025-07-28 Thread Russell Spitzer
I would love to have an event that's easier for non-us'ian members of the community to get to! I wouldn't mind just alternating NA and EU (and Asia?) but if we can find enough sponsorship and content I wouldn't mind multiple events. On Mon, Jul 28, 2025 at 1:57 PM Kevin Liu wrote: > +1 for both

Re: [EXTERNAL] Re: [DISCUSS] Iceberg Summit NA 2026 and Iceberg Summit EU 2026 ?

2025-07-28 Thread Kevin Liu
+1 for both NA and EU. I would love to attend both so I'd prefer they be on different days. Also +1 for hybrid. We should definitely keep the virtual days, they were helpful for the overall community to participate. I'm happy to be the MC again as needed :) Best, Kevin Liu On Mon, Jul 28, 2025 a

Re: [VOTE] Update the table statistics (puffin stats) spec

2025-07-28 Thread Russell Spitzer
I'm generally a +1 here since any implementation not using a long would have hit a bug a long time ago when interacting with any of the major engines I do want to make sure we let this vote go for at least a few more days to tease out any users with strong opinions. Unless we see a major implemen

Re: [EXTERNAL] Re: [DISCUSS] Iceberg Summit NA 2026 and Iceberg Summit EU 2026 ?

2025-07-28 Thread Roy Hasson
+2 for me (NA and EU) :) I, and Microsoft, are very much in favor of having two summits to accommodate Iceberg's large and growing international community. Roy I would love to participate in an Iceberg summit, and not have to fly long hours to get there. So +1

Re: [VOTE] Release Apache Iceberg Rust 0.6.0 RC1

2025-07-28 Thread Kevin Liu
Thanks for verifying and voting, Steven. We now have 3 binding votes 🚀 Kurtis has opened a PR to improve the docs around docker and podman https://github.com/apache/iceberg-rust/pull/1553 I'll also try to reproduce the issue with podman locally. Best, Kevin Liu On Mon, Jul 28, 2025 at 10:59 AM S

Updated invitation: Iceberg UDF sync @ Mon 2025-07-28 9am - 10am (PDT) (dev@iceberg.apache.org)

2025-07-28 Thread Yufei Gu
BEGIN:VCALENDAR PRODID:-//Google Inc//Google Calendar 70.9054//EN VERSION:2.0 CALSCALE:GREGORIAN METHOD:REQUEST BEGIN:VTIMEZONE TZID:America/Los_Angeles X-LIC-LOCATION:America/Los_Angeles BEGIN:DAYLIGHT TZOFFSETFROM:-0800 TZOFFSETTO:-0700 TZNAME:PDT DTSTART:19700308T02 RRULE:FREQ=YEARLY;BYMONTH

Re: [VOTE] Release Apache Iceberg Rust 0.6.0 RC1

2025-07-28 Thread Steven Wu
+1 (binding) Verified checksum, signature. Ran build and unit test on Mac OS (arm64). I wouldn't run the full test related to my container env setup. Tried podman (instead of docker desktop) per doc. Got the same failures. On Mon, Jul 28, 2025 at 10:21 AM Kevin Liu wrote: > Thanks for verifyi

Re: [VOTE] Update the table statistics (puffin stats) spec

2025-07-28 Thread Steven Wu
+1 for fixing the mistake in spec On Mon, Jul 28, 2025 at 10:41 AM Steve wrote: > +1 for using long type for snapshotId > > On Mon, Jul 28, 2025 at 6:24 AM Péter Váry > wrote: > >> +1 for long >> >> Given that it is implemented as a long in every known implementation, we >> might not even want

Re: [VOTE] Update the table statistics (puffin stats) spec

2025-07-28 Thread Steve
+1 for using long type for snapshotId On Mon, Jul 28, 2025 at 6:24 AM Péter Váry wrote: > +1 for long > > Given that it is implemented as a long in every known implementation, we > might not even want to handle the type difference in code > > Eduard Tudenhöfner ezt írta (időpont: 2025. > júl. 2

Re: [VOTE] Release Apache Iceberg Rust 0.6.0 RC1

2025-07-28 Thread Kevin Liu
Thanks for verifying the release, Kurtis! > Question, I noticed a naming difference between Github link and dist.apache.org link. Is there any concern this will affect the release? Yes this is intentional. The Apache link is under the `dev/` folder, https://dist.apache.org/repos/dist/dev/iceberg/

Re: [VOTE] Release Apache Iceberg Rust 0.6.0 RC1

2025-07-28 Thread Alex Stephen
+1 (non-binding) [x] Download links are valid. [x] Checksums and signatures. [x] LICENSE/NOTICE files exist [x] No unexpected binary files [x] Can compile from source Ran `make build && make test` on Mac (arm64) with Colima for integration tests. On Mon, Jul 28, 2025 at 7:12 AM Kurtis Wright wr

Re: [VOTE] Release Apache Iceberg Rust 0.6.0 RC1

2025-07-28 Thread Kurtis Wright
+1 (non-binding) [x] Download links are valid. [x] Checksums and signatures. [x] LICENSE/NOTICE files exist [x] No unexpected binary files [x] Can compile from source Ran `make build && make test` from both dist.apache.org and Github source links: - on Mac with M1 (arm64) CPU using Orbstack for

Re: [DISCUSS] Ambiguity over 'total-position-deletes' as V2 (Legacy) or V3 (Deletion Vectors) in Scan Planning

2025-07-28 Thread Anton Okolnychyi
There is no total number of DVs just like there is no total number of equality delete files or the total number of position delete files. Those types of snapshot metrics simply weren't tracked so we didn't provide an equivalent one for DVs when DVs were added. If we feel there is value in tracking

Re: [VOTE] Update the table statistics (puffin stats) spec

2025-07-28 Thread Péter Váry
+1 for long Given that it is implemented as a long in every known implementation, we might not even want to handle the type difference in code Eduard Tudenhöfner ezt írta (időpont: 2025. júl. 28., H, 12:47): > I agree that this should have been a long in the spec, so +1 to fixing the > spec. I

Re: [DISCUSS] Iceberg Summit NA 2026 and Iceberg Summit EU 2026 ?

2025-07-28 Thread Sung Yun
Hi JB, thanks for bringing this up. I think it's a great idea to start discussing the conferences early on as many organizations go through an scheduled process of approving and selecting OSS conferences to sponsor. I'm +1 to this idea as well, as it'll only help make the Iceberg Summit physica

Re: [DISCUSS] Iceberg Summit NA 2026 and Iceberg Summit EU 2026 ?

2025-07-28 Thread Péter Váry
I would love to participate in an Iceberg summit, and not have to fly long hours to get there. So +1 from me Gábor Kaszab ezt írta (időpont: 2025. júl. 28., H, 15:17): > Great to see there is interest for both NA and EU! > > +1 for both locations from me. > > Best Regards, > Gabor Kaszab > > Edu

Re: [DISCUSS] v4 - Improved column statistics

2025-07-28 Thread Péter Váry
If we focus strictly on file-level column statistics, then partition level column statistics is not a concern. However, looking ahead, we likely want to support column statistics at the table or partition level as well. It would be beneficial to adopt a consistent approach to ID generation and hand

Re: [DISCUSS] Iceberg Summit NA 2026 and Iceberg Summit EU 2026 ?

2025-07-28 Thread Gábor Kaszab
Great to see there is interest for both NA and EU! +1 for both locations from me. Best Regards, Gabor Kaszab Eduard Tudenhöfner ezt írta (időpont: 2025. júl. 28., H, 10:07): > +1 to having an EU summit > > On Mon, Jul 28, 2025 at 9:57 AM Claude Warren, Jr > wrote: > >> +1 >> >> I would be in

Re: [VOTE] Update the table statistics (puffin stats) spec

2025-07-28 Thread Eduard Tudenhöfner
I agree that this should have been a long in the spec, so +1 to fixing the spec. I checked and Trino also implements this as a long. On Mon, Jul 28, 2025 at 12:39 PM Ajantha Bhat wrote: > Hi everyone, > One of the users has raised a PR to update the table statistics (puffin > stats) spec. > http

[VOTE] Update the table statistics (puffin stats) spec

2025-07-28 Thread Ajantha Bhat
Hi everyone, One of the users has raised a PR to update the table statistics (puffin stats) spec. https://github.com/apache/iceberg/pull/13513 I have suggested a mailing list voting thread and also tagged the original spec author. Since there was no response from them for a long time, I am taking

Re: [DISCUSS] Limitation of storing UnknownType in MapType

2025-07-28 Thread Bart Samwel
On Sat, Jul 26, 2025 at 6:09 PM Kevin Liu wrote: > > My initial idea was to disallow the use of UnknownType as the element > in ListType and not allow the UnknownType as either a Key or Value of a > MapType. Any thoughts or concerns? > > That makes sense. I would also include `StructType` here to

Re: [DISCUSS] Iceberg Summit NA 2026 and Iceberg Summit EU 2026 ?

2025-07-28 Thread Eduard Tudenhöfner
+1 to having an EU summit On Mon, Jul 28, 2025 at 9:57 AM Claude Warren, Jr wrote: > +1 > > I would be inclined to attend an EU summit but not a NA one. > > I was chair of the ASF Community over Code EU 2024 and am willing to help > organize, though I do not have time to act as chair. > > Claud

Re: [DISCUSS] Iceberg Summit NA 2026 and Iceberg Summit EU 2026 ?

2025-07-28 Thread Claude Warren, Jr
+1 I would be inclined to attend an EU summit but not a NA one. I was chair of the ASF Community over Code EU 2024 and am willing to help organize, though I do not have time to act as chair. Claude On Mon, Jul 28, 2025 at 6:48 AM Viktor Kessler wrote: > Hi everyone, > > +1 to JB’s proposal —