I don't agree with the composition argument. The example query you provided doesn't make sense because there is no ON clause so you end up with a cartesian join. Luckily, since you're looking for a specific label, "owner", you end up with just one label and will aggregate the entire set of files. So you end up with a result that looks reasonable, but you're really just running two unrelated queries here: one to aggregate the total size of live files in the table, and one to select the owner.
I think that means that the only use case here is to expose this data to users. But I think that this reasoning is that we need a metadata table because we need SQL interaction and we need SQL interaction because . . . ? It's a nice-to-have, sure, but I'm not convinced that anyone would miss it if we didn't expose this directly to users. On Fri, Sep 25, 2026 at 2:40 PM Andrei Tserakhau via dev < [email protected]> wrote: > Hi Ryan, > > Agree here: the engine-facing consumption (cost attribution, policy > attachment) goes through SupportsLabels, no table needed. The table is for > the other consumer - SQL/people or AI Agent :). Some cases that i see here: > > 1) Exploration. "which columns are classified as X here" is a query a > person runs, not something an engine surfaces: > > SELECT field_name, key, value > FROM prod.db.orders.labels > WHERE scope = 'field' AND key = 'classification'; > > 2) Composition. .labels joins with .files / .partitions / .snapshots in > one query - e.g. attribute bytes to an owner label, i.e. cost attribution > as a report someone runs, not a log an engine emits: > > SELECT l.value AS owner, SUM(f.file_size_in_bytes) AS bytes > FROM prod.db.orders.files f, prod.db.orders.labels l > WHERE l.scope = 'object' AND l.key = 'owner' > GROUP BY l.value; > > I think the key value to have a metadata table is joinability, you can’t > have it with a programmatic label API. > > So there is a place for the SQL consumer, the engine path is a different > usecase and the co-live together. > > Thanks, > Andrei > > > On Fri, Sep 25, 2026 at 10:51 PM Ryan Blue <[email protected]> wrote: > >> > Are we OK with catalog-provided metadata tables as a separate category? >> >> I'm okay with providing metadata through a system table like this, as >> long as we think that people will want to access this data that way. >> Question 3, "If no, what should the SQL surface for labels be instead?" >> makes me think that a SQL surface is _assumed_ to be needed. >> >> I don't think it is necessarily the case that we need to expose these for >> SQL users. I thought that we wanted labels to expose additional context to >> engines for things like cost attribution logs or attaching an engine's >> policy to a table. That doesn't require a table-like user surface. >> >> I'm fine adding a metadata table if there's a use for it, but if we don't >> need one then it's simpler not to add and maintain it. And that avoids >> needing to answer questions like this as well. >> >> >> >> On Fri, Sep 25, 2026 at 1:13 PM Andrei Tserakhau via dev < >> [email protected]> wrote: >> >>> Hi all, >>> >>> Labels in the REST spec recently landed [1]. A catalog can now expose >>> object-level and per-field labels on load-table responses. As one of the >>> follow-ups, there is a proposal to add a .labels metadata table backed >>> by catalog data. >>> >>> During discussion of the follow-ups ([3], [4]), Peter raised a good >>> question on [3]: every metadata table today is derived from table metadata. >>> .labels would be different because the data comes from the catalog, may >>> vary by catalog, and may be absent if the catalog has nothing to return. >>> >>> I think this is less about Labels itself and more about a new *kind* of >>> data we would expose: metadata owned by the catalog rather than by storage. >>> Labels would be the first example, but later the same pattern could be used >>> to expose other catalog information as metadata tables. >>> >>> So the broader question is: do we want metadata tables to also expose >>> catalog-provided information? >>> >>> I think labels are a reasonable first case. They are structured, useful >>> to query via SQL, and fit the same access pattern as .snapshots or >>> .partitions. The table can stay read-only, limited to the spec-defined >>> shape, and empty when the catalog returns nothing. >>> >>> The tradeoff is that this breaks the current assumption that metadata >>> tables are deterministic projections of table metadata. It’s not explicitly >>> written anywhere, but that assumption exists today. >>> >>> To summarize the questions: >>> >>> 1. >>> >>> Are we OK with catalog-provided metadata tables as a separate >>> category? >>> 2. >>> >>> If yes, should we mark them somehow so they are clearly different >>> from spec-backed metadata tables? >>> 3. >>> >>> If no, what should the SQL surface for labels be instead? >>> >>> Thanks, >>> >>> Andrei >>> >>> [1] REST spec labels: https://github.com/apache/iceberg/pull/15750 >>> [2] SupportsLabels: https://github.com/apache/iceberg/pull/18046 >>> [3] Labels metadata table: https://github.com/apache/iceberg/pull/18048 >>> [4] Spark DESCRIBE: https://github.com/apache/iceberg/pull/18049 >>> [5] Design: >>> https://docs.google.com/document/d/1aj-6JlfBiMYEEVtNuh5WLMOrRQiMCcyYUGbouPM4hXI/edit?tab=t.0#heading=h.2w0kmp1v1gwv >>> >>
