My proposal definitely has some novelty that I could have explained better.
Part of my hope here is to understand if there's any interest from the Iceberg community in shipping a set of SQL functions as part of the Iceberg codebase. We have some precedent with the partition transform functions, but this is definitely more complex than those. If there's interest in that, there's a number of shapes that these functions could take: - Built into the Iceberg Spark / Flink connectors - Creating these as "default" UDFs that engines could then implement as part of the UDF spec I definitely understand the novelty here and that this may not be a perfect fit for the project. Just curious to see what everyone thinks On Mon, Nov 9, 2026 at 7:36 AM Russell Spitzer <[email protected]> wrote: > I'm not sure I understand the goal here. Is the goal to create a common > vocabulary between Spark and Flink? I'm a little hesitant just because i'm > not sure what the use case is here because you would most likely have a > number of engine specific things in each API outside of function names and > argos. > > On Mon, Nov 2, 2026 at 2:45 PM Alex Stephen via dev < > [email protected]> wrote: > >> Hello all, >> >> As part of our ongoing work with the FILE + VECTOR types, I'd like to >> propose adding a generative AI function + AI embedding function to the >> Iceberg Spark / Flink connectors. This would allow users to use a >> vendor-agnostic set of functions for interacting with their multimodal >> Iceberg data and creating AI embeddings. There's a wide variety of prior >> work across both the OSS and proprietary landscapes. >> >> Please take a look at the proposal document below. Thanks a lot! >> >> Proposal document: https://s.apache.org/iceberg-ai-functions >> >> -- Alex Stephen >> >
