RE: Re: Thoughts on Adding a `doc` Property for Schema Objects

2025-08-04 Thread Taeyun Kim
een table-level description and schema-level description, then I think you should consider it standardized. You can store the table description in the "comment" table property. On Sun, Aug 3, 2025 at 5:28 PM Taeyun Kim wrote: Hi, I’ve already explained my reasoning in earlier messag

RE: Re: Thoughts on Adding a `doc` Property for Schema Objects

2025-08-03 Thread Taeyun Kim
) 08:05:55 (UTC+09:00) Subject: Re: Thoughts on Adding a `doc` Property for Schema Objects Why would you need to version table descriptions? Are there cases where they are changing rapidly and inaccurate due to schema changes? On Thu, Jul 24, 2025 at 7:48 PM Taeyun Kim wrote: Thank you for

Re: Thoughts on Adding a `doc` Property for Schema Objects

2025-07-24 Thread Taeyun Kim
se text to each field may bloat the metadata storage, especially when there are a lot of duplicate `doc`s if schema evolution happens a lot. Best, Gang On Fri, Jul 25, 2025 at 9:25 AM Taeyun Kim wrote: Thank you for your response. As I understand it, the table description is currently st

Re: Thoughts on Adding a `doc` Property for Schema Objects

2025-07-24 Thread Taeyun Kim
tore table descriptions. The convention is to use a table property, "comment". While this isn't a schema-level doc/comment, I don't know of anything that makes a distinction between schema description and table description, so I think it should work for your use. On Tue, Jul 22, 2

Thoughts on Adding a `doc` Property for Schema Objects

2025-07-22 Thread Taeyun Kim
Hi, With the growing trend of using LLMs to automatically generate SQL, it feels increasingly important to manage descriptions of database tables and columns in a way that these tools can easily access. In the Iceberg specification, comments for schema fields (i.e., columns) can be specified u

Re: [DISCUSS] REST: Way to query if metadata pointer is the latest

2025-01-07 Thread Taeyun Kim
l take a look. >> >> Regards >> JB >> >> On Thu, Dec 12, 2024 at 2:52 PM Gabor Kaszab wrote: >> > >> > Hi Iceberg Community, >> > >> > It took me a while but I finally managed to upload the proposal for this >> > as an officia

Re: [DISCUSS] REST: Way to query if metadata pointer is the latest

2025-01-05 Thread Taeyun Kim
n official 'Iceberg improvement proposal'. Thanks for the feedback so far! > > https://github.com/apache/iceberg/issues/11766 > > Regards, > Gabor > > > On Fri, Nov 22, 2024 at 4:51 PM Taeyun Kim > wrote: >> >> Hi, >> >> Since ETags are o

Re: [DISCUSS] REST: Way to query if metadata pointer is the latest

2024-11-22 Thread Taeyun Kim
really low activity > on this thread from people with decision making privileges so let's do this: > Let me put together an improvement proposal as advised by Fokko, and let's > hope it will attract people having binding votes so that we can come to a > conclusion on

Re: [DISCUSS] REST: Way to query if metadata pointer is the latest

2024-11-21 Thread Taeyun Kim
itially load the table, e.g. "String > catalogVersion". In the case of the REST Catalog the catalogVersion > string would be the ETag. Other catalogs might not even need to add > anything, as the metadata_location of the Table object is sufficient. > > This way the API woul

Re: [DISCUSS] REST: Way to query if metadata pointer is the latest

2024-11-20 Thread Taeyun Kim
ld throw a NotImplementedException. I hope this makes sense and I haven't missed any details or previous comments. Regards, Gabor On Tue, Nov 19, 2024 at 5:17 AM Taeyun Kim wrote: Hi, Here are my thoughts: - HTTP Layer: To my knowledge, there isn’t a separate "HTTP layer"

Re: [DISCUSS] REST: Way to query if metadata pointer is the latest

2024-11-18 Thread Taeyun Kim
llow other approaches. Regards, Gabor On Mon, Nov 18, 2024 at 8:48 AM Shani Elharrar wrote: You're totally right. Perhaps using a "Content-Location" header might be a better fit for that.  Shani. On 18 Nov 2024, at 9:27, Taeyun Kim wrote:  Hi, Here are my thoughts: -

Re: [DISCUSS] REST: Way to query if metadata pointer is the latest

2024-11-17 Thread Taeyun Kim
be a significant factor.I’d also recommend considering the requirement to retrieve multiple tables(e.g., all tables under a namespace, or a list of table names) from the catalog. This requires a new endpoint and may not work with HTTP caching.Let me know your thoughts or if there’s anything else to cons

Re: [DISCUSS] REST: Way to query if metadata pointer is the latest

2024-11-17 Thread Taeyun Kim
like the idea and it makes sense. As soon as it's clearly stated in the spec (using If-Modified-Since header and 304 status code), it looks good to me. Thanks ! Regards JB On Fri, Nov 15, 2024 at 1:58 AM Taeyun Kim wrote: > > Hi, > > (Apologies if this email is a duplicate. This is

Re: [DISCUSS] REST: Way to query if metadata pointer is the latest

2024-11-14 Thread Taeyun Kim
Hi, (Apologies if this email is a duplicate. This is my third attempt.) I also need a way to ensure that my table data is up-to-date. For now, I’m handling this by setting an expiration period after which I fetch the data again, regardless of its freshness. Here are my thoughts on the current