> I know we have the RESTCatalogAdapter and RESTCatalogSevlet for unit tests, and technically we have a very similar Jetty server implementation in TestRESTCatalog. Should we think about making those components out of the tests into an iceberg-rest-server module for this use case, and merge with the implementation that Gravitino has?
I think that this would take the Iceberg project in the wrong direction. Iceberg has always been a library and I think it should continue to be. Concerns about runtime should be left to other projects that need to fit into existing infrastructure or skillsets of people maintaining them. The question of whether to use Jetty or Tomcat or whatever else is a serious consideration, as is how to monitor that application and send metrics. I think it would slow down the core purpose of Iceberg if we got distracted by these things. In fact, I think that this project shows that the library is getting the balance right: it is using `CatalogHandlers` for their intended purpose. It has opinions about how to run the actual HTTP service and people that agree can use it. Other people could use `CatalogHandlers` to build on a different foundation. On Thu, Jan 25, 2024 at 11:13 AM Jack Ye <yezhao...@gmail.com> wrote: > Really cool project! > > I browsed a bit of the codebase, and see this implementation of the REST > service backend: > - > https://github.com/datastrato/gravitino/blob/main/catalogs/catalog-lakehouse-iceberg/src/main/java/com/datastrato/gravitino/catalog/lakehouse/iceberg/IcebergRESTService.java#L39 > - > https://github.com/datastrato/gravitino/blob/main/catalogs/catalog-lakehouse-iceberg/src/main/java/com/datastrato/gravitino/catalog/lakehouse/iceberg/ops/IcebergTableOps.java#L42-L51 > > Looks like it is initializing a Jetty server that uses CatalogHandlers to > delegate the execution to a specific Java Catalog implementation. > > I think this is actually something that is lacking today in Iceberg, which > is an easy way for users to start an actual REST HTTP server. > > I know we have the RESTCatalogAdapter and RESTCatalogSevlet for unit > tests, and technically we have a very similar Jetty server implementation > in TestRESTCatalog. Should we think about making those components out of > the tests into an iceberg-rest-server module for this use case, and merge > with the implementation that Gravitino has? > > Best, > Jack Ye > > On Thu, Jan 25, 2024 at 10:47 AM Yufei Gu <flyrain...@gmail.com> wrote: > >> Thanks Justin for the sharing. >> >> It's pretty cool to see an open source REST catalog implementation in >> action. Having dabbled a bit in the early development of Gravitino myself, >> I'm really excited about its potential with the Iceberg REST catalog. >> >> The idea of Gravitino moving to an ASF project is promising. It’ll surely >> boost its visibility and open up more doors for collaboration and adoption. >> >> Looking forward to where this goes. Keep up the fantastic work! >> >> Yufei >> >> >> On Thu, Jan 25, 2024 at 5:55 AM Jean-Baptiste Onofré <j...@nanthrax.net> >> wrote: >> >>> Hi Justin, >>> >>> I talked with Junping a couple of months ago about Gravitino. Thanks >>> for sharing ! >>> >>> Regards >>> JB >>> >>> On Thu, Jan 25, 2024 at 12:15 AM Justin Mclean <jus...@classsoftware.com> >>> wrote: >>> > >>> > Hi, >>> > >>> > We open-sourced a new project, Gravitino, in December and have been >>> working on growing the community and adding new functionality. We plan to >>> donate the project to the ASF this year. Gravitino is a unified metadata >>> lake solution offering a unified approach to managing datasets from diverse >>> sources and regions across multiple cloud platforms. Its core is an Iceberg >>> REST catalog service implementation to manage Iceberg tables efficiently. >>> > >>> > If this sounds like something you would be interested in, then the >>> following resources will help: >>> > - Blog post: >>> https://datastrato.ai/blog/gravitino-iceberg-rest-catalog-service/ >>> > - Gravitino documentation: https://datastrato.ai/docs/0.3.1/ >>> > - Iceberg REST service documentation: >>> https://datastrato.ai/docs/0.3.1/iceberg-rest-service >>> > >>> > We welcome any feedback and suggestions you have, and as always, all >>> contributions are welcome. You can find the source code at >>> https://github.com/datastrato/gravitino. >>> > >>> > Kind Regards, >>> > Justin >>> >> -- Ryan Blue Tabular
