Lisoda, HadoopCatalog has many issues for production usage like Dan said. It has never been recommended in production. It was widely used in unit test code, which is also slowly moving toward InMemoryCatalog. As the community is aligned behind the REST catalog, it is preferable to limit the work related hadoop catalog.
On Sun, Jul 14, 2024 at 11:44 PM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Again, it's my "vision": if the community wants to maintain and move > forward on HadoopCatalog, that's fine (not sure it would be a good > idea regarding the "limitations" of filesystem based catalog :)). > > Let's see what the others are thinking. > > Regards > JB > > On Mon, Jul 15, 2024 at 8:29 AM lisoda <lis...@yeah.net> wrote: > > > > Okay. I see...... > > I‘m so sad. :( > > But anyway, thanks for answering all my questions. > > > > > > > > > > > > > > 在 2024-07-15 14:25:16，"Jean-Baptiste Onofré" <j...@nanthrax.net> 写道： > > >Hi > > > > > >HadoopCatalog is not a "recommended" catalog for production (at least > > >up to now). So, we should consider either to move it in a separate > > >repo (if we have the guarantee that it's gonna be maintained, else it > > >doesn't make sense) or remove it to avoid confusion. My take here is > > >the same (for several months :)): we should privilege the REST Catalog > > >API and users should use a REST Catalog server implementation. > > > > > >Regards > > >JB > > > > > >On Mon, Jul 15, 2024 at 8:13 AM lisoda <lis...@yeah.net> wrote: > > >> > > >> Sir. Even if the entire hadoopCatalog can be used without > lockManager, should we delete it? > > >> > > >> > > >> > > >> > > >> > > >> > > >> 在 2024-07-15 14:08:40，"Jean-Baptiste Onofré" <j...@nanthrax.net> 写道： > > >> >Hi > > >> > > > >> >My understanding is that lock manager is mostly used on the > > >> >HadoopCatalog. The other catalogs relays on a third party lock > > >> >mechanism: for instance, JDBC Catalog uses the RDBMS table/row > > >> >locking, REST Catalog uses implementation lock. > > >> >I would rather remove HadoopCatalog and the lock manager in favor of > > >> >the REST catalog and implementation lock mechanism. > > >> > > > >> >Just my $0.01 :) > > >> > > > >> >Regards > > >> >JB > > >> > > > >> >On Fri, Jul 12, 2024 at 7:41 AM lisoda <lis...@yeah.net> wrote: > > >> >> > > >> >> Currently, the only lockManager implementation in iceberg-core is > InMemoryLockManager. This PR extends two LockManager implementations, one > based on the Redis, and another based on the Rest-API. > > >> >> In general, most users use redisLockManager is sufficient to cope > with most of the scenarios, for redis can not meet the user's requirements, > we can let the user to provide a RestApi service to achieve this function. > I believe that, for a long time, these two lock-manager's will satisfy most > of the customer's needs. > > >> >> > > >> >> If someone could review this PR, that would be great. > > >> >> > > >> >> PR: https://github.com/apache/iceberg/pull/10688 > > >> >> SLACK: > https://apache-iceberg.slack.com/archives/C03LG1D563F/p1720761992982729 >
