[
https://issues.apache.org/jira/browse/HIVE-27473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17936765#comment-17936765
]
Seonggon Namgung commented on HIVE-27473:
-----------------------------------------
Hi, [~okumin]. After studying HIVE-28658 and HIVE-12679 (I particularly
appreciate the document you wrote, which helped me understand the design of
MetaStore related codes), I see that there is still a pending design decision
on how to introduce a pluggable interface for external Catalogs. Although the
discussion on HIVE-28658 is currently leaning toward Option 2 from your
document, I believe that HIVE-12679 + HIVE-27473 is also a viable approach
after a quick survey on HiveMetaStoreClient and its subclasses. Regardless of
the REST catalog implementation, I also see independent improvement points
within HIVE-27473. Below, I summarize my understanding and ideas. I would
appreciate it if you could take a look and share your insights, especially
based on your past experience working on this JIRA.
The circular dependency among SessionHiveMetaStoreClient(SHMSC),
HiveMetaStoreClientWithLocalCache(HMSCwLC), and HiveMetaStoreClient(HMSC) can
be understood in the following structure:
Type 1: Hook handling → Temporary Table → Query-level Cache → HS-level Cache →
Thrift Connection
Type 2: Temporary Table → Hook handling → Query-level Cache → HS-level Cache →
Thrift Connection
The circular dependency arises because SHMSC handles TmpTable and HS-level
cache, while HMSC handles Hook and Thrift. I believe this can be resolved by
separating them into five distinct layers. (Of course, I have not analyzed
every methods in these classes, so this may be infeasible.) Such a separation
would also improve the design of MetaStoreClient by better adhering to the
principle of separation of concerns.
Except for TmpTable, the hierarchy among the layers seems straightforward:
Thrift is the inner-most layer, Caches wrap Thrift, and Hook wraps caches.
However, in the case of TmpTable, I think it should wrap Thrift and Cache, but
the relationship between Hook and TmpTable remains unclear to me. I couldn't
find any documents or comments that declare the relationship, and I noticed
that both Type1 and Type2 dependencies are currently used in Hive. (Type1:
createTable, Type2: getTable) It seems that either implementation is
technically feasible, and I think the relationship can be easily defined once
the community initiates a discussion on it.
In conclusion, regardless of HIVE-12679 and HIVE-28658, I believe that we can
improve Hive within this JIRA by adhering to SoC and defining the relationship
between Hook and TmpTable. However, before proceeding with implementation, I
would like to ask some questions about this issue.
- Would it be possible for me to take on this issue? Since you are the
assignee, I will, of course, wait for your work if you are working on it.
- Do you think the approach I described above is possible? Given your prior
experience with this issue, I would appreciate any insights you can provide,
especially if I have overlooked something.
- Would resolving this issue contribute to HIVE-28658? While I find some
potential contributions within this JIRA, my ultimate goal is to support
resolving HIVE-28658. The community will decide whether to proceed with Option
1 (+HIVE-27473) or Option 2, but I would like to hear your thoughts on whether
addressing this issue could contribute to HIVE-12679 and, in turn, HIVE-28658.
Thank you for reading my lengthy opinion. I look forward to your thoughts.
> Make SessionHiveMetaStoreClient and HiveMetaStoreClientWithLocalCache
> composable
> --------------------------------------------------------------------------------
>
> Key: HIVE-27473
> URL: https://issues.apache.org/jira/browse/HIVE-27473
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Affects Versions: 4.0.0-alpha-2
> Reporter: Shohei Okumiya
> Assignee: Shohei Okumiya
> Priority: Major
>
> Currently, we implement the two classes using inheritance. It assumes we
> always use a single implementation of IMetaStoreClient.
> Some community members have been willing to make IMetaStoreClient pluggable
> as proposed in HIVE-12679. Considering the use case, we should provide the
> additional traits with composition over inheritance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
