> On Oct. 23, 2014, 9:50 p.m., Jason Dere wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java, line 465 > > <https://reviews.apache.org/r/26854/diff/1-3/?file=723909#file723909line465> > > > > There is no longer a way to query the metastore for UDFs apart from the > > static initialization. So if one CLI user creates a permanent UDF, another > > user on CLI, or HS2, will not be able to use that new UDF if the 2nd CLI or > > HS2 was initialized before this UDF was created. > > Navis Ryu wrote: > Permanent functions (persistent function seemed better name, imho) are > registered to system registry, which is shared to all clients. So if one user > creates new permanent function, it's shared to all clients. The time a user > accesses the function, the class is loaded with required resources and > registered to session registry as a temporary function. > > Jason Dere wrote: > So this would work if all clients are using hiveserver2, because all > clients in this scenario would share the same system registry. > But if one or more clients are using the Hive CLI, any persistent UDFs > created/dropped by this CLI client would not be reflected in the other > clients (or HS2), since it's a different process/system registry. > > Navis Ryu wrote: > I've missed this message. Yes, it will act like you've commented. But is > it a common case to expect things done in separate VM refleceted to others? > Should dropping presistent function by a jdbc client be reflected to other > CLI clients?
Would be good to have if it's possible. Certainly if one client creates a table, we would want the other client to be able to see it. Being able to have the same behavior with functions seems consistent. I suppose it is a bit harder for this to work in the case of drop function, unless we are looking up the UDF in the metastore each time it is used in a query. - Jason ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26854/#review57952 ----------------------------------------------------------- On Oct. 30, 2014, 11:41 p.m., Jason Dere wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/26854/ > ----------------------------------------------------------- > > (Updated Oct. 30, 2014, 11:41 p.m.) > > > Review request for hive, Navis Ryu and Thejas Nair. > > > Bugs: HIVE-2573 > https://issues.apache.org/jira/browse/HIVE-2573 > > > Repository: hive-git > > > Description > ------- > > Small updates to Navis' changes: > - session registry doesn't lookup metastore for UDFs > - my feedback from Navis' original patch > - metastore udfs should not be considered native. This allows them to be > added/removed from registry > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/common/JavaUtils.java 9aa917c > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7d8e5bc > contrib/src/test/results/clientnegative/invalid_row_sequence.q.out 8f3c0b3 > > itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestJdbcWithMiniKdc.java > 6647ce5 > > metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java > 88b0791 > ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 9ac540e > ql/src/java/org/apache/hadoop/hive/ql/exec/CommonFunctionInfo.java 93c15c0 > ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionInfo.java 074255b > ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e43a328 > ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionTask.java 569c125 > ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 5bdeb92 > ql/src/java/org/apache/hadoop/hive/ql/exec/WindowFunctionInfo.java efecb05 > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 4e3df75 > ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java b900627 > ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java 13277a9 > ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 211ab6c > ql/src/java/org/apache/hadoop/hive/ql/optimizer/IndexUtils.java e2768ff > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/optiq/translator/SqlFunctionConverter.java > 793f117 > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java > 1796b7b > ql/src/java/org/apache/hadoop/hive/ql/parse/FunctionSemanticAnalyzer.java > 22e5b47 > ql/src/java/org/apache/hadoop/hive/ql/parse/IndexUpdater.java 2b239ab > ql/src/java/org/apache/hadoop/hive/ql/session/SessionConf.java PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java af633cb > ql/src/test/org/apache/hadoop/hive/ql/parse/TestMacroSemanticAnalyzer.java > 46f8052 > ql/src/test/queries/clientnegative/drop_native_udf.q ae047bb > ql/src/test/results/clientnegative/create_function_nonexistent_class.q.out > c7405ed > ql/src/test/results/clientnegative/create_function_nonudf_class.q.out > d0dd50a > ql/src/test/results/clientnegative/drop_native_udf.q.out 9f0eaa5 > ql/src/test/results/clientnegative/udf_nonexistent_resource.q.out e184787 > service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java > fd38907 > > Diff: https://reviews.apache.org/r/26854/diff/ > > > Testing > ------- > > > Thanks, > > Jason Dere > >
