kgyrtkirk commented on a change in pull request #1876:
URL: https://github.com/apache/hive/pull/1876#discussion_r566284005
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java
##########
@@ -140,8 +141,23 @@ public ObjectInspector initialize(ObjectInspector
rowInspector) throws HiveExcep
childrenOIs[i] = children[i].initialize(rowInspector);
}
MapredContext context = MapredContext.get();
+ // It is possible that there is no context at this point. For example a
context is not created
+ // when "hive.fetch.task.conversion" occurs.
if (context != null) {
context.setup(genericUDF);
+ } else {
+ // It is a bit unfortunate that currently the UDF configuration
signature expects a
+ // MapredContext (even if execution is tez or another engine) - this
causes an
+ // impedence mismatch. For example: MapredContext has Reporter objects
that may or
+ // may not make sense for the current engine.
+ //
+ // We attempt to create a dummyContext that has at least access to the
currently set
+ // configuration. The other unfortunate issue is that some paths set the
configuration
+ // when creating ExprNodeGenericFuncEvaluator while others do not, so we
fallback to a
+ // "default" new HiveConf object which might be missing configuration
that changed during
+ // runtime.
+ MapredContext dummyContext = MapredContext.createDummy(getConf() != null
? getConf() : new HiveConf());
Review comment:
this is strange - during fetch task conversion we should have access to
the sessionstate; and when we are running it on the cluster
mapredcontext/tezcontext should be available.
There is one thing which might have came up in the failures: when we run
HS2/clidriver we *always* have a session state when we run queries; because of
this nature I think it makes sense to rely on this mechanism instead of working
around stuff here and there - however some test(s) may have "forgot" to start
the session (`SessionState.start`) - which could cause an NPE ; I think this
might be happening in `TestVectorizationContext`
but...there is also some stuff inside the metastore which can't have a
session - so you might need to handle the null case as well...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
