[
https://issues.apache.org/jira/browse/HIVE-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brock Noland updated HIVE-9335:
-------------------------------
Description:
I made a pass through HIVE-9257 and found the following issues:
{{HashTableSinkOperator.java}}
The fields EMPTY_OBJECT_ARRAY and EMPTY_ROW_CONTAINER are no longer constants
and should not be in upper case.
{{HivePairFlatMapFunction.java}}
We share NumberFormat accross threads and it's not thread safe.
{{KryoSerializer.java}}
we eat the stack trace in deserializeJobConf
{{SparkMapRecordHandler}}
in processRow we should not be using {{StringUtils.stringifyException}} since
LOG can handle stack traces.
in close:
{noformat}
// signal new failure to map-reduce
LOG.error("Hit error while closing operators - failing tree");
throw new IllegalStateException("Error while closing operators", e);
{noformat}
Should be:
{noformat}
String msg = "Error while closing operators: " + e;
throw new IllegalStateException(msg, e);
{noformat}
{{SparkSessionManagerImpl}} - the method {{canReuseSession}} is useless
{{GenSparkSkewJoinProcessor}}
{noformat}
+ // keep it as reference in case we need fetch work
+// localPlan.getAliasToFetchWork().put(small_alias.toString(),
+// new FetchWork(tblDir, tableDescList.get(small_alias)));
{noformat}
{{GenSparkWorkWalker}} trim ws
{{SparkCompiler}} remote init
{{SparkEdgeProperty}} trim ws
{{CounterStatsPublisher}} eat exception
{{Hadoop23Shims}} unused import of {{ResourceBundles}}
> Address review items on HIVE-9257 [Spark Branch]
> ------------------------------------------------
>
> Key: HIVE-9335
> URL: https://issues.apache.org/jira/browse/HIVE-9335
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Affects Versions: spark-branch
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: HIVE-9335.1-spark.patch
>
>
> I made a pass through HIVE-9257 and found the following issues:
> {{HashTableSinkOperator.java}}
> The fields EMPTY_OBJECT_ARRAY and EMPTY_ROW_CONTAINER are no longer constants
> and should not be in upper case.
> {{HivePairFlatMapFunction.java}}
> We share NumberFormat accross threads and it's not thread safe.
> {{KryoSerializer.java}}
> we eat the stack trace in deserializeJobConf
> {{SparkMapRecordHandler}}
> in processRow we should not be using {{StringUtils.stringifyException}} since
> LOG can handle stack traces.
> in close:
> {noformat}
> // signal new failure to map-reduce
> LOG.error("Hit error while closing operators - failing tree");
> throw new IllegalStateException("Error while closing operators", e);
> {noformat}
> Should be:
> {noformat}
> String msg = "Error while closing operators: " + e;
> throw new IllegalStateException(msg, e);
> {noformat}
> {{SparkSessionManagerImpl}} - the method {{canReuseSession}} is useless
> {{GenSparkSkewJoinProcessor}}
> {noformat}
> + // keep it as reference in case we need fetch work
> +// localPlan.getAliasToFetchWork().put(small_alias.toString(),
> +// new FetchWork(tblDir, tableDescList.get(small_alias)));
> {noformat}
> {{GenSparkWorkWalker}} trim ws
> {{SparkCompiler}} remote init
> {{SparkEdgeProperty}} trim ws
> {{CounterStatsPublisher}} eat exception
> {{Hadoop23Shims}} unused import of {{ResourceBundles}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
