[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101220#comment-15101220 ] Xuefu Zhang commented on HIVE-12828: I think we are okay. Please feel free to commit the patch. We will address the env issue tomorrow. > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Rui Li > Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch, > HIVE-12828.2-spark.patch, HIVE-12828.2-spark.patch, HIVE-12828.2-spark.patch, > mem.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101204#comment-15101204 ] Rui Li commented on HIVE-12828: --- [~xuefuz], do we need to make parquet_join pass here? > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Rui Li > Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch, > HIVE-12828.2-spark.patch, HIVE-12828.2-spark.patch, HIVE-12828.2-spark.patch, > mem.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101183#comment-15101183 ] Xuefu Zhang commented on HIVE-12828: +1 > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Rui Li > Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch, > HIVE-12828.2-spark.patch, HIVE-12828.2-spark.patch, HIVE-12828.2-spark.patch, > mem.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101178#comment-15101178 ] Rui Li commented on HIVE-12828: --- It passed on my machine too, with the updated tarball. > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Rui Li > Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch, > HIVE-12828.2-spark.patch, HIVE-12828.2-spark.patch, HIVE-12828.2-spark.patch, > mem.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101152#comment-15101152 ] Xuefu Zhang commented on HIVE-12828: I'm not sure if the test cleanup /thirdparty dir. I tested and worked for me. Could you give a try? > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Rui Li > Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch, > HIVE-12828.2-spark.patch, HIVE-12828.2-spark.patch, HIVE-12828.2-spark.patch, > mem.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[
https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101125#comment-15101125
]
Rui Li commented on HIVE-12828:
---
Looked at the log and error is
{noformat}
2016-01-14T14:38:11,889 - 16/01/14 14:38:11 WARN TaskSetManager: Lost task
0.0 in stage 136.0 (TID 238, ip-10-233-128-9.us-west-1.compute.internal):
java.io.IOException: java.lang.reflect.InvocationTargetException
2016-01-14T14:38:11,889 - at
org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
2016-01-14T14:38:11,889 - at
org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
2016-01-14T14:38:11,890 - at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:269)
2016-01-14T14:38:11,890 - at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.(HadoopShimsSecure.java:216)
2016-01-14T14:38:11,890 - at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:343)
2016-01-14T14:38:11,890 - at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:680)
2016-01-14T14:38:11,890 - at
org.apache.spark.rdd.HadoopRDD$$anon$1.(HadoopRDD.scala:237)
2016-01-14T14:38:11,890 - at
org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:208)
2016-01-14T14:38:11,890 - at
org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101)
2016-01-14T14:38:11,890 - at
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
2016-01-14T14:38:11,890 - at
org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
2016-01-14T14:38:11,890 - at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
2016-01-14T14:38:11,890 - at
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
2016-01-14T14:38:11,890 - at
org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
2016-01-14T14:38:11,890 - at
org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:87)
2016-01-14T14:38:11,890 - at
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
2016-01-14T14:38:11,890 - at
org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
2016-01-14T14:38:11,890 - at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
2016-01-14T14:38:11,890 - at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
2016-01-14T14:38:11,890 - at
org.apache.spark.scheduler.Task.run(Task.scala:89)
2016-01-14T14:38:11,890 - at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
2016-01-14T14:38:11,890 - at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
2016-01-14T14:38:11,890 - at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
2016-01-14T14:38:11,890 - at java.lang.Thread.run(Thread.java:744)
2016-01-14T14:38:11,890 - Caused by:
java.lang.reflect.InvocationTargetException
2016-01-14T14:38:11,890 - at
sun.reflect.GeneratedConstructorAccessor29.newInstance(Unknown Source)
2016-01-14T14:38:11,890 - at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2016-01-14T14:38:11,890 - at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)
2016-01-14T14:38:11,890 - at
org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:255)
2016-01-14T14:38:11,890 - ... 21 more
2016-01-14T14:38:11,891 - Caused by: java.lang.NoSuchMethodError:
org.apache.parquet.schema.Types$MessageTypeBuilder.addFields([Lorg/apache/parquet/schema/Type;)Lorg/apache/parquet/schema/Types$BaseGroupBuilder;
2016-01-14T14:38:11,891 - at
org.apache.hadoop.hive.ql.io.parquet.read.DataWritableReadSupport.getSchemaByName(DataWritableReadSupport.java:160)
2016-01-14T14:38:11,891 - at
org.apache.hadoop.hive.ql.io.parquet.read.DataWritableReadSupport.init(DataWritableReadSupport.java:223)
2016-01-14T14:38:11,891 - at
org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.getSplit(ParquetRecordReaderWrapper.java:248)
2016-01-14T14:38:11,891 - at
org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.(ParquetRecordReaderWrapper.java:94)
2016-01-14T14:38:11,891 - at
org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.(ParquetRecordReaderWrapper.java:80)
2016-01-14T14:38:11,891 - at
org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat.getRecordReader(MapredParquetInputFormat.java:72)
2016-01-14T14:38:11,891 - at
org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.(CombineHiveRecord
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[
https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15099092#comment-15099092
]
Hive QA commented on HIVE-12828:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12782359/HIVE-12828.2-spark.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 9866 tests executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_parquet_join
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testGetPartitionSpecs_WithAndWithoutPartitionGrouping
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/1030/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/1030/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-1030/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12782359 - PreCommit-HIVE-SPARK-Build
> Update Spark version to 1.6
> ---
>
> Key: HIVE-12828
> URL: https://issues.apache.org/jira/browse/HIVE-12828
> Project: Hive
> Issue Type: Task
> Components: Spark
>Reporter: Xuefu Zhang
>Assignee: Rui Li
> Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch,
> HIVE-12828.2-spark.patch, HIVE-12828.2-spark.patch, HIVE-12828.2-spark.patch,
> mem.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[
https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15098777#comment-15098777
]
Hive QA commented on HIVE-12828:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12782315/HIVE-12828.2-spark.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 45 failed/errored test(s), 9312 tests
executed
*Failed tests:*
{noformat}
TestCliDriver-auto_sortmerge_join_7.q-exim_04_evolved_parts.q-query_with_semi.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-bool_literal.q-authorization_cli_createtab.q-explain_ddl.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-bucketsortoptimize_insert_7.q-list_bucket_query_multiskew_1.q-skewjoin_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-cp_mj_rc.q-decimal_2.q-union32.q-and-12-more - did not produce a
TEST-*.xml file
TestCliDriver-groupby_map_ppr_multi_distinct.q-vectorization_16.q-union_remove_15.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-join39.q-exim_07_all_part_over_nonoverlap.q-cbo_windowing.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-join9.q-insert_values_partitioned.q-progress_1.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-join_cond_pushdown_unqual4.q-udf_var_samp.q-load_dyn_part2.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-metadata_export_drop.q-udf_sin.q-udf_reverse.q-and-12-more - did
not produce a TEST-*.xml file
TestCliDriver-orc_split_elimination.q-udf_xpath_string.q-partition_wise_fileformat.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-ptf_general_queries.q-unionDistinct_1.q-groupby1_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-push_or.q-infer_bucket_sort_list_bucket.q-vector_interval_2.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-script_pipe.q-auto_join24.q-cast1.q-and-12-more - did not produce
a TEST-*.xml file
TestCliDriver-show_conf.q-udaf_covar_samp.q-udf_md5.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-skewjoin_mapjoin4.q-groupby6_map.q-cbo_rp_union.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-smb_mapjoin_4.q-udf_asin.q-udf_to_unix_timestamp.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-stats13.q-join_parse.q-sort_merge_join_desc_2.q-and-12-more - did
not produce a TEST-*.xml file
TestCliDriver-timestamp_lazy.q-union29.q-groupby_ppd.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-timestamp_literal.q-inputddl8.q-runtime_skewjoin_mapjoin_spark.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-udf_current_user.q-join44.q-union2.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-udf_nvl.q-alter_char1.q-serde_reported_schema.q-and-12-more - did
not produce a TEST-*.xml file
TestCliDriver-union36.q-acid_join.q-part_inherit_tbl_props_empty.q-and-6-more -
did not produce a TEST-*.xml file
TestCliDriver-varchar_nested_types.q-leadlag.q-semicolon.q-and-12-more - did
not produce a TEST-*.xml file
TestCliDriver-vector_distinct_2.q-nullscript.q-vector_char_mapjoin1.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-vectorization_limit.q-union19.q-groupby_grouping_sets6.q-and-12-more
- did not produce a TEST-*.xml file
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniTezCliDriver-auto_join30.q-vector_decimal_10_0.q-schema_evol_orc_acidvec_mapwork_part.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_grouping_sets.q-mapjoin_mapjoin.q-update_all_partitioned.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_interval_2.q-constprog_dpp.q-dynamic_partition_pruning.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_non_string_partition.q-delete_where_non_partitioned.q-auto_sortmerge_join_16.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vectorization_13.q-tez_bmj_schema_evolution.q-bucket3.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-auto_join30.q-join9.q-input17.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-avro_joins.q-join36.q-join4.q-and-12-more - did not produce
a TEST-*.xml file
TestSparkCliDriver-bucketmapjoin3.q-enforce_order.q-union11.q-and-12-more - did
not produce a TEST-*.xml file
TestSparkCliDriver-groupby6_map.q-join13.q-join_reorder3.q-and-12-more - did
not produce a TEST-*.xml file
TestSparkCliDriver-groupby_grouping_id2.q-vectorization_13.q-auto_sortmerge_join_13.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-input1_limit.q-groupby8_map.q-varchar_join1.q-and-12-more -
did not produce a TEST-*.xml file
TestSparkCliDriver-join_cond_pushdown_3.q-groupby7.q-auto_join17.q-and-12-more
- did not produc
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097621#comment-15097621 ] Rui Li commented on HIVE-12828: --- OK. Thanks for taking care of this, Xuefu. > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Rui Li > Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch, > HIVE-12828.2-spark.patch, mem.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097618#comment-15097618 ] Xuefu Zhang commented on HIVE-12828: Hi [~lirui], it seems that cloudfront still has the old tarball (judged from file size). Not sure what happened last time, but I will upload a new tarball tomorrow. Thanks. > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Rui Li > Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch, > HIVE-12828.2-spark.patch, mem.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[
https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097583#comment-15097583
]
Xuefu Zhang commented on HIVE-12828:
[~lirui], I used this command to generate the tarball. Maybe I have missed
anything.
{code}
./make-distribution.sh --name hadoop2-without-hive --tgz -Phadoop-2.4 -Pyarn
-Pparquet-provided
{code}
I will try locally to see if that test fails.
> Update Spark version to 1.6
> ---
>
> Key: HIVE-12828
> URL: https://issues.apache.org/jira/browse/HIVE-12828
> Project: Hive
> Issue Type: Task
> Components: Spark
>Reporter: Xuefu Zhang
>Assignee: Rui Li
> Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch,
> HIVE-12828.2-spark.patch, mem.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096325#comment-15096325 ] Rui Li commented on HIVE-12828: --- The parquet_join passes on my machine with a locally built tar ball. [~xuefuz] - Is it possible the test isn't picking the updated tar ball? > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Rui Li > Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch, > HIVE-12828.2-spark.patch, mem.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[
https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095607#comment-15095607
]
Hive QA commented on HIVE-12828:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12781969/HIVE-12828.2-spark.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 9866 tests executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_parquet_join
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testGetPartitionSpecs_WithAndWithoutPartitionGrouping
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/1028/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/1028/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-1028/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12781969 - PreCommit-HIVE-SPARK-Build
> Update Spark version to 1.6
> ---
>
> Key: HIVE-12828
> URL: https://issues.apache.org/jira/browse/HIVE-12828
> Project: Hive
> Issue Type: Task
> Components: Spark
>Reporter: Xuefu Zhang
>Assignee: Rui Li
> Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch,
> HIVE-12828.2-spark.patch, mem.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095181#comment-15095181 ] Xuefu Zhang commented on HIVE-12828: Okay. I forgot about it. The tarball is updated with the profile when building it. Thanks. > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Rui Li > Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch, > mem.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094077#comment-15094077 ] Rui Li commented on HIVE-12828: --- We found the profile is needed when we updated to spark 1.5. I have also noted that in our wiki:) > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Rui Li > Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch, > mem.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093957#comment-15093957 ] Xuefu Zhang commented on HIVE-12828: Yeah, I didn't enable that profile as I had never done that in the past when I generated the tarballs. I'm wondering if there is parquet version difference between spark and hive previously. Certainly, I will try the test myself to make sure the parquet test passes. Thanks. > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Rui Li > Attachments: HIVE-12828.1-spark.patch, HIVE-12828.2-spark.patch, > mem.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[
https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093611#comment-15093611
]
Hive QA commented on HIVE-12828:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12781757/HIVE-12828.2-spark.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 41 failed/errored test(s), 8688 tests
executed
*Failed tests:*
{noformat}
TestCliDriver-alter_table_not_sorted.q-cbo_udf_max.q-groupby4_map.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-columnstats_partlvl_dp.q-input31.q-udaf_percentile.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-compute_stats_string.q-skewjoin_union_remove_2.q-avro_nullable_fields.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-groupby4.q-varchar_serde.q-ivyDownload.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-groupby_grouping_id2.q-insert_values_nonascii.q-vectorization_13.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-groupby_map_ppr_multi_distinct.q-vectorization_16.q-union_remove_15.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-groupby_sort_7.q-vector_decimal_10_0.q-partition_vs_table_metadata.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-index_compact_2.q-bucketmapjoin12.q-udf_percentile.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-join13.q-udf_bitwise_shiftleft.q-parquet_decimal.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-join36.q-authorization_index.q-parquet_mixed_partition_formats2.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-metadata_export_drop.q-udf_sin.q-udf_reverse.q-and-12-more - did
not produce a TEST-*.xml file
TestCliDriver-orc_ppd_decimal.q-alter_partition_clusterby_sortby.q-udf_radians.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-ppd_join4.q-union27.q-show_indexes_edge_cases.q-and-12-more - did
not produce a TEST-*.xml file
TestCliDriver-ppd_transform.q-orc_merge9.q-ba_table3.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-schema_evol_text_nonvec_mapwork_table.q-auto_join23.q-auto_join19.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-skewjoinopt15.q-fileformat_text.q-load_dyn_part12.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-udaf_collect_set_2.q-ba_table_union.q-orc_vectorization_ppd.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-update_orig_table.q-order.q-auto_join18_multi_distinct.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-vector_acid3.q-vector_custom_udf_configure.q-udf4.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-vectorization_10.q-orc_merge5.q-union26.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-vectorization_limit.q-union19.q-groupby_grouping_sets6.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-view.q-udf2.q-groupby7.q-and-12-more - did not produce a
TEST-*.xml file
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniTezCliDriver-schema_evol_text_nonvec_mapwork_table.q-vector_decimal_trailing.q-subquery_in.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-tez_union_group_by.q-vector_auto_smb_mapjoin_14.q-union_fast_stats.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-transform_ppr2.q-hybridgrace_hashjoin_2.q-orc_ppd_basic.q-and-1-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_decimal_2.q-explainuser_1.q-explainuser_3.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_distinct_2.q-tez_joins_explain.q-vector_char_mapjoin1.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_interval_2.q-constprog_dpp.q-dynamic_partition_pruning.q-and-12-more
- did not produce a TEST-*.xml file
TestNegativeCliDriver-udf_invalid.q-nopart_insert.q-join_nonexistent_part.q-and-677-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-auto_join_reordering_values.q-ptf_seqfile.q-auto_join18.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-groupby3_map.q-skewjoinopt8.q-union_remove_1.q-and-12-more -
did not produce a TEST-*.xml file
TestSparkCliDriver-join2.q-script_pipe.q-auto_join24.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-skewjoinopt3.q-union27.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-skewjoinopt15.q-join39.q-load_dyn_part12.q-and-12-more - did
not produce a TEST-*.xml file
TestSparkCliDriver-smb_mapjoin_15.q-mapreduce2.q-mapreduce1.q-and-12-more - did
not produce a TEST-*.xml file
TestSparkCliDriver-vector_distinct_2.q-join15.q-load_dyn_part3.q-and-12-more -
did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093090#comment-15093090 ] Rui Li commented on HIVE-12828: --- Thanks Xuefu for the patch. I'll try it out. > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Rui Li > Attachments: HIVE-12828.1-spark.patch, mem.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[
https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15092844#comment-15092844
]
Hive QA commented on HIVE-12828:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12781545/HIVE-12828.1-spark.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 57 failed/errored test(s), 9866 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_partitioned
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_bucketizedhiveinputformat
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_bucketmapjoin6
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_bucketmapjoin7
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_disable_merge_for_bucketing
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_empty_dir_in_table
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_external_table_with_space_in_location_path
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_file_with_header_footer
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_gen_udf_example_add10
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_import_exported_table
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap_auto
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_infer_bucket_sort_bucketed_table
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_infer_bucket_sort_map_operators
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_infer_bucket_sort_merge
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_infer_bucket_sort_num_buckets
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_input16_cc
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_leftsemijoin_mr
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_list_bucket_dml_10
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_load_fs2
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge2
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge3
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge4
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge8
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge9
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge_incompat1
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge_incompat2
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_parallel_orderby
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_quotedid_smb
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_reduce_deduplicate
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_remote_script
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_schemeAuthority
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_schemeAuthority2
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_scriptfile1
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_temp_table_external
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_truncate_column_buckets
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_uber_reduce
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_inner_join
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join0
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join1
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join2
org.apache.hadoop.h
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15092560#comment-15092560 ] Xuefu Zhang commented on HIVE-12828: Thanks for working on this, [~lirui]! The spark tar ball is uploaded. I submitted the patch for a test run. > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Rui Li > Attachments: HIVE-12828.1-spark.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12828) Update Spark version to 1.6
[ https://issues.apache.org/jira/browse/HIVE-12828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090652#comment-15090652 ] Xuefu Zhang commented on HIVE-12828: cc: [~lirui] > Update Spark version to 1.6 > --- > > Key: HIVE-12828 > URL: https://issues.apache.org/jira/browse/HIVE-12828 > Project: Hive > Issue Type: Task > Components: Spark >Reporter: Xuefu Zhang >Assignee: Xuefu Zhang > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
