[jira] [Created] (HIVE-19992) Vectorization: Follow-on to HIVE-19951 --> add call to SchemaEvolution.isOnlyImplicitConversion to disable encoded LLAP I/O for ORC only when data type conversion is not
Matt McCline created HIVE-19992: --- Summary: Vectorization: Follow-on to HIVE-19951 --> add call to SchemaEvolution.isOnlyImplicitConversion to disable encoded LLAP I/O for ORC only when data type conversion is not implicit Key: HIVE-19992 URL: https://issues.apache.org/jira/browse/HIVE-19992 Project: Hive Issue Type: Bug Components: Hive Reporter: Matt McCline Assignee: Matt McCline When ORC-380 that adds the SchemaEvolution.isOnlyImplicitConversion call is available in the ORC release used by Apache master (and branch-3), then update LlapRecordReader (see comments in HIVE-19951 change). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19991) msck repair table command not able to retrieve achieved data.
Manpreet Singh created HIVE-19991: - Summary: msck repair table command not able to retrieve achieved data. Key: HIVE-19991 URL: https://issues.apache.org/jira/browse/HIVE-19991 Project: Hive Issue Type: Bug Affects Versions: 1.1.0 Reporter: Manpreet Singh Observed an issue when customer used msck repair tablename command on a archeived data (data copied via hadoop fs -cp from another location) by the command did not loaded the partitions in the table and showed zero results. Please find below complete steps for both archived and unarchived. Copied table's partition data to another location & define a similar table structure for new location an ran "msck repair table" command on the destination and then ran a select * from table it produced zero results, same thing is working for non achieved data. 1. Created two table with similar structure and different loctation. create table sau_test1 ( a int, b string) partitioned by (dt string) stored as parquet location '/user/hive/warehouse/sau_test1'; create table sau_arch ( a int, b string) partitioned by (dt string) stored as parquet location'/user/hive/warehouse/sau_arch'; 2. Inserted data in source table insert into sau_test1 partition(dt='dt1') select 1,'A1'; select * from sau_test1; +---+-++-+ |sau_test1.a|sau_test1.b|sau_test1.dt| +---+-++-+ |1|A1|dt1| +---+-++-+ 3. Copied the content of directory source directory to destination directory. hadoop fs -cp /user/hive/warehouse/sau_test1/* /user/hive/warehouse/sau_arch/ 4. Running msck repair table and checking results. — for unachieved data msck repair table sau_arch ; select * from sau_arch; select * from sau_arch; +--++++ |sau_arch.a|sau_arch.b|sau_arch.dt| +--++++ |1|A1|dt1| +--++++ 5. Customer wants the same functionality for archived data and hence tried below steps. a) Dropped table partition in destination table "alter table sau_arch drop partition(dt='dt1');" b) set hive.archive.enabled=true; alter table sau_test1 archive partition ( dt='dt1'); c) copied the hdfs files from source table to destination tables. hdfs dfs -ls /user/hive/warehouse/sau_test1/dt=dt1/ drwxr-xr-x - hive supergroup 0 2018-06-08 13:26 /user/hive/warehouse/sau_test1/dt=dt1/data.har -rw-r--r-- 3 hive supergroup 0 2018-06-08 13:26 /user/hive/warehouse/sau_test1/dt=dt1/data.har/_SUCCESS -rw-r--r-- 3 hive supergroup 305 2018-06-08 13:26 /user/hive/warehouse/sau_test1/dt=dt1/data.har/_index -rw-r--r-- 3 hive supergroup 23 2018-06-08 13:26 /user/hive/warehouse/sau_test1/dt=dt1/data.har/_masterindex -rw-r--r-- 3 hive supergroup 286 2018-06-08 13:26 /user/hive/warehouse/sau_test1/dt=dt1/data.har/part-0 $ hdfs dfs -ls /user/hive/warehouse/sau_arch/dt=dt1/ drwxr-xr-x - ngdb supergroup 0 2018-06-08 13:27 /user/hive/warehouse/sau_arch/dt=dt1/data.har -rw-r--r-- 3 ngdb supergroup 0 2018-06-08 13:27 /user/hive/warehouse/sau_arch/dt=dt1/data.har/_SUCCESS -rw-r--r-- 3 ngdb supergroup 305 2018-06-08 13:27 /user/hive/warehouse/sau_arch/dt=dt1/data.har/_index -rw-r--r-- 3 ngdb supergroup 23 2018-06-08 13:27 /user/hive/warehouse/sau_arch/dt=dt1/data.har/_masterindex -rw-r--r-- 3 ngdb supergroup 286 2018-06-08 13:27 /user/hive/warehouse/sau_arch/dt=dt1/data.har/part-0 d) msck repair table sau_arch; e)select * from sau_arch . - No results shown +--++++ |sau_arch.a|sau_arch.b|sau_arch.dt| +--++++ +--++++ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19990) Query with interval literal in join condition fails
Vineet Garg created HIVE-19990:
--
Summary: Query with interval literal in join condition fails
Key: HIVE-19990
URL: https://issues.apache.org/jira/browse/HIVE-19990
Project: Hive
Issue Type: Bug
Components: Query Planning
Reporter: Vineet Garg
Assignee: Vineet Garg
*Reproducer*
{code:sql}
SELECT
d1.d_week_seq
FROM
date_dim d1
WHERE
Cast(d1.d_date AS date) > INTERVAL '5' day ;
{code}
*Exception*
{code}
org.apache.hadoop.hive.ql.parse.SemanticException: '5 00:00:00.0'
encountered with 0 children
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.parseJoinCondPopulateAlias(SemanticAnalyzer.java:2780)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.parseJoinCondPopulateAlias(SemanticAnalyzer.java:2775)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.parseJoinCondition(SemanticAnalyzer.java:3060)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.parseJoinCondition(SemanticAnalyzer.java:2959)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genJoinTree(SemanticAnalyzer.java:9633)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11380)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11285)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:12071)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:593)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12150)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:330)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:658)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1829)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1776)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1771)
at
org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
at
org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:832)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:770)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:694)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Apache Hive 3.1. release preparation
I think it would be useful to do a metastore 3.1 release as well along with the release. In order to do that we should deploy metastore in standalone mode, make sure it works as expected and also document how to install and use metastore as a standalone module. On Mon, Jun 25, 2018 at 3:00 PM, Vineet Garg wrote: > Hello folks, > > It has been more than one month since Hive 3.0 release. Plenty of bug > fixes and minor features have been pushed in to branch-3 and therefore I > believe it is time for us to release Hive 3.1. > I plan to cut off branch 3.1 off branch-3 tomorrow at end of day. Once > the branch is cut please do not commit anything in there. If you absolutely > must please check in with me first. I plan to prepare a RC within a week of > cutting the branch. > > Thanks, > Vineet > >
[jira] [Created] (HIVE-19989) Metastore uses wrong application name for HADOOP2 metrics
Vineet Garg created HIVE-19989: -- Summary: Metastore uses wrong application name for HADOOP2 metrics Key: HIVE-19989 URL: https://issues.apache.org/jira/browse/HIVE-19989 Project: Hive Issue Type: Improvement Components: Standalone Metastore Reporter: Vineet Garg Assignee: Vineet Garg -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Apache Hive 3.1. release preparation
Hello folks, It has been more than one month since Hive 3.0 release. Plenty of bug fixes and minor features have been pushed in to branch-3 and therefore I believe it is time for us to release Hive 3.1. I plan to cut off branch 3.1 off branch-3 tomorrow at end of day. Once the branch is cut please do not commit anything in there. If you absolutely must please check in with me first. I plan to prepare a RC within a week of cutting the branch. Thanks, Vineet
Re: Hive QA logs not accessible
It looks like a infrastructure issue. Whenever H19 node tries to execute the pre-commit job it fails because it cannot download maven artifacts which fails the compilation process for Ptest Client. On Mon, Jun 25, 2018 at 2:23 PM, Vihang Karajgaonkar wrote: > Hmm .. you are right. The builds are broken. Its not able to compile the > code but it works for me locally. Let me take a look. Thanks for reporting. > I have created https://issues.apache.org/jira/browse/HIVE-19988 > > On Mon, Jun 25, 2018 at 2:00 PM, Deepak Jaiswal > wrote: > >> It was too soon. Looks like it is broken again. One of my runs, >> >> https://builds.apache.org/view/H-L/view/Hive/job/PreCommit-H >> IVE-Build/12114/console >> >> Regards, >> Deepak >> >> On 6/25/18, 1:57 PM, "Deepak Jaiswal" wrote: >> >> Hi Vihang, >> >> It took a while but tests started to appear, so all is good now. >> >> Regards, >> Deepak >> >> On 6/25/18, 12:24 PM, "Vihang Karajgaonkar" >> wrote: >> >> I see there are 6 builds in the queue right now (which is >> unusually small). >> What is the JIRA number where you submitted the patch? >> >> On Mon, Jun 25, 2018 at 11:05 AM, Deepak Jaiswal < >> djais...@hortonworks.com> >> wrote: >> >> > Hi Vihang, >> > >> > I am looking for logs of failed test runs. Thanks for >> optimizing this for >> > successful runs. However, I think there is a problem with Hive >> QA, the >> > queue is gone and I submitted a patch more than 10 minutes ago >> and it >> > hasn’t started or enqueued yet. >> > >> > https://builds.apache.org/view/H-L/view/Hive/job/PreCommit-H >> IVE-Build/ >> > >> > Regards, >> > Deepak >> > >> > On 6/25/18, 10:53 AM, "Vihang Karajgaonkar" >> >> > wrote: >> > >> > Are you looking for logs for successful tests? I had >> submitted a change >> > recently which stops skips downloading logs for successful >> tests to >> > shave >> > off ~10 min time from each run. I found that the job was >> spending too >> > much >> > time copying over ~20G of logs from worker nodes to the >> server. Can you >> > give the JIRA number so that I can take a look? >> > >> > On Mon, Jun 25, 2018 at 10:38 AM, Deepak Jaiswal < >> > djais...@hortonworks.com> >> > wrote: >> > >> > > The Hive QA logs are not accessible for yesterday night’s >> run. Also, >> > I >> > > don’t see any test running. >> > > Is the disk full again? >> > > >> > > Regards, >> > > Deepak >> > > >> > >> > >> > >> >> >> >> >> >
Re: Hive QA logs not accessible
Hmm .. you are right. The builds are broken. Its not able to compile the code but it works for me locally. Let me take a look. Thanks for reporting. I have created https://issues.apache.org/jira/browse/HIVE-19988 On Mon, Jun 25, 2018 at 2:00 PM, Deepak Jaiswal wrote: > It was too soon. Looks like it is broken again. One of my runs, > > https://builds.apache.org/view/H-L/view/Hive/job/PreCommit- > HIVE-Build/12114/console > > Regards, > Deepak > > On 6/25/18, 1:57 PM, "Deepak Jaiswal" wrote: > > Hi Vihang, > > It took a while but tests started to appear, so all is good now. > > Regards, > Deepak > > On 6/25/18, 12:24 PM, "Vihang Karajgaonkar" > wrote: > > I see there are 6 builds in the queue right now (which is > unusually small). > What is the JIRA number where you submitted the patch? > > On Mon, Jun 25, 2018 at 11:05 AM, Deepak Jaiswal < > djais...@hortonworks.com> > wrote: > > > Hi Vihang, > > > > I am looking for logs of failed test runs. Thanks for optimizing > this for > > successful runs. However, I think there is a problem with Hive > QA, the > > queue is gone and I submitted a patch more than 10 minutes ago > and it > > hasn’t started or enqueued yet. > > > > https://builds.apache.org/view/H-L/view/Hive/job/PreCommit- > HIVE-Build/ > > > > Regards, > > Deepak > > > > On 6/25/18, 10:53 AM, "Vihang Karajgaonkar" > > > wrote: > > > > Are you looking for logs for successful tests? I had > submitted a change > > recently which stops skips downloading logs for successful > tests to > > shave > > off ~10 min time from each run. I found that the job was > spending too > > much > > time copying over ~20G of logs from worker nodes to the > server. Can you > > give the JIRA number so that I can take a look? > > > > On Mon, Jun 25, 2018 at 10:38 AM, Deepak Jaiswal < > > djais...@hortonworks.com> > > wrote: > > > > > The Hive QA logs are not accessible for yesterday night’s > run. Also, > > I > > > don’t see any test running. > > > Is the disk full again? > > > > > > Regards, > > > Deepak > > > > > > > > > > > > > >
[jira] [Created] (HIVE-19988) Precommit jobs erroring out
Vihang Karajgaonkar created HIVE-19988:
--
Summary: Precommit jobs erroring out
Key: HIVE-19988
URL: https://issues.apache.org/jira/browse/HIVE-19988
Project: Hive
Issue Type: Bug
Reporter: Vihang Karajgaonkar
Assignee: Vihang Karajgaonkar
{code}
+ mvn clean package -B -DskipTests -Drat.numUnapprovedLicenses=1000
-Dmaven.repo.local=/home/jenkins/jenkins-slave/workspace/PreCommit-HIVE-Build/.m2/repository
[INFO] Scanning for projects...
[INFO]
[INFO]
[INFO] Building hive-ptest 3.0
[INFO]
[INFO] Downloading from central:
https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 0.925 s
[INFO] Finished at: 2018-06-25T20:46:27Z
[INFO] Final Memory: 24M/1447M
[INFO]
[ERROR] Plugin org.apache.maven.plugins:maven-clean-plugin:2.5 or one of its
dependencies could not be resolved: Failed to read artifact descriptor for
org.apache.maven.plugins:maven-clean-plugin:jar:2.5: Could not transfer
artifact org.apache.maven.plugins:maven-clean-plugin:pom:2.5 from/to central
(https://repo.maven.apache.org/maven2): Received fatal alert: protocol_version
-> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
+ return 1
+ ret=1
+ unpack_test_results
+ '[' -z /home/jenkins/jenkins-slave/workspace/PreCommit-HIVE-Build/hive/build
']'
+ cd
/home/jenkins/jenkins-slave/workspace/PreCommit-HIVE-Build/hive/build/hive/testutils/ptest2/target
jenkins-execute-build.sh: line 61: cd:
/home/jenkins/jenkins-slave/workspace/PreCommit-HIVE-Build/hive/build/hive/testutils/ptest2/target:
No such file or directory
+ [[ -f test-results.tar.gz ]]
+ exit 1
+ rm -f /tmp/tmp.LFKzzyYwIt
Build step 'Execute shell' marked build as failure
Recording test results
ERROR: Step ?Publish JUnit test result report? failed: No test report files
were found. Configuration error?
[description-setter] Description set: HIVE-19980 / master-mr2
Finished: FAILURE
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HIVE-19987) Add logging of runtime statistics indicating when Hdfs Erasure Coding is used by Spark
Andrew Sherman created HIVE-19987: - Summary: Add logging of runtime statistics indicating when Hdfs Erasure Coding is used by Spark Key: HIVE-19987 URL: https://issues.apache.org/jira/browse/HIVE-19987 Project: Hive Issue Type: Task Reporter: Andrew Sherman Assignee: Andrew Sherman -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19986) Add logging of runtime statistics indicating when Hdfs Erasure Coding is used by MR
Andrew Sherman created HIVE-19986: - Summary: Add logging of runtime statistics indicating when Hdfs Erasure Coding is used by MR Key: HIVE-19986 URL: https://issues.apache.org/jira/browse/HIVE-19986 Project: Hive Issue Type: Task Reporter: Andrew Sherman Assignee: Andrew Sherman -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Hive QA logs not accessible
It was too soon. Looks like it is broken again. One of my runs, https://builds.apache.org/view/H-L/view/Hive/job/PreCommit-HIVE-Build/12114/console Regards, Deepak On 6/25/18, 1:57 PM, "Deepak Jaiswal" wrote: Hi Vihang, It took a while but tests started to appear, so all is good now. Regards, Deepak On 6/25/18, 12:24 PM, "Vihang Karajgaonkar" wrote: I see there are 6 builds in the queue right now (which is unusually small). What is the JIRA number where you submitted the patch? On Mon, Jun 25, 2018 at 11:05 AM, Deepak Jaiswal wrote: > Hi Vihang, > > I am looking for logs of failed test runs. Thanks for optimizing this for > successful runs. However, I think there is a problem with Hive QA, the > queue is gone and I submitted a patch more than 10 minutes ago and it > hasn’t started or enqueued yet. > > https://builds.apache.org/view/H-L/view/Hive/job/PreCommit-HIVE-Build/ > > Regards, > Deepak > > On 6/25/18, 10:53 AM, "Vihang Karajgaonkar" > wrote: > > Are you looking for logs for successful tests? I had submitted a change > recently which stops skips downloading logs for successful tests to > shave > off ~10 min time from each run. I found that the job was spending too > much > time copying over ~20G of logs from worker nodes to the server. Can you > give the JIRA number so that I can take a look? > > On Mon, Jun 25, 2018 at 10:38 AM, Deepak Jaiswal < > djais...@hortonworks.com> > wrote: > > > The Hive QA logs are not accessible for yesterday night’s run. Also, > I > > don’t see any test running. > > Is the disk full again? > > > > Regards, > > Deepak > > > > >
Re: Hive QA logs not accessible
Hi Vihang, It took a while but tests started to appear, so all is good now. Regards, Deepak On 6/25/18, 12:24 PM, "Vihang Karajgaonkar" wrote: I see there are 6 builds in the queue right now (which is unusually small). What is the JIRA number where you submitted the patch? On Mon, Jun 25, 2018 at 11:05 AM, Deepak Jaiswal wrote: > Hi Vihang, > > I am looking for logs of failed test runs. Thanks for optimizing this for > successful runs. However, I think there is a problem with Hive QA, the > queue is gone and I submitted a patch more than 10 minutes ago and it > hasn’t started or enqueued yet. > > https://builds.apache.org/view/H-L/view/Hive/job/PreCommit-HIVE-Build/ > > Regards, > Deepak > > On 6/25/18, 10:53 AM, "Vihang Karajgaonkar" > wrote: > > Are you looking for logs for successful tests? I had submitted a change > recently which stops skips downloading logs for successful tests to > shave > off ~10 min time from each run. I found that the job was spending too > much > time copying over ~20G of logs from worker nodes to the server. Can you > give the JIRA number so that I can take a look? > > On Mon, Jun 25, 2018 at 10:38 AM, Deepak Jaiswal < > djais...@hortonworks.com> > wrote: > > > The Hive QA logs are not accessible for yesterday night’s run. Also, > I > > don’t see any test running. > > Is the disk full again? > > > > Regards, > > Deepak > > > > >
[jira] [Created] (HIVE-19985) ACID: Skip decoding the ROW__ID sections for read-only queries
Gopal V created HIVE-19985: -- Summary: ACID: Skip decoding the ROW__ID sections for read-only queries Key: HIVE-19985 URL: https://issues.apache.org/jira/browse/HIVE-19985 Project: Hive Issue Type: Improvement Components: Transactions Reporter: Gopal V For a base_n file there are no aborted transactions within the file and if there are no pending delete deltas, the entire ACID ROW__ID can be skipped for all read-only queries (i.e SELECT), though it still needs to be projected out for MERGE, UPDATE and DELETE queries. This patch tries to entirely ignore the ACID ROW__ID fields for all tables where there are no possible deletes or aborted transactions for an ACID split. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Review Request 67731: HIVE-19902: Provide Metastore micro-benchmarks
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67731/ --- Review request for hive, Andrew Sherman, Alan Gates, Janaki Lahorani, Peter Vary, Sahil Takiar, and Vihang Karajgaonkar. Bugs: HIVE-19902 https://issues.apache.org/jira/browse/HIVE-19902 Repository: hive-git Description --- HIVE-19902: Provide Metastore micro-benchmarks Diffs - metastore-tools/data/conf/.keepme PRE-CREATION metastore-tools/metastore-benchmarks/README.md PRE-CREATION metastore-tools/metastore-benchmarks/pom.xml PRE-CREATION metastore-tools/metastore-benchmarks/src/main/java/org/apache/hadoop/hive/metastore/tools/BenchmarkSuite.java PRE-CREATION metastore-tools/metastore-benchmarks/src/main/java/org/apache/hadoop/hive/metastore/tools/BenchmarkTool.java PRE-CREATION metastore-tools/metastore-benchmarks/src/main/java/org/apache/hadoop/hive/metastore/tools/HMSBenchmarks.java PRE-CREATION metastore-tools/metastore-benchmarks/src/main/java/org/apache/hadoop/hive/metastore/tools/MicroBenchmark.java PRE-CREATION metastore-tools/metastore-benchmarks/src/main/resources/log4j.properties PRE-CREATION metastore-tools/metastore-benchmarks/src/main/resources/log4j2.xml PRE-CREATION metastore-tools/metastore-cli/README.md PRE-CREATION metastore-tools/metastore-cli/pom.xml PRE-CREATION metastore-tools/metastore-cli/src/main/java/org/apache/hadoop/hive/metastore/tools/HMSTool.java PRE-CREATION metastore-tools/pom.xml PRE-CREATION metastore-tools/tools-common/pom.xml PRE-CREATION metastore-tools/tools-common/src/main/java/org/apache/hadoop/hive/metastore/tools/Constants.java PRE-CREATION metastore-tools/tools-common/src/main/java/org/apache/hadoop/hive/metastore/tools/HMSClient.java PRE-CREATION metastore-tools/tools-common/src/main/java/org/apache/hadoop/hive/metastore/tools/Util.java PRE-CREATION metastore-tools/tools-common/src/test/java/org/apache/hadoop/hive/metastore/tools/HMSClientTest.java PRE-CREATION metastore-tools/tools-common/src/test/java/org/apache/hadoop/hive/metastore/tools/UtilTest.java PRE-CREATION pom.xml 5202248315f771385aa0aaa0d13907f00f3e329d Diff: https://reviews.apache.org/r/67731/diff/1/ Testing --- Thanks, Alexander Kolbasov
[jira] [Created] (HIVE-19984) Backport HIVE-15976 to branch-3
Alan Gates created HIVE-19984: - Summary: Backport HIVE-15976 to branch-3 Key: HIVE-19984 URL: https://issues.apache.org/jira/browse/HIVE-19984 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 3.1.0 Reporter: Alan Gates Assignee: Alan Gates -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19983) Backport HIVE-19769 to branch-3
Alan Gates created HIVE-19983: - Summary: Backport HIVE-19769 to branch-3 Key: HIVE-19983 URL: https://issues.apache.org/jira/browse/HIVE-19983 Project: Hive Issue Type: Bug Components: storage-api Affects Versions: 3.1.0 Reporter: Alan Gates Assignee: Alan Gates This patch will be needed for other catalog related work to be backported to branch-3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Hive QA logs not accessible
I see there are 6 builds in the queue right now (which is unusually small). What is the JIRA number where you submitted the patch? On Mon, Jun 25, 2018 at 11:05 AM, Deepak Jaiswal wrote: > Hi Vihang, > > I am looking for logs of failed test runs. Thanks for optimizing this for > successful runs. However, I think there is a problem with Hive QA, the > queue is gone and I submitted a patch more than 10 minutes ago and it > hasn’t started or enqueued yet. > > https://builds.apache.org/view/H-L/view/Hive/job/PreCommit-HIVE-Build/ > > Regards, > Deepak > > On 6/25/18, 10:53 AM, "Vihang Karajgaonkar" > wrote: > > Are you looking for logs for successful tests? I had submitted a change > recently which stops skips downloading logs for successful tests to > shave > off ~10 min time from each run. I found that the job was spending too > much > time copying over ~20G of logs from worker nodes to the server. Can you > give the JIRA number so that I can take a look? > > On Mon, Jun 25, 2018 at 10:38 AM, Deepak Jaiswal < > djais...@hortonworks.com> > wrote: > > > The Hive QA logs are not accessible for yesterday night’s run. Also, > I > > don’t see any test running. > > Is the disk full again? > > > > Regards, > > Deepak > > > > >
[jira] [Created] (HIVE-19982) Make JDBC work with Catalogs
Alan Gates created HIVE-19982: - Summary: Make JDBC work with Catalogs Key: HIVE-19982 URL: https://issues.apache.org/jira/browse/HIVE-19982 Project: Hive Issue Type: Sub-task Components: JDBC Affects Versions: 3.0.0 Reporter: Alan Gates Many JDBC methods include a catalog specifier in the call. We need to update these to work with multiple catalogs. Also we will need to also support the JDBC calls for catalogs. This at least includes Connection: getCatalog() and setCatalog() DatabaseMetaData: getCatalogs(), getCatalogSeparator(), and getCatalogTerm() and maybe others. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19981) Managed tables converted to external tables by the HiveStrictManagedMigration utility should be set to delete data when the table is dropped
Jason Dere created HIVE-19981: - Summary: Managed tables converted to external tables by the HiveStrictManagedMigration utility should be set to delete data when the table is dropped Key: HIVE-19981 URL: https://issues.apache.org/jira/browse/HIVE-19981 Project: Hive Issue Type: Bug Reporter: Jason Dere Assignee: Jason Dere Using the HiveStrictManagedMigration utility, tables can be converted to conform to the Hive strict managed tables mode. For managed tables that are converted to external tables by the utility, these tables should keep the "drop data on delete" semantics they had when they were managed tables. One way to do this is to introduce a table property "external.table.purge", which if true (and if the table is an external table), will let Hive know to delete the table data when the table is dropped. This property will be set by the HiveStrictManagedMigration utility when managed tables are converted to external tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19980) GenericUDTFGetSplits fails when order by query returns 0 rows
Prasanth Jayachandran created HIVE-19980:
Summary: GenericUDTFGetSplits fails when order by query returns 0
rows
Key: HIVE-19980
URL: https://issues.apache.org/jira/browse/HIVE-19980
Project: Hive
Issue Type: Bug
Affects Versions: 3.1.0, 4.0.0
Reporter: Prasanth Jayachandran
Assignee: Prasanth Jayachandran
When order by query returns 0 rows, there will not be any files in temporary
table location for GenericUDTFGetSplits
which results in the following exception
{code:java}
Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:217)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDTFGetSplits.getSplits(GenericUDTFGetSplits.java:420)
... 52 more{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Review Request 67710: HIVE-19481
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67710/ --- (Updated June 25, 2018, 6:15 p.m.) Review request for hive, Jason Dere and Sergey Shelukhin. Changes --- Updated results for failed tests. Bugs: HIVE-19481 https://issues.apache.org/jira/browse/HIVE-19481 Repository: hive-git Description --- sample10.q returns wrong results. Multiple issues were fixed 1. Instead of using old MR logic which assumes there is 1 file for each bucket, lookup buckets by name(non-managed tables) 2. Skip bucket pruning for managed tables. Diffs (updated) - itests/src/test/resources/testconfiguration.properties 517b413839 ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java 9dbd869d57 ql/src/java/org/apache/hadoop/hive/ql/optimizer/SamplePruner.java 8200e6a237 ql/src/test/queries/clientpositive/sample10_mm.q PRE-CREATION ql/src/test/results/clientpositive/archive_excludeHadoop20.q.out e4b390c9cd ql/src/test/results/clientpositive/beeline/smb_mapjoin_11.q.out 9f946e0b50 ql/src/test/results/clientpositive/llap/sample10.q.out ce3c2880a6 ql/src/test/results/clientpositive/llap/sample10_mm.q.out PRE-CREATION ql/src/test/results/clientpositive/masking_5.q.out 498fc117c7 ql/src/test/results/clientpositive/sample6.q.out 7f853e55c5 ql/src/test/results/clientpositive/sample7.q.out 0e2fc287d4 ql/src/test/results/clientpositive/sample9.q.out 0de49a698a ql/src/test/results/clientpositive/smb_mapjoin_11.q.out a83f3e66c4 ql/src/test/results/clientpositive/spark/infer_bucket_sort_bucketed_table.q.out 8fab7ecbd0 ql/src/test/results/clientpositive/spark/sample10.q.out 555e5f43ec ql/src/test/results/clientpositive/spark/sample2.q.out 8b73fdf874 ql/src/test/results/clientpositive/spark/sample4.q.out 3269b015ec ql/src/test/results/clientpositive/spark/sample6.q.out 36532d7fbe ql/src/test/results/clientpositive/spark/sample7.q.out d0b52bcdce Diff: https://reviews.apache.org/r/67710/diff/2/ Changes: https://reviews.apache.org/r/67710/diff/1-2/ Testing --- Thanks, Deepak Jaiswal
Re: Hive QA logs not accessible
Hi Vihang, I am looking for logs of failed test runs. Thanks for optimizing this for successful runs. However, I think there is a problem with Hive QA, the queue is gone and I submitted a patch more than 10 minutes ago and it hasn’t started or enqueued yet. https://builds.apache.org/view/H-L/view/Hive/job/PreCommit-HIVE-Build/ Regards, Deepak On 6/25/18, 10:53 AM, "Vihang Karajgaonkar" wrote: Are you looking for logs for successful tests? I had submitted a change recently which stops skips downloading logs for successful tests to shave off ~10 min time from each run. I found that the job was spending too much time copying over ~20G of logs from worker nodes to the server. Can you give the JIRA number so that I can take a look? On Mon, Jun 25, 2018 at 10:38 AM, Deepak Jaiswal wrote: > The Hive QA logs are not accessible for yesterday night’s run. Also, I > don’t see any test running. > Is the disk full again? > > Regards, > Deepak >
Re: Hive QA logs not accessible
Are you looking for logs for successful tests? I had submitted a change recently which stops skips downloading logs for successful tests to shave off ~10 min time from each run. I found that the job was spending too much time copying over ~20G of logs from worker nodes to the server. Can you give the JIRA number so that I can take a look? On Mon, Jun 25, 2018 at 10:38 AM, Deepak Jaiswal wrote: > The Hive QA logs are not accessible for yesterday night’s run. Also, I > don’t see any test running. > Is the disk full again? > > Regards, > Deepak >
Hive QA logs not accessible
The Hive QA logs are not accessible for yesterday night’s run. Also, I don’t see any test running. Is the disk full again? Regards, Deepak
[jira] [Created] (HIVE-19979) Backport HIVE-19304 to branch-3
Daniel Voros created HIVE-19979: --- Summary: Backport HIVE-19304 to branch-3 Key: HIVE-19979 URL: https://issues.apache.org/jira/browse/HIVE-19979 Project: Hive Issue Type: Task Reporter: Daniel Voros Assignee: Daniel Voros Needs HIVE-19978 (backport of HIVE-18037) to land first. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19978) Backport HIVE-18037 to branch-3
Daniel Voros created HIVE-19978: --- Summary: Backport HIVE-18037 to branch-3 Key: HIVE-19978 URL: https://issues.apache.org/jira/browse/HIVE-19978 Project: Hive Issue Type: Task Reporter: Daniel Voros Assignee: Daniel Voros -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-19977) Improve Output of SHOW PARTITIONS
BELUGA BEHR created HIVE-19977:
--
Summary: Improve Output of SHOW PARTITIONS
Key: HIVE-19977
URL: https://issues.apache.org/jira/browse/HIVE-19977
Project: Hive
Issue Type: Improvement
Components: Hive
Affects Versions: 3.0.0, 4.0.0
Reporter: BELUGA BEHR
{code:sql}
> create table partition_test (a string) partitioned by (b string, c string);
> insert into table partition_test partition (b='z', c='z') VALUES ('top');
> show partitions partition_test;
b=z/c=z
{code}
I think it would be more informative in a table format:
||b||z||
|z|z|
This clearly provides the information and prevents users from doing something
like...
{code:sql}
> alter table partition_test drop partition ("b=z/c=z");
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: New committer announcement : Marta Kuczora
Thank you all!! On Thu, Jun 21, 2018 at 9:03 AM Lefty Leverenz wrote: > Congratulations Marta! > > -- Lefty > > > On Thu, Jun 21, 2018 at 1:46 AM Prasanth Jayachandran < > pjayachand...@hortonworks.com> wrote: > > > Congratulations! > > > > Thanks > > Prasanth > > > > > > > > On Wed, Jun 20, 2018 at 10:44 PM -0700, "Vihang Karajgaonkar" > > mailto:vih...@cloudera.com.INVALID>> wrote: > > > > > > Congrats Marta! > > > > On Wed, Jun 20, 2018 at 8:46 PM, Zoltan Haindrich wrote: > > > > > Congratulations Márta! > > > > > > On 20 June 2018 22:20:30 CEST, Deepak Jaiswal > > > wrote: > > > >Congratulations Marta. > > > > > > > >On 6/20/18, 12:06 PM, "Ashutosh Chauhan" wrote: > > > > > > > >Apache Hive's Project Management Committee (PMC) has invited Marta > > > >Kuczora > > > >to become a committer, and we are pleased to announce that he has > > > >accepted. > > > > > > > >Marta, welcome, thank you for your contributions, and we look forward > > > >your > > > >further interactions with the community! > > > > > > > >Ashutosh Chauhan (on behalf of the Apache Hive PMC) > > > > > > > > > > > >
Re: Review Request 66667: HIVE-19046: Refactor the common parts of the HiveMetastore add_partition_core and add_partitions_pspec_core methods
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7/ --- (Updated June 25, 2018, 12:01 p.m.) Review request for hive, Peter Vary, Sahil Takiar, and Adam Szita. Changes --- Rebased the patch Bugs: HIVE-19046 https://issues.apache.org/jira/browse/HIVE-19046 Repository: hive-git Description --- The biggest part of these methods use the same code. Refactored these code parts to common methods. Diffs (updated) - standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java e9d7e7c standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/client/TestAddPartitions.java bf559b4 standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/client/TestAddPartitionsFromPartSpec.java 4f11a55 Diff: https://reviews.apache.org/r/7/diff/7/ Changes: https://reviews.apache.org/r/7/diff/6-7/ Testing --- Thanks, Marta Kuczora
[jira] [Created] (HIVE-19976) Fix TestMiniDruidKafkaCliDriver[druidkafkamini_basic] is flakiness
Peter Vary created HIVE-19976: - Summary: Fix TestMiniDruidKafkaCliDriver[druidkafkamini_basic] is flakiness Key: HIVE-19976 URL: https://issues.apache.org/jira/browse/HIVE-19976 Project: Hive Issue Type: Test Components: Druid integration Affects Versions: 4.0.0 Reporter: Peter Vary HIVE-19922, HIVE-19509 disabled TestMiniDruidKafkaCliDriver[druidkafkamini_basic] because of flakiness. This is a follow-up jira, to fix it and enable it back -- This message was sent by Atlassian JIRA (v7.6.3#76005)
