Build failed in Hudson: Hive-trunk-h0.18 #8
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/8/changes
--
[...truncated 18998 lines...]
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/build/ql/test/logs/negative/unknown_column2.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/src/test/results/compiler/errors/unknown_column2.q.out
[junit] Done query: unknown_column2.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902170624_585112001.txt
[junit] Begin query: unknown_column3.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/build/ql/test/logs/negative/unknown_column3.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/src/test/results/compiler/errors/unknown_column3.q.out
[junit] Done query: unknown_column3.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902170624_1570155427.txt
[junit] Begin query: unknown_column4.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/build/ql/test/logs/negative/unknown_column4.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/src/test/results/compiler/errors/unknown_column4.q.out
[junit] Done query: unknown_column4.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902170624_-809642801.txt
[junit] Begin query: unknown_column5.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/build/ql/test/logs/negative/unknown_column5.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/src/test/results/compiler/errors/unknown_column5.q.out
[junit] Done query: unknown_column5.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902170624_1534433669.txt
[junit] Begin query: unknown_column6.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/build/ql/test/logs/negative/unknown_column6.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/src/test/results/compiler/errors/unknown_column6.q.out
[junit] Done query: unknown_column6.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902170624_1525541092.txt
[junit] Begin query: unknown_function1.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition
Build failed in Hudson: Hive-trunk-h0.19 #7
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/7/changes
--
[...truncated 18617 lines...]
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/build/ql/test/logs/negative/unknown_column2.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/src/test/results/compiler/errors/unknown_column2.q.out
[junit] Done query: unknown_column2.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902170724_-61617800.txt
[junit] Begin query: unknown_column3.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/build/ql/test/logs/negative/unknown_column3.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/src/test/results/compiler/errors/unknown_column3.q.out
[junit] Done query: unknown_column3.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902170724_-338386608.txt
[junit] Begin query: unknown_column4.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/build/ql/test/logs/negative/unknown_column4.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/src/test/results/compiler/errors/unknown_column4.q.out
[junit] Done query: unknown_column4.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902170724_-1811458827.txt
[junit] Begin query: unknown_column5.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/build/ql/test/logs/negative/unknown_column5.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/src/test/results/compiler/errors/unknown_column5.q.out
[junit] Done query: unknown_column5.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902170724_-34330565.txt
[junit] Begin query: unknown_column6.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] OK
[junit] Loading data to table src
[junit] OK
[junit] diff
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/build/ql/test/logs/negative/unknown_column6.q.out
http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/src/test/results/compiler/errors/unknown_column6.q.out
[junit] Done query: unknown_column6.q
[junit] Hive history
file=http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/ws/hive/ql/../build/ql/tmp/hive_job_log_hudson_200902170724_1633814606.txt
[junit] Begin query: unknown_function1.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] OK
[junit] Loading data to table srcpart partition
[jira] Commented: (HIVE-79) Print number of rows inserted to table(s) when the query is finished.
[ https://issues.apache.org/jira/browse/HIVE-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12674327#action_12674327 ] Suresh Antony commented on HIVE-79: --- if so can I commit this patch. Print number of rows inserted to table(s) when the query is finished. -- Key: HIVE-79 URL: https://issues.apache.org/jira/browse/HIVE-79 Project: Hadoop Hive Issue Type: New Feature Components: Logging Reporter: Suresh Antony Assignee: Suresh Antony Priority: Minor Fix For: 0.2.0 Attachments: patch_79_1.txt, patch_79_2.txt, patch_79_3.txt It is good to print the number of rows inserted into each table at end of query. insert overwrite table tab1 select a.* from tab2 a where a.col1 = 10; This query can print something like: tab1 rows=100 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-270) Add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types
[ https://issues.apache.org/jira/browse/HIVE-270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Shao resolved HIVE-270. - Resolution: Fixed Fix Version/s: 0.3.0 Release Note: HIVE-270. Add a lazy-deserialized SerDe for efficient deserialization of rows with primitive types. (zshao) Hadoop Flags: [Reviewed] Committed to trunk only. Committed revision 745212. Add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types Key: HIVE-270 URL: https://issues.apache.org/jira/browse/HIVE-270 Project: Hadoop Hive Issue Type: New Feature Components: Serializers/Deserializers Reporter: Zheng Shao Assignee: Zheng Shao Fix For: 0.3.0 Attachments: HIVE-270.1.patch, HIVE-270.3.patch, HIVE-270.4.patch, HIVE-270.5.patch We want to add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types. This SerDe will share the same format as MetadataTypedColumnsetSerDe/TCTLSeparatedProtocol to be backward compatible. This SerDe will be used to replace the default table SerDe, and the SerDe used to communicate with user scripts. For simplicity, we don't plan to support nested structure with this SerDe. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Need help on Hive.g and parser!
Hello,
Someone please explain the following build.xml spec for grammar build
(required and not required) ::
===
uptodate property=grammarBuild.notRequired
srcfiles dir= ${src.dir}/org/apache/hadoop/hive/ql/parse
includes=**/*.g/
mapper type=merge
to=${build.dir.hive}/ql/gen-java/org/apache/hadoop/hive/ql/parse/HiveParser.java/
/uptodate
target name=build-grammar unless=grammarBuild.notRequired
echoBuilding Grammar ${src.dir}/org/apache/hadoop/hive/ql/parse/Hive.g
/echo
java classname=org.antlr.Tool classpathref=classpath fork=true
arg value=-fo /
arg
value=${build.dir.hive}/ql/gen-java/org/apache/hadoop/hive/ql/parse /
arg value=${src.dir}/org/apache/hadoop/hive/ql/parse/Hive.g /
/java
/target
=
Also can someone tell me which parser generator is used? I used JavaCC
in the past.
Thanks,
shyam_sar...@yahoo.com
*UNIT TEST FAILURE for apache HIVE* Hadoop.Version=0.17.1 based on SVN Rev# 745216.53
[junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED BUILD FAILED [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED BUILD FAILED
[jira] Commented: (HIVE-79) Print number of rows inserted to table(s) when the query is finished.
[ https://issues.apache.org/jira/browse/HIVE-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12674371#action_12674371 ] Joydeep Sen Sarma commented on HIVE-79: --- +1 Print number of rows inserted to table(s) when the query is finished. -- Key: HIVE-79 URL: https://issues.apache.org/jira/browse/HIVE-79 Project: Hadoop Hive Issue Type: New Feature Components: Logging Reporter: Suresh Antony Assignee: Suresh Antony Priority: Minor Fix For: 0.2.0 Attachments: patch_79_1.txt, patch_79_2.txt, patch_79_3.txt It is good to print the number of rows inserted into each table at end of query. insert overwrite table tab1 select a.* from tab2 a where a.col1 = 10; This query can print something like: tab1 rows=100 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Need help on Hive.g and parser!
We are using antlr.
Basically, the rule checks the timestamp of HiveParser.java. If it's newer
than Hive.g, then we don't need to regenerate HiveParse.java from Hive.g
again.
Zheng
On Tue, Feb 17, 2009 at 12:15 PM, Shyam Sarkar shyam_sar...@yahoo.comwrote:
Hello,
Someone please explain the following build.xml spec for grammar build
(required and not required) ::
===
uptodate property=grammarBuild.notRequired
srcfiles dir= ${src.dir}/org/apache/hadoop/hive/ql/parse
includes=**/*.g/
mapper type=merge
to=${build.dir.hive}/ql/gen-java/org/apache/hadoop/hive/ql/parse/HiveParser.java/
/uptodate
target name=build-grammar unless=grammarBuild.notRequired
echoBuilding Grammar ${src.dir}/org/apache/hadoop/hive/ql/parse/Hive.g
/echo
java classname=org.antlr.Tool classpathref=classpath fork=true
arg value=-fo /
arg
value=${build.dir.hive}/ql/gen-java/org/apache/hadoop/hive/ql/parse /
arg value=${src.dir}/org/apache/hadoop/hive/ql/parse/Hive.g /
/java
/target
=
Also can someone tell me which parser generator is used? I used JavaCC
in the past.
Thanks,
shyam_sar...@yahoo.com
--
Yours,
Zheng
[jira] Updated: (HIVE-223) when using map-side aggregates - perform single map-reduce group-by
[ https://issues.apache.org/jira/browse/HIVE-223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-223: Status: Patch Available (was: Open) fixed the bug that joy spotted, and also performed some code cleanup when using map-side aggregates - perform single map-reduce group-by --- Key: HIVE-223 URL: https://issues.apache.org/jira/browse/HIVE-223 Project: Hadoop Hive Issue Type: Improvement Components: Query Processor Reporter: Joydeep Sen Sarma Assignee: Namit Jain Attachments: 223.2.txt, 223.patch1.txt today even when we do map side aggregates - we do multiple map-reduce jobs. however - the reason for doing multiple map-reduce group-bys (for single group-bys) was the fear of skews. When we are doing map side aggregates - skews should not exist for the most part. There can be two reason for skews: - large number of entries for a single grouping set - map side aggregates should take care of this - badness in hash function that sends too much stuff to one reducer - we should be able to take care of this by having good hash functions (and prime number reducer counts) So i think we should be able to do a single stage map-reduce when doing map-side aggregates. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-223) when using map-side aggregates - perform single map-reduce group-by
[ https://issues.apache.org/jira/browse/HIVE-223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-223: Attachment: 223.2.txt when using map-side aggregates - perform single map-reduce group-by --- Key: HIVE-223 URL: https://issues.apache.org/jira/browse/HIVE-223 Project: Hadoop Hive Issue Type: Improvement Components: Query Processor Reporter: Joydeep Sen Sarma Assignee: Namit Jain Attachments: 223.2.txt, 223.patch1.txt today even when we do map side aggregates - we do multiple map-reduce jobs. however - the reason for doing multiple map-reduce group-bys (for single group-bys) was the fear of skews. When we are doing map side aggregates - skews should not exist for the most part. There can be two reason for skews: - large number of entries for a single grouping set - map side aggregates should take care of this - badness in hash function that sends too much stuff to one reducer - we should be able to take care of this by having good hash functions (and prime number reducer counts) So i think we should be able to do a single stage map-reduce when doing map-side aggregates. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
You are voted to be a Hive committer
Hi Johan, The Hadoop PMC has voted to make you a committer for the Hive subproject. Please complete and sign the ICLA at http://www.apache.org/licenses/icla.txtand fax it to the number specified in the form. Once the form is processed, you would be granted an apache account. thanks, dhruba
JIRA_223.2.txt_UNIT_TEST_SUCCEEDED
SUCCESS: BUILD AND UNIT TEST using PATCH 223.2.txt PASSED!!
