[incubator-hawq-site] Git Push Summary
Repository: incubator-hawq-site Updated Branches: refs/heads/HAWQ-683 [deleted] e45ad3097
incubator-hawq-site git commit: HAWQ-1554. Add registered trademark symbol (®) to website.
Repository: incubator-hawq-site Updated Branches: refs/heads/asf-site 5832adbbb -> bfcc79a5d HAWQ-1554. Add registered trademark symbol (®) to website. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/commit/bfcc79a5 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/tree/bfcc79a5 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/diff/bfcc79a5 Branch: refs/heads/asf-site Commit: bfcc79a5d5e1fc7ab945f57bfb77a368f8a3dd1e Parents: 5832adb Author: Ed Espino Authored: Mon Nov 27 00:00:08 2017 -0800 Committer: Ed Espino Committed: Mon Nov 27 00:00:08 2017 -0800 -- index.html | 44 ++-- 1 file changed, 22 insertions(+), 22 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/blob/bfcc79a5/index.html -- diff --git a/index.html b/index.html index 4a61fa3..c5731f1 100644 --- a/index.html +++ b/index.html @@ -10,7 +10,7 @@ http://hawq.incubator.apache.org/"/> -Apache HAWQ +Apache HAWQ® @@ -78,7 +78,7 @@ - + Apache Hadoop Native SQL. Advanced, MPP, elastic query engine and analytic database for enterprises. @@ -97,20 +97,20 @@ -Apache Hadoop Native SQL. Advanced Analytics MPP Database for Enterprises. +Apache HAWQ® (incubating) is Apache Hadoop Native SQL. Advanced Analytics MPP Database for Enterprises. -In a class by itself, only Apache HAWQ (incubating) combines exceptional MPP-based analytics performance, robust ANSI SQL compliance, Hadoop ecosystem integration and manageability, and flexible data-store format support. All natively in Hadoop. No connectors required. +In a class by itself, only Apache HAWQ® (incubating) combines exceptional MPP-based analytics performance, robust ANSI SQL compliance, Hadoop ecosystem integration and manageability, and flexible data-store format support. All natively in Hadoop. No connectors required. -Built from a decadeâs worth of massively parallel processing (MPP) expertise developed through the creation of the Pivotal Greenplum® enterprise database and open source PostgreSQL, HAWQ enables to you to swiftly and interactively query Hadoop data, natively via HDFS. +Built from a decadeâs worth of massively parallel processing (MPP) expertise developed through the creation of the Pivotal Greenplum® enterprise database and open source PostgreSQL, HAWQ® enables to you to swiftly and interactively query Hadoop data, natively via HDFS. -HAWQ is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the http://incubator.apache.org";>Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. +HAWQ® is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the http://incubator.apache.org";>Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. Exceptional performance - HAWQâs parallel processing architecture delivers high performance throughput and low latency - potentially near real time - query responses that can scale to petabyte-sized datasets. Operate natively in Hadoop. + HAWQ®âs parallel processing architecture delivers high performance throughput and low latency - potentially near real time - query responses that can scale to petabyte-sized datasets. Operate natively in Hadoop. Robust ANSI SQL compliance @@ -118,7 +118,7 @@ Hadoop ecosystem manageability and integration - Integrate and manage with YARN. Provision with Ambari. Interface with HCatalog. HAWQ supports Parquet, AVRO, HBase, and others. Easily scale nodes up or down to meet performance or capacity requirements. + Integrate and manage with YARN. Provision with Ambari. Interface with HCatalog. HAWQ® supports Parquet, AVRO, HBas
incubator-hawq git commit: HAWQ-1496. Add DISCLAIMER, LICENSE and NOTICE files into PXF rpm.
Repository: incubator-hawq Updated Branches: refs/heads/master 1aafa0c88 -> 8d18e6b25 HAWQ-1496. Add DISCLAIMER, LICENSE and NOTICE files into PXF rpm. The files will be installed into /opt/pxf-${project.version} directory. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/8d18e6b2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/8d18e6b2 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/8d18e6b2 Branch: refs/heads/master Commit: 8d18e6b25820618599946035c5b9f83f8d728c30 Parents: 1aafa0c Author: Ed Espino Authored: Wed Oct 4 15:06:06 2017 -0700 Committer: Ed Espino Committed: Tue Nov 7 21:21:38 2017 -0800 -- pxf/build.gradle | 12 1 file changed, 12 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/8d18e6b2/pxf/build.gradle -- diff --git a/pxf/build.gradle b/pxf/build.gradle index 78c5249..706caa9 100644 --- a/pxf/build.gradle +++ b/pxf/build.gradle @@ -336,6 +336,18 @@ project('pxf') { link('/etc/pxf', "/etc/pxf-${project.version}") link('/opt/pxf', "/opt/pxf-${project.version}") link('/etc/init.d/pxf-service', "/opt/pxf/pxf-service") + +from('../resources/META-INF/DISCLAIMER') { +into "/opt/pxf-${project.version}" +} + +from('../resources/META-INF/LICENSE') { +into "/opt/pxf-${project.version}" +} + +from('../resources/META-INF/NOTICE') { +into "/opt/pxf-${project.version}" +} } }
svn commit: r21274 - /dev/incubator/hawq/2.0.0.0-incubating/
Author: espino Date: Wed Aug 23 06:19:19 2017 New Revision: 21274 Log: Cleanup: remove 2.0.0.0-incubating dev release. Removed: dev/incubator/hawq/2.0.0.0-incubating/
svn commit: r21280 - /dev/incubator/hawq/2.1.0.0-incubating.RC4/
Author: espino Date: Wed Aug 23 06:20:59 2017 New Revision: 21280 Log: Cleanup: remove 2.1.0.0-incubating.RC4 dev release. Removed: dev/incubator/hawq/2.1.0.0-incubating.RC4/
svn commit: r21278 - /dev/incubator/hawq/2.1.0.0-incubating.RC2/
Author: espino Date: Wed Aug 23 06:20:38 2017 New Revision: 21278 Log: Cleanup: remove 2.1.0.0-incubating.RC2 dev release. Removed: dev/incubator/hawq/2.1.0.0-incubating.RC2/
svn commit: r21279 - /dev/incubator/hawq/2.1.0.0-incubating.RC3/
Author: espino Date: Wed Aug 23 06:20:47 2017 New Revision: 21279 Log: Cleanup: remove 2.1.0.0-incubating.RC3 dev release. Removed: dev/incubator/hawq/2.1.0.0-incubating.RC3/
svn commit: r21277 - /dev/incubator/hawq/2.1.0.0-incubating.RC1/
Author: espino Date: Wed Aug 23 06:20:30 2017 New Revision: 21277 Log: Cleanup: remove 2.1.0.0-incubating.RC1 dev release. Removed: dev/incubator/hawq/2.1.0.0-incubating.RC1/
svn commit: r21275 - /dev/incubator/hawq/2.0.0.0-incubating.RC2/
Author: espino Date: Wed Aug 23 06:19:50 2017 New Revision: 21275 Log: Cleanup: remove 2.0.0.0-incubating.RC2 dev release. Removed: dev/incubator/hawq/2.0.0.0-incubating.RC2/
svn commit: r21276 - /dev/incubator/hawq/2.0.0.0-incubating.RC3/
Author: espino Date: Wed Aug 23 06:20:10 2017 New Revision: 21276 Log: Cleanup: remove 2.0.0.0-incubating.RC3 dev release. Removed: dev/incubator/hawq/2.0.0.0-incubating.RC3/
svn commit: r21272 - /dev/incubator/hawq/2.0.0-beta-incubating.RC4/
Author: espino Date: Wed Aug 23 05:18:50 2017 New Revision: 21272 Log: Cleanup: remove 2.0.0-beta-incubating.RC4 dev release. Removed: dev/incubator/hawq/2.0.0-beta-incubating.RC4/
svn commit: r21271 - /dev/incubator/hawq/2.0.0-beta-incubating.RC3/
Author: espino Date: Wed Aug 23 05:18:26 2017 New Revision: 21271 Log: Cleanup: remove 2.0.0-beta-incubating.RC3 dev release. Removed: dev/incubator/hawq/2.0.0-beta-incubating.RC3/
incubator-hawq git commit: HAWQ-1499. Update README.md Travis CI status to report only master branch status.
Repository: incubator-hawq Updated Branches: refs/heads/master f6bfaaacc -> 7e4edd4d2 HAWQ-1499. Update README.md Travis CI status to report only master branch status. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/7e4edd4d Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/7e4edd4d Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/7e4edd4d Branch: refs/heads/master Commit: 7e4edd4d22687bda04119408cf4ba9517c457096 Parents: f6bfaaa Author: Ed Espino Authored: Thu Jul 6 12:31:59 2017 -0700 Committer: Ed Espino Committed: Thu Jul 6 12:31:59 2017 -0700 -- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/7e4edd4d/README.md -- diff --git a/README.md b/README.md index 5bf1e1b..3e1333c 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ |CI Process|Status| |---|---| -|Travis CI Build|[![https://travis-ci.org/apache/incubator-hawq.png](https://travis-ci.org/apache/incubator-hawq.png)](https://travis-ci.org/apache/incubator-hawq)| +|Travis CI Build|[![https://travis-ci.org/apache/incubator-hawq.svg?branch=master](https://travis-ci.org/apache/incubator-hawq.png?branch=master)](https://travis-ci.org/apache/incubator-hawq?branch=master)| |Apache Release Audit Tool ([RAT](https://creadur.apache.org/rat/))|[![Rat Status](https://builds.apache.org/buildStatus/icon?job=HAWQ-rat)](https://builds.apache.org/view/HAWQ/job/HAWQ-rat/)| |Coverity Static Analysis |[![Coverity Scan Build](https://scan.coverity.com/projects/apache-incubator-hawq/badge.svg)](https://scan.coverity.com/projects/apache-incubator-hawq)|
[incubator-hawq] Git Push Summary
Repository: incubator-hawq Updated Tags: refs/tags/rel/v2.1.0.0-incubating [created] f42f25b33
incubator-hawq git commit: HAWQ-1489. Add LICENSE, DISCLAIMER and NOTICE to pxf jar files.
Repository: incubator-hawq Updated Branches: refs/heads/2.2.0.0-incubating 32e01c76b -> fbc03d82e HAWQ-1489. Add LICENSE, DISCLAIMER and NOTICE to pxf jar files. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/fbc03d82 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/fbc03d82 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/fbc03d82 Branch: refs/heads/2.2.0.0-incubating Commit: fbc03d82e653e535cb634994d606673f13cb45bb Parents: 32e01c7 Author: Ed Espino Authored: Thu Jun 22 21:35:51 2017 -0700 Committer: Ed Espino Committed: Thu Jun 22 21:35:51 2017 -0700 -- pxf/build.gradle | 10 + pxf/resources/META-INF/DISCLAIMER | 11 + pxf/resources/META-INF/LICENSE| 380 + pxf/resources/META-INF/NOTICE | 5 + 4 files changed, 406 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/fbc03d82/pxf/build.gradle -- diff --git a/pxf/build.gradle b/pxf/build.gradle index 802b1e4..88085cc 100644 --- a/pxf/build.gradle +++ b/pxf/build.gradle @@ -79,6 +79,16 @@ subprojects { subProject -> "-Xlint:-fallthrough", "-Xlint:-deprecation", "-Xlint:-unchecked", "-Xlint:-options" ] +// Add LICENSE, DISCLAIMER and NOTICE to generated jar files. +sourceSets { +main { +resources { +srcDir '../resources' +include '**/*' +} +} +} + repositories { // mavenCentral without https: maven { url 'http://repo1.maven.org/maven2' } http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/fbc03d82/pxf/resources/META-INF/DISCLAIMER -- diff --git a/pxf/resources/META-INF/DISCLAIMER b/pxf/resources/META-INF/DISCLAIMER new file mode 100644 index 000..8c2058c --- /dev/null +++ b/pxf/resources/META-INF/DISCLAIMER @@ -0,0 +1,11 @@ +Apache HAWQ is an effort undergoing incubation at the Apache Software +Foundation (ASF), sponsored by the Apache Incubator PMC. + +Incubation is required of all newly accepted projects until a further +review indicates that the infrastructure, communications, and decision +making process have stabilized in a manner consistent with other +successful ASF projects. + +While incubation status is not necessarily a reflection of the +completeness or stability of the code, it does indicate that the +project has yet to be fully endorsed by the ASF. http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/fbc03d82/pxf/resources/META-INF/LICENSE -- diff --git a/pxf/resources/META-INF/LICENSE b/pxf/resources/META-INF/LICENSE new file mode 100644 index 000..fed4f91 --- /dev/null +++ b/pxf/resources/META-INF/LICENSE @@ -0,0 +1,380 @@ + Apache License + Version 2.0, January 2004 +http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or +
incubator-hawq git commit: HAWQ-1489. Add LICENSE, DISCLAIMER and NOTICE to pxf jar files.
Repository: incubator-hawq Updated Branches: refs/heads/master 7df7f96f2 -> f157d59fa HAWQ-1489. Add LICENSE, DISCLAIMER and NOTICE to pxf jar files. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/f157d59f Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/f157d59f Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/f157d59f Branch: refs/heads/master Commit: f157d59fa8d16329ba0166e36085980b1a0ef6ff Parents: 7df7f96 Author: Ed Espino Authored: Thu Jun 22 16:16:43 2017 -0700 Committer: Ed Espino Committed: Thu Jun 22 16:16:43 2017 -0700 -- pxf/build.gradle | 10 + pxf/resources/META-INF/DISCLAIMER | 11 + pxf/resources/META-INF/LICENSE| 380 + pxf/resources/META-INF/NOTICE | 5 + 4 files changed, 406 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/f157d59f/pxf/build.gradle -- diff --git a/pxf/build.gradle b/pxf/build.gradle index 80d30df..213b2dd 100644 --- a/pxf/build.gradle +++ b/pxf/build.gradle @@ -79,6 +79,16 @@ subprojects { subProject -> "-Xlint:-fallthrough", "-Xlint:-deprecation", "-Xlint:-unchecked", "-Xlint:-options" ] +// Add LICENSE, DISCLAIMER and NOTICE to generated jar files. +sourceSets { +main { +resources { +srcDir '../resources' +include '**/*' +} +} +} + repositories { // mavenCentral without https: maven { url 'http://repo1.maven.org/maven2' } http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/f157d59f/pxf/resources/META-INF/DISCLAIMER -- diff --git a/pxf/resources/META-INF/DISCLAIMER b/pxf/resources/META-INF/DISCLAIMER new file mode 100644 index 000..8c2058c --- /dev/null +++ b/pxf/resources/META-INF/DISCLAIMER @@ -0,0 +1,11 @@ +Apache HAWQ is an effort undergoing incubation at the Apache Software +Foundation (ASF), sponsored by the Apache Incubator PMC. + +Incubation is required of all newly accepted projects until a further +review indicates that the infrastructure, communications, and decision +making process have stabilized in a manner consistent with other +successful ASF projects. + +While incubation status is not necessarily a reflection of the +completeness or stability of the code, it does indicate that the +project has yet to be fully endorsed by the ASF. http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/f157d59f/pxf/resources/META-INF/LICENSE -- diff --git a/pxf/resources/META-INF/LICENSE b/pxf/resources/META-INF/LICENSE new file mode 100644 index 000..fed4f91 --- /dev/null +++ b/pxf/resources/META-INF/LICENSE @@ -0,0 +1,380 @@ + Apache License + Version 2.0, January 2004 +http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made a
incubator-hawq git commit: HAWQ-1481. Rework Maven "3" major version check.
Repository: incubator-hawq Updated Branches: refs/heads/master ed5029c76 -> 3b55bfd67 HAWQ-1481. Rework Maven "3" major version check. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/3b55bfd6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/3b55bfd6 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/3b55bfd6 Branch: refs/heads/master Commit: 3b55bfd67956ca71f5605ebfdabfa30a04868728 Parents: ed5029c Author: Ed Espino Authored: Mon Jun 5 23:49:56 2017 -0700 Committer: Ed Espino Committed: Mon Jun 5 23:49:56 2017 -0700 -- config/maven.m4 | 8 configure | 6 -- 2 files changed, 8 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/3b55bfd6/config/maven.m4 -- diff --git a/config/maven.m4 b/config/maven.m4 index a9ccef7..28320a7 100644 --- a/config/maven.m4 +++ b/config/maven.m4 @@ -13,9 +13,11 @@ fi if test "$MAVEN"; then pgac_maven_version=`$MAVEN --version 2>/dev/null | sed q | $AWK '{print [$]1" "[$]2" "[$]3;}'` + pgac_maven_major_version=`$MAVEN --version 2>/dev/null | sed q | $AWK '{print [$]3;}' | sed -e 's|\([[0-9]]\).*|\1|g'` + AC_MSG_NOTICE([using $pgac_maven_version]) - if echo "$pgac_maven_version" | sed ['s/[^0-9]/ /g'] | $AWK '{ if ([$]1 < 3) exit 0; else exit 1;}' - then + + if test $pgac_maven_major_version -lt 3; then AC_MSG_WARN([ *** The installed version of Maven, $MAVEN, is too old to use with HAWQ. *** Maven version 3.0.0 or later is required, but this is $pgac_maven_version.]) @@ -31,5 +33,3 @@ fi # We don't need AC_SUBST(MAVEN) because AC_PATH_PROG did it. AC_SUBST(MAVEN) ])# PGAC_PATH_MAVEN - - http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/3b55bfd6/configure -- diff --git a/configure b/configure index 4230fa1..ddd57de 100755 --- a/configure +++ b/configure @@ -7587,10 +7587,12 @@ fi if test "$MAVEN"; then pgac_maven_version=`$MAVEN --version 2>/dev/null | sed q | $AWK '{print $1" "$2" "$3;}'` + pgac_maven_major_version=`$MAVEN --version 2>/dev/null | sed q | $AWK '{print $3;}' | sed -e 's|\([0-9]\).*|\1|g'` + { $as_echo "$as_me:${as_lineno-$LINENO}: using $pgac_maven_version" >&5 $as_echo "$as_me: using $pgac_maven_version" >&6;} - if echo "$pgac_maven_version" | sed 's/[^0-9]/ /g' | $AWK '{ if ($1 < 3) exit 0; else exit 1;}' - then + + if test $pgac_maven_major_version -lt 3; then { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: *** The installed version of Maven, $MAVEN, is too old to use with HAWQ. *** Maven version 3.0.0 or later is required, but this is $pgac_maven_version." >&5
incubator-hawq-site git commit: Update ASF copyright year and introduce new Apache Incubator logo.
Repository: incubator-hawq-site Updated Branches: refs/heads/asf-site 0a7945626 -> 3cfac27fd Update ASF copyright year and introduce new Apache Incubator logo. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/commit/3cfac27f Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/tree/3cfac27f Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/diff/3cfac27f Branch: refs/heads/asf-site Commit: 3cfac27fd5cd87ea55b2414c15b8c2e1a3de27f7 Parents: 0a79456 Author: Ed Espino Authored: Sat Apr 22 01:56:22 2017 +0800 Committer: Ed Espino Committed: Sat Apr 22 01:56:22 2017 +0800 -- images/incubator_feather_egg_logo.png | Bin 0 -> 30207 bytes index.html| 4 ++-- 2 files changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/blob/3cfac27f/images/incubator_feather_egg_logo.png -- diff --git a/images/incubator_feather_egg_logo.png b/images/incubator_feather_egg_logo.png new file mode 100644 index 000..338169e Binary files /dev/null and b/images/incubator_feather_egg_logo.png differ http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/blob/3cfac27f/index.html -- diff --git a/index.html b/index.html index 9072ebc..746f1cb 100644 --- a/index.html +++ b/index.html @@ -429,7 +429,7 @@ - + @@ -437,7 +437,7 @@ newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. - Copyright © 2016 The Apache Software Foundation + Copyright © 2016-2017 The Apache Software Foundation Apache, Apache HAWQ (incubating), the Apache feather and the HAWQ logo are trademarks of The Apache Software Foundation
incubator-hawq git commit: HAWQ-1415. Set the default_value of JAVA_HOME for running RPS (closes #1206).
Repository: incubator-hawq Updated Branches: refs/heads/2.2.0.0-incubating fe6c0e9c8 -> dc41e4c51 HAWQ-1415. Set the default_value of JAVA_HOME for running RPS (closes #1206). Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/dc41e4c5 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/dc41e4c5 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/dc41e4c5 Branch: refs/heads/2.2.0.0-incubating Commit: dc41e4c5131351a5260788e14f6cebd611e4762a Parents: fe6c0e9 Author: ljainpivotalio Authored: Tue Mar 28 19:05:52 2017 -0700 Committer: Ed Espino Committed: Tue Mar 28 20:03:41 2017 -0700 -- ranger-plugin/conf/rps.properties | 3 +++ ranger-plugin/scripts/enable-ranger-plugin.sh | 25 +- ranger-plugin/scripts/rps.sh | 1 + 3 files changed, 28 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/dc41e4c5/ranger-plugin/conf/rps.properties -- diff --git a/ranger-plugin/conf/rps.properties b/ranger-plugin/conf/rps.properties index 81a6986..7565885 100644 --- a/ranger-plugin/conf/rps.properties +++ b/ranger-plugin/conf/rps.properties @@ -36,3 +36,6 @@ RPS_HEAP_SIZE=128m # use CATALINA_HOME in default and use /usr/lib/bigtop-tomcat if CATALINA_HOME not set CATALINA_HOME=${CATALINA_HOME:-/usr/lib/bigtop-tomcat} + +# use JAVA_HOME in default and use /usr/java/default if JAVA_HOME not set +JAVA_HOME=${JAVA_HOME:-/usr/java/default} http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/dc41e4c5/ranger-plugin/scripts/enable-ranger-plugin.sh -- diff --git a/ranger-plugin/scripts/enable-ranger-plugin.sh b/ranger-plugin/scripts/enable-ranger-plugin.sh index 05a0b1d..2fa7162 100755 --- a/ranger-plugin/scripts/enable-ranger-plugin.sh +++ b/ranger-plugin/scripts/enable-ranger-plugin.sh @@ -150,7 +150,7 @@ function validate_params() { get_hawq_password echo "RANGER URL = ${RANGER_URL}" echo "RANGER User = ${RANGER_USER}" - echo "RANGER Password = $(mask ${RANGER_PASSWORD})" + echo "RANGER Password = $(mask ${RANGER_PASSWORD})" echo "HAWQ HOST = ${HAWQ_HOST}" echo "HAWQ PORT = ${HAWQ_PORT}" echo "HAWQ User = ${HAWQ_USER}" @@ -211,6 +211,28 @@ function update_ranger_url() { echo "Updated POLICY_MGR_URL to ${policy_mgr_url} in ${prop_file}" } +function update_java_home() { + local jdk64="/usr/jdk64" + local java_sdk="/etc/alternatives/java_sdk" + + if [[ -d ${jdk64} ]]; then +local DIR_NAME=$(ls ${jdk64} | sort -r | head -1) +if [[ ${DIR_NAME} ]]; then + JAVA_HOME_DIR="${jdk64}/${DIR_NAME}" +fi + elif [[ -d ${java_sdk} ]]; then +JAVA_HOME_DIR="${java_sdk}" + fi + + if [[ ${JAVA_HOME_DIR} ]]; then +local prop_file=$(dirname ${SCRIPT_DIR})/etc/rps.properties +sed -i -e "s|/usr/java/default|${JAVA_HOME_DIR}|g" ${prop_file} +echo "Updated default value of JAVA_HOME to ${JAVA_HOME_DIR} in ${prop_file}" + elif [[ ! ${JAVA_HOME} ]]; then +echo "Unable to locate JAVA_HOME on this machine. Please modify the default value of JAVA_HOME in ${prop_file}." + fi +} + main() { if [[ $# -lt 1 ]]; then usage @@ -221,5 +243,6 @@ main() { create_hawq_service_definition create_hawq_service_instance update_ranger_url + update_java_home } main "$@" http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/dc41e4c5/ranger-plugin/scripts/rps.sh -- diff --git a/ranger-plugin/scripts/rps.sh b/ranger-plugin/scripts/rps.sh index fa268c9..70aa547 100755 --- a/ranger-plugin/scripts/rps.sh +++ b/ranger-plugin/scripts/rps.sh @@ -32,6 +32,7 @@ BASEDIR=$( dirname ${CWDIR} ) # read properties from the file source ${BASEDIR}/etc/rps.properties +export JAVA_HOME export CATALINA_HOME export CATALINA_BASE=${BASEDIR}/plugin-service export CATALINA_PID=${CATALINA_BASE}/work/rps.pid
incubator-hawq git commit: HAWQ-1415. Set the default_value of JAVA_HOME for running RPS (closes #1206).
Repository: incubator-hawq Updated Branches: refs/heads/master fe6c0e9c8 -> dc41e4c51 HAWQ-1415. Set the default_value of JAVA_HOME for running RPS (closes #1206). Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/dc41e4c5 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/dc41e4c5 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/dc41e4c5 Branch: refs/heads/master Commit: dc41e4c5131351a5260788e14f6cebd611e4762a Parents: fe6c0e9 Author: ljainpivotalio Authored: Tue Mar 28 19:05:52 2017 -0700 Committer: Ed Espino Committed: Tue Mar 28 20:03:41 2017 -0700 -- ranger-plugin/conf/rps.properties | 3 +++ ranger-plugin/scripts/enable-ranger-plugin.sh | 25 +- ranger-plugin/scripts/rps.sh | 1 + 3 files changed, 28 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/dc41e4c5/ranger-plugin/conf/rps.properties -- diff --git a/ranger-plugin/conf/rps.properties b/ranger-plugin/conf/rps.properties index 81a6986..7565885 100644 --- a/ranger-plugin/conf/rps.properties +++ b/ranger-plugin/conf/rps.properties @@ -36,3 +36,6 @@ RPS_HEAP_SIZE=128m # use CATALINA_HOME in default and use /usr/lib/bigtop-tomcat if CATALINA_HOME not set CATALINA_HOME=${CATALINA_HOME:-/usr/lib/bigtop-tomcat} + +# use JAVA_HOME in default and use /usr/java/default if JAVA_HOME not set +JAVA_HOME=${JAVA_HOME:-/usr/java/default} http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/dc41e4c5/ranger-plugin/scripts/enable-ranger-plugin.sh -- diff --git a/ranger-plugin/scripts/enable-ranger-plugin.sh b/ranger-plugin/scripts/enable-ranger-plugin.sh index 05a0b1d..2fa7162 100755 --- a/ranger-plugin/scripts/enable-ranger-plugin.sh +++ b/ranger-plugin/scripts/enable-ranger-plugin.sh @@ -150,7 +150,7 @@ function validate_params() { get_hawq_password echo "RANGER URL = ${RANGER_URL}" echo "RANGER User = ${RANGER_USER}" - echo "RANGER Password = $(mask ${RANGER_PASSWORD})" + echo "RANGER Password = $(mask ${RANGER_PASSWORD})" echo "HAWQ HOST = ${HAWQ_HOST}" echo "HAWQ PORT = ${HAWQ_PORT}" echo "HAWQ User = ${HAWQ_USER}" @@ -211,6 +211,28 @@ function update_ranger_url() { echo "Updated POLICY_MGR_URL to ${policy_mgr_url} in ${prop_file}" } +function update_java_home() { + local jdk64="/usr/jdk64" + local java_sdk="/etc/alternatives/java_sdk" + + if [[ -d ${jdk64} ]]; then +local DIR_NAME=$(ls ${jdk64} | sort -r | head -1) +if [[ ${DIR_NAME} ]]; then + JAVA_HOME_DIR="${jdk64}/${DIR_NAME}" +fi + elif [[ -d ${java_sdk} ]]; then +JAVA_HOME_DIR="${java_sdk}" + fi + + if [[ ${JAVA_HOME_DIR} ]]; then +local prop_file=$(dirname ${SCRIPT_DIR})/etc/rps.properties +sed -i -e "s|/usr/java/default|${JAVA_HOME_DIR}|g" ${prop_file} +echo "Updated default value of JAVA_HOME to ${JAVA_HOME_DIR} in ${prop_file}" + elif [[ ! ${JAVA_HOME} ]]; then +echo "Unable to locate JAVA_HOME on this machine. Please modify the default value of JAVA_HOME in ${prop_file}." + fi +} + main() { if [[ $# -lt 1 ]]; then usage @@ -221,5 +243,6 @@ main() { create_hawq_service_definition create_hawq_service_instance update_ranger_url + update_java_home } main "$@" http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/dc41e4c5/ranger-plugin/scripts/rps.sh -- diff --git a/ranger-plugin/scripts/rps.sh b/ranger-plugin/scripts/rps.sh index fa268c9..70aa547 100755 --- a/ranger-plugin/scripts/rps.sh +++ b/ranger-plugin/scripts/rps.sh @@ -32,6 +32,7 @@ BASEDIR=$( dirname ${CWDIR} ) # read properties from the file source ${BASEDIR}/etc/rps.properties +export JAVA_HOME export CATALINA_HOME export CATALINA_BASE=${BASEDIR}/plugin-service export CATALINA_PID=${CATALINA_BASE}/work/rps.pid
incubator-hawq git commit: HAWQ-1397. Support Flex versions 2.6 and above.
Repository: incubator-hawq Updated Branches: refs/heads/master efaf24ea4 -> c89b3e2ca HAWQ-1397. Support Flex versions 2.6 and above. * The FLEX configure snippet came from Postgres upstream in the form of two commits (see below). * Misc change: commit tomcat version check for Mac OSX platform using brew. This change was out of sync with generated "configure" script. commit 7d7b129277eb545286aecf29ec22b5bb7fdf46bd Author: Tom Lane Date: Mon May 2 11:18:10 2016 -0400 Fix configure's incorrect version tests for flex and perl. awk's equality-comparison operator is "==" not "=". We got this right in many places, but not in configure's checks for supported version numbers of flex and perl. It hadn't been noticed because unsupported versions are so old as to be basically extinct in the wild, and because the only consequence is whether or not a WARNING flies by during configure. Daniel Gustafsson noted the problem with respect to the test for flex, I found the other by reviewing other awk calls. commit 32f15d05c80044335f97347b5406f6736c06a033 Author: Tom Lane Date: Wed Nov 18 17:45:05 2015 -0500 Accept flex > 2.5.x in configure. Per buildfarm member anchovy, 2.6.0 exists in the wild now. Hopefully it works with Postgres; if not, we'll have to do something about that, but in any case claiming it's "too old" is pretty silly. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/c89b3e2c Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/c89b3e2c Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/c89b3e2c Branch: refs/heads/master Commit: c89b3e2cafb099e46661a477f5fe64b908791824 Parents: efaf24e Author: Ed Espino Authored: Tue Mar 21 10:30:33 2017 -0700 Committer: Ed Espino Committed: Tue Mar 21 20:57:52 2017 -0700 -- config/programs.m4 | 2 +- config/tomcat.m4 | 2 ++ configure | 2 +- 3 files changed, 4 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/c89b3e2c/config/programs.m4 -- diff --git a/config/programs.m4 b/config/programs.m4 index 0ebfc65..637f47a 100644 --- a/config/programs.m4 +++ b/config/programs.m4 @@ -69,7 +69,7 @@ else echo '%%' > conftest.l if $pgac_candidate -t conftest.l 2>/dev/null | grep FLEX_SCANNER >/dev/null 2>&1; then pgac_flex_version=`$pgac_candidate --version 2>/dev/null` - if echo "$pgac_flex_version" | sed ['s/[^0-9]/ /g'] | $AWK '{ if ([$]1 = 2 && [$]2 = 5 && [$]3 >= 4) exit 0; else exit 1;}' + if echo "$pgac_flex_version" | sed ['s/[.a-z]/ /g'] | $AWK '{ if ([$]1 == 2 && ([$]2 > 5 || ([$]2 == 5 && [$]3 >= 4))) exit 0; else exit 1;}' then pgac_cv_path_flex=$pgac_candidate break 2 http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/c89b3e2c/config/tomcat.m4 -- diff --git a/config/tomcat.m4 b/config/tomcat.m4 index 9e122a3..f2d744a 100644 --- a/config/tomcat.m4 +++ b/config/tomcat.m4 @@ -13,6 +13,8 @@ AC_DEFUN([PGAC_CATALINA_HOME], dnl /usr/lib/bigtop-tomcat if test -x "${CATALINA_HOME}/bin/catalina.sh"; then TOMCAT="${CATALINA_HOME}" + elif test -x "/usr/local/Cellar/tomcat@6/6.0.45/libexec/bin/catalina.sh"; then +TOMCAT="/usr/local/Cellar/tomcat@6/6.0.45/libexec/" elif test -x "/usr/lib/bigtop-tomcat/bin/catalina.sh"; then TOMCAT="/usr/lib/bigtop-tomcat/" else http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/c89b3e2c/configure -- diff --git a/configure b/configure index 14f88e4..4230fa1 100755 --- a/configure +++ b/configure @@ -7488,7 +7488,7 @@ else echo '%%' > conftest.l if $pgac_candidate -t conftest.l 2>/dev/null | grep FLEX_SCANNER >/dev/null 2>&1; then pgac_flex_version=`$pgac_candidate --version 2>/dev/null` - if echo "$pgac_flex_version" | sed 's/[^0-9]/ /g' | $AWK '{ if ($1 = 2 && $2 = 5 && $3 >= 4) exit 0; else exit 1;}' + if echo "$pgac_flex_version" | sed 's/[.a-z]/ /g' | $AWK '{ if ($1 == 2 && ($2 > 5 || ($2 == 5 && $3 >= 4))) exit 0; else exit 1;}' then pgac_cv_path_flex=$pgac_candidate break 2
incubator-hawq git commit: HAWQ-1387. Add support for CentOS/RHEL 7 in HAWQ Ambari plugin.
Repository: incubator-hawq Updated Branches: refs/heads/master 4eb6f0ab0 -> 13df9e1a7 HAWQ-1387. Add support for CentOS/RHEL 7 in HAWQ Ambari plugin. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/13df9e1a Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/13df9e1a Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/13df9e1a Branch: refs/heads/master Commit: 13df9e1a7cba62e4d3779ce1b66fd90bf1f4e3ac Parents: 4eb6f0a Author: Ed Espino Authored: Wed Mar 15 15:58:07 2017 -0700 Committer: Ed Espino Committed: Wed Mar 15 15:58:07 2017 -0700 -- contrib/hawq-ambari-plugin/src/main/resources/utils/add-hawq.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/13df9e1a/contrib/hawq-ambari-plugin/src/main/resources/utils/add-hawq.py -- diff --git a/contrib/hawq-ambari-plugin/src/main/resources/utils/add-hawq.py b/contrib/hawq-ambari-plugin/src/main/resources/utils/add-hawq.py index d8ba361..4ae07a7 100755 --- a/contrib/hawq-ambari-plugin/src/main/resources/utils/add-hawq.py +++ b/contrib/hawq-ambari-plugin/src/main/resources/utils/add-hawq.py @@ -30,7 +30,7 @@ from optparse import OptionParser PLUGIN_VERSION = '${release}' DEFAULT_STACK = '${default.stack}' -SUPPORTED_OS_LIST = ['redhat6'] +SUPPORTED_OS_LIST = ['redhat6', 'redhat7'] HAWQ_LIB_STAGING_DIR = '${hawq.lib.staging.dir}' REPO_VERSION = '${repository.version}' HAWQ_REPO = '${hawq.repo.prefix}' @@ -496,4 +496,4 @@ def main(): if __name__ == '__main__': - main() \ No newline at end of file + main()
incubator-hawq git commit: Add diagnostic scriptlet to clean up Apache Jenkins slave.
Repository: incubator-hawq Updated Branches: refs/heads/master 1ad7dcdf4 -> e5bb0d99b Add diagnostic scriptlet to clean up Apache Jenkins slave. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/e5bb0d99 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/e5bb0d99 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/e5bb0d99 Branch: refs/heads/master Commit: e5bb0d99b83c3d54bdcc6daceb4a4a744cd36637 Parents: 1ad7dcd Author: Ed Espino Authored: Wed Mar 8 11:33:52 2017 -0800 Committer: Ed Espino Committed: Wed Mar 8 11:33:52 2017 -0800 -- contrib/jenkins/HAWQ-build-pullrequest.sh | 37 ++ 1 file changed, 37 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/e5bb0d99/contrib/jenkins/HAWQ-build-pullrequest.sh -- diff --git a/contrib/jenkins/HAWQ-build-pullrequest.sh b/contrib/jenkins/HAWQ-build-pullrequest.sh index d5d2440..3f99af9 100644 --- a/contrib/jenkins/HAWQ-build-pullrequest.sh +++ b/contrib/jenkins/HAWQ-build-pullrequest.sh @@ -26,6 +26,43 @@ work_dir=$(pwd) echo "working dir is ${work_dir}" +## -- +## Diagnostic scriptlet to fix cloning permission denied issue for HAWQ +## PR Project (HAWQ-build-pullrequest). +## +## 0) Open two browser "configure" tab's for project: +## +##https://builds.apache.org/job/HAWQ-build-pullrequest/configure +## +##One will be used to make the updates to the project's +##configuration that will be reverted by the second unmodified tab +##when the slave has been fixed. +## +## 1) Change "Source Code Management" to None +## 2) Uncomment to following section +## 3) Ensure the "Restrict where this project can be run" --> "Label +##Expression" specifies the slave with the build issue. +## 4) Manually trigger project "HAWQ-build-pullrequest" +## 5) Restore project in 2nd "configure" browser tab. +## +## -- + +## ls -al +## +## # Remove previous source artifacts from slave. The operation is +## # performed in the container to have sufficient privileges. +## +## docker run -v $(pwd):/data/hawq-src \ +##-u root rlei/mydocker:centos7-build \ +##/bin/sh -c "rm -rf /data/hawq-src/*" +## +## ls -al +## exit 1 + +## -- +## End Diagnositc scriptlet +## -- + git diff git log -n 5 git status
incubator-hawq git commit: HAWQ-1369. Update for RAT status and external doc link in README.md
Repository: incubator-hawq Updated Branches: refs/heads/master aece96726 -> 1470d5302 HAWQ-1369. Update for RAT status and external doc link in README.md Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/1470d530 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/1470d530 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/1470d530 Branch: refs/heads/master Commit: 1470d5302e956a46674c233d0e3723c7d936b5d6 Parents: aece967 Author: Ed Espino Authored: Thu Jan 12 16:20:24 2017 +0800 Committer: Ed Espino Committed: Thu Mar 2 00:40:26 2017 -0800 -- README.md | 16 +--- 1 file changed, 13 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/1470d530/README.md -- diff --git a/README.md b/README.md index 8237435..5bf1e1b 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,18 @@ -![HAWQ](http://hawq.incubator.apache.org/images/logo-hawq.png) [![https://travis-ci.org/apache/incubator-hawq.png](https://travis-ci.org/apache/incubator-hawq.png)](https://travis-ci.org/apache/incubator-hawq) [![Coverity Scan Build](https://scan.coverity.com/projects/apache-incubator-hawq/badge.svg)](https://scan.coverity.com/projects/apache-incubator-hawq) +![HAWQ](http://hawq.incubator.apache.org/images/logo-hawq.png) + +--- + +|CI Process|Status| +|---|---| +|Travis CI Build|[![https://travis-ci.org/apache/incubator-hawq.png](https://travis-ci.org/apache/incubator-hawq.png)](https://travis-ci.org/apache/incubator-hawq)| +|Apache Release Audit Tool ([RAT](https://creadur.apache.org/rat/))|[![Rat Status](https://builds.apache.org/buildStatus/icon?job=HAWQ-rat)](https://builds.apache.org/view/HAWQ/job/HAWQ-rat/)| +|Coverity Static Analysis |[![Coverity Scan Build](https://scan.coverity.com/projects/apache-incubator-hawq/badge.svg)](https://scan.coverity.com/projects/apache-incubator-hawq)| + +--- [Website](http://hawq.incubator.apache.org/) | -[Wiki](https://cwiki.apache.org/confluence/display/HAWQ/Apache+HAWQ+Home) | -[Documentation](http://hdb.docs.pivotal.io/) | +[Wiki](https://cwiki.apache.org/confluence/display/HAWQ) | +[Documentation](http://hawq.incubator.apache.org/docs/userguide/2.1.0.0-incubating/overview/HAWQOverview.html) | [Developer Mailing List](mailto:d...@hawq.incubator.apache.org) | [User Mailing List](mailto:u...@hawq.incubator.apache.org) | [Q&A Collections](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65144284) |
incubator-hawq git commit: HAWQ-1364. Store Apache Jenkins build scripts.
Repository: incubator-hawq Updated Branches: refs/heads/master 8f4d0f522 -> 4820193c9 HAWQ-1364. Store Apache Jenkins build scripts. The following Jenkins build scripts are being stored for easier maintenance. * HAWQ-build-pullrequest * HAWQ-rat Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/4820193c Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/4820193c Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/4820193c Branch: refs/heads/master Commit: 4820193c943a3b647ec59a798c63098b8c685e74 Parents: 8f4d0f5 Author: Ed Espino Authored: Mon Feb 27 12:04:22 2017 -0800 Committer: Ed Espino Committed: Mon Feb 27 23:04:07 2017 -0800 -- contrib/jenkins/HAWQ-build-pullrequest.sh | 67 ++ contrib/jenkins/HAWQ-rat.sh | 61 +++ 2 files changed, 128 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/4820193c/contrib/jenkins/HAWQ-build-pullrequest.sh -- diff --git a/contrib/jenkins/HAWQ-build-pullrequest.sh b/contrib/jenkins/HAWQ-build-pullrequest.sh new file mode 100644 index 000..d5d2440 --- /dev/null +++ b/contrib/jenkins/HAWQ-build-pullrequest.sh @@ -0,0 +1,67 @@ +#!/bin/bash +# -- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# -- +#HAWQ-build-pullrequest +# +# This file captures the Apache Jenkins build script +# -- + +work_dir=$(pwd) + +echo "working dir is ${work_dir}" + +git diff +git log -n 5 +git status + +# +# Create compile build script to be executed in Docker container. +# + +cat > compile_hawq.sh <http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/4820193c/contrib/jenkins/HAWQ-rat.sh -- diff --git a/contrib/jenkins/HAWQ-rat.sh b/contrib/jenkins/HAWQ-rat.sh new file mode 100644 index 000..a32e0a9 --- /dev/null +++ b/contrib/jenkins/HAWQ-rat.sh @@ -0,0 +1,61 @@ +# -- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# -- +# HAWQ-rat +# +# This file captures the Apache Jenkins build script +# -- + +set -exu + +# Check if NOTICE file year is current (HAWQ-1267) + +grep "Copyright $(date +"%Y") The Apache Software Foundation." NOTICE + +# Check if pom.xml file version is current (HAWQ-1268) + +grep "$(./getversion | cut -c1-3)" pom.xml + +set +x + +badfile_extentions="class jar tar tgz zip" +badfiles_found=false + +for extension in ${badfile_extentions}; do +echo "Searching for ${extension} files:" +badfile_count=$(find . -name "*.${extension}" | wc -l) +if [ ${badfile_count} != 0 ]; then +echo "-
incubator-hawq-site git commit: Add Apache HAWQ 2.1.0.0-incubating download links.
Repository: incubator-hawq-site Updated Branches: refs/heads/asf-site a8c9d8218 -> 085e55bfa Add Apache HAWQ 2.1.0.0-incubating download links. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/commit/085e55bf Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/tree/085e55bf Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/diff/085e55bf Branch: refs/heads/asf-site Commit: 085e55bfa3124f28295a5c39dcb7105e1ed3a244 Parents: a8c9d82 Author: Ed Espino Authored: Mon Feb 27 21:11:18 2017 -0800 Committer: Ed Espino Committed: Mon Feb 27 21:11:18 2017 -0800 -- index.html | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq-site/blob/085e55bf/index.html -- diff --git a/index.html b/index.html index a3b7575..ed750ec 100644 --- a/index.html +++ b/index.html @@ -323,11 +323,20 @@ +http://apache.org/dyn/closer.cgi/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz"; class="type">Source + + Version 2.1.0.0 | + http://www-eu.apache.org/dist/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc";>PGP | + http://www-eu.apache.org/dist/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256";>SHA-256 | + http://www-eu.apache.org/dist/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5";>MD5 + + + http://apache.org/dyn/closer.cgi/incubator/hawq/2.0.0.0-incubating/apache-hawq-src-2.0.0.0-incubating.tar.gz"; class="type">Source Version 2.0.0.0 | http://www-eu.apache.org/dist/incubator/hawq/2.0.0.0-incubating/apache-hawq-src-2.0.0.0-incubating.tar.gz.asc";>PGP | - http://www-eu.apache.org/dist/incubator/hawq/2.0.0.0-incubating/apache-hawq-src-2.0.0.0-incubating.tar.gz.sha1";>SHA-256 | + http://www-eu.apache.org/dist/incubator/hawq/2.0.0.0-incubating/apache-hawq-src-2.0.0.0-incubating.tar.gz.sha1";>SHA-1 | http://www-eu.apache.org/dist/incubator/hawq/2.0.0.0-incubating/apache-hawq-src-2.0.0.0-incubating.tar.gz.md5";>MD5
svn commit: r18497 - in /release/incubator/hawq/2.1.0.0-incubating: ./ apache-hawq-src-2.1.0.0-incubating.tar.gz apache-hawq-src-2.1.0.0-incubating.tar.gz.asc apache-hawq-src-2.1.0.0-incubating.tar.gz
Author: espino Date: Tue Feb 28 04:52:51 2017 New Revision: 18497 Log: Release Apache HAWQ (incubating) 2.1.0.0 Added: release/incubator/hawq/2.1.0.0-incubating/ release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz (with props) release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5 release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256 Added: release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz == Binary file - no diff available. Propchange: release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz -- svn:mime-type = application/octet-stream Added: release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc == --- release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc (added) +++ release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc Tue Feb 28 04:52:51 2017 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2 + +iQIcBAABCgAGBQJYqidEAAoJEPwGYvJXMlUivNoQAKiLNii+5mYl6hsqwqq1z2Di +buhLhaCUiNFZ08onRn9Y1xUTsafaGyZF5cNLPAGgCkPHTN3jcxpaDck5G3xZ3KCC +KxkGHjv2Tasoeb8vIxwNo508lpWA/sBn1rwshf4T+xowNewNX9q7uLzZiDd0yWV+ +3yaftI1zhe9FyAlqKP/3TD/5FfpYTJ50UqmxwBIkQ80acMsYoLmyEexj9z/0E76U +3VbCBXOqrE4f7g7giQ4URbSaKGGbgwszqfvrHO01TJNFZ2EcarN5uCDuYSoZp++k +DvnsKAJ+d9vLsZ4ozPHSDBMdS7ogL2Qd7qS0m3EI8xozaIC38KRrAMpwbVo0kga0 +Gj6BBN+i7STqYUdjN33g3X64vt1eXXyrmGLc1srOhP7PQDuq6k+ofJJOxKsHwcnD +G7uEor75QGvPsT9MyXcMzGayA6X+PEtvl40IqC6PJCdKn4+TPTpF1e53muMne/V3 +BHHw56J5wZi2B4Ah7dPAsEoIqCqYjfYQHvbwl0oc2ZDVjcVt+wvdutf8qz4F/5vC +s189qqDwFU8UaOmQ5IIVo1nrGJ4pkx1Yd2vu+IKiPeuyCFtpjRhF1sn5MrPi8NA5 +1I1AaT1EINmP3DTVaVm+1vtmyN8kQ32WqhmgRrhg+9yH8iRl86WeqC0hwT2KswUi +0XyAVRSr55bP6xY2Fw3Y +=aWi0 +-END PGP SIGNATURE- Added: release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5 == --- release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5 (added) +++ release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5 Tue Feb 28 04:52:51 2017 @@ -0,0 +1 @@ +MD5 (apache-hawq-src-2.1.0.0-incubating.tar.gz) = f8afc7bc9d69a1e9289a8ebfbaf1e92f Added: release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256 == --- release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256 (added) +++ release/incubator/hawq/2.1.0.0-incubating/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256 Tue Feb 28 04:52:51 2017 @@ -0,0 +1 @@ +91944ed45e33540409e4d998dc5d2e721c5a5eaa16ba153ce1804bcb0086dbb9 apache-hawq-src-2.1.0.0-incubating.tar.gz
incubator-hawq git commit: HAWQ-1351. Remove tweets.tar.gz & generate it dynamically.
Repository: incubator-hawq Updated Branches: refs/heads/master 6508ebb84 -> 80ea89d2c HAWQ-1351. Remove tweets.tar.gz & generate it dynamically. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/80ea89d2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/80ea89d2 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/80ea89d2 Branch: refs/heads/master Commit: 80ea89d2cc3ac192549510674983c52c40c2ee38 Parents: 6508ebb Author: Ed Espino Authored: Wed Feb 22 23:42:38 2017 -0800 Committer: Ed Espino Committed: Fri Feb 24 11:41:08 2017 -0800 -- pom.xml | 4 pxf/build.gradle | 6 ++ pxf/pxf-json/.gitignore | 1 + pxf/pxf-json/src/test/resources/tweets.tar.gz | Bin 796 -> 0 bytes 4 files changed, 7 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/80ea89d2/pom.xml -- diff --git a/pom.xml b/pom.xml index 6410958..1ccaeee 100644 --- a/pom.xml +++ b/pom.xml @@ -242,10 +242,6 @@ tools/demo/gpfdist_transform/*.yaml tools/bin/gppylib/operations/madlib_depcheck/changelist.yaml - - - pxf/pxf-json/src/test/resources/tweets.tar.gz -
[1/2] incubator-hawq git commit: improve hawq check command line help
Repository: incubator-hawq Updated Branches: refs/heads/master 974e35258 -> b37f18768 improve hawq check command line help - add several missing options - update content for accuracy and clarity Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/c0d086d0 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/c0d086d0 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/c0d086d0 Branch: refs/heads/master Commit: c0d086d02ba6d7f925e35439165189abfcb23200 Parents: 974e352 Author: Lisa Owen Authored: Thu Sep 15 16:53:04 2016 -0700 Committer: Ed Espino Committed: Wed Feb 22 11:00:49 2017 -0800 -- tools/doc/gpcheck_help | 99 - 1 file changed, 53 insertions(+), 46 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/c0d086d0/tools/doc/gpcheck_help -- diff --git a/tools/doc/gpcheck_help b/tools/doc/gpcheck_help index 587f32f..3ff35b5 100644 --- a/tools/doc/gpcheck_help +++ b/tools/doc/gpcheck_help @@ -6,9 +6,11 @@ Verifies and validates HAWQ platform settings. SYNOPSIS * -hawq check -f +hawq check -f | + (-h | --host ) [--hadoop | --hadoop-home ] [--stdout | --zipout] [--config ] +[--kerberos] [--hdfs-ha] [--yarn] [--yarn-ha] hawq check --zipin @@ -25,24 +27,26 @@ The hawq check utility determines the platform on which you are running HAWQ and validates various platform-specific configuration settings as well as HAWQ and HDFS specific configuration settings. In order to perform HAWQ configuration -checks, make sure HAWQ has been already started and gpconfig -works. For HDFS checks, you should either set the HADOOP_HOME -environment variable or give the hadoop installation location -using --hadoop option. - -hawq check can use a host file or a file previously created with -the --zipout option to validate platform settings. If -GPCHECK_ERROR displays, one or more validation checks failed. +checks, make sure HAWQ has been already started and hawq config +works. For HDFS checks, you should either set the $HADOOP_HOME +environment variable or provide the full path to the hadoop +installation location using the --hadoop option. + +The hawq check utility can use a host file or a file previously +created with the --zipout option to validate platform settings. +If GPCHECK_ERROR displays, one or more validation checks failed. You can also use hawq check to gather and view platform settings on hosts without running validation checks. When running checks, hawq check compares your actual configuration -setting with expected value listed in a config file (which is -$GPHOME/etc/hawq check.cnf by default). You are supposed to modify -values for "mount.points" and "diskusage.monitor.mounts" to -the actual mount points you want to check separated by comma, -otherwise it only checks the root directory which may not be -helpful, here is an example: +setting with an expected value listed in a configuration file +($GPHOME/etc/hawq_check.cnf by default). You must modify your +configuration values for "mount.points" and "diskusage.monitor.mounts" +to the actual mount points you want to check, as a comma-separated +list. Otherwise the utility only checks the root directory, which +may not be helpful. + +An example is shown below: [linux.mount] mount.points = /,/data1,/data2 @@ -57,10 +61,7 @@ OPTIONS --config config_file The name of a configuration file to use instead of the default - file $GPHOME/etc/hawq_check.cnf (or ~/gpconfigs/hawq_check_dca_config - on the EMC Greenplum Data Computing Appliance). This file specifies - the OS-specific checks and HDFS-specific checks to run. - + file $GPHOME/etc/hawq_check.cnf. -f hostfile_hawq_check @@ -69,56 +70,62 @@ OPTIONS contain a single host name for all hosts in your HAWQ system (master, standby master, and segments). +-h/--host hostname + Specifies a single host on which platform-specific settings will + be validated. + --hadoop/--hadoop-home hadoop_home - Use this option to specify your hadoop installation location so that - hawq check can validate HDFS settings. This option is not needed when - HADOOP_HOME environment variable is set. + Use this option to specify the full path to your hadoop installation + location so that hawq check can validate HDFS settings. This option + is not needed when the $HADOOP_HOME environment variable is set. --kerberos - Use this option to specify HDFS/YARN enables Kerberos mode, so - that hawq check can validate HDFS/YARN settings with Kerberos - enabled. +
[2/2] incubator-hawq git commit: use <> in options/arguments section
use <> in options/arguments section This closes #903 PR. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/b37f1876 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/b37f1876 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/b37f1876 Branch: refs/heads/master Commit: b37f18768dceec5600f8a90f6f8b17045954629c Parents: c0d086d Author: Lisa Owen Authored: Wed Feb 22 09:52:58 2017 -0800 Committer: Ed Espino Committed: Wed Feb 22 11:02:41 2017 -0800 -- tools/doc/gpcheck_help | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b37f1876/tools/doc/gpcheck_help -- diff --git a/tools/doc/gpcheck_help b/tools/doc/gpcheck_help index 3ff35b5..1e35b95 100644 --- a/tools/doc/gpcheck_help +++ b/tools/doc/gpcheck_help @@ -58,23 +58,23 @@ An example is shown below: OPTIONS * ---config config_file +--config The name of a configuration file to use instead of the default file $GPHOME/etc/hawq_check.cnf. --f hostfile_hawq_check +-f The name of a file that contains a list of hosts that hawq check uses to validate platform-specific settings. This file should contain a single host name for all hosts in your HAWQ system (master, standby master, and segments). --h/--host hostname +-h/--host Specifies a single host on which platform-specific settings will be validated. ---hadoop/--hadoop-home hadoop_home +--hadoop/--hadoop-home Use this option to specify the full path to your hadoop installation location so that hawq check can validate HDFS settings. This option @@ -121,7 +121,7 @@ OPTIONS performed. ---zipin hawq_check_zipfile +--zipin Use this option to decompress and check a .zip file created with the --zipout option. hawq check performs validation tasks against @@ -142,7 +142,7 @@ OPTIONS EXAMPLES * -Verify and validate the HAWQ platform settings by specifying +Verify and validate the HAWQ platform settings by specifying a host file and identifying the full hadoop install path: # hawq check -f hostfile_hawq_check --hadoop /usr/local/hadoop-/
incubator-hawq git commit: HAWQ-1063. HAWQ Python library missing import (DatabaseError).
Repository: incubator-hawq Updated Branches: refs/heads/master 78dc733f4 -> 974e35258 HAWQ-1063. HAWQ Python library missing import (DatabaseError). This closes #920 PR. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/974e3525 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/974e3525 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/974e3525 Branch: refs/heads/master Commit: 974e3525802f2b1568a19f6a6c3e78fca0388130 Parents: 78dc733 Author: Ed Espino Authored: Tue Feb 21 23:52:22 2017 -0800 Committer: Ed Espino Committed: Tue Feb 21 23:52:22 2017 -0800 -- tools/bin/hawqpylib/hawqlib.py | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/974e3525/tools/bin/hawqpylib/hawqlib.py -- diff --git a/tools/bin/hawqpylib/hawqlib.py b/tools/bin/hawqpylib/hawqlib.py index d9bd85d..08a8caf 100755 --- a/tools/bin/hawqpylib/hawqlib.py +++ b/tools/bin/hawqpylib/hawqlib.py @@ -22,6 +22,7 @@ import threading import Queue from xml.dom import minidom from xml.etree.ElementTree import ElementTree +from pygresql.pg import DatabaseError import shutil from gppylib.db import dbconn from gppylib.commands.base import WorkerPool, REMOTE
[incubator-hawq] Git Push Summary
Repository: incubator-hawq Updated Tags: refs/tags/2.1.0.0-incubating-rc4 [created] 1a69c459c
svn commit: r18394 - in /dev/incubator/hawq/2.1.0.0-incubating.RC4: ./ apache-hawq-src-2.1.0.0-incubating.tar.gz apache-hawq-src-2.1.0.0-incubating.tar.gz.asc apache-hawq-src-2.1.0.0-incubating.tar.gz
Author: espino Date: Sun Feb 19 23:23:32 2017 New Revision: 18394 Log: Adding 2.1.0.0-incubating RC4 candidate release artifacts Added: dev/incubator/hawq/2.1.0.0-incubating.RC4/ dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz (with props) dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5 dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256 Added: dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz == Binary file - no diff available. Propchange: dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz -- svn:mime-type = application/octet-stream Added: dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc == --- dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc (added) +++ dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc Sun Feb 19 23:23:32 2017 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2 + +iQIcBAABCgAGBQJYqidEAAoJEPwGYvJXMlUivNoQAKiLNii+5mYl6hsqwqq1z2Di +buhLhaCUiNFZ08onRn9Y1xUTsafaGyZF5cNLPAGgCkPHTN3jcxpaDck5G3xZ3KCC +KxkGHjv2Tasoeb8vIxwNo508lpWA/sBn1rwshf4T+xowNewNX9q7uLzZiDd0yWV+ +3yaftI1zhe9FyAlqKP/3TD/5FfpYTJ50UqmxwBIkQ80acMsYoLmyEexj9z/0E76U +3VbCBXOqrE4f7g7giQ4URbSaKGGbgwszqfvrHO01TJNFZ2EcarN5uCDuYSoZp++k +DvnsKAJ+d9vLsZ4ozPHSDBMdS7ogL2Qd7qS0m3EI8xozaIC38KRrAMpwbVo0kga0 +Gj6BBN+i7STqYUdjN33g3X64vt1eXXyrmGLc1srOhP7PQDuq6k+ofJJOxKsHwcnD +G7uEor75QGvPsT9MyXcMzGayA6X+PEtvl40IqC6PJCdKn4+TPTpF1e53muMne/V3 +BHHw56J5wZi2B4Ah7dPAsEoIqCqYjfYQHvbwl0oc2ZDVjcVt+wvdutf8qz4F/5vC +s189qqDwFU8UaOmQ5IIVo1nrGJ4pkx1Yd2vu+IKiPeuyCFtpjRhF1sn5MrPi8NA5 +1I1AaT1EINmP3DTVaVm+1vtmyN8kQ32WqhmgRrhg+9yH8iRl86WeqC0hwT2KswUi +0XyAVRSr55bP6xY2Fw3Y +=aWi0 +-END PGP SIGNATURE- Added: dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5 == --- dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5 (added) +++ dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5 Sun Feb 19 23:23:32 2017 @@ -0,0 +1 @@ +MD5 (apache-hawq-src-2.1.0.0-incubating.tar.gz) = f8afc7bc9d69a1e9289a8ebfbaf1e92f Added: dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256 == --- dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256 (added) +++ dev/incubator/hawq/2.1.0.0-incubating.RC4/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256 Sun Feb 19 23:23:32 2017 @@ -0,0 +1 @@ +91944ed45e33540409e4d998dc5d2e721c5a5eaa16ba153ce1804bcb0086dbb9 apache-hawq-src-2.1.0.0-incubating.tar.gz
[15/35] incubator-hawq git commit: HAWQ-1329. pg_catalog view fallback failed.
HAWQ-1329. pg_catalog view fallback failed. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/17a9dc1e Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/17a9dc1e Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/17a9dc1e Branch: refs/heads/2.1.0.0-incubating Commit: 17a9dc1e570fdbb3dc550b46c96c3b57c4df7013 Parents: 492b1a7 Author: stanlyxiang Authored: Tue Feb 14 16:13:52 2017 +0800 Committer: Wen Lin Committed: Tue Feb 14 16:52:28 2017 +0800 -- src/backend/catalog/aclchk.c | 22 ++ 1 file changed, 10 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/17a9dc1e/src/backend/catalog/aclchk.c -- diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c index 416e0c4..6712cc8 100644 --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -2707,6 +2707,8 @@ List *getActionName(AclMode mask) return actions; } +#define FALLBACK_IS_TRUE(x)x == PG_CATALOG_NAMESPACE || x == information_schema_namespcace_oid \ +|| x == PG_AOSEGMENT_NAMESPACE || x == PG_TOAST_NAMESPACE || x == PG_BITMAPINDEX_NAMESPACE bool fallBackToNativeCheck(AclObjectKind objkind, Oid obj_oid, Oid roleid) { @@ -2720,20 +2722,18 @@ bool fallBackToNativeCheck(AclObjectKind objkind, Oid obj_oid, Oid roleid) /* for heap table, we fall back to native check. */ if (objkind == ACL_KIND_CLASS) { -char relstorage = get_rel_relstorage(obj_oid); -if (relstorage == 'h') -{ - return true; -} + Oid namespaceid = get_rel_namespace(obj_oid); + if(FALLBACK_IS_TRUE(namespaceid)) + { + return true; + } } else if (objkind == ACL_KIND_NAMESPACE) { /* native check build-in schemas. */ -if (obj_oid == PG_CATALOG_NAMESPACE || obj_oid == information_schema_namespcace_oid -|| obj_oid == PG_AOSEGMENT_NAMESPACE || obj_oid == PG_TOAST_NAMESPACE -|| obj_oid == PG_BITMAPINDEX_NAMESPACE) +if(FALLBACK_IS_TRUE(obj_oid)) { - return true; +return true; } else if (obj_oid == PG_PUBLIC_NAMESPACE && superuser()) { @@ -2745,9 +2745,7 @@ bool fallBackToNativeCheck(AclObjectKind objkind, Oid obj_oid, Oid roleid) { /* native check functions under build-in schemas. */ Oid namespaceid = get_func_namespace(obj_oid); -if (namespaceid == PG_CATALOG_NAMESPACE || namespaceid == information_schema_namespcace_oid -|| namespaceid == PG_AOSEGMENT_NAMESPACE || namespaceid == PG_TOAST_NAMESPACE -|| namespaceid == PG_BITMAPINDEX_NAMESPACE) +if (FALLBACK_IS_TRUE(namespaceid)) { return true; }
[01/35] incubator-hawq git commit: HAWQ-1292. Change GUC enable_ranger(bool) to a text GUC(hawq_acl_type), which can allow other kinds of ACL. [Forced Update!]
Repository: incubator-hawq Updated Branches: refs/heads/2.1.0.0-incubating 12c7df017 -> f5033eaa3 (forced update) HAWQ-1292. Change GUC enable_ranger(bool) to a text GUC(hawq_acl_type), which can allow other kinds of ACL. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/e4ac516b Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/e4ac516b Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/e4ac516b Branch: refs/heads/2.1.0.0-incubating Commit: e4ac516b24853a83f3c0c7c66d858e92437f8d46 Parents: 7d02472 Author: stanlyxiang Authored: Sat Jan 14 15:31:28 2017 +0800 Committer: Wen Lin Committed: Sat Feb 4 16:21:19 2017 +0800 -- src/backend/catalog/aclchk.c| 22 +++--- src/backend/catalog/namespace.c | 4 +-- src/backend/parser/parse_relation.c | 5 +-- src/backend/tcop/postgres.c | 17 +-- src/backend/utils/adt/acl.c | 3 +- src/backend/utils/misc/guc.c| 52 src/include/utils/acl.h | 14 - src/include/utils/guc.h | 4 ++- 8 files changed, 75 insertions(+), 46 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/e4ac516b/src/backend/catalog/aclchk.c -- diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c index 33fa9ab..667aa61 100644 --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -228,7 +228,7 @@ restrict_and_check_grant(bool is_grant, AclMode avail_goptions, bool all_privs, */ if (avail_goptions == ACL_NO_RIGHTS && Gp_role != GP_ROLE_EXECUTE) { - if (enable_ranger && !fallBackToNativeCheck(objkind, objectId, grantorId)) { + if (aclType == HAWQ_ACL_RANGER && !fallBackToNativeCheck(objkind, objectId, grantorId)) { if (pg_rangercheck(objkind, objectId, grantorId, whole_mask | ACL_GRANT_OPTION_FOR(whole_mask), ACLMASK_ANY) != ACLCHECK_OK) @@ -3851,7 +3851,7 @@ pg_class_aclcheck(Oid table_oid, Oid roleid, AclMode mode) if (Gp_role == GP_ROLE_EXECUTE) return ACLCHECK_OK; - if(enable_ranger && !fallBackToNativeCheck(ACL_KIND_CLASS, table_oid, roleid)) + if(aclType == HAWQ_ACL_RANGER && !fallBackToNativeCheck(ACL_KIND_CLASS, table_oid, roleid)) { return pg_rangercheck(ACL_KIND_CLASS, table_oid, roleid, mode, ACLMASK_ANY); } @@ -3871,7 +3871,7 @@ pg_database_aclcheck(Oid db_oid, Oid roleid, AclMode mode) if (Gp_role == GP_ROLE_EXECUTE) return ACLCHECK_OK; - if(enable_ranger && !fallBackToNativeCheck(ACL_KIND_DATABASE, db_oid, roleid)) + if(aclType == HAWQ_ACL_RANGER && !fallBackToNativeCheck(ACL_KIND_DATABASE, db_oid, roleid)) { return pg_rangercheck(ACL_KIND_DATABASE, db_oid, roleid, mode, ACLMASK_ANY); } @@ -3891,7 +3891,7 @@ pg_proc_aclcheck(Oid proc_oid, Oid roleid, AclMode mode) if (Gp_role == GP_ROLE_EXECUTE) return ACLCHECK_OK; - if(enable_ranger && !fallBackToNativeCheck(ACL_KIND_PROC, proc_oid, roleid)) + if(aclType == HAWQ_ACL_RANGER && !fallBackToNativeCheck(ACL_KIND_PROC, proc_oid, roleid)) { return pg_rangercheck(ACL_KIND_PROC, proc_oid, roleid, mode, ACLMASK_ANY); } @@ -3911,7 +3911,7 @@ pg_language_aclcheck(Oid lang_oid, Oid roleid, AclMode mode) if (Gp_role == GP_ROLE_EXECUTE) return ACLCHECK_OK; - if(enable_ranger && !fallBackToNativeCheck(ACL_KIND_LANGUAGE, lang_oid, roleid)) + if(aclType == HAWQ_ACL_RANGER && !fallBackToNativeCheck(ACL_KIND_LANGUAGE, lang_oid, roleid)) { return pg_rangercheck(ACL_KIND_LANGUAGE, lang_oid, roleid, mode, ACLMASK_ANY); } @@ -3931,7 +3931,7 @@ pg_namespace_aclcheck(Oid nsp_oid, Oid roleid, AclMode mode) if (Gp_role == GP_ROLE_EXECUTE) return ACLCHECK_OK; - if(enable_ranger && !fallBackToNativeCheck(ACL_KIND_NAMESPACE, nsp_oid, roleid)) + if(aclType == HAWQ_ACL_RANGER && !fallBackToNativeCheck(ACL_KIND_NAMESPACE, nsp_oid, roleid)) { return pg_rangercheck(ACL_KIND_NAMESPACE, nsp_oid, roleid, mode, ACLMASK_ANY); } @@ -3951,7 +3951,7 @@ pg_tablespace_aclcheck(Oid spc_oid, Oid roleid, AclMode mode) if (Gp_role == GP_ROLE_EXECUTE) return ACLCHECK_OK; - if(enable_ranger && !fallBackToNativeCheck(ACL_KIND_TABLESPACE, spc_oid, roleid)) + if(aclType == HAWQ_ACL_RANGER && !fallBackToNativeCheck(ACL_KIND_TABLESPACE, spc_oid, roleid)) { return pg_rangercheck(ACL_KIND_TABLESPACE, spc_oid, roleid, mode, ACLMASK_ANY); } @@ -3972,7 +3972,7 @@ pg_foreign_data_wrapper_aclcheck(Oid fdw_oid, Oid roleid, AclMode mode) if (Gp_role == GP_ROLE_EXECUTE) return ACLCHECK_OK; - if(enable_ranger && !fallBackToNativeCheck(ACL_KIND_FDW, fd
[13/35] incubator-hawq git commit: HAWQ-1323. Add license header added to each source file in test directory
http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/492b1a78/src/test/feature/query/test_insert.cpp -- diff --git a/src/test/feature/query/test_insert.cpp b/src/test/feature/query/test_insert.cpp index 1e05b9c..dc1e361 100644 --- a/src/test/feature/query/test_insert.cpp +++ b/src/test/feature/query/test_insert.cpp @@ -1,3 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + #include #include #include http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/492b1a78/src/test/feature/query/test_nested_case_null.cpp -- diff --git a/src/test/feature/query/test_nested_case_null.cpp b/src/test/feature/query/test_nested_case_null.cpp index 3d5894f..5fd547f 100644 --- a/src/test/feature/query/test_nested_case_null.cpp +++ b/src/test/feature/query/test_nested_case_null.cpp @@ -1,3 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + #include #include #include http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/492b1a78/src/test/feature/query/test_parser.cpp -- diff --git a/src/test/feature/query/test_parser.cpp b/src/test/feature/query/test_parser.cpp index c2aec26..597dbf1 100644 --- a/src/test/feature/query/test_parser.cpp +++ b/src/test/feature/query/test_parser.cpp @@ -1,3 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + #include #include #include http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/492b1a78/src/test/feature/query/test_polymorphism.cpp -- diff --git a/src/test/feature/query/test_polymorphism.cpp b/src/test/feature/query/test_polymorphism.cpp index 42d5a6a..21bb663 100644 --- a/src/test/feature/query/test_polymorphism.cpp +++ b/src/test/feature/query/test_polymorphism.cpp @@ -1,3 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + *
[03/35] incubator-hawq git commit: HAWQ-1312. Forbid partial grant/revoke command in HAWQ side once Ranger is configured.
HAWQ-1312. Forbid partial grant/revoke command in HAWQ side once Ranger is configured. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/35ed3ad3 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/35ed3ad3 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/35ed3ad3 Branch: refs/heads/2.1.0.0-incubating Commit: 35ed3ad38c7ad0207f4e594fdbcdf4e324449c38 Parents: 31aff87 Author: Wen Lin Authored: Wed Feb 8 10:05:56 2017 +0800 Committer: Wen Lin Committed: Wed Feb 8 10:05:56 2017 +0800 -- src/backend/catalog/aclchk.c| 41 + src/test/feature/UDF/TestUDF.cpp| 24 +- .../UDF/ans/function_basics.ranger.ans.orca | 1088 ++ .../UDF/ans/function_basics.ranger.ans.planner | 1076 + 4 files changed, 2225 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/35ed3ad3/src/backend/catalog/aclchk.c -- diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c index 667aa61..3ab3248 100644 --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -285,6 +285,31 @@ restrict_and_check_grant(bool is_grant, AclMode avail_goptions, bool all_privs, } /* + * Check if this object's ACL is checked natively. + */ +bool checkACLNative(GrantObjectType type, Oid oid) +{ + AclObjectKind kind = MAX_ACL_KIND; + if (type == ACL_OBJECT_RELATION) + { + kind = ACL_KIND_CLASS; + } + else if (type == ACL_OBJECT_NAMESPACE) + { + kind = ACL_KIND_NAMESPACE; + } + else if (type == ACL_OBJECT_FUNCTION) + { + kind = ACL_KIND_PROC; + } + else + { + return false; + } + return fallBackToNativeCheck(kind, oid, GetUserId()); +} + +/* * Called to execute the utility commands GRANT and REVOKE */ void @@ -304,6 +329,22 @@ ExecuteGrantStmt(GrantStmt *stmt) istmt.objtype = stmt->objtype; istmt.objects = objectNamesToOids(stmt->objtype, stmt->objects); + /* +* Don't allow GRANT/REVOKE for objects managed by Ranger +* if in ranger mode. +*/ + if (aclType == HAWQ_ACL_RANGER) + { + foreach(cell, istmt.objects) + { + Oid oid = lfirst_oid(cell); + if (!checkACLNative(stmt->objtype, oid)) + { + elog(ERROR, "GRANT/REVOKE is not allowed for this object in ranger mode"); + } + } + } + /* If this is a GRANT/REVOKE on a table, expand partition references */ if (istmt.objtype == ACL_OBJECT_RELATION) { http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/35ed3ad3/src/test/feature/UDF/TestUDF.cpp -- diff --git a/src/test/feature/UDF/TestUDF.cpp b/src/test/feature/UDF/TestUDF.cpp index 7107b63..95509fb 100755 --- a/src/test/feature/UDF/TestUDF.cpp +++ b/src/test/feature/UDF/TestUDF.cpp @@ -18,13 +18,29 @@ TEST_F(TestUDF, TestUDFBasics) hawq::test::SQLUtility util; if (util.getGUCValue("optimizer") == "on") { - util.execSQLFile("UDF/sql/function_basics.sql", -"UDF/ans/function_basics.ans.orca"); + if (util.getGUCValue("hawq_acl_type") == "standalone") + { + util.execSQLFile("UDF/sql/function_basics.sql", + "UDF/ans/function_basics.ans.orca"); + } + else + { + util.execSQLFile("UDF/sql/function_basics.sql", + "UDF/ans/function_basics.ranger.ans.orca"); + } } else { - util.execSQLFile("UDF/sql/function_basics.sql", -"UDF/ans/function_basics.ans.planner"); + if (util.getGUCValue("hawq_acl_type") == "standalone") + { + util.execSQLFile("UDF/sql/function_basics.sql", + "UDF/ans/function_basics.ans.planner"); + } + else + { + util.execSQLFile("UDF/sql/function_basics.sql", + "UDF/ans/function_basics.ranger.ans.planner"); + } } } http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/35ed3ad3/src/test/feature/UDF/ans/function_basic
[07/35] incubator-hawq git commit: HAWQ-1318. Fix the bug of cannot start/stop master successfully if ranger is enable and with a wrong RPS address
HAWQ-1318. Fix the bug of cannot start/stop master successfully if ranger is enable and with a wrong RPS address Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/63c85628 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/63c85628 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/63c85628 Branch: refs/heads/2.1.0.0-incubating Commit: 63c856280c1fea655f99a9aea2c2196116cc2ab3 Parents: 8c9b45a Author: Wen Lin Authored: Thu Feb 9 16:03:53 2017 +0800 Committer: Wen Lin Committed: Fri Feb 10 10:18:19 2017 +0800 -- src/backend/catalog/aclchk.c | 32 +++- 1 file changed, 19 insertions(+), 13 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/63c85628/src/backend/catalog/aclchk.c -- diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c index 3ab3248..416e0c4 100644 --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -2715,9 +2715,9 @@ bool fallBackToNativeCheck(AclObjectKind objkind, Oid obj_oid, Oid roleid) */ if (information_schema_namespcace_oid == 0) { - information_schema_namespcace_oid = (int)get_namespace_oid("information_schema"); + information_schema_namespcace_oid = (int)get_namespace_oid("information_schema"); } - /*for heap table, we fall back to native check.*/ + /* for heap table, we fall back to native check. */ if (objkind == ACL_KIND_CLASS) { char relstorage = get_rel_relstorage(obj_oid); @@ -2728,21 +2728,26 @@ bool fallBackToNativeCheck(AclObjectKind objkind, Oid obj_oid, Oid roleid) } else if (objkind == ACL_KIND_NAMESPACE) { - /*native check build-in schemas.*/ +/* native check build-in schemas. */ if (obj_oid == PG_CATALOG_NAMESPACE || obj_oid == information_schema_namespcace_oid - || obj_oid == PG_AOSEGMENT_NAMESPACE || obj_oid == PG_TOAST_NAMESPACE - || obj_oid == PG_BITMAPINDEX_NAMESPACE) +|| obj_oid == PG_AOSEGMENT_NAMESPACE || obj_oid == PG_TOAST_NAMESPACE +|| obj_oid == PG_BITMAPINDEX_NAMESPACE) { return true; } +else if (obj_oid == PG_PUBLIC_NAMESPACE && superuser()) +{ + /* superuser's access to PUBLIC */ + return true; +} } else if (objkind == ACL_KIND_PROC) { - /*native check functions under build-in schemas.*/ +/* native check functions under build-in schemas. */ Oid namespaceid = get_func_namespace(obj_oid); if (namespaceid == PG_CATALOG_NAMESPACE || namespaceid == information_schema_namespcace_oid - || namespaceid == PG_AOSEGMENT_NAMESPACE || namespaceid == PG_TOAST_NAMESPACE - || namespaceid == PG_BITMAPINDEX_NAMESPACE) +|| namespaceid == PG_AOSEGMENT_NAMESPACE || namespaceid == PG_TOAST_NAMESPACE +|| namespaceid == PG_BITMAPINDEX_NAMESPACE) { return true; } @@ -2848,16 +2853,17 @@ pg_rangercheck(AclObjectKind objkind, Oid object_oid, Oid roleid, List* actions = getActionName(mask); bool isAll = (how == ACLMASK_ALL) ? true: false; - elog(DEBUG3, "ranger acl check kind: %d, object name: %s, role: %s, mask: %u\n", objkind, objectname, rolename, mask); + elog(DEBUG3, "ranger acl check kind: %d, object name: %s, object oid:%d, role: %s, mask: %u\n", + objkind, objectname, object_oid, rolename, mask); List *resultargs = NIL; -RangerPrivilegeResults *aclresult = (RangerPrivilegeResults *) palloc(sizeof(RangerPrivilegeResults)); -aclresult->result = RANGERCHECK_NO_PRIV; -aclresult->relOid = object_oid; + RangerPrivilegeResults *aclresult = (RangerPrivilegeResults *) palloc(sizeof(RangerPrivilegeResults)); + aclresult->result = RANGERCHECK_NO_PRIV; + aclresult->relOid = object_oid; /* this two sign fields will be set in function create_ranger_request_json */ aclresult->resource_sign = 0; aclresult->privilege_sign = 0; -resultargs = lappend(resultargs, aclresult); + resultargs = lappend(resultargs, aclresult); List *requestargs = NIL; RangerRequestJsonArgs *requestarg = (RangerRequestJsonArgs *) palloc(sizeof(RangerRequestJsonArgs));
[12/35] incubator-hawq git commit: HAWQ-1327. Move ID from struct SegStatData to struct SegInfoData so that ID could be used in QD.
HAWQ-1327. Move ID from struct SegStatData to struct SegInfoData so that ID could be used in QD. This also fixes some warnings and some minor bugs which are found during warning fixing. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/9918443b Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/9918443b Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/9918443b Branch: refs/heads/2.1.0.0-incubating Commit: 9918443b6327cf859d148af38cc0ec90b7ec02ee Parents: 305e2bc Author: Paul Guo Authored: Sat Feb 4 18:09:14 2017 +0800 Committer: Paul Guo Committed: Tue Feb 14 13:08:07 2017 +0800 -- .../communication/rmcomm_RM2RMSEG.c | 8 +-- .../resourcemanager/include/resourcepool.h | 7 ++- src/backend/resourcemanager/requesthandler.c| 11 ++-- .../resourcemanager/requesthandler_RMSEG.c | 28 +- .../resourcebroker/resourcebroker_LIBYARN.c | 4 +- .../resourcebroker_LIBYARN_proc.c | 4 +- src/backend/resourcemanager/resourcemanager.c | 8 +-- src/backend/resourcemanager/resourcepool.c | 57 8 files changed, 53 insertions(+), 74 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/9918443b/src/backend/resourcemanager/communication/rmcomm_RM2RMSEG.c -- diff --git a/src/backend/resourcemanager/communication/rmcomm_RM2RMSEG.c b/src/backend/resourcemanager/communication/rmcomm_RM2RMSEG.c index a79bd6c..2539356 100644 --- a/src/backend/resourcemanager/communication/rmcomm_RM2RMSEG.c +++ b/src/backend/resourcemanager/communication/rmcomm_RM2RMSEG.c @@ -191,10 +191,10 @@ void receivedRUAliveResponse(AsyncCommMessageHandlerContext context, if (Gp_role != GP_ROLE_UTILITY) { SimpStringPtr description = build_segment_status_description(segres->Stat); - update_segment_status(segres->Stat->ID + REGISTRATION_ORDER_OFFSET, + update_segment_status(segres->Stat->Info.ID + REGISTRATION_ORDER_OFFSET, SEGMENT_STATUS_DOWN, (description->Len > 0)?description->Str:""); - add_segment_history_row(segres->Stat->ID + REGISTRATION_ORDER_OFFSET, + add_segment_history_row(segres->Stat->Info.ID + REGISTRATION_ORDER_OFFSET, GET_SEGRESOURCE_HOSTNAME(segres), description->Str); @@ -257,10 +257,10 @@ void sentRUAliveError(AsyncCommMessageHandlerContext context) if (Gp_role != GP_ROLE_UTILITY) { SimpStringPtr description = build_segment_status_description(segres->Stat); - update_segment_status(segres->Stat->ID + REGISTRATION_ORDER_OFFSET, + update_segment_status(segres->Stat->Info.ID + REGISTRATION_ORDER_OFFSET, SEGMENT_STATUS_DOWN, (description->Len > 0)?description->Str:""); - add_segment_history_row(segres->Stat->ID + REGISTRATION_ORDER_OFFSET, + add_segment_history_row(segres->Stat->Info.ID + REGISTRATION_ORDER_OFFSET, GET_SEGRESOURCE_HOSTNAME(segres), description->Str); http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/9918443b/src/backend/resourcemanager/include/resourcepool.h -- diff --git a/src/backend/resourcemanager/include/resourcepool.h b/src/backend/resourcemanager/include/resourcepool.h index 718e5fc..c906d01 100644 --- a/src/backend/resourcemanager/include/resourcepool.h +++ b/src/backend/resourcemanager/include/resourcepool.h @@ -69,10 +69,11 @@ struct SegInfoData { uint32_tHostAddrCount; uint32_tFailedTmpDirOffset; uint32_tFailedTmpDirLen; + int32_t ID; /* Global unique ID. */ uint8_t master; uint8_t standby; uint8_t alive; - uint8_t Reserve
[02/35] incubator-hawq git commit: HAWQ-1316. Fix feature test build on CentOS 7.3
HAWQ-1316. Fix feature test build on CentOS 7.3 Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/31aff875 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/31aff875 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/31aff875 Branch: refs/heads/2.1.0.0-incubating Commit: 31aff875ca969e9c8d74bd4eafe56806dc83dad4 Parents: e4ac516 Author: rlei Authored: Tue Feb 7 10:46:10 2017 +0800 Committer: rlei Committed: Tue Feb 7 15:05:30 2017 +0800 -- src/test/feature/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/31aff875/src/test/feature/Makefile -- diff --git a/src/test/feature/Makefile b/src/test/feature/Makefile index d12b751..06c3b61 100644 --- a/src/test/feature/Makefile +++ b/src/test/feature/Makefile @@ -9,7 +9,7 @@ gmock_lib_path = $(abs_top_srcdir)/depends/thirdparty/googletest/build/googlemoc override CXX = g++ override CXXFLAGS = -Wall -O0 -g -std=c++11 override CPPFLAGS := -I/usr/include -I/usr/local/include -I/usr/include/libxml2 -I$(abs_top_srcdir)/src/test/feature/ -I$(abs_top_srcdir)/src/test/feature/ManagementTool/ -I$(abs_top_srcdir)/src/test/feature/lib/ -I$(abs_top_srcdir)/src/interfaces/libpq -I$(abs_top_srcdir)/src/interfaces -I$(abs_top_srcdir)/src/include -I$(gtest_include) -I$(gmock_include) -override LIBS := $(LIBS) -lgtest -lpq -lxml2 -ltest +override LIBS := $(LIBS) -lgtest -lpq -lxml2 -ltest -lpthread override LDFLAGS += -L/usr/local/lib -L/usr/lib -L$(abs_top_srcdir)/src/test/feature/ -L$(abs_top_srcdir)/src/test/feature/lib/ -L$(abs_top_srcdir)/src/interfaces/libpq -L$(gtest_lib_path) -L$(gmock_lib_path) PROG = test_main.cpp $(wildcard */*.cpp)
[10/35] incubator-hawq git commit: HAWQ-1321. failNames wrongly uses memory context to build message when ANALYZE failed
HAWQ-1321. failNames wrongly uses memory context to build message when ANALYZE failed Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/256c0668 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/256c0668 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/256c0668 Branch: refs/heads/2.1.0.0-incubating Commit: 256c0668c94137565c24dc2057fbe3282187165c Parents: 1c36c28 Author: Yi Authored: Tue Feb 14 09:43:16 2017 +1100 Committer: Yi Committed: Tue Feb 14 09:43:16 2017 +1100 -- src/backend/commands/analyze.c | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/256c0668/src/backend/commands/analyze.c -- diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index ae869da..65d9bd3 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -309,8 +309,6 @@ void analyzeStmt(VacuumStmt *stmt, List *relids, int preferred_seg_num) int successCount = 0, failCount = 0; StringInfoData failNames; - initStringInfo(&failNames); - /** * Ensure that an ANALYZE is requested. */ @@ -362,6 +360,7 @@ void analyzeStmt(VacuumStmt *stmt, List *relids, int preferred_seg_num) MemoryContextSwitchTo(analyzeStatementContext); + initStringInfo(&failNames); /* * This is a per relation context. @@ -678,8 +677,12 @@ void analyzeStmt(VacuumStmt *stmt, List *relids, int preferred_seg_num) RelationGetRelationName(candidateRelation), edata->message); failCount += 1; + + /* failNames uses memory from statement level context */ + MemoryContextSwitchTo(analyzeStatementContext); appendStringInfo(&failNames, "%s", failCount == 1 ? "(" : ", "); appendStringInfo(&failNames, "%s", RelationGetRelationName(candidateRelation)); + MemoryContextSwitchTo(oldcontext); /* rollback this table's sub-transaction */ RollbackAndReleaseCurrentSubTransaction();
[06/35] incubator-hawq git commit: HAWQ-1298. Updated RPS properties and scripts (closes #1109)
HAWQ-1298. Updated RPS properties and scripts (closes #1109) Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/8c9b45a4 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/8c9b45a4 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/8c9b45a4 Branch: refs/heads/2.1.0.0-incubating Commit: 8c9b45a40ec55bd7ad1589de894962aaf1540f3a Parents: 23c45c7 Author: Alexander Denissov Authored: Tue Jan 31 14:31:06 2017 -0800 Committer: Alexander Denissov Committed: Thu Feb 9 11:13:54 2017 -0800 -- ranger-plugin/conf/catalina.properties | 83 +++ ranger-plugin/conf/log4j.properties | 58 +++ ranger-plugin/conf/ranger-hawq-audit.xml| 47 ++ ranger-plugin/conf/ranger-hawq-security.xml | 85 ranger-plugin/conf/rps.properties | 32 ++ ranger-plugin/conf/server.xml | 38 ++ ranger-plugin/conf/tomcat-server.xml| 60 --- .../service/tests/common/ServiceTestBase.java | 5 +- ranger-plugin/pom.xml | 67 ++- ranger-plugin/scripts/catalina.sh | 507 +++ ranger-plugin/scripts/enable-ranger-plugin.sh | 225 ranger-plugin/scripts/register_hawq.sh | 217 ranger-plugin/scripts/rps.sh| 82 ++- ranger-plugin/scripts/rps_env.sh| 30 -- ranger-plugin/scripts/setenv.sh | 23 + ranger-plugin/service/pom.xml | 42 +- .../authorization/RangerHawqAuthorizer.java | 6 +- .../apache/hawq/ranger/authorization/Utils.java | 17 +- .../service/src/main/resources/log4j.properties | 42 -- .../src/main/resources/ranger-hawq-security.xml | 92 .../service/src/main/resources/rps.properties | 17 - .../service/src/main/webapp/WEB-INF/web.xml | 20 +- .../hawq/ranger/authorization/UtilsTest.java| 20 +- .../src/test/resources/ranger-hawq-security.xml | 85 .../service/src/test/resources/rps.properties | 4 +- 25 files changed, 1363 insertions(+), 541 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/8c9b45a4/ranger-plugin/conf/catalina.properties -- diff --git a/ranger-plugin/conf/catalina.properties b/ranger-plugin/conf/catalina.properties new file mode 100644 index 000..7e10ef5 --- /dev/null +++ b/ranger-plugin/conf/catalina.properties @@ -0,0 +1,83 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# List of comma-separated packages that start with or equal this string +# will cause a security exception to be thrown when +# passed to checkPackageAccess unless the +# corresponding RuntimePermission ("accessClassInPackage."+package) has +# been granted. +package.access=sun.,org.apache.catalina.,org.apache.coyote.,org.apache.jasper.,\ +org.apache.naming.resources.,org.apache.tomcat.,sun.beans. +# +# List of comma-separated packages that start with or equal this string +# will cause a security exception to be thrown when +# passed to checkPackageDefinition unless the +# corresponding RuntimePermission ("defineClassInPackage."+package) has +# been granted. +# +# by default, no packages are restricted for definition, and none of +# the class loaders supplied with the JDK call checkPackageDefinition. +# +package.definition=sun.,java.,org.apache.catalina.,org.apache.coyote.,\ +org.apache.jasper.,org.apache.naming.,org.apache.tomcat. + +# +# +# List of comma-separated paths defining the contents of the "common" +# classloader. Prefixes should be used to define what is the repository type. +# Path may be relative to the CATALINA_HOME or CATALINA_BASE path or absolute. +# If left as blank,the JVM system loader will be used as Catalina's "common" +# loader. +# Examples: +# "foo": Add this folder as a class repository +# "foo/*.jar": Add all the JARs of the specified folder as class +# repositories +# "foo/bar.jar": Add bar.jar as a class repository +common.loader=${c
[11/35] incubator-hawq git commit: HAWQ-1324. Fixed crash at query cancel, signal hanler cannot call unsafe elog()
HAWQ-1324. Fixed crash at query cancel, signal hanler cannot call unsafe elog() Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/305e2bc7 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/305e2bc7 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/305e2bc7 Branch: refs/heads/2.1.0.0-incubating Commit: 305e2bc78c86e5b0f38fdce349e78e908b3afb65 Parents: 256c066 Author: Ming LI Authored: Mon Feb 13 15:16:53 2017 +0800 Committer: Ming LI Committed: Tue Feb 14 11:15:04 2017 +0800 -- src/backend/tcop/postgres.c | 2 -- 1 file changed, 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/305e2bc7/src/backend/tcop/postgres.c -- diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 7cbe206..2450136 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -3608,8 +3608,6 @@ ProcessInterrupts(void) if (QueryCancelPending) { - elog(LOG,"Process interrupt for 'query cancel pending'."); - QueryCancelPending = false; ImmediateInterruptOK = false; /* not idle anymore */ DisableNotifyInterrupt();
[17/35] incubator-hawq git commit: HAWQ-1323. Fix access mode of files in adding license header added to each source file in test directory
HAWQ-1323. Fix access mode of files in adding license header added to each source file in test directory Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/9a86ed8c Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/9a86ed8c Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/9a86ed8c Branch: refs/heads/2.1.0.0-incubating Commit: 9a86ed8c9356fc2133d36421116961b7a8ca2265 Parents: 55e5ab5 Author: amyrazz44 Authored: Tue Feb 14 17:00:37 2017 +0800 Committer: Ruilong Huo Committed: Tue Feb 14 17:18:53 2017 +0800 -- src/test/bench/create.sh | 0 src/test/bench/runwisc.sh| 0 src/test/bench/wholebench.sh | 0 src/test/feature/PreparedStatement/TestPreparedStatement.cpp | 0 src/test/feature/UDF/TestUDF.cpp | 0 src/test/feature/UDF/lib/function.c | 0 src/test/feature/ao/TestAoSnappy.cpp | 0 src/test/feature/parallel-run-feature-test.sh| 0 src/test/feature/partition/test_partition.cpp| 0 src/test/feature/test_main.cpp | 0 src/test/locale/sort-test.pl | 0 src/test/locale/sort-test.py | 0 src/test/mb/mbregress.sh | 0 src/test/performance/runtests.pl | 0 src/test/performance/start-pgsql.sh | 0 src/test/regress/atmsort.pl | 0 src/test/regress/checkinc.py | 0 src/test/regress/dld.pl | 0 src/test/regress/explain.pl | 0 src/test/regress/get_ereport.pl | 0 src/test/regress/gpdiff.pl | 0 src/test/regress/gpexclude.pl| 0 src/test/regress/gpsourcify.pl | 0 src/test/regress/gpstringsubs.pl | 0 src/test/regress/gptorment.pl| 0 src/test/regress/maketestschedule.py | 0 src/test/regress/regressplans.sh | 0 src/test/regress/upg2_wizard.pl | 0 src/test/unit/cmockery/cmockery.h| 0 src/test/unit/mock/mocker.py | 0 30 files changed, 0 insertions(+), 0 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/9a86ed8c/src/test/bench/create.sh -- diff --git a/src/test/bench/create.sh b/src/test/bench/create.sh old mode 100644 new mode 100755 http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/9a86ed8c/src/test/bench/runwisc.sh -- diff --git a/src/test/bench/runwisc.sh b/src/test/bench/runwisc.sh old mode 100644 new mode 100755 http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/9a86ed8c/src/test/bench/wholebench.sh -- diff --git a/src/test/bench/wholebench.sh b/src/test/bench/wholebench.sh old mode 100644 new mode 100755 http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/9a86ed8c/src/test/feature/PreparedStatement/TestPreparedStatement.cpp -- diff --git a/src/test/feature/PreparedStatement/TestPreparedStatement.cpp b/src/test/feature/PreparedStatement/TestPreparedStatement.cpp old mode 100644 new mode 100755 http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/9a86ed8c/src/test/feature/UDF/TestUDF.cpp -- diff --git a/src/test/feature/UDF/TestUDF.cpp b/src/test/feature/UDF/TestUDF.cpp old mode 100644 new mode 100755 http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/9a86ed8c/src/test/feature/UDF/lib/function.c -- diff --git a/src/test/feature/UDF/lib/function.c b/src/test/feature/UDF/lib/function.c old mode 100644 new mode 100755 http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/9a86ed8c/src/test/feature/ao/TestAoSnappy.cpp -- diff --git a/src/test/feature/ao/TestAoSnappy.cpp b/src/test/feature/ao/TestAoSnappy.cpp old mode 100644 new mode 100755 http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/9a86ed8c/src/test/feature/parallel-run-feature-test.sh ---
[08/35] incubator-hawq git commit: HAWQ-1319. Add ASF header to java files and remove jar files in function suite
HAWQ-1319. Add ASF header to java files and remove jar files in function suite Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/d7e6c196 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/d7e6c196 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/d7e6c196 Branch: refs/heads/2.1.0.0-incubating Commit: d7e6c196fcfb4badd50feab330ad85cf717cb171 Parents: 63c8562 Author: Ruilong Huo Authored: Fri Feb 10 13:34:21 2017 +0800 Committer: Ruilong Huo Committed: Fri Feb 10 17:25:10 2017 +0800 -- src/test/feature/UDF/TestUDF.cpp | 26 ++ src/test/feature/UDF/lib/function.c | 18 ++ src/test/feature/UDF/sql/PLJavaAdd.jar | Bin 654 -> 0 bytes src/test/feature/UDF/sql/PLJavaAdd.java | 18 ++ src/test/feature/UDF/sql/PLJavauAdd.jar | Bin 657 -> 0 bytes src/test/feature/UDF/sql/PLJavauAdd.java | 18 ++ 6 files changed, 80 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/d7e6c196/src/test/feature/UDF/TestUDF.cpp -- diff --git a/src/test/feature/UDF/TestUDF.cpp b/src/test/feature/UDF/TestUDF.cpp index 95509fb..f79843b 100755 --- a/src/test/feature/UDF/TestUDF.cpp +++ b/src/test/feature/UDF/TestUDF.cpp @@ -1,3 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + #include "gtest/gtest.h" #include "lib/command.h" @@ -200,6 +218,10 @@ TEST_F(TestUDF, TestUDFPljava) // run test if pljava language is enabled if (util.getQueryResult("SELECT lanname FROM pg_language WHERE lanname = 'java'") == "java") { + // prepare jar files + hawq::test::Command cmd("cd " + d_feature_test_root + "/UDF/sql/; javac PLJavaAdd.java; jar cf PLJavaAdd.jar PLJavaAdd.class"); + cmd.run(); + // copy jar files over hawq cluster std::string query = "SELECT string_agg('-h ' || hostname, ' ' ORDER BY hostname) FROM gp_segment_configuration;"; std::string hosts = util.getQueryResult(query); @@ -230,6 +252,10 @@ TEST_F(TestUDF, TestUDFPljavau) // run test if pljavau language is enabled if (util.getQueryResult("SELECT lanname FROM pg_language WHERE lanname = 'javau'") == "javau") { + // prepare jar files + hawq::test::Command cmd("cd " + d_feature_test_root + "/UDF/sql/; javac PLJavauAdd.java; jar cf PLJavauAdd.jar PLJavauAdd.class"); + cmd.run(); + // copy jar files over hawq cluster std::string query = "SELECT string_agg('-h ' || hostname, ' ' ORDER BY hostname) FROM gp_segment_configuration;"; std::string hosts = util.getQueryResult(query); http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/d7e6c196/src/test/feature/UDF/lib/function.c -- diff --git a/src/test/feature/UDF/lib/function.c b/src/test/feature/UDF/lib/function.c index d9a842b..b2d4b81 100755 --- a/src/test/feature/UDF/lib/function.c +++ b/src/test/feature/UDF/lib/function.c @@ -1,3 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the L
[14/35] incubator-hawq git commit: HAWQ-1323. Add license header added to each source file in test directory
HAWQ-1323. Add license header added to each source file in test directory Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/492b1a78 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/492b1a78 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/492b1a78 Branch: refs/heads/2.1.0.0-incubating Commit: 492b1a782351b8cf1701703773a6cacc6bd1a0cb Parents: 9918443 Author: amyrazz44 Authored: Tue Feb 14 10:08:03 2017 +0800 Committer: amyrazz44 Committed: Tue Feb 14 16:18:46 2017 +0800 -- pom.xml | 34 +++- src/test/amsd/testframe.h | 18 +++ src/test/bench/create.sh| 18 +++ src/test/bench/runwisc.sh | 18 +++ src/test/bench/wholebench.sh| 18 +++ src/test/examples/testlibpq.c | 18 +++ src/test/examples/testlibpq2.c | 18 +++ src/test/examples/testlibpq3.c | 18 +++ src/test/examples/testlibpq4.c | 18 +++ src/test/examples/testlo.c | 18 +++ src/test/feature/ExternalSource/lib/function.c | 18 +++ .../feature/ExternalSource/test_errortbl.cpp| 18 +++ .../ExternalSource/test_external_oid.cpp| 18 +++ src/test/feature/ExternalSource/test_exttab.cpp | 18 +++ .../feature/ManagementTool/test_hawq_register.h | 18 +++ .../test_hawq_register_partition.cpp| 18 +++ .../test_hawq_register_rollback.cpp | 18 +++ .../test_hawq_register_usage1.cpp | 18 +++ .../test_hawq_register_usage2_case1.cpp | 18 +++ .../test_hawq_register_usage2_case2.cpp | 18 +++ .../PreparedStatement/TestPreparedStatement.cpp | 18 +++ src/test/feature/UDF/TestUDF.cpp| 18 +++ src/test/feature/UDF/lib/function.c | 18 +++ src/test/feature/ao/TestAoSnappy.cpp| 18 +++ src/test/feature/catalog/test_alter_owner.cpp | 18 +++ src/test/feature/catalog/test_alter_table.cpp | 18 +++ src/test/feature/catalog/test_create_table.cpp | 18 +++ src/test/feature/catalog/test_guc.cpp | 18 +++ src/test/feature/catalog/test_type.cpp | 18 +++ src/test/feature/ddl/test_database.cpp | 18 +++ src/test/feature/lib/command.cpp| 18 +++ src/test/feature/lib/command.h | 18 +++ src/test/feature/lib/data_gen.cpp | 18 +++ src/test/feature/lib/data_gen.h | 18 +++ src/test/feature/lib/file_replace.cpp | 18 +++ src/test/feature/lib/file_replace.h | 18 +++ src/test/feature/lib/hawq_config.cpp| 18 +++ src/test/feature/lib/hawq_config.h | 18 +++ src/test/feature/lib/hawq_scp.cpp | 18 +++ src/test/feature/lib/hawq_scp.h | 18 +++ src/test/feature/lib/hdfs_config.cpp| 18 +++ src/test/feature/lib/hdfs_config.h | 18 +++ src/test/feature/lib/psql.cpp | 18 +++ src/test/feature/lib/psql.h | 18 +++ src/test/feature/lib/sql_util.cpp | 18 +++ src/test/feature/lib/sql_util.h | 18 +++ src/test/feature/lib/string_util.cpp| 18 +++ src/test/feature/lib/string_util.h | 18 +++ src/test/feature/lib/xml_parser.cpp | 18 +++ src/test/feature/lib/xml_parser.h | 18 +++ src/test/feature/lib/yarn_config.cpp| 18 +++ src/test/feature/lib/yarn_config.h | 18 +++ src/test/feature/parallel-run-feature-test.sh | 20 +++- src/test/feature/parquet/test_parquet.cpp | 18 +++ src/test/feature/partition/test_partition.cpp | 18 +++ src/test/feature/planner/test_subplan.cpp | 18 +++ src/test/feature/query/test_aggregate.cpp | 18 +++ .../query/test_create_type_composite.cpp| 18 +++ src/test/feature/query/test_gp_dist_random.cpp | 18 +++ .../feature/query/test_information_schema.cpp | 18 +++ src/test/feature/query/test_insert.cpp | 18 +++ .../feature/query/test_nested_case_null.cpp | 18 +++ src/test/feature/query/test_parser.cpp | 18 +++ src/test/feature/query/test_polymorphism.cpp| 18 +++ src/test/feature/query/test_portal.cpp | 18 +++ src/test/feature/query/test
[09/35] incubator-hawq git commit: HAWQ-1320. Remove reference to PXF rpm version in source
HAWQ-1320. Remove reference to PXF rpm version in source Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/1c36c28c Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/1c36c28c Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/1c36c28c Branch: refs/heads/2.1.0.0-incubating Commit: 1c36c28c82a05ac696806f136d912c5d5f4bcb47 Parents: d7e6c19 Author: shivzone Authored: Fri Feb 10 13:48:00 2017 -0800 Committer: shivzone Committed: Fri Feb 10 13:48:00 2017 -0800 -- pxf/Makefile | 4 pxf/gradle.properties | 1 - 2 files changed, 4 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/1c36c28c/pxf/Makefile -- diff --git a/pxf/Makefile b/pxf/Makefile index 81298d0..9095ccb 100644 --- a/pxf/Makefile +++ b/pxf/Makefile @@ -38,6 +38,10 @@ ifneq "$(VENDOR)" "" BUILD_PARAMS+= -Pvendor="$(VENDOR)" endif +ifneq "$(PXF_VERSION)" "" +BUILD_PARAMS+= -Pversion="$(PXF_VERSION)" +endif + help: @echo @echo"help it is then" http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/1c36c28c/pxf/gradle.properties -- diff --git a/pxf/gradle.properties b/pxf/gradle.properties index 2af17ef..3a02baf 100644 --- a/pxf/gradle.properties +++ b/pxf/gradle.properties @@ -15,7 +15,6 @@ # specific language governing permissions and limitations # under the License. -version=3.1.0.0 license=ASL 2.0 vendor=Apache HAWQ Incubating hadoopVersion=2.7.1
[24/35] incubator-hawq git commit: HAWQ-1338. Fixed writer process doesn't exit nicely in some case
HAWQ-1338. Fixed writer process doesn't exit nicely in some case Signed-off-by: Ivan Weng Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/946fe58e Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/946fe58e Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/946fe58e Branch: refs/heads/2.1.0.0-incubating Commit: 946fe58ee83eb445849a76524be894e34f822114 Parents: 571c50b Author: Ming LI Authored: Thu Feb 16 12:34:21 2017 +0800 Committer: Ivan Weng Committed: Thu Feb 16 12:47:11 2017 +0800 -- src/backend/storage/ipc/ipc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/946fe58e/src/backend/storage/ipc/ipc.c -- diff --git a/src/backend/storage/ipc/ipc.c b/src/backend/storage/ipc/ipc.c index dd7aa85..bff9a2c 100644 --- a/src/backend/storage/ipc/ipc.c +++ b/src/backend/storage/ipc/ipc.c @@ -149,7 +149,7 @@ proc_exit_prepare(int code) * desirable bail-out, and whenever you should see this situation, you * should consider to resolve the actual programming error. */ - if (CritSectionCount > 0) + if (CritSectionCount > 0 && !SuppressPanic) elog(PANIC, "process is dying from critical section"); /*
[20/35] incubator-hawq git commit: HAWQ-1331. Update hawq and pxf versions
http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/286049d7/tools/bin/gppylib/data/2.2.json -- diff --git a/tools/bin/gppylib/data/2.2.json b/tools/bin/gppylib/data/2.2.json new file mode 100644 index 000..019c378 --- /dev/null +++ b/tools/bin/gppylib/data/2.2.json @@ -0,0 +1,10458 @@ +{ + "__comment" : "Generated by tidycat.pl version 34 on Sat Jan 2 21:08:04 2016 CATALOG_VERSION_NO=201507221", + "__info" : { + "CATALOG_VERSION_NO" : "201507221" + }, + "gp_configuration" : { + "CamelCaseRelationId" : "GpConfigurationRelationId", + "colh" : { + "content" : "int2", + "datadir" : "text", + "dbid" : "int2", + "definedprimary" : "bool", + "hostname" : "NameData", + "isprimary" : "bool", + "port" : "int4", + "valid" : "bool" + }, + "cols" : [ + { +"colname" : "content", +"ctype" : "int2", +"precomment" : "\n**TK_BLANK_LINE**", +"sqltype" : "smallint" + }, + { +"colname" : "definedprimary", +"ctype" : "bool", +"sqltype" : "boolean" + }, + { +"colname" : "dbid", +"ctype" : "int2", +"sqltype" : "smallint" + }, + { +"colname" : "isprimary", +"ctype" : "bool", +"sqltype" : "boolean" + }, + { +"colname" : "valid", +"ctype" : "bool", +"sqltype" : "boolean" + }, + { +"colname" : "hostname", +"ctype" : "NameData", +"sqltype" : "name" + }, + { +"colname" : "port", +"ctype" : "int4", +"sqltype" : "integer" + }, + { +"colname" : "datadir", +"ctype" : "text", +"sqltype" : "text" + } + ], + "filename" : "gp_configuration.h", + "indexes" : [ + { +"CamelCaseIndexId" : "GpConfigurationContentDefinedprimaryIndexId", +"cols" : [ + [ + "content", + "int2_ops" + ], + [ + "definedprimary", + "bool_ops" + ] +], +"indexid" : "6101", +"unique" : "1", +"with" : { + "indexid" : "6101" +} + }, + { +"CamelCaseIndexId" : "GpConfigurationDbidIndexId", +"cols" : [ + [ + "dbid", + "int2_ops" + ] +], +"indexid" : "6102", +"unique" : "1", +"with" : { + "indexid" : "6102" +} + } + ], + "relid_comment_tag" : "/* relation id: 5000 - gp_configuration */\n", + "tabdef_text" : "\n CREATE TABLE gp_configuration\n with (shared=true, oid=false, relid=5000, content=MASTER_ONLY)\n (\n content smallint,\n definedprimary boolean,\n dbidsmallint,\n isprimary boolean,\n valid boolean,\n hostname name,\n portinteger,\n datadir text\n )", + "with" : { + "bootstrap" : 0, + "camelcase" : "GpConfiguration", + "content" : "MASTER_ONLY", + "oid" : "", + "relid" : "5000", + "shared" : "1", + "text" : "with (shared=true, oid=false, relid=5000, content=MASTER_ONLY)" + } + }, + "gp_configuration_history" : { + "CamelCaseRelationId" : "GpConfigHistoryRelationId", + "UppercaseReltypeOid" : "GP_CONFIGURATION_HISTORY_RELTYPE_OID", + "colh" : { + "description" : "text", + "hostname" : "text", + "registration_order" : "int4", + "time" : "timestamptz" + }, + "cols" : [ + { +"colname" : "time", +"ctype" : "timestamptz", +"precomment" : "\n**TK_BLANK_LINE**", +"sqltype" : "timestamp_with_time_zone" + }, + { +"colname" : "registration_order", +"ctype" : "int4", +"sqltype" : "integer" + }, + { +"colname" : "hostname", +"ctype" : "text", +"sqltype" : "text" + }, + { +"colname" : "description", +"ctype" : "text", +"sqltype" : "text" + } + ], + "filename" : "gp_configuration.h", + "relid_comment_tag" : "/* relation id: 5006 - gp_configuration_history */\n", + "tabdef_text" : "\n CREATE TABLE gp_configuration_history\n with (camelcase=GpConfigHistory, shared=true, oid=false, relid=5006, reltype_oid=6434, content=MASTER_ONLY)\n (\n time timestamp with time zone,\n registration_
[21/35] incubator-hawq git commit: HAWQ-1331. Update hawq and pxf versions
HAWQ-1331. Update hawq and pxf versions Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/286049d7 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/286049d7 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/286049d7 Branch: refs/heads/2.1.0.0-incubating Commit: 286049d7039c952e2769c03b2c01b43199435b97 Parents: d55014d Author: shivzone Authored: Wed Feb 15 14:41:12 2017 -0800 Committer: shivzone Committed: Wed Feb 15 14:41:12 2017 -0800 -- contrib/hawq-ambari-plugin/build.properties | 8 +- contrib/hawq-ambari-plugin/pom.xml | 2 +- getversion | 2 +- pxf/gradle.properties | 1 + ranger-plugin/admin-plugin/pom.xml | 4 +- ranger-plugin/integration/admin/pom.xml | 2 +- ranger-plugin/integration/pom.xml | 2 +- ranger-plugin/integration/service/pom.xml | 2 +- ranger-plugin/pom.xml | 4 +- ranger-plugin/service/pom.xml | 4 +- tools/bin/gppylib/data/2.2.json | 10458 + 11 files changed, 10474 insertions(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/286049d7/contrib/hawq-ambari-plugin/build.properties -- diff --git a/contrib/hawq-ambari-plugin/build.properties b/contrib/hawq-ambari-plugin/build.properties index ac9fcb9..aeb4259 100644 --- a/contrib/hawq-ambari-plugin/build.properties +++ b/contrib/hawq-ambari-plugin/build.properties @@ -1,8 +1,8 @@ -hawq.release.version=2.1.0 +hawq.release.version=2.2.0 hawq.common.services.version=2.0.0 -pxf.release.version=3.1.0 +pxf.release.version=3.2.0 pxf.common.services.version=3.0.0 hawq.repo.prefix=hawq hawq.addons.repo.prefix=hawq-add-ons -repository.version=2.1.0.0 -default.stack=HDP-2.5 \ No newline at end of file +repository.version=2.2.0.0 +default.stack=HDP-2.5 http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/286049d7/contrib/hawq-ambari-plugin/pom.xml -- diff --git a/contrib/hawq-ambari-plugin/pom.xml b/contrib/hawq-ambari-plugin/pom.xml index 2367a57..ad55e84 100644 --- a/contrib/hawq-ambari-plugin/pom.xml +++ b/contrib/hawq-ambari-plugin/pom.xml @@ -21,7 +21,7 @@ 4.0.0 org.apache.hawq hawq-ambari-plugin - 2.1.0.0 + 2.2.0.0 hawq-ambari-plugin http://maven.apache.org http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/286049d7/getversion -- diff --git a/getversion b/getversion index 6aa6adf..5617db4 100755 --- a/getversion +++ b/getversion @@ -18,7 +18,7 @@ # under the License. # -GP_VERSION="2.1.0.0" +GP_VERSION="2.2.0.0" GP_BUILDNUMBER=dev if [ -f BUILD_NUMBER ] ; then http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/286049d7/pxf/gradle.properties -- diff --git a/pxf/gradle.properties b/pxf/gradle.properties index 3a02baf..ba8ad2e 100644 --- a/pxf/gradle.properties +++ b/pxf/gradle.properties @@ -15,6 +15,7 @@ # specific language governing permissions and limitations # under the License. +version=3.2.0.0 license=ASL 2.0 vendor=Apache HAWQ Incubating hadoopVersion=2.7.1 http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/286049d7/ranger-plugin/admin-plugin/pom.xml -- diff --git a/ranger-plugin/admin-plugin/pom.xml b/ranger-plugin/admin-plugin/pom.xml index 67c0824..a444e97 100644 --- a/ranger-plugin/admin-plugin/pom.xml +++ b/ranger-plugin/admin-plugin/pom.xml @@ -28,7 +28,7 @@ org.apache.hawq ranger-plugin -2.1.0.0 +2.2.0.0 .. @@ -87,4 +87,4 @@ - \ No newline at end of file + http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/286049d7/ranger-plugin/integration/admin/pom.xml -- diff --git a/ranger-plugin/integration/admin/pom.xml b/ranger-plugin/integration/admin/pom.xml index 0a81941..3544c6f 100644 --- a/ranger-plugin/integration/admin/pom.xml +++ b/ranger-plugin/integration/admin/pom.xml @@ -29,7 +29,7 @@ org.apache.hawq ranger-plugin-integration -2.1.0.0 +2.2.0.0 .. http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/286049d7/ranger-plugin/integration/pom.xml -- diff --git a/ranger-plugin/integration/pom.xml b/ranger-plugin/integration/pom.xml index b6aac80..c2f144f 100644 --- a/range
[25/35] incubator-hawq git commit: HAWQ-1325. Allow queries related to pg_temp if ranger is enable
HAWQ-1325. Allow queries related to pg_temp if ranger is enable Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/d5238532 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/d5238532 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/d5238532 Branch: refs/heads/2.1.0.0-incubating Commit: d5238532e65f6b79dfcedf631d4138a109d32087 Parents: 946fe58 Author: Wen Lin Authored: Thu Feb 16 14:20:53 2017 +0800 Committer: Wen Lin Committed: Thu Feb 16 15:28:13 2017 +0800 -- src/backend/catalog/aclchk.c | 63 --- src/include/utils/acl.h | 1 + 2 files changed, 53 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/d5238532/src/backend/catalog/aclchk.c -- diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c index 6712cc8..f6ac590 100644 --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -2551,6 +2551,28 @@ char *getNamespaceNameFromOid(Oid object_oid) return tname.data; } +char *getNamespaceNameByOid(Oid object_oid) +{ + + StringInfoData tname; + initStringInfo(&tname); + + Assert(OidIsValid(object_oid)); + char* schema_name = caql_getcstring( + NULL, + cql("SELECT nspname FROM pg_namespace " + " WHERE oid = :1", + ObjectIdGetDatum(object_oid))); + if (schema_name == NULL) + { +return NULL; + } + + appendStringInfo(&tname, "%s", schema_name); + pfree(schema_name); + + return tname.data; +} char *getConversionNameFromOid(Oid object_oid) { Assert(OidIsValid(object_oid)); @@ -2707,8 +2729,27 @@ List *getActionName(AclMode mask) return actions; } -#define FALLBACK_IS_TRUE(x)x == PG_CATALOG_NAMESPACE || x == information_schema_namespcace_oid \ -|| x == PG_AOSEGMENT_NAMESPACE || x == PG_TOAST_NAMESPACE || x == PG_BITMAPINDEX_NAMESPACE +bool checkNamespaceFallback(Oid x) +{ + if (x == PG_CATALOG_NAMESPACE || x == information_schema_namespcace_oid + || x == PG_AOSEGMENT_NAMESPACE || x == PG_TOAST_NAMESPACE || x == PG_BITMAPINDEX_NAMESPACE) + { +return true; + } + else + { +char* name = getNamespaceNameByOid(x); +if (name != NULL && pg_strncasecmp(name, "pg_temp", strlen("pg_temp")) == 0) +{ + /* fall back pg_temp_XX check to native*/ + return true; +} +else +{ + return false; +} + } +} bool fallBackToNativeCheck(AclObjectKind objkind, Oid obj_oid, Oid roleid) { @@ -2717,23 +2758,23 @@ bool fallBackToNativeCheck(AclObjectKind objkind, Oid obj_oid, Oid roleid) */ if (information_schema_namespcace_oid == 0) { - information_schema_namespcace_oid = (int)get_namespace_oid("information_schema"); +information_schema_namespcace_oid = (int)get_namespace_oid("information_schema"); } /* for heap table, we fall back to native check. */ if (objkind == ACL_KIND_CLASS) { - Oid namespaceid = get_rel_namespace(obj_oid); - if(FALLBACK_IS_TRUE(namespaceid)) - { - return true; - } +Oid namespaceid = get_rel_namespace(obj_oid); +if(checkNamespaceFallback(namespaceid)) +{ + return true; +} } else if (objkind == ACL_KIND_NAMESPACE) { /* native check build-in schemas. */ -if(FALLBACK_IS_TRUE(obj_oid)) +if(checkNamespaceFallback(obj_oid)) { -return true; + return true; } else if (obj_oid == PG_PUBLIC_NAMESPACE && superuser()) { @@ -2745,7 +2786,7 @@ bool fallBackToNativeCheck(AclObjectKind objkind, Oid obj_oid, Oid roleid) { /* native check functions under build-in schemas. */ Oid namespaceid = get_func_namespace(obj_oid); -if (FALLBACK_IS_TRUE(namespaceid)) +if (checkNamespaceFallback(namespaceid)) { return true; } http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/d5238532/src/include/utils/acl.h -- diff --git a/src/include/utils/acl.h b/src/include/utils/acl.h index 863f5ae..d476e4d 100644 --- a/src/include/utils/acl.h +++ b/src/include/utils/acl.h @@ -327,6 +327,7 @@ extern char *getOperNameFromOid(Oid object_oid); extern char *getTypeNameFromOid(Oid object_oid); extern char *getLanguageNameFromOid(Oid object_oid); extern char *getNamespaceNameFromOid(Oid object_oid); +extern char *getNamespaceNameByOid(Oid object_oid); extern char *getConversionNameFromOid(Oid object_oid); extern char *getTablespaceNameFromOid(Oid object_oid); extern char *getFilespaceNameFromOid(Oid object_oid);
[22/35] incubator-hawq git commit: HAWQ-1331. Update hawq version in pom.xml
HAWQ-1331. Update hawq version in pom.xml Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/7ce195a3 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/7ce195a3 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/7ce195a3 Branch: refs/heads/2.1.0.0-incubating Commit: 7ce195a3f3fb553e8785f93cf03c8462ef61b402 Parents: 286049d Author: shivzone Authored: Wed Feb 15 16:03:39 2017 -0800 Committer: shivzone Committed: Wed Feb 15 16:03:39 2017 -0800 -- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/7ce195a3/pom.xml -- diff --git a/pom.xml b/pom.xml index 23ff24e..499d5bf 100644 --- a/pom.xml +++ b/pom.xml @@ -22,7 +22,7 @@ org.apache.hawq hawq - 2.1 + 2.2 pom
[19/35] incubator-hawq git commit: HAWQ-1315. Function validateResourcePoolStatus() in resourcepool.c is logging the wrong information
HAWQ-1315. Function validateResourcePoolStatus() in resourcepool.c is logging the wrong information Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/d55014db Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/d55014db Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/d55014db Branch: refs/heads/2.1.0.0-incubating Commit: d55014dba9026e77be7c315c6884c227130e4b7f Parents: 103373b Author: amyrazz44 Authored: Wed Feb 15 13:32:13 2017 +0800 Committer: Yi Committed: Wed Feb 15 22:45:09 2017 +1100 -- src/backend/resourcemanager/resourcepool.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/d55014db/src/backend/resourcemanager/resourcepool.c -- diff --git a/src/backend/resourcemanager/resourcepool.c b/src/backend/resourcemanager/resourcepool.c index 9485dbf..c8355d5 100644 --- a/src/backend/resourcemanager/resourcepool.c +++ b/src/backend/resourcemanager/resourcepool.c @@ -4143,8 +4143,8 @@ void validateResourcePoolStatus(bool refquemgr) "pool (%d MB, %lf CORE), maximum capacity (%d MB, %d CORE)", totalallocmem, totalalloccore, - core, - mem); + mem, + core); } /*
[05/35] incubator-hawq git commit: HAWQ-1298. Updated RPS properties and scripts (closes #1109)
http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/8c9b45a4/ranger-plugin/service/src/test/resources/ranger-hawq-security.xml -- diff --git a/ranger-plugin/service/src/test/resources/ranger-hawq-security.xml b/ranger-plugin/service/src/test/resources/ranger-hawq-security.xml new file mode 100644 index 000..6141df7 --- /dev/null +++ b/ranger-plugin/service/src/test/resources/ranger-hawq-security.xml @@ -0,0 +1,85 @@ + + + + + +http://www.w3.org/2001/XInclude";> + +ranger.plugin.hawq.service.name +hawq + +Name of the Ranger service containing policies for this HAWQ instance + + + + +ranger.plugin.hawq.policy.source.impl +org.apache.ranger.admin.client.RangerAdminRESTClient + +Class to retrieve policies from the source + + + + +ranger.plugin.hawq.policy.rest.url +http://localhost:6080 + +URL to Ranger Admin + + + + +ranger.plugin.hawq.policy.rest.ssl.config.file +/usr/local/hawq/ranger/etc/ranger-policymgr-ssl.xml + +Path to the file containing SSL details to contact Ranger Admin + + + + +ranger.plugin.hawq.policy.pollIntervalMs +3 + +How often to poll for changes in policies? + + + + +ranger.plugin.hawq.policy.cache.dir +/usr/local/hawq/ranger/policycache + +Directory where Ranger policies are cached after successful retrieval from the source + + + + +ranger.plugin.hawq.policy.rest.client.connection.timeoutMs +12 + +RangerRESTClient Connection Timeout in Milliseconds + + + + +ranger.plugin.hawq.policy.rest.client.read.timeoutMs +3 + +RangerRESTClient read Timeout in Milliseconds + + + http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/8c9b45a4/ranger-plugin/service/src/test/resources/rps.properties -- diff --git a/ranger-plugin/service/src/test/resources/rps.properties b/ranger-plugin/service/src/test/resources/rps.properties index 1fd50e5..042e4ff 100644 --- a/ranger-plugin/service/src/test/resources/rps.properties +++ b/ranger-plugin/service/src/test/resources/rps.properties @@ -13,5 +13,5 @@ # See the License for the specific language governing permissions and # limitations under the License. -ranger.hawq.instance=instance-test -version=version-test \ No newline at end of file +RANGER_HAWQ_INSTANCE=instance-test +RPS_VERSION=version-test \ No newline at end of file
[28/35] incubator-hawq git commit: HAWQ-1330. More ASF header updates (including pom.xml update)
HAWQ-1330. More ASF header updates (including pom.xml update) Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/b9dd7b59 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/b9dd7b59 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/b9dd7b59 Branch: refs/heads/2.1.0.0-incubating Commit: b9dd7b59e9220201f60e7281b5a4f565bb17489f Parents: 089210e Author: Ed Espino Authored: Tue Feb 14 11:26:16 2017 -0800 Committer: Ed Espino Committed: Wed Feb 15 23:57:13 2017 -0800 -- .travis.yml | 14 +++ pom.xml | 103 ++- .../src/main/resources/pxf-private.classpath| 15 +++ .../src/main/resources/pxf-privatehdp.classpath | 14 +++ .../src/main/resources/pxf-privatephd.classpath | 14 +++ .../src/main/resources/pxf-public.classpath | 14 +++ src/test/amsd/Makefile | 15 +++ src/test/examples/Makefile | 15 +++ src/test/feature/ExternalSource/lib/Makefile| 15 +++ src/test/feature/Makefile | 15 +++ .../feature/ManagementTool/files_incomplete.yml | 15 +++ .../ManagementTool/missing_bucketnum.yml| 15 +++ .../feature/ManagementTool/missing_checksum.yml | 15 +++ .../missing_constraint_partition.yml| 15 +++ .../feature/ManagementTool/missing_filesize.yml | 15 +++ .../feature/ManagementTool/missing_pagesize.yml | 15 +++ .../ManagementTool/missing_rowgroupsize.yml | 15 +++ .../constraint_not_exists_in_catalog_tpl.yml| 15 +++ .../different_partition_policy_tpl.yml | 15 +++ .../duplicate_partition_constraint_tpl.yml | 15 +++ .../partition/force_mode_normal_tpl.yml | 15 +++ .../partition/sub_set_catalog_table_tpl.yml | 15 +++ .../table_exists_list_partition_tpl.yml | 15 +++ .../table_exists_range_partition_tpl.yml| 15 +++ .../table_exists_table_file_not_exists_tpl.yml | 15 +++ .../table_not_exists_list_partition_tpl.yml | 15 +++ .../table_not_exists_range_partition_tpl.yml| 15 +++ ...ble_not_exists_table_file_not_exists_tpl.yml | 15 +++ .../usage2_include1/includedirectory_tpl.yml| 15 +++ .../usage2_include2/includedirectory_tpl.yml| 15 +++ .../ManagementTool/usage2case1/bucket0_tpl.yml | 15 +++ .../usage2case1/contain_error_symbol_tpl.yml| 15 +++ .../usage2case1/error_blocksize_tpl.yml | 15 +++ .../usage2case1/error_checksum_tpl.yml | 15 +++ .../usage2case1/error_columnnum_tpl.yml | 15 +++ .../usage2case1/error_compresslevel_tpl.yml | 15 +++ .../usage2case1/error_compresstype_tpl.yml | 15 +++ .../usage2case1/error_encoding_tpl.yml | 15 +++ .../usage2case1/error_format_tpl.yml| 15 +++ .../usage2case1/error_pagesize_tpl.yml | 15 +++ .../usage2case1/error_rowgroupsize_tpl.yml | 15 +++ .../usage2case1/error_schema_tpl.yml| 15 +++ .../file_under_table_directory_tpl.yml | 15 +++ .../usage2case1/float_eof_tpl.yml | 15 +++ .../usage2case1/includedirectory_tpl.yml| 15 +++ .../usage2case1/larger_eof_tpl.yml | 15 +++ .../usage2case1/minus_eof_tpl.yml | 15 +++ .../wrong_distributed_policy_tpl.yml| 15 +++ .../ManagementTool/usage2case1/zero_eof_tpl.yml | 15 +++ .../ManagementTool/usage2case2/bucket0_tpl.yml | 15 +++ .../usage2case2/contain_error_symbol_tpl.yml| 15 +++ .../usage2case2/error_blocksize_tpl.yml | 15 +++ .../usage2case2/error_checksum_tpl.yml | 15 +++ .../usage2case2/error_columnnum_tpl.yml | 15 +++ .../usage2case2/error_compresslevel_tpl.yml | 15 +++ .../usage2case2/error_compresstype_tpl.yml | 15 +++ .../usage2case2/error_encoding_tpl.yml | 15 +++ .../usage2case2/error_format_tpl.yml| 15 +++ .../usage2case2/error_pagesize_tpl.yml | 15 +++ .../usage2case2/error_rowgroupsize_tpl.yml | 15 +++ .../usage2case2/file_in_yaml_not_exist_tpl.yml | 15 +++ .../file_not_included_in_yaml_config_tpl.yml| 15 +++ .../usage2case2/float_eof_tpl.yml | 15 +++ .../usage2case2/hash_to_random_tpl.yml | 15 +++ .../usage2case2/larger_eof_tpl.yml | 15 +++ .../usage2case2/minus_eof_tpl.yml | 15 +++ .../usage2case2/normal_yaml_config_tpl.yml | 15 +++ .../normal_yaml_no_update_config_tpl.yml| 15 +++ .../usage2case2/random_to_hash_tpl.yml | 15 +++ .../usage2case2/same_path_yaml_config_tpl.yml | 15 +++ .../ManagementTool/usage2case2/t_tpl_1.yml | 15 +++ .../ManagementTool/usage2case2/t_tpl_2.yml | 15 +++ .../ManagementTool/usage2case2/t_tpl_3.yml | 15
[18/35] incubator-hawq git commit: HAWQ-1314. Added upgrade for function pxf_get_item_fields.
HAWQ-1314. Added upgrade for function pxf_get_item_fields. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/103373b7 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/103373b7 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/103373b7 Branch: refs/heads/2.1.0.0-incubating Commit: 103373b7f7cfabef181bd624a6ceb4d6e5095954 Parents: 9a86ed8 Author: Oleksandr Diachenko Authored: Tue Feb 14 18:20:02 2017 -0800 Committer: Oleksandr Diachenko Committed: Tue Feb 14 18:20:16 2017 -0800 -- tools/bin/hawqupgrade | 48 +++--- 1 file changed, 45 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/103373b7/tools/bin/hawqupgrade -- diff --git a/tools/bin/hawqupgrade b/tools/bin/hawqupgrade index a71f4d6..0352018 100755 --- a/tools/bin/hawqupgrade +++ b/tools/bin/hawqupgrade @@ -89,12 +89,52 @@ def register_func(options, args): except pg.DatabaseError as e: logger.error('%s', e) logger.info('Hawq Upgrade Failed.') -return 1 +sys.exit(1) except pg.InternalError: logger.error('Fail to connect to database, this script can only be run when database is up.') logger.info('Hawq Upgrade Failed.') -return 1 -logger.info('Hawq Upgrade Succeed.') +sys.exit(1) + +def pxf_get_item_fields_func(options, args): +# connect db +try: +url = dbconn.DbURL(hostname=options.host, port=options.port, + dbname="template1", username=options.user) +utility_conn = pg.connect(dbname=url.pgdb, host=url.pghost, port=url.pgport, + user=url.pguser, passwd=url.pgpass, opt='-c gp_session_role=utility') + + +# update function for template1 +update_func_query = """ +SET allow_system_table_mods = 'dml'; +UPDATE pg_proc +SET proallargtypes = '{25,25,25,25,25,25,25}', proargmodes = '{i,i,o,o,o,o,o}', proargnames = '{profile,pattern,path,itemname,fieldname,fieldtype,sourcefieldtype}' +WHERE proname = 'pxf_get_item_fields'; +""" + +utility_conn.query(update_func_query) +logger.info('Function pxf_get_item_fields successfully updated into database template1.') + # pxf_get_item_fields function for other databases +query = """select datname from pg_database;""" +dbs = utility_conn.query(query).dictresult() +for cur_db in dbs: +if cur_db['datname'] not in ['template1', 'template0', 'hcatalog']: +url = dbconn.DbURL(hostname=options.host, port=options.port, + dbname=cur_db['datname'], username=options.user) +existed_db_conn = pg.connect(dbname=cur_db['datname'], host=url.pghost, + port=url.pgport, user=url.pguser, passwd=url.pgpass) +existed_db_conn.query(update_func_query); +logger.info('Function pxf_get_item_fields successfully updated in database %s.',cur_db['datname']) +existed_db_conn.close() +utility_conn.close() +except pg.DatabaseError as e: +logger.error('%s', e) +logger.info('Hawq Upgrade Failed.') +sys.exit(1) +except pg.InternalError: +logger.error('Fail to connect to database, this script can only be run when database is up.') +logger.info('Hawq Upgrade Failed.') +sys.exit(1) if __name__ == '__main__': parser = option_parser() @@ -103,3 +143,5 @@ if __name__ == '__main__': parser.print_help(sys.stderr) sys.exit(1) register_func(options, args) +pxf_get_item_fields_func(options, args) +logger.info('Hawq Upgrade Succeed.') \ No newline at end of file
[32/35] incubator-hawq git commit: HAWQ-1322. Changed RPS RPM name to hawq-ranger-plugin_
HAWQ-1322. Changed RPS RPM name to hawq-ranger-plugin_ (closes #1131) Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/86084994 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/86084994 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/86084994 Branch: refs/heads/2.1.0.0-incubating Commit: 86084994c2a36d7726a93cb5c15801947cf69f0a Parents: 14df7bc Author: Alexander Denissov Authored: Fri Feb 10 12:13:23 2017 -0800 Committer: Alexander Denissov Committed: Fri Feb 17 12:05:03 2017 -0800 -- ranger-plugin/pom.xml | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/86084994/ranger-plugin/pom.xml -- diff --git a/ranger-plugin/pom.xml b/ranger-plugin/pom.xml index bf24df9..0846c48 100644 --- a/ranger-plugin/pom.xml +++ b/ranger-plugin/pom.xml @@ -68,18 +68,22 @@ ASL 2.0 org.apache.hawq.ranger +hawq-ranger-plugin_${hawq.name.version} gpadmin gpadmin HAWQ Ranger plugin. -${release.version} +${release.version}%{?dist} bigtop-tomcat hawq_${hawq.name.version} >= ${project.version} + /usr/local/hawq_${hawq.name.version}/ranger + + /usr/local/hawq_${hawq.name.version}/ranger/bin 750 @@ -177,7 +181,6 @@ -
[23/35] incubator-hawq git commit: HAWQ-1334. QD thread should set error code if failing so that the main process for the query could exit soon
HAWQ-1334. QD thread should set error code if failing so that the main process for the query could exit soon Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/571c50b0 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/571c50b0 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/571c50b0 Branch: refs/heads/2.1.0.0-incubating Commit: 571c50b07ec8a45a2203ec061fb6fbadfa7f3047 Parents: 7ce195a Author: Paul Guo Authored: Wed Feb 15 16:50:07 2017 +0800 Committer: Paul Guo Committed: Thu Feb 16 11:30:23 2017 +0800 -- src/backend/cdb/dispatcher_mgt.c | 44 + src/backend/cdb/executormgr.c| 62 +-- src/include/cdb/executormgr.h| 1 + 3 files changed, 69 insertions(+), 38 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/571c50b0/src/backend/cdb/dispatcher_mgt.c -- diff --git a/src/backend/cdb/dispatcher_mgt.c b/src/backend/cdb/dispatcher_mgt.c index 31c483e..5b1622d 100644 --- a/src/backend/cdb/dispatcher_mgt.c +++ b/src/backend/cdb/dispatcher_mgt.c @@ -281,22 +281,26 @@ dispmgt_thread_func_run(QueryExecutorGroup *group, struct WorkerMgrState *state) QueryExecutorInGroupIteratoriterator; struct DispatchData *data = group->team->refDispatchData; struct QueryExecutor*executor; + struct QueryExecutor*err_handle_executor = NULL; /* Assume the connections are already set up. */ dispmgt_init_query_executor_in_group_iterator(group, &iterator, false); while ((executor = dispmgt_get_query_executor_in_group_iterator(group, &iterator)) != NULL) { + if (err_handle_executor == NULL) + err_handle_executor = executor; + if (workermgr_should_query_stop(state)) { - write_log("+%s meets should query " - "stop before dispatching, entering error_cleanup", - __func__); + write_log("%s(): query is canceled before dispatching. " + "Will exit and clean up.", __func__); + err_handle_executor = executor; goto error_cleanup; } if (!executormgr_dispatch_and_run(data, executor)) { - write_log("+%s meets dispatch_and_run " - "problem when dispatching, entering error_cleanup", - __func__); + write_log("%s(): query cannot dispatch and run. " + "Will exit and clean up.", __func__); + err_handle_executor = executor; goto error_cleanup; } } @@ -310,9 +314,8 @@ dispmgt_thread_func_run(QueryExecutorGroup *group, struct WorkerMgrState *state) /* Check global state to abort query, this let poll process easier. */ if (workermgr_should_query_stop(state)){ - write_log("%s meets should query stop when " - "polling executors, entering error_cleanup", - __func__); + write_log("%s(): query is canceled before polling executors." + "Will exit and clean up.", __func__); goto error_cleanup; } /* Skip the stopped executor make the logic easy to understand. */ @@ -321,9 +324,10 @@ dispmgt_thread_func_run(QueryExecutorGroup *group, struct WorkerMgrState *state) { if (!executormgr_check_segment_status(executor)) { - write_log("Detected one segment (Global ID: %d) is down, so " - "abort the query that is running or will run on it", - executormgr_get_ID(executor)); + write_log("%s(): detected one segment (Global ID: %d) is down, " + "so abort the query that is running or will run on it", + __func__, executormgr_get_ID(executor)); + err_handle_executor = executor; goto error_cleanup; } @@ -359,6 +363,8 @@ dispmgt_thread_func_run(QueryExecut
[26/35] incubator-hawq git commit: HAWQ-1336. Travis CI "brew" updates
HAWQ-1336. Travis CI "brew" updates * Display brew version. * Display installed brew components and version after brew update. * Perform "reinstall" instead of "install" to avoid pre-existing component version errors. * Display installed brew components and version after brew work is complete. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/089210e8 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/089210e8 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/089210e8 Branch: refs/heads/2.1.0.0-incubating Commit: 089210e8e61a3e3797740837a81d41e0f793a50c Parents: d523853 Author: Ed Espino Authored: Wed Feb 15 11:03:43 2017 -0800 Committer: Ed Espino Committed: Wed Feb 15 23:47:28 2017 -0800 -- .travis.yml | 27 +++ 1 file changed, 15 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/089210e8/.travis.yml -- diff --git a/.travis.yml b/.travis.yml index 4dce250..c3a66a4 100644 --- a/.travis.yml +++ b/.travis.yml @@ -7,27 +7,30 @@ compiler: - clang before_install: + - brew --version - "brew update 2>&1 > /tmp/brew-update.txt || (cat /tmp/brew-update.txt && false)" + - brew list --versions install: - - brew install -protobuf -protobuf-c + - brew reinstall Gsasl -openssl -thrift -ccache -snappy -libevent bison +ccache cpanm +libevent maven - - brew reinstall python +openssl +protobuf +protobuf-c +python +snappy +thrift - brew outdated libyaml || brew upgrade libyaml - - brew outdated json-c || brew upgrade json-c - - brew outdated boost || brew upgrade boost - - brew outdated maven || brew upgrade maven + - brew outdated json-c || brew upgrade json-c + - brew outdated boost || brew upgrade boost + - brew outdated maven || brew upgrade maven - brew install iproute2mac + - brew list --versions - sudo pip install pycrypto - sudo cpanm install JSON
[16/35] incubator-hawq git commit: HAWQ-1326. Cancel the query earlier if one of the segments for the query crashes
HAWQ-1326. Cancel the query earlier if one of the segments for the query crashes Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/55e5ab5a Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/55e5ab5a Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/55e5ab5a Branch: refs/heads/2.1.0.0-incubating Commit: 55e5ab5a3eadfa9ba022db86826980cda870f1ce Parents: 17a9dc1 Author: Paul Guo Authored: Mon Feb 6 19:54:57 2017 +0800 Committer: Paul Guo Committed: Tue Feb 14 17:12:27 2017 +0800 -- src/backend/cdb/dispatcher_mgt.c | 32 + src/backend/cdb/executormgr.c | 32 - src/backend/resourcemanager/include/dynrm.h | 8 +++ src/backend/resourcemanager/resourcemanager.c | 77 ++ src/backend/resourcemanager/resourcepool.c| 30 + src/backend/storage/ipc/ipci.c| 6 +- src/backend/tcop/pquery.c | 13 ++-- src/include/cdb/cdbutil.h | 3 + src/include/cdb/executormgr.h | 2 + 9 files changed, 181 insertions(+), 22 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/55e5ab5a/src/backend/cdb/dispatcher_mgt.c -- diff --git a/src/backend/cdb/dispatcher_mgt.c b/src/backend/cdb/dispatcher_mgt.c index 8ed3402..31c483e 100644 --- a/src/backend/cdb/dispatcher_mgt.c +++ b/src/backend/cdb/dispatcher_mgt.c @@ -42,7 +42,6 @@ #endif #include "cdb/cdbconn.h" /* SOCK_ERRNO */ - typedef enum DispMgtConstant { DISPMGT_POLL_TIME = 2 * 1000, } DispMgtConstant; @@ -288,14 +287,16 @@ dispmgt_thread_func_run(QueryExecutorGroup *group, struct WorkerMgrState *state) while ((executor = dispmgt_get_query_executor_in_group_iterator(group, &iterator)) != NULL) { if (workermgr_should_query_stop(state)) { - write_log("+dispmgr_thread_func_run meets should query " - "stop before dispatching, entering error_cleanup"); + write_log("+%s meets should query " + "stop before dispatching, entering error_cleanup", + __func__); goto error_cleanup; } if (!executormgr_dispatch_and_run(data, executor)) { - write_log("+dispmgr_thread_func_run meets dispatch_and_run " - "problem when dispatching, entering error_cleanup"); + write_log("+%s meets dispatch_and_run " + "problem when dispatching, entering error_cleanup", + __func__); goto error_cleanup; } } @@ -309,14 +310,23 @@ dispmgt_thread_func_run(QueryExecutorGroup *group, struct WorkerMgrState *state) /* Check global state to abort query, this let poll process easier. */ if (workermgr_should_query_stop(state)){ - write_log("dispmgr_thread_func_run meets should query stop when " - "polling executors, entering error_cleanup"); + write_log("%s meets should query stop when " + "polling executors, entering error_cleanup", + __func__); goto error_cleanup; } /* Skip the stopped executor make the logic easy to understand. */ dispmgt_init_query_executor_in_group_iterator(group, &iterator, true); while ((executor = dispmgt_get_query_executor_in_group_iterator(group, &iterator)) != NULL) { + if (!executormgr_check_segment_status(executor)) + { + write_log("Detected one segment (Global ID: %d) is down, so " + "abort the query that is running or will run on it", + executormgr_get_ID(executor)); + goto error_cleanup; + } + /* * The fds array may shorter than executor array. * DO NOT mark executor stop! @@ -367,7 +377,7 @@ dispmgt_thread_func_run(QueryExecutorGroup *group, struct WorkerMgrState *state) dispmgt_init_query_executor_in_group_iterator(group, &iterator, true); while ((execu
[35/35] incubator-hawq git commit: HAWQ-1343. - Install pip and minor formatting change.
HAWQ-1343. - Install pip and minor formatting change. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/f5033eaa Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/f5033eaa Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/f5033eaa Branch: refs/heads/2.1.0.0-incubating Commit: f5033eaa3c7c1d9f85bbcc56e9d921d96337831a Parents: c7b3c62 Author: Ed Espino Authored: Fri Feb 17 07:50:00 2017 -0800 Committer: Ed Espino Committed: Sat Feb 18 14:31:09 2017 -0800 -- .travis.yml | 26 ++ 1 file changed, 14 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/f5033eaa/.travis.yml -- diff --git a/.travis.yml b/.travis.yml index b38852d..79d8bdc 100644 --- a/.travis.yml +++ b/.travis.yml @@ -27,24 +27,26 @@ before_install: install: - brew reinstall -Gsasl -bison -ccache -cpanm -libevent -maven -openssl -protobuf -protobuf-c -python -snappy -thrift + Gsasl + bison + ccache + cpanm + libevent + maven + openssl + python + snappy + - brew reinstall + protobuf + protobuf-c + thrift - brew outdated libyaml || brew upgrade libyaml - brew outdated json-c || brew upgrade json-c - brew outdated boost || brew upgrade boost - brew outdated maven || brew upgrade maven - brew install iproute2mac - brew list --versions + - sudo easy_install pip - sudo pip install pycrypto - sudo cpanm install JSON
[29/35] incubator-hawq git commit: HAWQ-1340. Remove unused file - tools/bin/disqual.ksh
HAWQ-1340. Remove unused file - tools/bin/disqual.ksh This closes #339 PR. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/93ed1847 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/93ed1847 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/93ed1847 Branch: refs/heads/2.1.0.0-incubating Commit: 93ed1847b0bdd65c8af49e1433999b361ff60572 Parents: b9dd7b5 Author: Ed Espino Authored: Thu Feb 16 10:18:06 2017 -0800 Committer: Ed Espino Committed: Thu Feb 16 10:18:06 2017 -0800 -- tools/bin/diskqual.ksh | 41 - 1 file changed, 41 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/93ed1847/tools/bin/diskqual.ksh -- diff --git a/tools/bin/diskqual.ksh b/tools/bin/diskqual.ksh deleted file mode 100644 index 7da0ab2..000 --- a/tools/bin/diskqual.ksh +++ /dev/null @@ -1,41 +0,0 @@ -#!/bin/ksh -# -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. -# - -disks=`format &1 | - nawk '$1 == "real" { printf("%.0f\n", 67.108864 / $2) }' -} - -getspeed() -{ - for iter in 1 2 3 - do - getspeed1 $1 - done | sort -n | tail -2 | head -1 -} - -for disk in $disks -do - echo $disk `getspeed $disk` MB/sec -done -
[27/35] incubator-hawq git commit: HAWQ-1330. More ASF header updates (including pom.xml update)
http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b9dd7b59/src/test/feature/ManagementTool/usage2case2/error_blocksize_tpl.yml -- diff --git a/src/test/feature/ManagementTool/usage2case2/error_blocksize_tpl.yml b/src/test/feature/ManagementTool/usage2case2/error_blocksize_tpl.yml index 02405da..1238a78 100644 --- a/src/test/feature/ManagementTool/usage2case2/error_blocksize_tpl.yml +++ b/src/test/feature/ManagementTool/usage2case2/error_blocksize_tpl.yml @@ -1,3 +1,18 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + AO_FileLocations: Blocksize: 32767 Checksum: false http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b9dd7b59/src/test/feature/ManagementTool/usage2case2/error_checksum_tpl.yml -- diff --git a/src/test/feature/ManagementTool/usage2case2/error_checksum_tpl.yml b/src/test/feature/ManagementTool/usage2case2/error_checksum_tpl.yml index e6f928e..29d413d 100644 --- a/src/test/feature/ManagementTool/usage2case2/error_checksum_tpl.yml +++ b/src/test/feature/ManagementTool/usage2case2/error_checksum_tpl.yml @@ -1,3 +1,18 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + Bucketnum: 6 DBVersion: PostgreSQL 8.2.15 (Greenplum Database 4.2.0 build 1) (HAWQ 2.0.1.0 build dev) on x86_64-apple-darwin14.5.0, compiled by GCC Apple LLVM version 6.1.0 (clang-602.0.53) http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b9dd7b59/src/test/feature/ManagementTool/usage2case2/error_columnnum_tpl.yml -- diff --git a/src/test/feature/ManagementTool/usage2case2/error_columnnum_tpl.yml b/src/test/feature/ManagementTool/usage2case2/error_columnnum_tpl.yml index d941316..f0380d3 100644 --- a/src/test/feature/ManagementTool/usage2case2/error_columnnum_tpl.yml +++ b/src/test/feature/ManagementTool/usage2case2/error_columnnum_tpl.yml @@ -1,3 +1,18 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + Bucketnum: 6 DBVersion: PostgreSQL 8.2.15 (Greenplum Database 4.2.0 build 1) (HAWQ 2.0.1.0 build dev) on x86_64-apple-darwin14.5.0, compiled by GCC Apple LLVM version 6.1.0 (clang-602.0.53) http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b9dd7b59/src/test/feature/ManagementTool/usage2case2/error_compresslevel_tpl.yml -- diff --git a/src/test/feature/ManagementTool/usage2case2/error_compresslevel_tpl.yml b/src/test/feature/ManagementTool/usage2case2/error_compresslevel_tpl.yml index fb127c2..4733770 100644 --- a/src/test/feature/ManagementTool/usage2case2/error_compresslevel_tpl.yml +++ b/src/test
[30/35] incubator-hawq git commit: HAWQ-1134. Add Bigtop layout specific pxf-private classpath
HAWQ-1134. Add Bigtop layout specific pxf-private classpath This closes #993 PR. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/f512b6a1 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/f512b6a1 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/f512b6a1 Branch: refs/heads/2.1.0.0-incubating Commit: f512b6a1ae7e7a471deca8c4fef87b949baf0516 Parents: 93ed184 Author: Roman Shaposhnik Authored: Tue Nov 1 16:03:49 2016 -0700 Committer: Roman Shaposhnik Committed: Fri Feb 17 09:39:16 2017 -0800 -- .../main/resources/pxf-privatebigtop.classpath | 47 1 file changed, 47 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/f512b6a1/pxf/pxf-service/src/main/resources/pxf-privatebigtop.classpath -- diff --git a/pxf/pxf-service/src/main/resources/pxf-privatebigtop.classpath b/pxf/pxf-service/src/main/resources/pxf-privatebigtop.classpath new file mode 100644 index 000..0241897 --- /dev/null +++ b/pxf/pxf-service/src/main/resources/pxf-privatebigtop.classpath @@ -0,0 +1,47 @@ +## +# This file contains the internal classpaths required to run PXF. +# WARNING: DO NOT EDIT! +# Any change in this file can result in PXF failing to run. +# Adding resources should be done using pxf-public.classpath file. +## +/etc/pxf/conf +/etc/hadoop/conf +/etc/hbase/conf +/etc/hive/conf +/usr/lib/hadoop/client/zookeeper.jar +/usr/lib/hadoop-hdfs/hadoop-hdfs.jar +/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar +/usr/lib/hadoop/client/hadoop-auth.jar +/usr/lib/hadoop/client/hadoop-common.jar +/usr/lib/hadoop/lib/asm-*[0-9].jar +/usr/lib/hadoop/lib/jersey-core-*[0-9].jar +/usr/lib/hadoop/lib/jersey-server-*[0-9].jar +/usr/lib/hadoop/client/avro-*[0-9].jar +/usr/lib/hadoop/client/commons-cli-*[0-9].jar +/usr/lib/hadoop/client/commons-codec-*[0-9].jar +/usr/lib/hadoop/client/commons-collections-*[0-9].jar +/usr/lib/hadoop/client/commons-configuration-*[0-9].jar +/usr/lib/hadoop/client/commons-io-*[0-9].jar +/usr/lib/hadoop/client/commons-lang-*[0-9].jar +/usr/lib/hadoop/client/commons-logging-*[0-9].jar +/usr/lib/hadoop/client/guava-*[0-9].jar +/usr/lib/hadoop/client/htrace-core-*[0-9]*.jar +/usr/lib/hadoop/client/hadoop-lzo-*[0-9].jar +/usr/lib/hadoop/client/jackson-core-asl-*[0-9].jar +/usr/lib/hadoop/client/jackson-mapper-asl-*[0-9].jar +/usr/lib/hadoop/client/jetty-*.jar +/usr/lib/hadoop/client/log4j-*[0-9].jar +/usr/lib/hadoop/client/protobuf-java-*[0-9].jar +/usr/lib/hadoop/client/slf4j-api-*[0-9].jar +/usr/lib/hadoop/client/snappy-java-*[0-9].jar +/usr/lib/hive/lib/antlr-runtime-*[0-9].jar +/usr/lib/hive/lib/datanucleus-api-jdo-*[0-9].jar +/usr/lib/hive/lib/datanucleus-core-*[0-9].jar +/usr/lib/hive/lib/hive-exec.jar +/usr/lib/hive/lib/hive-metastore.jar +/usr/lib/hive/lib/jdo-api-*[0-9].jar +/usr/lib/hive/lib/libfb303-*[0-9].jar +/usr/lib/pxf/pxf-hbase-*[0-9].jar +/usr/lib/pxf/pxf-hdfs-*[0-9].jar +/usr/lib/pxf/pxf-hive-*[0-9].jar +/usr/lib/pxf/pxf-json-*[0-9].jar
[33/35] incubator-hawq git commit: HAWQ-1344. Rollback Apache HAWQ version to 2.1, update PXF to 3.2
http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/c7b3c626/tools/bin/gppylib/data/2.2.json -- diff --git a/tools/bin/gppylib/data/2.2.json b/tools/bin/gppylib/data/2.2.json deleted file mode 100644 index 019c378..000 --- a/tools/bin/gppylib/data/2.2.json +++ /dev/null @@ -1,10458 +0,0 @@ -{ - "__comment" : "Generated by tidycat.pl version 34 on Sat Jan 2 21:08:04 2016 CATALOG_VERSION_NO=201507221", - "__info" : { - "CATALOG_VERSION_NO" : "201507221" - }, - "gp_configuration" : { - "CamelCaseRelationId" : "GpConfigurationRelationId", - "colh" : { - "content" : "int2", - "datadir" : "text", - "dbid" : "int2", - "definedprimary" : "bool", - "hostname" : "NameData", - "isprimary" : "bool", - "port" : "int4", - "valid" : "bool" - }, - "cols" : [ - { -"colname" : "content", -"ctype" : "int2", -"precomment" : "\n**TK_BLANK_LINE**", -"sqltype" : "smallint" - }, - { -"colname" : "definedprimary", -"ctype" : "bool", -"sqltype" : "boolean" - }, - { -"colname" : "dbid", -"ctype" : "int2", -"sqltype" : "smallint" - }, - { -"colname" : "isprimary", -"ctype" : "bool", -"sqltype" : "boolean" - }, - { -"colname" : "valid", -"ctype" : "bool", -"sqltype" : "boolean" - }, - { -"colname" : "hostname", -"ctype" : "NameData", -"sqltype" : "name" - }, - { -"colname" : "port", -"ctype" : "int4", -"sqltype" : "integer" - }, - { -"colname" : "datadir", -"ctype" : "text", -"sqltype" : "text" - } - ], - "filename" : "gp_configuration.h", - "indexes" : [ - { -"CamelCaseIndexId" : "GpConfigurationContentDefinedprimaryIndexId", -"cols" : [ - [ - "content", - "int2_ops" - ], - [ - "definedprimary", - "bool_ops" - ] -], -"indexid" : "6101", -"unique" : "1", -"with" : { - "indexid" : "6101" -} - }, - { -"CamelCaseIndexId" : "GpConfigurationDbidIndexId", -"cols" : [ - [ - "dbid", - "int2_ops" - ] -], -"indexid" : "6102", -"unique" : "1", -"with" : { - "indexid" : "6102" -} - } - ], - "relid_comment_tag" : "/* relation id: 5000 - gp_configuration */\n", - "tabdef_text" : "\n CREATE TABLE gp_configuration\n with (shared=true, oid=false, relid=5000, content=MASTER_ONLY)\n (\n content smallint,\n definedprimary boolean,\n dbidsmallint,\n isprimary boolean,\n valid boolean,\n hostname name,\n portinteger,\n datadir text\n )", - "with" : { - "bootstrap" : 0, - "camelcase" : "GpConfiguration", - "content" : "MASTER_ONLY", - "oid" : "", - "relid" : "5000", - "shared" : "1", - "text" : "with (shared=true, oid=false, relid=5000, content=MASTER_ONLY)" - } - }, - "gp_configuration_history" : { - "CamelCaseRelationId" : "GpConfigHistoryRelationId", - "UppercaseReltypeOid" : "GP_CONFIGURATION_HISTORY_RELTYPE_OID", - "colh" : { - "description" : "text", - "hostname" : "text", - "registration_order" : "int4", - "time" : "timestamptz" - }, - "cols" : [ - { -"colname" : "time", -"ctype" : "timestamptz", -"precomment" : "\n**TK_BLANK_LINE**", -"sqltype" : "timestamp_with_time_zone" - }, - { -"colname" : "registration_order", -"ctype" : "int4", -"sqltype" : "integer" - }, - { -"colname" : "hostname", -"ctype" : "text", -"sqltype" : "text" - }, - { -"colname" : "description", -"ctype" : "text", -"sqltype" : "text" - } - ], - "filename" : "gp_configuration.h", - "relid_comment_tag" : "/* relation id: 5006 - gp_configuration_history */\n", - "tabdef_text" : "\n CREATE TABLE gp_configuration_history\n with (camelcase=GpConfigHistory, shared=true, oid=false, relid=5006, reltype_oid=6434, content=MASTER_ONLY)\n (\n time timestamp with time zone,\n registrat
[34/35] incubator-hawq git commit: HAWQ-1344. Rollback Apache HAWQ version to 2.1, update PXF to 3.2
HAWQ-1344. Rollback Apache HAWQ version to 2.1, update PXF to 3.2 Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/c7b3c626 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/c7b3c626 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/c7b3c626 Branch: refs/heads/2.1.0.0-incubating Commit: c7b3c626fab3460ef3f5ca0d89a1c257491c518f Parents: 8608499 Author: shivzone Authored: Sat Feb 18 01:17:01 2017 -0800 Committer: shivzone Committed: Sat Feb 18 01:17:01 2017 -0800 -- contrib/hawq-ambari-plugin/build.properties | 4 +- contrib/hawq-ambari-plugin/pom.xml | 2 +- getversion | 2 +- pom.xml | 2 +- ranger-plugin/admin-plugin/pom.xml | 2 +- ranger-plugin/integration/admin/pom.xml | 2 +- ranger-plugin/integration/pom.xml | 2 +- ranger-plugin/integration/service/pom.xml | 2 +- ranger-plugin/pom.xml | 2 +- ranger-plugin/service/pom.xml | 2 +- tools/bin/gppylib/data/2.2.json | 10458 - 11 files changed, 11 insertions(+), 10469 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/c7b3c626/contrib/hawq-ambari-plugin/build.properties -- diff --git a/contrib/hawq-ambari-plugin/build.properties b/contrib/hawq-ambari-plugin/build.properties index aeb4259..696a550 100644 --- a/contrib/hawq-ambari-plugin/build.properties +++ b/contrib/hawq-ambari-plugin/build.properties @@ -1,8 +1,8 @@ -hawq.release.version=2.2.0 +hawq.release.version=2.1.0 hawq.common.services.version=2.0.0 pxf.release.version=3.2.0 pxf.common.services.version=3.0.0 hawq.repo.prefix=hawq hawq.addons.repo.prefix=hawq-add-ons -repository.version=2.2.0.0 +repository.version=2.1.0.0 default.stack=HDP-2.5 http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/c7b3c626/contrib/hawq-ambari-plugin/pom.xml -- diff --git a/contrib/hawq-ambari-plugin/pom.xml b/contrib/hawq-ambari-plugin/pom.xml index ad55e84..2367a57 100644 --- a/contrib/hawq-ambari-plugin/pom.xml +++ b/contrib/hawq-ambari-plugin/pom.xml @@ -21,7 +21,7 @@ 4.0.0 org.apache.hawq hawq-ambari-plugin - 2.2.0.0 + 2.1.0.0 hawq-ambari-plugin http://maven.apache.org http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/c7b3c626/getversion -- diff --git a/getversion b/getversion index 5617db4..ac43748 100755 --- a/getversion +++ b/getversion @@ -18,7 +18,7 @@ # under the License. # -GP_VERSION="2.2.0.0" +GP_VERSION="2.1.0.0-incubating" GP_BUILDNUMBER=dev if [ -f BUILD_NUMBER ] ; then http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/c7b3c626/pom.xml -- diff --git a/pom.xml b/pom.xml index 65813a3..6410958 100644 --- a/pom.xml +++ b/pom.xml @@ -22,7 +22,7 @@ org.apache.hawq hawq - 2.2 + 2.1 pom http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/c7b3c626/ranger-plugin/admin-plugin/pom.xml -- diff --git a/ranger-plugin/admin-plugin/pom.xml b/ranger-plugin/admin-plugin/pom.xml index a444e97..22ac044 100644 --- a/ranger-plugin/admin-plugin/pom.xml +++ b/ranger-plugin/admin-plugin/pom.xml @@ -28,7 +28,7 @@ org.apache.hawq ranger-plugin -2.2.0.0 +2.1.0.0 .. http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/c7b3c626/ranger-plugin/integration/admin/pom.xml -- diff --git a/ranger-plugin/integration/admin/pom.xml b/ranger-plugin/integration/admin/pom.xml index 3544c6f..0a81941 100644 --- a/ranger-plugin/integration/admin/pom.xml +++ b/ranger-plugin/integration/admin/pom.xml @@ -29,7 +29,7 @@ org.apache.hawq ranger-plugin-integration -2.2.0.0 +2.1.0.0 .. http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/c7b3c626/ranger-plugin/integration/pom.xml -- diff --git a/ranger-plugin/integration/pom.xml b/ranger-plugin/integration/pom.xml index c2f144f..b6aac80 100644 --- a/ranger-plugin/integration/pom.xml +++ b/ranger-plugin/integration/pom.xml @@ -29,7 +29,7 @@ org.apache.hawq ranger-plugin -2.2.0.0 +2.1.0.0 .. http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/c7b3c626/ranger-plugin/in
[31/35] incubator-hawq git commit: HAWQ-1134. Add Bigtop layout specific pxf-private classpath
HAWQ-1134. Add Bigtop layout specific pxf-private classpath Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/14df7bca Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/14df7bca Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/14df7bca Branch: refs/heads/2.1.0.0-incubating Commit: 14df7bca15eb77cdd705e5ee0f66f8e530f75c6c Parents: f512b6a Author: Roman Shaposhnik Authored: Fri Feb 17 09:48:01 2017 -0800 Committer: Roman Shaposhnik Committed: Fri Feb 17 09:48:01 2017 -0800 -- .../src/main/resources/pxf-privatebigtop.classpath | 15 +++ 1 file changed, 15 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/14df7bca/pxf/pxf-service/src/main/resources/pxf-privatebigtop.classpath -- diff --git a/pxf/pxf-service/src/main/resources/pxf-privatebigtop.classpath b/pxf/pxf-service/src/main/resources/pxf-privatebigtop.classpath index 0241897..8eff55c 100644 --- a/pxf/pxf-service/src/main/resources/pxf-privatebigtop.classpath +++ b/pxf/pxf-service/src/main/resources/pxf-privatebigtop.classpath @@ -1,3 +1,18 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + ## # This file contains the internal classpaths required to run PXF. # WARNING: DO NOT EDIT!
[incubator-hawq] Git Push Summary
Repository: incubator-hawq Updated Branches: refs/heads/2.1.0.0-incubating-RC3 [created] 12c7df017
incubator-hawq git commit: HAWQ-1343. - Install pip and minor formatting change.
Repository: incubator-hawq Updated Branches: refs/heads/master c7b3c626f -> f5033eaa3 HAWQ-1343. - Install pip and minor formatting change. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/f5033eaa Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/f5033eaa Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/f5033eaa Branch: refs/heads/master Commit: f5033eaa3c7c1d9f85bbcc56e9d921d96337831a Parents: c7b3c62 Author: Ed Espino Authored: Fri Feb 17 07:50:00 2017 -0800 Committer: Ed Espino Committed: Sat Feb 18 14:31:09 2017 -0800 -- .travis.yml | 26 ++ 1 file changed, 14 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/f5033eaa/.travis.yml -- diff --git a/.travis.yml b/.travis.yml index b38852d..79d8bdc 100644 --- a/.travis.yml +++ b/.travis.yml @@ -27,24 +27,26 @@ before_install: install: - brew reinstall -Gsasl -bison -ccache -cpanm -libevent -maven -openssl -protobuf -protobuf-c -python -snappy -thrift + Gsasl + bison + ccache + cpanm + libevent + maven + openssl + python + snappy + - brew reinstall + protobuf + protobuf-c + thrift - brew outdated libyaml || brew upgrade libyaml - brew outdated json-c || brew upgrade json-c - brew outdated boost || brew upgrade boost - brew outdated maven || brew upgrade maven - brew install iproute2mac - brew list --versions + - sudo easy_install pip - sudo pip install pycrypto - sudo cpanm install JSON
incubator-hawq git commit: HAWQ-1340. Remove unused file - tools/bin/disqual.ksh
Repository: incubator-hawq Updated Branches: refs/heads/master b9dd7b59e -> 93ed1847b HAWQ-1340. Remove unused file - tools/bin/disqual.ksh This closes #339 PR. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/93ed1847 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/93ed1847 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/93ed1847 Branch: refs/heads/master Commit: 93ed1847b0bdd65c8af49e1433999b361ff60572 Parents: b9dd7b5 Author: Ed Espino Authored: Thu Feb 16 10:18:06 2017 -0800 Committer: Ed Espino Committed: Thu Feb 16 10:18:06 2017 -0800 -- tools/bin/diskqual.ksh | 41 - 1 file changed, 41 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/93ed1847/tools/bin/diskqual.ksh -- diff --git a/tools/bin/diskqual.ksh b/tools/bin/diskqual.ksh deleted file mode 100644 index 7da0ab2..000 --- a/tools/bin/diskqual.ksh +++ /dev/null @@ -1,41 +0,0 @@ -#!/bin/ksh -# -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. -# - -disks=`format &1 | - nawk '$1 == "real" { printf("%.0f\n", 67.108864 / $2) }' -} - -getspeed() -{ - for iter in 1 2 3 - do - getspeed1 $1 - done | sort -n | tail -2 | head -1 -} - -for disk in $disks -do - echo $disk `getspeed $disk` MB/sec -done -
[1/2] incubator-hawq git commit: HAWQ-1330. More ASF header updates (including pom.xml update)
Repository: incubator-hawq Updated Branches: refs/heads/master 089210e8e -> b9dd7b59e http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b9dd7b59/src/test/feature/ManagementTool/usage2case2/error_blocksize_tpl.yml -- diff --git a/src/test/feature/ManagementTool/usage2case2/error_blocksize_tpl.yml b/src/test/feature/ManagementTool/usage2case2/error_blocksize_tpl.yml index 02405da..1238a78 100644 --- a/src/test/feature/ManagementTool/usage2case2/error_blocksize_tpl.yml +++ b/src/test/feature/ManagementTool/usage2case2/error_blocksize_tpl.yml @@ -1,3 +1,18 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + AO_FileLocations: Blocksize: 32767 Checksum: false http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b9dd7b59/src/test/feature/ManagementTool/usage2case2/error_checksum_tpl.yml -- diff --git a/src/test/feature/ManagementTool/usage2case2/error_checksum_tpl.yml b/src/test/feature/ManagementTool/usage2case2/error_checksum_tpl.yml index e6f928e..29d413d 100644 --- a/src/test/feature/ManagementTool/usage2case2/error_checksum_tpl.yml +++ b/src/test/feature/ManagementTool/usage2case2/error_checksum_tpl.yml @@ -1,3 +1,18 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + Bucketnum: 6 DBVersion: PostgreSQL 8.2.15 (Greenplum Database 4.2.0 build 1) (HAWQ 2.0.1.0 build dev) on x86_64-apple-darwin14.5.0, compiled by GCC Apple LLVM version 6.1.0 (clang-602.0.53) http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b9dd7b59/src/test/feature/ManagementTool/usage2case2/error_columnnum_tpl.yml -- diff --git a/src/test/feature/ManagementTool/usage2case2/error_columnnum_tpl.yml b/src/test/feature/ManagementTool/usage2case2/error_columnnum_tpl.yml index d941316..f0380d3 100644 --- a/src/test/feature/ManagementTool/usage2case2/error_columnnum_tpl.yml +++ b/src/test/feature/ManagementTool/usage2case2/error_columnnum_tpl.yml @@ -1,3 +1,18 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + Bucketnum: 6 DBVersion: PostgreSQL 8.2.15 (Greenplum Database 4.2.0 build 1) (HAWQ 2.0.1.0 build dev) on x86_64-apple-darwin14.5.0, compiled by GCC Apple LLVM version 6.1.0 (clang-602.0.53) http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b9dd7b59/src/test/feature/ManagementTool/usage2case2/error_compresslevel_tpl.yml -- diff --git a/src/test/feature/ManagementTool/usage2case2/error_compresslevel_tpl.yml b/src/test/feature/ManagementTool/usage2case2/error_compresslevel_tpl.yml index fb127c2..4733770 100644 --
[2/2] incubator-hawq git commit: HAWQ-1330. More ASF header updates (including pom.xml update)
HAWQ-1330. More ASF header updates (including pom.xml update) Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/b9dd7b59 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/b9dd7b59 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/b9dd7b59 Branch: refs/heads/master Commit: b9dd7b59e9220201f60e7281b5a4f565bb17489f Parents: 089210e Author: Ed Espino Authored: Tue Feb 14 11:26:16 2017 -0800 Committer: Ed Espino Committed: Wed Feb 15 23:57:13 2017 -0800 -- .travis.yml | 14 +++ pom.xml | 103 ++- .../src/main/resources/pxf-private.classpath| 15 +++ .../src/main/resources/pxf-privatehdp.classpath | 14 +++ .../src/main/resources/pxf-privatephd.classpath | 14 +++ .../src/main/resources/pxf-public.classpath | 14 +++ src/test/amsd/Makefile | 15 +++ src/test/examples/Makefile | 15 +++ src/test/feature/ExternalSource/lib/Makefile| 15 +++ src/test/feature/Makefile | 15 +++ .../feature/ManagementTool/files_incomplete.yml | 15 +++ .../ManagementTool/missing_bucketnum.yml| 15 +++ .../feature/ManagementTool/missing_checksum.yml | 15 +++ .../missing_constraint_partition.yml| 15 +++ .../feature/ManagementTool/missing_filesize.yml | 15 +++ .../feature/ManagementTool/missing_pagesize.yml | 15 +++ .../ManagementTool/missing_rowgroupsize.yml | 15 +++ .../constraint_not_exists_in_catalog_tpl.yml| 15 +++ .../different_partition_policy_tpl.yml | 15 +++ .../duplicate_partition_constraint_tpl.yml | 15 +++ .../partition/force_mode_normal_tpl.yml | 15 +++ .../partition/sub_set_catalog_table_tpl.yml | 15 +++ .../table_exists_list_partition_tpl.yml | 15 +++ .../table_exists_range_partition_tpl.yml| 15 +++ .../table_exists_table_file_not_exists_tpl.yml | 15 +++ .../table_not_exists_list_partition_tpl.yml | 15 +++ .../table_not_exists_range_partition_tpl.yml| 15 +++ ...ble_not_exists_table_file_not_exists_tpl.yml | 15 +++ .../usage2_include1/includedirectory_tpl.yml| 15 +++ .../usage2_include2/includedirectory_tpl.yml| 15 +++ .../ManagementTool/usage2case1/bucket0_tpl.yml | 15 +++ .../usage2case1/contain_error_symbol_tpl.yml| 15 +++ .../usage2case1/error_blocksize_tpl.yml | 15 +++ .../usage2case1/error_checksum_tpl.yml | 15 +++ .../usage2case1/error_columnnum_tpl.yml | 15 +++ .../usage2case1/error_compresslevel_tpl.yml | 15 +++ .../usage2case1/error_compresstype_tpl.yml | 15 +++ .../usage2case1/error_encoding_tpl.yml | 15 +++ .../usage2case1/error_format_tpl.yml| 15 +++ .../usage2case1/error_pagesize_tpl.yml | 15 +++ .../usage2case1/error_rowgroupsize_tpl.yml | 15 +++ .../usage2case1/error_schema_tpl.yml| 15 +++ .../file_under_table_directory_tpl.yml | 15 +++ .../usage2case1/float_eof_tpl.yml | 15 +++ .../usage2case1/includedirectory_tpl.yml| 15 +++ .../usage2case1/larger_eof_tpl.yml | 15 +++ .../usage2case1/minus_eof_tpl.yml | 15 +++ .../wrong_distributed_policy_tpl.yml| 15 +++ .../ManagementTool/usage2case1/zero_eof_tpl.yml | 15 +++ .../ManagementTool/usage2case2/bucket0_tpl.yml | 15 +++ .../usage2case2/contain_error_symbol_tpl.yml| 15 +++ .../usage2case2/error_blocksize_tpl.yml | 15 +++ .../usage2case2/error_checksum_tpl.yml | 15 +++ .../usage2case2/error_columnnum_tpl.yml | 15 +++ .../usage2case2/error_compresslevel_tpl.yml | 15 +++ .../usage2case2/error_compresstype_tpl.yml | 15 +++ .../usage2case2/error_encoding_tpl.yml | 15 +++ .../usage2case2/error_format_tpl.yml| 15 +++ .../usage2case2/error_pagesize_tpl.yml | 15 +++ .../usage2case2/error_rowgroupsize_tpl.yml | 15 +++ .../usage2case2/file_in_yaml_not_exist_tpl.yml | 15 +++ .../file_not_included_in_yaml_config_tpl.yml| 15 +++ .../usage2case2/float_eof_tpl.yml | 15 +++ .../usage2case2/hash_to_random_tpl.yml | 15 +++ .../usage2case2/larger_eof_tpl.yml | 15 +++ .../usage2case2/minus_eof_tpl.yml | 15 +++ .../usage2case2/normal_yaml_config_tpl.yml | 15 +++ .../normal_yaml_no_update_config_tpl.yml| 15 +++ .../usage2case2/random_to_hash_tpl.yml | 15 +++ .../usage2case2/same_path_yaml_config_tpl.yml | 15 +++ .../ManagementTool/usage2case2/t_tpl_1.yml | 15 +++ .../ManagementTool/usage2case2/t_tpl_2.yml | 15 +++ .../ManagementTool/usage2case2/t_tpl_3.yml | 15 +++ .../ManagementTool
incubator-hawq git commit: HAWQ-1336. Travis CI "brew" updates
Repository: incubator-hawq Updated Branches: refs/heads/master d5238532e -> 089210e8e HAWQ-1336. Travis CI "brew" updates * Display brew version. * Display installed brew components and version after brew update. * Perform "reinstall" instead of "install" to avoid pre-existing component version errors. * Display installed brew components and version after brew work is complete. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/089210e8 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/089210e8 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/089210e8 Branch: refs/heads/master Commit: 089210e8e61a3e3797740837a81d41e0f793a50c Parents: d523853 Author: Ed Espino Authored: Wed Feb 15 11:03:43 2017 -0800 Committer: Ed Espino Committed: Wed Feb 15 23:47:28 2017 -0800 -- .travis.yml | 27 +++ 1 file changed, 15 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/089210e8/.travis.yml -- diff --git a/.travis.yml b/.travis.yml index 4dce250..c3a66a4 100644 --- a/.travis.yml +++ b/.travis.yml @@ -7,27 +7,30 @@ compiler: - clang before_install: + - brew --version - "brew update 2>&1 > /tmp/brew-update.txt || (cat /tmp/brew-update.txt && false)" + - brew list --versions install: - - brew install -protobuf -protobuf-c + - brew reinstall Gsasl -openssl -thrift -ccache -snappy -libevent bison +ccache cpanm +libevent maven - - brew reinstall python +openssl +protobuf +protobuf-c +python +snappy +thrift - brew outdated libyaml || brew upgrade libyaml - - brew outdated json-c || brew upgrade json-c - - brew outdated boost || brew upgrade boost - - brew outdated maven || brew upgrade maven + - brew outdated json-c || brew upgrade json-c + - brew outdated boost || brew upgrade boost + - brew outdated maven || brew upgrade maven - brew install iproute2mac + - brew list --versions - sudo pip install pycrypto - sudo cpanm install JSON
svn commit: r18094 - in /dev/incubator/hawq/2.1.0.0-incubating.RC3: ./ apache-hawq-src-2.1.0.0-incubating.tar.gz apache-hawq-src-2.1.0.0-incubating.tar.gz.asc apache-hawq-src-2.1.0.0-incubating.tar.gz
Author: espino Date: Fri Feb 3 09:25:22 2017 New Revision: 18094 Log: Adding 2.1.0.0-incubating RC3 candidate release artifacts Added: dev/incubator/hawq/2.1.0.0-incubating.RC3/ dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz (with props) dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5 dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256 Added: dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz == Binary file - no diff available. Propchange: dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz -- svn:mime-type = application/octet-stream Added: dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc == --- dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc (added) +++ dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.asc Fri Feb 3 09:25:22 2017 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2 + +iQIcBAABCgAGBQJYlEppAAoJEPwGYvJXMlUiP0IP/iTljA++cBeS3ZtTy/6N5sQL +uJ06iSAzKXtmbK4T+wi5xtY1MHam78qzOzTsv+HsDuH9pKzsdZ/bNS21iHhcCHrO +SbfrVzfY0qp32PVpmJhIKVXvyGUIzGfzj1BuFqRYFp7MDv1FJVz5rtB9FzMik25B +uDb1pmtqoeaMPSeOx6OQusBAAyExiJ5dg7+zWf+UD8XsdYglG0Cbi00y//pe9A9n +xhDA5KaxoaCO+iGCA6UHNzNr7jUuFBFmWvjJchFh1DKAnkOLbpG4nAXgIbYJENuN +s/wsiYNoMzgNQOipLleU1eagasX4TvzYpNNuA85v3u4C3utQdE85FtvX59hZecdF +1ar+eLm8sUF7IU4OUHpENW1XF+cg5XCJT4luJ0fkWvvNbOZCrUlqoUHcGEAk7wRs +ybRRCr7pzJGtjn5vkyVdQoCZBf6lBkaEZu5IcG7TSEE+SM5B/9pNqYmNS4oijhXP +AnRAGslHcu83nww2NtTPaQxlFmyD7BYvbZeUetuGG2SA7K1RzE9umlFi+2OuOpRK +R5/6Edw9T2pObMUak2BSswojToflgswvJEMgFdchYcpGzw2PWtEOEWlDJzTPC3iH +pOsRbuX+3NGYkBnQQTI4W+3YRv7d8/oOboifdzW5H19cm9Lg4Rmo0zX3966Y/08W +sPD7XOMg+g5mktDHTdv+ +=RxYX +-END PGP SIGNATURE- Added: dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5 == --- dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5 (added) +++ dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.md5 Fri Feb 3 09:25:22 2017 @@ -0,0 +1 @@ +MD5 (apache-hawq-src-2.1.0.0-incubating.tar.gz) = 696b40766301e76ddf632a7187b0a667 Added: dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256 == --- dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256 (added) +++ dev/incubator/hawq/2.1.0.0-incubating.RC3/apache-hawq-src-2.1.0.0-incubating.tar.gz.sha256 Fri Feb 3 09:25:22 2017 @@ -0,0 +1 @@ +9c1e9e54e55a97744b6397c5ce7025a6d99dfd656b8d6ff7e65903b0cc02dfb6 apache-hawq-src-2.1.0.0-incubating.tar.gz
[incubator-hawq] Git Push Summary
Repository: incubator-hawq Updated Tags: refs/tags/2.1.0.0-incubating-rc3 [created] 699bb9cb7
[50/50] [abbrv] incubator-hawq git commit: [HAWQ-1236] - Update HAWQ DB version to 2.1.0.0-incubating.
[HAWQ-1236] - Update HAWQ DB version to 2.1.0.0-incubating. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/12c7df01 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/12c7df01 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/12c7df01 Branch: refs/heads/2.1.0.0-incubating Commit: 12c7df017551f1c3b0deb38c7243db3e018ef62c Parents: 7d02472 Author: Ed Espino Authored: Fri Dec 23 20:32:44 2016 -0800 Committer: Ed Espino Committed: Fri Feb 3 00:47:14 2017 -0800 -- getversion | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/12c7df01/getversion -- diff --git a/getversion b/getversion index 6aa6adf..ac43748 100755 --- a/getversion +++ b/getversion @@ -18,7 +18,7 @@ # under the License. # -GP_VERSION="2.1.0.0" +GP_VERSION="2.1.0.0-incubating" GP_BUILDNUMBER=dev if [ -f BUILD_NUMBER ] ; then
[20/50] [abbrv] incubator-hawq git commit: Revert "HAWQ-1248. Merge Dockerfiles for HAWQ Dev into HAWQ code base."
http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/1cb29096/contrib/hawq-docker/centos7-docker/hawq-test/conf/kms-site.xml -- diff --git a/contrib/hawq-docker/centos7-docker/hawq-test/conf/kms-site.xml b/contrib/hawq-docker/centos7-docker/hawq-test/conf/kms-site.xml deleted file mode 100644 index a810ca4..000 --- a/contrib/hawq-docker/centos7-docker/hawq-test/conf/kms-site.xml +++ /dev/null @@ -1,173 +0,0 @@ - - - - - - - -hadoop.kms.key.provider.uri -jceks://file@/${user.home}/kms.keystore - - URI of the backing KeyProvider for the KMS. - - - - -hadoop.security.keystore.JavaKeyStoreProvider.password -none - - If using the JavaKeyStoreProvider, the password for the keystore file. - - - - - - -hadoop.kms.cache.enable -true - - Whether the KMS will act as a cache for the backing KeyProvider. - When the cache is enabled, operations like getKeyVersion, getMetadata, - and getCurrentKey will sometimes return cached data without consulting - the backing KeyProvider. Cached values are flushed when keys are deleted - or modified. - - - - -hadoop.kms.cache.timeout.ms -60 - - Expiry time for the KMS key version and key metadata cache, in - milliseconds. This affects getKeyVersion and getMetadata. - - - - -hadoop.kms.current.key.cache.timeout.ms -3 - - Expiry time for the KMS current key cache, in milliseconds. This - affects getCurrentKey operations. - - - - - - -hadoop.kms.audit.aggregation.window.ms -1 - - Duplicate audit log events within the aggregation window (specified in - ms) are quashed to reduce log traffic. A single message for aggregated - events is printed at the end of the window, along with a count of the - number of aggregated events. - - - - - - -hadoop.kms.authentication.type -simple - - Authentication type for the KMS. Can be either "simple" - or "kerberos". - - - - -hadoop.kms.authentication.kerberos.keytab -${user.home}/kms.keytab - - Path to the keytab with credentials for the configured Kerberos principal. - - - - -hadoop.kms.authentication.kerberos.principal -HTTP/localhost - - The Kerberos principal to use for the HTTP endpoint. - The principal must start with 'HTTP/' as per the Kerberos HTTP SPNEGO specification. - - - - -hadoop.kms.authentication.kerberos.name.rules -DEFAULT - - Rules used to resolve Kerberos principal names. - - - - - - -hadoop.kms.authentication.signer.secret.provider -random - - Indicates how the secret to sign the authentication cookies will be - stored. Options are 'random' (default), 'string' and 'zookeeper'. - If using a setup with multiple KMS instances, 'zookeeper' should be used. - - - - - - - hadoop.kms.authentication.signer.secret.provider.zookeeper.path -/hadoop-kms/hadoop-auth-signature-secret - - The Zookeeper ZNode path where the KMS instances will store and retrieve - the secret from. - - - - - hadoop.kms.authentication.signer.secret.provider.zookeeper.connection.string -#HOSTNAME#:#PORT#,... - - The Zookeeper connection string, a list of hostnames and port comma - separated. - - - - - hadoop.kms.authentication.signer.secret.provider.zookeeper.auth.type -kerberos - - The Zookeeper authentication type, 'none' or 'sasl' (Kerberos). - - - - - hadoop.kms.authentication.signer.secret.provider.zookeeper.kerberos.keytab -/etc/hadoop/conf/kms.keytab - - The absolute path for the Kerberos keytab with the credentials to - connect to Zookeeper. - - - - - hadoop.kms.authentication.signer.secret.provider.zookeeper.kerberos.principal -kms/#HOSTNAME# - - The Kerberos service principal used to connect to Zookeeper. - - - - http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/1cb29096/contrib/hawq-docker/centos7-docker/hawq-test/conf/log4j.properties -- diff --git a/contrib/hawq-docker/centos7-docker/hawq-test/conf/log4j.properties b/contrib/hawq-docker/centos7-docker/hawq-test/conf/log4j.properties deleted file mode 100644 index c901ab1..000 --- a/contrib/hawq-docker/centos7-docker/hawq-test/conf/log4j.properties +++ /dev/null @@ -1,291 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License.
[07/50] [abbrv] incubator-hawq git commit: HAWQ-1246. Add generation of RequestID, ClientIP, queryContext(SQL Statement) and encapsulate these contents to JSON request to RPS.
HAWQ-1246. Add generation of RequestID, ClientIP, queryContext(SQL Statement) and encapsulate these contents to JSON request to RPS. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/60f09337 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/60f09337 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/60f09337 Branch: refs/heads/2.1.0.0-incubating Commit: 60f093372888fdead70ffea1d2b035c7d2bc343d Parents: 94239f5 Author: stanlyxiang Authored: Thu Dec 22 10:28:17 2016 +0800 Committer: hzhang2 Committed: Tue Jan 3 10:39:51 2017 +0800 -- src/backend/catalog/aclchk.c | 57 --- src/backend/libpq/rangerrest.c | 318 +++- src/include/utils/rangerrest.h | 11 +- 3 files changed, 134 insertions(+), 252 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/60f09337/src/backend/catalog/aclchk.c -- diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c index d3e4b64..d19a045 100644 --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -2732,7 +2732,7 @@ List *pg_rangercheck_batch(List *arg_list) } // foreach - RangerACLResult ret = check_privilege_from_ranger_batch(requestargs); + RangerACLResult ret = check_privilege_from_ranger(requestargs); ListCell *result; int k = 0; @@ -2752,7 +2752,7 @@ List *pg_rangercheck_batch(List *arg_list) (RangerRequestJsonArgs*)lfirst(tmp); pfree(requestarg->user); pfree(requestarg->object); - pfree(requestarg->actions); + list_free_deep(requestarg->actions); } list_free_deep(requestargs); @@ -2770,27 +2770,38 @@ AclResult pg_rangercheck(AclObjectKind objkind, Oid object_oid, Oid roleid, AclMode mask, AclMaskHow how) { - char* objectname = getNameFromOid(objkind, object_oid); - char* rolename = getRoleName(roleid); - List* actions = getActionName(mask); - bool isAll = (how == ACLMASK_ALL) ? true: false; - - elog(LOG, "rangeraclcheck kind:%d,objectname:%s,role:%s,mask:%u\n",objkind,objectname,rolename,mask); - int ret = check_privilege_from_ranger(rolename, objkind, objectname, actions, isAll); - - if(objectname){ -pfree(objectname); -objectname = NULL; - } - if(rolename){ -pfree(rolename); -rolename = NULL; - } - if(actions){ -list_free_deep(actions); -actions = NIL; - } - return ret; + char* objectname = getNameFromOid(objkind, object_oid); + char* rolename = getRoleName(roleid); + List* actions = getActionName(mask); + bool isAll = (how == ACLMASK_ALL) ? true: false; + + elog(LOG, "rangeraclcheck kind:%d,objectname:%s,role:%s,mask:%u\n",objkind,objectname,rolename,mask); + List *requestargs = NIL; + RangerRequestJsonArgs *requestarg = (RangerRequestJsonArgs *) palloc(sizeof(RangerRequestJsonArgs)); + requestarg->user = rolename; + requestarg->kind = objkind; + requestarg->object = objectname; + requestarg->actions = actions; + requestarg->isAll = isAll; + requestargs = lappend(requestargs, requestarg); + int ret = check_privilege_from_ranger(requestargs); + + if (requestargs) + { + ListCell *cell = list_head(requestargs); + while (cell != NULL) + { + ListCell *tmp = cell; + cell = lnext(cell); + RangerRequestJsonArgs* requestarg = (RangerRequestJsonArgs*) lfirst(tmp); + pfree(requestarg->user); + pfree(requestarg->object); + list_free_deep(requestarg->actions); + } + list_free_deep(requestargs); + requestargs = NULL; + } + return ret; } /* http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/60f09337/src/backend/libpq/rangerrest.c -- diff --git a/src/backend/libpq/rangerrest.c b/src/backend/libpq/rangerrest.c index 120f64f..56d30b5 100644 --- a/src/backend/libpq/rangerrest.c +++ b/src/backend/libpq/rangerrest.c @@ -49,10 +49,28 @@ char* AclObjectKindStr[] = "none" /* MUST BE LAST */ }; +static int request_id = 1; + +static void getClientIP(char *remote_host) +{ + if( MyProcPort->remote_host == NULL || strlen(MyProcPort->remote_host) == 0 ) + { + snprintf(remote_host, HOST_BUFFER_SIZE, "%s", "UNKNOWN"); + return; + } + if (strcmp(MyProcPort->remote_host, "[local]") == 0) + { + snprintf(remote_host, HOST_BUFFER_SIZE, "%s", "127.0.0.1"); + } + else +
[40/50] [abbrv] incubator-hawq git commit: HAWQ-1291. Fix the name of privilege when create temp table.
HAWQ-1291. Fix the name of privilege when create temp table. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/eb2ea907 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/eb2ea907 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/eb2ea907 Branch: refs/heads/2.1.0.0-incubating Commit: eb2ea9074460d911eeed5205970bd4076188c3fc Parents: d2608de Author: hzhang2 Authored: Tue Jan 24 14:57:13 2017 +0800 Committer: hzhang2 Committed: Tue Jan 24 14:57:13 2017 +0800 -- src/backend/catalog/aclchk.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/eb2ea907/src/backend/catalog/aclchk.c -- diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c index ed36330..33fa9ab 100644 --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -2646,7 +2646,7 @@ char *getNameFromOid(AclObjectKind objkind, Oid object_oid) char actionName[12][12] = {"INSERT","SELECT","UPDATE", "DELETE", "TRUNCATE", "REFERENCES", "TRIGGER", "EXECUTE", "USAGE", -"CREATE", "CREATE_TEMP", "CONNECT"}; +"CREATE", "TEMP", "CONNECT"}; List *getActionName(AclMode mask) {
[46/50] [abbrv] incubator-hawq git commit: HAWQ-1297. Fixed pxf classpath to work with custom hadoop distros
HAWQ-1297. Fixed pxf classpath to work with custom hadoop distros Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/524e2e50 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/524e2e50 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/524e2e50 Branch: refs/heads/2.1.0.0-incubating Commit: 524e2e501511b8fbd1c9d517e785b6f8d5ae883c Parents: 6fa1ced Author: Kavinder Dhaliwal Authored: Tue Jan 31 15:26:03 2017 -0800 Committer: Kavinder Dhaliwal Committed: Tue Jan 31 15:26:03 2017 -0800 -- pxf/build.gradle | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/524e2e50/pxf/build.gradle -- diff --git a/pxf/build.gradle b/pxf/build.gradle index 3c6d591..f273469 100644 --- a/pxf/build.gradle +++ b/pxf/build.gradle @@ -232,6 +232,7 @@ project('pxf-service') { from("src/main/resources") { into("/etc/pxf-${project.version}/conf") include("**/pxf-private*.classpath") +exclude("**/pxf-private.classpath") } from("src/main/resources/pxf-private${hddist}.classpath") {
[35/50] [abbrv] incubator-hawq git commit: HAWQ-1276. hawq should error out directly when ranger plugin service is unavailable, not a warning.
HAWQ-1276. hawq should error out directly when ranger plugin service is unavailable, not a warning. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/326fa4f9 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/326fa4f9 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/326fa4f9 Branch: refs/heads/2.1.0.0-incubating Commit: 326fa4f918b2be14adb3d667aea6cd707a24fa16 Parents: 21f1e29 Author: stanlyxiang Authored: Wed Jan 18 15:40:12 2017 +0800 Committer: hubertzhang Committed: Thu Jan 19 11:54:40 2017 +0800 -- src/backend/libpq/rangerrest.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/326fa4f9/src/backend/libpq/rangerrest.c -- diff --git a/src/backend/libpq/rangerrest.c b/src/backend/libpq/rangerrest.c index dc5d193..c920575 100644 --- a/src/backend/libpq/rangerrest.c +++ b/src/backend/libpq/rangerrest.c @@ -425,7 +425,7 @@ static int call_ranger_rest(CURL_HANDLE curl_handle, const char* request) /* check for errors */ if(res != CURLE_OK) { - elog(WARNING, "ranger plugin service from http://%s:%d/%s is unavailable : %s.\n", + elog(ERROR, "ranger plugin service from http://%s:%d/%s is unavailable : %s.\n", rps_addr_host, rps_addr_port, rps_addr_suffix, curl_easy_strerror(res)); } else @@ -467,7 +467,7 @@ int check_privilege_from_ranger(List *request_list, List *result_list) int ret = parse_ranger_response(curl_context_ranger.response.buffer, result_list); if (ret < 0) { - elog(WARNING, "parse ranger response failed, ranger response content is %s", + elog(ERROR, "parse ranger response failed, ranger response content is %s", curl_context_ranger.response.buffer == NULL? "empty.":curl_context_ranger.response.buffer); } if (curl_context_ranger.response.buffer != NULL)
[31/50] [abbrv] incubator-hawq git commit: HAWQ-1203. Ranger Plugin Service Implementation. (with contributions by Lav Jain and Leslie Chang) (close #1092)
http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/7f36b35b/ranger-plugin/integration/admin/src/test/java/org/apache/hawq/ranger/integration/admin/ListSchemasTest.java -- diff --git a/ranger-plugin/integration/admin/src/test/java/org/apache/hawq/ranger/integration/admin/ListSchemasTest.java b/ranger-plugin/integration/admin/src/test/java/org/apache/hawq/ranger/integration/admin/ListSchemasTest.java new file mode 100644 index 000..94372aa --- /dev/null +++ b/ranger-plugin/integration/admin/src/test/java/org/apache/hawq/ranger/integration/admin/ListSchemasTest.java @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific schema governing permissions and limitations + * under the License. + */ + +package org.apache.hawq.ranger.integration.admin; + +import com.google.common.collect.Sets; +import org.junit.Before; +import org.junit.Test; + +import java.util.List; +import java.util.Set; +import java.util.Map; +import java.util.HashMap; +import java.util.Arrays; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +public class ListSchemasTest extends LookupTestBase { + +private static final Set DEFAULT_SCHEMAS = Sets.newHashSet("public"); +private static final Set EAST_SCHEMAS = Sets.newHashSet("common", "japan", "public"); +private static final Set WEST_SCHEMAS = Sets.newHashSet("common", "france", "jamaica", "public"); +private static final Set ALL_SCHEMAS = Sets.newHashSet("common", "japan", "france", "jamaica", "public"); + +private Map> resources; + +@Before +public void setUp() { +resources = new HashMap<>(); +} + +@Test +public void testListSchema_NoResources() throws Exception { +resources.put("database", Arrays.asList("noschema_db")); +List result = service.lookupResource(getContext("schema", "*", resources)); +assertEquals(DEFAULT_SCHEMAS.size(), result.size()); +assertEquals(DEFAULT_SCHEMAS, Sets.newHashSet(result)); +} + +@Test +public void testListSchemas_SingleDb_AllFilter() throws Exception { +resources.put("database", Arrays.asList("east")); +List result = service.lookupResource(getContext("schema", "*", resources)); +assertEquals(EAST_SCHEMAS.size(), result.size()); +assertEquals(EAST_SCHEMAS, Sets.newHashSet(result)); +} + +@Test +public void testListSchemas_TwoDb_AllFilter() throws Exception { +resources.put("database", Arrays.asList("east", "west")); +List result = service.lookupResource(getContext("schema", "*", resources)); +assertEquals(ALL_SCHEMAS.size(), result.size()); +assertEquals(ALL_SCHEMAS, Sets.newHashSet(result)); +} + +@Test +public void testListSchemas_AllDb_AllFilter() throws Exception { +resources.put("database", Arrays.asList("*")); +List result = service.lookupResource(getContext("schema", "*", resources)); +assertEquals(ALL_SCHEMAS.size(), result.size()); +assertEquals(ALL_SCHEMAS, Sets.newHashSet(result)); +} + +@Test +public void testListSchemas_SingleDb_FilteredAbsent() throws Exception { +resources.put("database", Arrays.asList("east")); +List result = service.lookupResource(getContext("schema", "z", resources)); +assertTrue(result.isEmpty()); +} + +@Test +public void testListSchemas_TwoDb_FilteredAbsent() throws Exception { +resources.put("database", Arrays.asList("east", "west")); +List result = service.lookupResource(getContext("schema", "z", resources)); +assertTrue(result.isEmpty()); +} + +@Test +public void testListSchemas_AllDb_FilteredAbsent() throws Exception { +resources.put("database", Arrays.asList("*")); +List result = service.lookupResource(getContext("schema", "z", resources)); +assertTrue(result.isEmpty()); +} + +@Test +public void testListSchemas_SingleDb_FilteredPresent() throws Exception { +resources.put("database", Arrays.asList("east")); +List result = service.lookupResource(getContext("schema", "ja", resources)); +assertEquals(1, result.size()); +assertEquals(
[45/50] [abbrv] incubator-hawq git commit: HAWQ-1228. Use profile based on file format in HCatalog integration(HiveRC, HiveText profiles).
HAWQ-1228. Use profile based on file format in HCatalog integration(HiveRC, HiveText profiles). Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/6fa1ced2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/6fa1ced2 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/6fa1ced2 Branch: refs/heads/2.1.0.0-incubating Commit: 6fa1ced20e8bb2820b73e6904f77c4b4a1ed6de2 Parents: aac8868 Author: Oleksandr Diachenko Authored: Mon Jan 30 23:38:06 2017 -0800 Committer: Oleksandr Diachenko Committed: Mon Jan 30 23:38:50 2017 -0800 -- pxf/gradle.properties | 2 +- .../java/org/apache/hawq/pxf/api/Metadata.java | 51 +++- .../org/apache/hawq/pxf/api/OutputFormat.java | 37 ++- .../hawq/pxf/api/utilities/InputData.java | 1 + .../org/apache/hawq/pxf/api/MetadataTest.java | 2 +- .../hawq/pxf/plugins/hive/HiveAccessor.java | 14 +- .../plugins/hive/HiveColumnarSerdeResolver.java | 60 ++--- .../pxf/plugins/hive/HiveDataFragmenter.java| 28 +- .../plugins/hive/HiveInputFormatFragmenter.java | 41 --- .../pxf/plugins/hive/HiveLineBreakAccessor.java | 10 +- .../pxf/plugins/hive/HiveMetadataFetcher.java | 85 +++--- .../hawq/pxf/plugins/hive/HiveORCAccessor.java | 9 +- .../pxf/plugins/hive/HiveORCSerdeResolver.java | 44 +--- .../pxf/plugins/hive/HiveRCFileAccessor.java| 10 +- .../hawq/pxf/plugins/hive/HiveResolver.java | 107 .../plugins/hive/HiveStringPassResolver.java| 39 ++- .../hawq/pxf/plugins/hive/HiveUserData.java | 135 ++ .../hive/utilities/EnumHiveToHawqType.java | 31 ++- .../plugins/hive/utilities/HiveUtilities.java | 263 +++ .../plugins/hive/utilities/ProfileFactory.java | 61 + .../plugins/hive/HiveMetadataFetcherTest.java | 3 + .../pxf/plugins/hive/HiveORCAccessorTest.java | 9 +- .../hive/utilities/HiveUtilitiesTest.java | 53 .../hive/utilities/ProfileFactoryTest.java | 65 + .../hawq/pxf/service/BridgeOutputBuilder.java | 8 +- .../pxf/service/MetadataResponseFormatter.java | 3 +- .../apache/hawq/pxf/service/ProfileFactory.java | 45 .../hawq/pxf/service/rest/MetadataResource.java | 9 +- .../hawq/pxf/service/rest/VersionResource.java | 2 +- .../pxf/service/utilities/ProtocolData.java | 22 +- .../src/main/resources/pxf-profiles-default.xml | 14 +- .../service/MetadataResponseFormatterTest.java | 16 +- src/backend/access/external/fileam.c| 3 + src/backend/access/external/pxfheaders.c| 21 +- .../access/external/test/pxfheaders_test.c | 18 ++ src/backend/catalog/external/externalmd.c | 137 +++--- src/bin/gpfusion/gpbridgeapi.c | 6 +- src/include/access/hd_work_mgr.h| 2 + src/include/access/pxfheaders.h | 1 + src/include/access/pxfuriparser.h | 2 +- src/include/catalog/external/itemmd.h | 5 + src/include/catalog/pg_exttable.h | 14 +- .../regress/data/hcatalog/single_table.json | 2 +- .../data/hcatalog/single_table_text.json| 1 + src/test/regress/input/json_load.source | 12 +- src/test/regress/json_utils.c | 24 +- src/test/regress/output/json_load.source| 35 ++- 47 files changed, 1109 insertions(+), 453 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/6fa1ced2/pxf/gradle.properties -- diff --git a/pxf/gradle.properties b/pxf/gradle.properties index b003c56..2af17ef 100644 --- a/pxf/gradle.properties +++ b/pxf/gradle.properties @@ -23,5 +23,5 @@ hiveVersion=1.2.1 hbaseVersionJar=1.1.2 hbaseVersionRPM=1.1.2 tomcatVersion=7.0.62 -pxfProtocolVersion=v14 +pxfProtocolVersion=v15 osFamily=el6 http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/6fa1ced2/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java -- diff --git a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java index 9e1c137..bb22d41 100644 --- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java +++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java @@ -22,6 +22,8 @@ package org.apache.hawq.pxf.api; import java.util.ArrayList; import java.util.List; +import java.util.Map; +import java.util.Set; import org.apache.hawq.pxf.api.utilities.EnumHawqType; import org.apache.commons.lang.StringUtils; @@ -68,14 +70,16 @@ public class Metadata { } /** - * Class representing item field - name, type, source type,
[44/50] [abbrv] incubator-hawq git commit: HAWQ-1228. Use profile based on file format in HCatalog integration(HiveRC, HiveText profiles).
http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/6fa1ced2/pxf/pxf-hive/src/test/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilitiesTest.java -- diff --git a/pxf/pxf-hive/src/test/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilitiesTest.java b/pxf/pxf-hive/src/test/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilitiesTest.java index b736bba..05ef001 100644 --- a/pxf/pxf-hive/src/test/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilitiesTest.java +++ b/pxf/pxf-hive/src/test/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilitiesTest.java @@ -23,10 +23,18 @@ package org.apache.hawq.pxf.plugins.hive.utilities; import static org.junit.Assert.*; import java.util.Arrays; +import java.util.Collections; import com.google.common.base.Joiner; + import org.apache.hawq.pxf.api.io.DataType; import org.apache.hadoop.hive.metastore.api.FieldSchema; +import org.apache.hadoop.hive.metastore.api.SerDeInfo; +import org.apache.hadoop.hive.metastore.api.StorageDescriptor; +import org.apache.hadoop.hive.serde.serdeConstants; +import org.apache.hadoop.hive.ql.io.orc.OrcSerde; +import org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe; +import org.apache.hadoop.hive.serde2.*; import org.junit.Test; import org.apache.hawq.pxf.api.Metadata; import org.apache.hawq.pxf.api.UnsupportedTypeException; @@ -399,4 +407,49 @@ public class HiveUtilitiesTest { assertEquals(errorMsg, e.getMessage()); } } + +@Test +public void createDeserializer() throws Exception { +SerDe serde = HiveUtilities.createDeserializer(HiveUtilities.PXF_HIVE_SERDES.ORC_SERDE, HiveUtilities.PXF_HIVE_SERDES.ORC_SERDE); +assertTrue(serde instanceof OrcSerde); + +serde = HiveUtilities.createDeserializer(HiveUtilities.PXF_HIVE_SERDES.LAZY_BINARY_COLUMNAR_SERDE, HiveUtilities.PXF_HIVE_SERDES.LAZY_BINARY_COLUMNAR_SERDE); +assertTrue(serde instanceof org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe); + +serde = HiveUtilities.createDeserializer(HiveUtilities.PXF_HIVE_SERDES.COLUMNAR_SERDE, HiveUtilities.PXF_HIVE_SERDES.COLUMNAR_SERDE); +assertTrue(serde instanceof org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe); + +try { +serde = HiveUtilities.createDeserializer(HiveUtilities.PXF_HIVE_SERDES.COLUMNAR_SERDE, HiveUtilities.PXF_HIVE_SERDES.ORC_SERDE); +fail("shouldn't be able to create deserializer with not allowed serde"); +} catch (UnsupportedTypeException e) { +assertTrue(e.getMessage().equals("Unsupported Hive Serde: " + HiveUtilities.PXF_HIVE_SERDES.COLUMNAR_SERDE.name())); +} +} + +@Test +public void getDelimiterCode() { + +//Default delimiter code should be 44(comma) +Integer delimiterCode = HiveUtilities.getDelimiterCode(null); +char defaultDelim = ','; +assertTrue(delimiterCode == (int) defaultDelim); + +//Some serdes use FIELD_DELIM key +char expectedDelim = '%'; +StorageDescriptor sd = new StorageDescriptor(); +SerDeInfo si = new SerDeInfo(); +si.setParameters(Collections.singletonMap(serdeConstants.FIELD_DELIM, String.valueOf(expectedDelim))); +sd.setSerdeInfo(si); +delimiterCode = HiveUtilities.getDelimiterCode(sd); +assertTrue(delimiterCode == (int) expectedDelim); + +//Some serdes use SERIALIZATION_FORMAT key +sd = new StorageDescriptor(); +si = new SerDeInfo(); + si.setParameters(Collections.singletonMap(serdeConstants.SERIALIZATION_FORMAT, String.valueOf((int)expectedDelim))); +sd.setSerdeInfo(si); +delimiterCode = HiveUtilities.getDelimiterCode(sd); +assertTrue(delimiterCode == (int) expectedDelim); +} } http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/6fa1ced2/pxf/pxf-hive/src/test/java/org/apache/hawq/pxf/plugins/hive/utilities/ProfileFactoryTest.java -- diff --git a/pxf/pxf-hive/src/test/java/org/apache/hawq/pxf/plugins/hive/utilities/ProfileFactoryTest.java b/pxf/pxf-hive/src/test/java/org/apache/hawq/pxf/plugins/hive/utilities/ProfileFactoryTest.java new file mode 100644 index 000..d588d35 --- /dev/null +++ b/pxf/pxf-hive/src/test/java/org/apache/hawq/pxf/plugins/hive/utilities/ProfileFactoryTest.java @@ -0,0 +1,65 @@ +package org.apache.hawq.pxf.plugins.hive.utilities; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the L
[14/50] [abbrv] incubator-hawq git commit: HAWQ-1257. Prompt all tables which user doesn't have right once
HAWQ-1257. Prompt all tables which user doesn't have right once Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/bf4742cb Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/bf4742cb Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/bf4742cb Branch: refs/heads/2.1.0.0-incubating Commit: bf4742cb6c4cd71c9174f030c54e28b8c2595942 Parents: 3b15739 Author: interma Authored: Wed Jan 11 14:51:21 2017 +0800 Committer: Wen Lin Committed: Thu Jan 12 17:45:20 2017 +0800 -- src/backend/catalog/aclchk.c| 60 +--- src/backend/libpq/rangerrest.c | 139 -- src/backend/parser/parse_relation.c | 163 ++- src/include/utils/rangerrest.h | 13 ++- 4 files changed, 255 insertions(+), 120 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/bf4742cb/src/backend/catalog/aclchk.c -- diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c index 01a4f94..73de11b 100644 --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -2710,17 +2710,23 @@ List *pg_rangercheck_batch(List *arg_list) List *aclresults = NIL; List *requestargs = NIL; ListCell *arg; + elog(LOG, "rangeracl batch check, acl list length:%d\n", arg_list->length); foreach(arg, arg_list) { RangerPrivilegeArgs *arg_ptr = (RangerPrivilegeArgs *) lfirst(arg); + AclObjectKind objkind = arg_ptr->objkind; Oid object_oid = arg_ptr->object_oid; char *objectname = getNameFromOid(objkind, object_oid); char *rolename = getRoleName(arg_ptr->roleid); List* actions = getActionName(arg_ptr->mask); bool isAll = (arg_ptr->how == ACLMASK_ALL) ? true: false; + RangerPrivilegeResults *aclresult = (RangerPrivilegeResults *) palloc(sizeof(RangerPrivilegeResults)); -aclresult->result = -1; +aclresult->result = RANGERCHECK_NO_PRIV; aclresult->relOid = object_oid; +// this two sign fields will be set in create_ranger_request_json() +aclresult->resource_sign = 0; +aclresult->privilege_sign = 0; aclresults = lappend(aclresults, aclresult); RangerRequestJsonArgs *requestarg = (RangerRequestJsonArgs *) palloc(sizeof(RangerRequestJsonArgs)); @@ -2733,14 +2739,15 @@ List *pg_rangercheck_batch(List *arg_list) } // foreach - RangerACLResult ret = check_privilege_from_ranger(requestargs); - - ListCell *result; - int k = 0; - foreach(result, aclresults) { -RangerPrivilegeResults *result_ptr = (RangerPrivilegeResults *) lfirst(result); -result_ptr->result = ret; -++k; + int ret = check_privilege_from_ranger(requestargs, aclresults); + if (ret < 0) + { + elog(WARNING, "ranger service unavailable or unexpected error\n"); + ListCell *result; + foreach(result, aclresults) { + RangerPrivilegeResults *result_ptr = (RangerPrivilegeResults *) lfirst(result); + result_ptr->result = RANGERCHECK_NO_PRIV; + } } if(requestargs) { @@ -2760,10 +2767,6 @@ List *pg_rangercheck_batch(List *arg_list) requestargs = NULL; } - if(ret != RANGERCHECK_OK){ -elog(ERROR, "ACL check failed\n"); - } - elog(LOG, "oids%d\n", arg_list->length); return aclresults; } @@ -2777,6 +2780,16 @@ pg_rangercheck(AclObjectKind objkind, Oid object_oid, Oid roleid, bool isAll = (how == ACLMASK_ALL) ? true: false; elog(LOG, "rangeraclcheck kind:%d,objectname:%s,role:%s,mask:%u\n",objkind,objectname,rolename,mask); + + List *resultargs = NIL; +RangerPrivilegeResults *aclresult = (RangerPrivilegeResults *) palloc(sizeof(RangerPrivilegeResults)); +aclresult->result = RANGERCHECK_NO_PRIV; +aclresult->relOid = object_oid; + // this two sign fields will be set in create_ranger_request_json() + aclresult->resource_sign = 0; + aclresult->privilege_sign = 0; +resultargs = lappend(resultargs, aclresult); + List *requestargs = NIL; RangerRequestJsonArgs *requestarg = (RangerRequestJsonArgs *) palloc(sizeof(RangerRequestJsonArgs)); requestarg->user = rolename; @@ -2785,8 +2798,25 @@ pg_rangercheck(AclObjectKind objkind, Oid object_oid, Oid roleid, requestarg->actions = actions; requestarg->isAll = isAll; requestargs = lappend(requestargs, requestarg); - int ret = check_privilege_from_ranger(requestargs); + AclResult result = ACLCHECK_NO_PRIV; + int ret = check_privilege_from_ranger(requestargs, resultargs); + if (ret == 0) + { + ListCell *arg; + foreach(arg, resultargs) { + // only one element +
[09/50] [abbrv] incubator-hawq git commit: HAWQ-1249. Don't do ACL checks on segments
HAWQ-1249. Don't do ACL checks on segments Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/2f5910f2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/2f5910f2 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/2f5910f2 Branch: refs/heads/2.1.0.0-incubating Commit: 2f5910f2b0c2877e524c4c428ed963255c176378 Parents: 8d22582 Author: Chunling Wang Authored: Mon Jan 9 14:35:11 2017 +0800 Committer: Chunling Wang Committed: Mon Jan 9 14:35:11 2017 +0800 -- src/backend/catalog/aclchk.c| 85 +++- src/backend/executor/execMain.c | 37 +- src/backend/parser/parse_relation.c | 35 +++-- 3 files changed, 72 insertions(+), 85 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/2f5910f2/src/backend/catalog/aclchk.c -- diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c index d19a045..01a4f94 100644 --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -224,8 +224,9 @@ restrict_and_check_grant(bool is_grant, AclMode avail_goptions, bool all_privs, * If we found no grant options, consider whether to issue a hard error. * Per spec, having any privilege at all on the object will get you by * here. +* QE bypass all permission checking. */ - if (avail_goptions == ACL_NO_RIGHTS) + if (avail_goptions == ACL_NO_RIGHTS && Gp_role != GP_ROLE_EXECUTE) { if (enable_ranger && !fallBackToNativeCheck(objkind, objectId, grantorId)) { if (pg_rangercheck(objkind, objectId, grantorId, @@ -2948,9 +2949,9 @@ pg_class_aclmask(Oid table_oid, Oid roleid, } } /* -* Otherwise, superusers or on QE bypass all permission-checking. +* Otherwise, superusers bypass all permission-checking. */ - if (GP_ROLE_EXECUTE == Gp_role || superuser_arg(roleid)) + if (superuser_arg(roleid)) { #ifdef ACLDEBUG elog(DEBUG2, "OID %u is superuser, home free", roleid); @@ -3006,8 +3007,8 @@ pg_database_aclmask(Oid db_oid, Oid roleid, Oid ownerId; cqContext *pcqCtx; - /* Superusers or on QE bypass all permission checking. */ - if (GP_ROLE_EXECUTE == Gp_role || superuser_arg(roleid)) + /* Superusers bypass all permission checking. */ + if (superuser_arg(roleid)) return mask; /* @@ -3069,8 +3070,8 @@ pg_proc_aclmask(Oid proc_oid, Oid roleid, Oid ownerId; cqContext *pcqCtx; - /* Superusers or on QE bypass all permission checking. */ - if (GP_ROLE_EXECUTE == Gp_role || superuser_arg(roleid)) + /* Superusers bypass all permission checking. */ + if (superuser_arg(roleid)) return mask; /* @@ -3131,8 +3132,8 @@ pg_language_aclmask(Oid lang_oid, Oid roleid, Oid ownerId; cqContext *pcqCtx; - /* Superusers or on QE bypass all permission checking. */ - if (GP_ROLE_EXECUTE == Gp_role || superuser_arg(roleid)) + /* Superusers bypass all permission checking. */ + if (superuser_arg(roleid)) return mask; /* @@ -3194,8 +3195,8 @@ pg_namespace_aclmask(Oid nsp_oid, Oid roleid, Oid ownerId; cqContext *pcqCtx; - /* Superusers or on QE bypass all permission checking. */ - if (GP_ROLE_EXECUTE == Gp_role || superuser_arg(roleid)) + /* Superusers bypass all permission checking. */ + if (superuser_arg(roleid)) return mask; /* @@ -3293,8 +3294,8 @@ pg_tablespace_aclmask(Oid spc_oid, Oid roleid, if (spc_oid == GLOBALTABLESPACE_OID && !(IsBootstrapProcessingMode()||gp_upgrade_mode)) return 0; - /* Otherwise, superusers or on QE bypass all permission checking. */ - if (GP_ROLE_EXECUTE == Gp_role || superuser_arg(roleid)) + /* Superusers bypass all permission checking. */ + if (superuser_arg(roleid)) return mask; /* @@ -3366,8 +3367,8 @@ pg_foreign_data_wrapper_aclmask(Oid fdw_oid, Oid roleid, Form_pg_foreign_data_wrapper fdwForm; - /* Bypass permission checks for superusers or on QE */ - if (GP_ROLE_EXECUTE == Gp_role || superuser_arg(roleid)) + /* Superusers bypass all permission checking. */ + if (superuser_arg(roleid)) return mask; /* @@ -3435,8 +3436,8 @@ pg_foreign_server_aclmask(Oid srv_oid, Oid roleid, Form_pg_foreign_server srvForm; - /* Bypass permission checks for s
[08/50] [abbrv] incubator-hawq git commit: HAWQ-1242. hawq-site.xml default content has wrong guc variable names
HAWQ-1242. hawq-site.xml default content has wrong guc variable names Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/8d22582c Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/8d22582c Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/8d22582c Branch: refs/heads/2.1.0.0-incubating Commit: 8d22582c76309184cdb961abcd2b66f65e6bebb1 Parents: 60f0933 Author: Yi Authored: Wed Jan 4 14:10:09 2017 +1100 Committer: Yi Committed: Wed Jan 4 14:10:09 2017 +1100 -- src/backend/utils/misc/guc.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/8d22582c/src/backend/utils/misc/guc.c -- diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c index 00b9bad..dccd599 100644 --- a/src/backend/utils/misc/guc.c +++ b/src/backend/utils/misc/guc.c @@ -8195,7 +8195,7 @@ static struct config_string ConfigureNamesString[] = }, { - {"standby_address_host", PGC_POSTMASTER, PRESET_OPTIONS, + {"hawq_standby_address_host", PGC_POSTMASTER, PRESET_OPTIONS, gettext_noop("standby server address hostname"), NULL }, @@ -8204,7 +8204,7 @@ static struct config_string ConfigureNamesString[] = }, { - {"dfs_url", PGC_POSTMASTER, PRESET_OPTIONS, + {"hawq_dfs_url", PGC_POSTMASTER, PRESET_OPTIONS, gettext_noop("hdfs url"), NULL }, @@ -8213,7 +8213,7 @@ static struct config_string ConfigureNamesString[] = }, { - {"master_directory", PGC_POSTMASTER, PRESET_OPTIONS, + {"hawq_master_directory", PGC_POSTMASTER, PRESET_OPTIONS, gettext_noop("master server data directory"), NULL }, @@ -8222,7 +8222,7 @@ static struct config_string ConfigureNamesString[] = }, { - {"segment_directory", PGC_POSTMASTER, PRESET_OPTIONS, + {"hawq_segment_directory", PGC_POSTMASTER, PRESET_OPTIONS, gettext_noop("segment data directory"), NULL },
[33/50] [abbrv] incubator-hawq git commit: HAWQ-762. Login to kerberos if credentials are no longer valid
HAWQ-762. Login to kerberos if credentials are no longer valid Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/8261c13e Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/8261c13e Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/8261c13e Branch: refs/heads/2.1.0.0-incubating Commit: 8261c13ef73de9109ec5340304471871f544fa17 Parents: 7f36b35 Author: Kavinder Dhaliwal Authored: Fri Jan 6 11:56:29 2017 -0800 Committer: Kavinder Dhaliwal Committed: Wed Jan 18 14:06:32 2017 -0800 -- .../hawq/pxf/service/utilities/SecuredHDFS.java | 11 +++-- .../pxf/service/utilities/SecuredHDFSTest.java | 45 ++-- 2 files changed, 31 insertions(+), 25 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/8261c13e/pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/utilities/SecuredHDFS.java -- diff --git a/pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/utilities/SecuredHDFS.java b/pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/utilities/SecuredHDFS.java index f442a6d..1e1bcd3 100644 --- a/pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/utilities/SecuredHDFS.java +++ b/pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/utilities/SecuredHDFS.java @@ -53,6 +53,14 @@ public class SecuredHDFS { public static void verifyToken(ProtocolData protData, ServletContext context) { try { if (UserGroupInformation.isSecurityEnabled()) { +/* + * HAWQ-1215: The verify token method validates that the token sent from + * Hawq to PXF is valid. However, this token is for a user other than + * 'pxf'. The following line ensures that before attempting any secure communication + * PXF tries to relogin in the case that its own ticket is about to expire + * #reloginFromKeytab is a no-op if the ticket is not near expiring + */ +UserGroupInformation.getLoginUser().reloginFromKeytab(); Token token = new Token(); String tokenString = protData.getToken(); token.decodeFromUrlString(tokenString); @@ -103,9 +111,6 @@ public class SecuredHDFS { LOG.debug("user " + userGroupInformation.getUserName() + " (" + userGroupInformation.getShortUserName() + ") authenticated"); - -// re-login if necessary -userGroupInformation.checkTGTAndReloginFromKeytab(); } catch (IOException e) { throw new SecurityException("Failed to verify delegation token " + e, e); http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/8261c13e/pxf/pxf-service/src/test/java/org/apache/hawq/pxf/service/utilities/SecuredHDFSTest.java -- diff --git a/pxf/pxf-service/src/test/java/org/apache/hawq/pxf/service/utilities/SecuredHDFSTest.java b/pxf/pxf-service/src/test/java/org/apache/hawq/pxf/service/utilities/SecuredHDFSTest.java index 4944a35..9aecce0 100644 --- a/pxf/pxf-service/src/test/java/org/apache/hawq/pxf/service/utilities/SecuredHDFSTest.java +++ b/pxf/pxf-service/src/test/java/org/apache/hawq/pxf/service/utilities/SecuredHDFSTest.java @@ -29,24 +29,25 @@ import org.powermock.core.classloader.annotations.PrepareForTest; import org.powermock.modules.junit4.PowerMockRunner; import javax.servlet.ServletContext; -import java.util.HashMap; -import java.util.Map; +import java.io.IOException; import static org.junit.Assert.*; import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; @RunWith(PowerMockRunner.class) @PrepareForTest({UserGroupInformation.class}) public class SecuredHDFSTest { -Map parameters; ProtocolData mockProtocolData; ServletContext mockContext; @Test -public void invalidTokenThrows() { +public void invalidTokenThrows() throws IOException { when(UserGroupInformation.isSecurityEnabled()).thenReturn(true); +UserGroupInformation ugi = mock(UserGroupInformation.class); +when(UserGroupInformation.getLoginUser()).thenReturn(ugi); when(mockProtocolData.getToken()).thenReturn("This is odd"); try { @@ -57,30 +58,30 @@ public class SecuredHDFSTest { } } +@Test +public void loggedOutUser() throws IOException { +when(UserGroupInformation.isSecurityEnabled()).thenReturn(true); +UserGroupInformation ugi = mock(UserGroupInformation.class); +when(UserGroupInformation.getLoginUser())
[13/50] [abbrv] incubator-hawq git commit: HAWQ-1215. Support Complextypes with HiveORC
HAWQ-1215. Support Complextypes with HiveORC Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/3b15739a Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/3b15739a Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/3b15739a Branch: refs/heads/2.1.0.0-incubating Commit: 3b15739a009601049f7343131abe889b204b4d62 Parents: aa5792d Author: Kavinder Dhaliwal Authored: Fri Dec 23 16:27:58 2016 -0800 Committer: Kavinder Dhaliwal Committed: Wed Jan 11 14:14:36 2017 -0800 -- .../plugins/hive/HiveInputFormatFragmenter.java | 5 +-- .../pxf/plugins/hive/HiveORCSerdeResolver.java | 32 .../plugins/hive/utilities/HiveUtilities.java | 3 +- 3 files changed, 37 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/3b15739a/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveInputFormatFragmenter.java -- diff --git a/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveInputFormatFragmenter.java b/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveInputFormatFragmenter.java index 051a246..ca4501b 100644 --- a/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveInputFormatFragmenter.java +++ b/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveInputFormatFragmenter.java @@ -26,7 +26,6 @@ import org.apache.hawq.pxf.api.UserDataException; import org.apache.hawq.pxf.api.io.DataType; import org.apache.hawq.pxf.api.utilities.ColumnDescriptor; import org.apache.hawq.pxf.api.utilities.InputData; -import org.apache.hawq.pxf.plugins.hive.utilities.EnumHiveToHawqType; import org.apache.hawq.pxf.plugins.hive.utilities.HiveUtilities; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; @@ -35,6 +34,7 @@ import org.apache.hadoop.hive.metastore.api.Table; import java.util.Arrays; import java.util.List; +import java.util.Properties; /** * Specialized Hive fragmenter for RC and Text files tables. Unlike the @@ -55,10 +55,11 @@ import java.util.List; */ public class HiveInputFormatFragmenter extends HiveDataFragmenter { private static final Log LOG = LogFactory.getLog(HiveInputFormatFragmenter.class); -private static final int EXPECTED_NUM_OF_TOKS = 3; +private static final int EXPECTED_NUM_OF_TOKS = 4; public static final int TOK_SERDE = 0; public static final int TOK_KEYS = 1; public static final int TOK_FILTER_DONE = 2; +public static final int TOK_COL_TYPES = 3; /** Defines the Hive input formats currently supported in pxf */ public enum PXF_HIVE_INPUT_FORMATS { http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/3b15739a/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveORCSerdeResolver.java -- diff --git a/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveORCSerdeResolver.java b/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveORCSerdeResolver.java index 7673713..93aa474 100644 --- a/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveORCSerdeResolver.java +++ b/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveORCSerdeResolver.java @@ -45,6 +45,7 @@ public class HiveORCSerdeResolver extends HiveResolver { private static final Log LOG = LogFactory.getLog(HiveORCSerdeResolver.class); private OrcSerde deserializer; private HiveInputFormatFragmenter.PXF_HIVE_SERDES serdeType; +private String typesString; public HiveORCSerdeResolver(InputData input) throws Exception { super(input); @@ -61,6 +62,7 @@ public class HiveORCSerdeResolver extends HiveResolver { throw new UnsupportedTypeException("Unsupported Hive Serde: " + serdeEnumStr); } partitionKeys = toks[HiveInputFormatFragmenter.TOK_KEYS]; +typesString = toks[HiveInputFormatFragmenter.TOK_COL_TYPES]; collectionDelim = input.getUserProperty("COLLECTION_DELIM") == null ? COLLECTION_DELIM : input.getUserProperty("COLLECTION_DELIM"); mapkeyDelim = input.getUserProperty("MAPKEY_DELIM") == null ? MAPKEY_DELIM @@ -102,11 +104,19 @@ public class HiveORCSerdeResolver extends HiveResolver { StringBuilder columnNames = new StringBuilder(numberOfDataColumns * 2); // column + delimiter StringBuilder columnTypes = new StringBuilder(numberOfDataColumns * 2); // column + delimiter +String[] cols = typesString.split(":"); +String[] hiveColTypes = new String[numberOfDataColumns]; +parseColTypes(cols, hiveColTypes); + String delim = ","; for (
[06/50] [abbrv] incubator-hawq git commit: Revert "HAWQ-1242. hawq-site.xml default content has wrong guc variable names"
Revert "HAWQ-1242. hawq-site.xml default content has wrong guc variable names" This reverts commit 61646cd55fddb4ef3feafe9fd125763796d518c0. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/94239f5e Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/94239f5e Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/94239f5e Branch: refs/heads/2.1.0.0-incubating Commit: 94239f5ed5ecf39e1650ec4b5d348ec9a907edc9 Parents: 61646cd Author: Yi Authored: Fri Dec 30 16:06:44 2016 +1100 Committer: Yi Committed: Fri Dec 30 16:06:44 2016 +1100 -- src/backend/utils/misc/etc/hawq-site.xml | 8 src/backend/utils/misc/etc/template-hawq-site.xml | 8 2 files changed, 8 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/94239f5e/src/backend/utils/misc/etc/hawq-site.xml -- diff --git a/src/backend/utils/misc/etc/hawq-site.xml b/src/backend/utils/misc/etc/hawq-site.xml index 012ccbf..b327ab4 100644 --- a/src/backend/utils/misc/etc/hawq-site.xml +++ b/src/backend/utils/misc/etc/hawq-site.xml @@ -33,7 +33,7 @@ under the License. - standby_address_host + hawq_standby_address_host none The host name of hawq standby master. @@ -45,19 +45,19 @@ under the License. - dfs_url + hawq_dfs_url localhost:8020/hawq_default URL for accessing HDFS. - master_directory + hawq_master_directory ~/hawq-data-directory/masterdd The directory of hawq master. - segment_directory + hawq_segment_directory ~/hawq-data-directory/segmentdd The directory of hawq segment. http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/94239f5e/src/backend/utils/misc/etc/template-hawq-site.xml -- diff --git a/src/backend/utils/misc/etc/template-hawq-site.xml b/src/backend/utils/misc/etc/template-hawq-site.xml index 1901db3..cf6273a 100644 --- a/src/backend/utils/misc/etc/template-hawq-site.xml +++ b/src/backend/utils/misc/etc/template-hawq-site.xml @@ -33,7 +33,7 @@ under the License. - standby_address_host + hawq_standby_address_host %standby.host% The host name of hawq standby master. @@ -45,19 +45,19 @@ under the License. - dfs_url + hawq_dfs_url %namenode.host%:%namenode.port%/%hawq.file.space% URL for accessing HDFS. - master_directory + hawq_master_directory %master.directory% The directory of hawq master. - segment_directory + hawq_segment_directory %segment.directory% The directory of hawq segment.
[49/50] [abbrv] incubator-hawq git commit: HAWQ-1308. Fixed Javadoc warnings.
HAWQ-1308. Fixed Javadoc warnings. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/7d02472b Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/7d02472b Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/7d02472b Branch: refs/heads/2.1.0.0-incubating Commit: 7d02472b82e841254d63215c9acdfc51a2ecd7e3 Parents: f6452d2 Author: Oleksandr Diachenko Authored: Thu Feb 2 14:43:45 2017 -0800 Committer: Oleksandr Diachenko Committed: Thu Feb 2 14:44:19 2017 -0800 -- .../java/org/apache/hawq/pxf/api/OutputFormat.java | 3 ++- .../pxf/plugins/hive/utilities/HiveUtilities.java| 15 --- 2 files changed, 10 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/7d02472b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/OutputFormat.java -- diff --git a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/OutputFormat.java b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/OutputFormat.java index 89c4b30..4d2d806 100644 --- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/OutputFormat.java +++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/OutputFormat.java @@ -44,7 +44,8 @@ public enum OutputFormat { /** * Looks up output format for given class name if it exists. - * + * + * @param className class name implementing certain output format * @throws UnsupportedTypeException if output format with given class wasn't found * @return an output format with given class name */ http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/7d02472b/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilities.java -- diff --git a/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilities.java b/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilities.java index 37f4ac2..3328c9f 100644 --- a/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilities.java +++ b/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilities.java @@ -455,11 +455,11 @@ public class HiveUtilities { /** * The method which serializes fragment-related attributes, needed for reading and resolution to string * - * @param fragmenterClassName - * @param partData - * @param filterInFragmenter + * @param fragmenterClassName fragmenter class name + * @param partData partition data + * @param filterInFragmenter whether filtering was done in fragmenter * @return serialized representation of fragment-related attributes - * @throws Exception + * @throws Exception when error occurred during serialization */ @SuppressWarnings("unchecked") public static byte[] makeUserData(String fragmenterClassName, HiveTablePartition partData, boolean filterInFragmenter) throws Exception { @@ -494,7 +494,7 @@ public class HiveUtilities { * @param input input data * @param supportedSerdes list of allowed serdes in current context * @return instance of HiveUserData class - * @throws UserDataException + * @throws UserDataException when incorrect number of tokens in Hive user data received */ public static HiveUserData parseHiveUserData(InputData input, PXF_HIVE_SERDES... supportedSerdes) throws UserDataException{ String userData = new String(input.getFragmentUserData()); @@ -610,10 +610,11 @@ public class HiveUtilities { /** * Creates an instance of a given serde type * - * @param serdeType - * @param allowedSerdes + * @param serdeType SerDe type + * @param allowedSerdes allowed serdes in current context * @return instance of a given serde * @throws UnsupportedTypeException if given serde is not allowed in current context + * @throws Exception if other error occurred during creation of SerDe instance */ @SuppressWarnings("deprecation") public static SerDe createDeserializer(PXF_HIVE_SERDES serdeType, PXF_HIVE_SERDES... allowedSerdes) throws Exception{
[21/50] [abbrv] incubator-hawq git commit: Revert "HAWQ-1248. Merge Dockerfiles for HAWQ Dev into HAWQ code base."
http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/1cb29096/contrib/hawq-docker/centos6-docker/hawq-test/conf/log4j.properties -- diff --git a/contrib/hawq-docker/centos6-docker/hawq-test/conf/log4j.properties b/contrib/hawq-docker/centos6-docker/hawq-test/conf/log4j.properties deleted file mode 100644 index c901ab1..000 --- a/contrib/hawq-docker/centos6-docker/hawq-test/conf/log4j.properties +++ /dev/null @@ -1,291 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# Define some default values that can be overridden by system properties -hadoop.root.logger=INFO,console -hadoop.log.dir=. -hadoop.log.file=hadoop.log - -# Define the root logger to the system property "hadoop.root.logger". -log4j.rootLogger=${hadoop.root.logger}, EventCounter - -# Logging Threshold -log4j.threshold=ALL - -# Null Appender -log4j.appender.NullAppender=org.apache.log4j.varia.NullAppender - -# -# Rolling File Appender - cap space usage at 5gb. -# -hadoop.log.maxfilesize=256MB -hadoop.log.maxbackupindex=20 -log4j.appender.RFA=org.apache.log4j.RollingFileAppender -log4j.appender.RFA.File=${hadoop.log.dir}/${hadoop.log.file} - -log4j.appender.RFA.MaxFileSize=${hadoop.log.maxfilesize} -log4j.appender.RFA.MaxBackupIndex=${hadoop.log.maxbackupindex} - -log4j.appender.RFA.layout=org.apache.log4j.PatternLayout - -# Pattern format: Date LogLevel LoggerName LogMessage -log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n -# Debugging Pattern format -#log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} (%F:%M(%L)) - %m%n - - -# -# Daily Rolling File Appender -# - -log4j.appender.DRFA=org.apache.log4j.DailyRollingFileAppender -log4j.appender.DRFA.File=${hadoop.log.dir}/${hadoop.log.file} - -# Rollover at midnight -log4j.appender.DRFA.DatePattern=.-MM-dd - -log4j.appender.DRFA.layout=org.apache.log4j.PatternLayout - -# Pattern format: Date LogLevel LoggerName LogMessage -log4j.appender.DRFA.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n -# Debugging Pattern format -#log4j.appender.DRFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} (%F:%M(%L)) - %m%n - - -# -# console -# Add "console" to rootlogger above if you want to use this -# - -log4j.appender.console=org.apache.log4j.ConsoleAppender -log4j.appender.console.target=System.err -log4j.appender.console.layout=org.apache.log4j.PatternLayout -log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{2}: %m%n - -# -# TaskLog Appender -# - -#Default values -hadoop.tasklog.taskid=null -hadoop.tasklog.iscleanup=false -hadoop.tasklog.noKeepSplits=4 -hadoop.tasklog.totalLogFileSize=100 -hadoop.tasklog.purgeLogSplits=true -hadoop.tasklog.logsRetainHours=12 - -log4j.appender.TLA=org.apache.hadoop.mapred.TaskLogAppender -log4j.appender.TLA.taskId=${hadoop.tasklog.taskid} -log4j.appender.TLA.isCleanup=${hadoop.tasklog.iscleanup} -log4j.appender.TLA.totalLogFileSize=${hadoop.tasklog.totalLogFileSize} - -log4j.appender.TLA.layout=org.apache.log4j.PatternLayout -log4j.appender.TLA.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n - -# -# HDFS block state change log from block manager -# -# Uncomment the following to suppress normal block state change -# messages from BlockManager in NameNode. -#log4j.logger.BlockStateChange=WARN - -# -#Security appender -# -hadoop.security.logger=INFO,NullAppender -hadoop.security.log.maxfilesize=256MB -hadoop.security.log.maxbackupindex=20 -log4j.category.SecurityLogger=${hadoop.security.logger} -hadoop.security.log.file=SecurityAuth-${user.name}.audit -log4j.appender.RFAS=org.apache.log4j.RollingFileAppender -log4j.appender.RFAS.File=${hadoop.log.dir}/${hadoop.security.log.file} -log4j.appender.RFAS.layout=org.apache.log4j.PatternLayout -log4j.appender.RFAS.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n -log4j.appender.RFAS.MaxFileSize=${hadoop.security.log.maxfilesize} -log4j.appender.RFAS.MaxBackupIndex=${hadoop.security.log.maxbackupindex} - -# -# Daily Rolling Security appender -# -log4j.appender.DRFAS=org.apache.log4j.DailyRollingFileAppender -log4j.appender.DRFAS.File=${hadoop.log.dir}/${hadoop.security.log.file} -log4j.appender.DRFAS.layout=org.apache.log4j.PatternLayout -log4j.a
[37/50] [abbrv] incubator-hawq git commit: HAWQ-1193. Add createEncryption, getEZForPath, listEncryptionZones RPC for libhdfs3.
HAWQ-1193. Add createEncryption, getEZForPath, listEncryptionZones RPC for libhdfs3. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/517e6d26 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/517e6d26 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/517e6d26 Branch: refs/heads/2.1.0.0-incubating Commit: 517e6d26cadff3a0fc03af1f36e8302bf454e573 Parents: efa1230 Author: ivan Authored: Tue Dec 20 16:51:18 2016 +0800 Committer: ivan Committed: Thu Jan 19 16:10:32 2017 +0800 -- depends/libhdfs3/mock/MockFileSystemInter.h | 5 + .../libhdfs3/src/client/EncryptionZoneInfo.h| 80 + .../src/client/EncryptionZoneIterator.cpp | 86 ++ .../src/client/EncryptionZoneIterator.h | 56 + .../libhdfs3/src/client/FileEncryptionInfo.h| 93 +++ depends/libhdfs3/src/client/FileStatus.h| 19 +++- depends/libhdfs3/src/client/FileSystem.cpp | 54 + depends/libhdfs3/src/client/FileSystem.h| 32 ++ depends/libhdfs3/src/client/FileSystemImpl.cpp | 88 ++ depends/libhdfs3/src/client/FileSystemImpl.h| 40 +++ depends/libhdfs3/src/client/FileSystemInter.h | 40 +++ depends/libhdfs3/src/client/Hdfs.cpp| 114 ++- depends/libhdfs3/src/client/hdfs.h | 63 ++ .../src/proto/ClientNamenodeProtocol.proto | 8 ++ depends/libhdfs3/src/proto/datatransfer.proto | 1 + depends/libhdfs3/src/proto/encryption.proto | 67 +++ depends/libhdfs3/src/proto/hdfs.proto | 60 ++ depends/libhdfs3/src/rpc/RpcAuth.h | 2 +- depends/libhdfs3/src/rpc/RpcChannel.cpp | 4 +- depends/libhdfs3/src/server/Namenode.h | 35 +- depends/libhdfs3/src/server/NamenodeImpl.cpp| 73 depends/libhdfs3/src/server/NamenodeImpl.h | 9 ++ depends/libhdfs3/src/server/NamenodeProxy.cpp | 24 depends/libhdfs3/src/server/NamenodeProxy.h | 7 ++ depends/libhdfs3/src/server/RpcHelper.h | 31 + .../libhdfs3/test/function/TestCInterface.cpp | 42 +++ .../libhdfs3/test/function/TestFileSystem.cpp | 33 ++ .../libhdfs3/test/function/TestOutputStream.cpp | 12 ++ 28 files changed, 1171 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/517e6d26/depends/libhdfs3/mock/MockFileSystemInter.h -- diff --git a/depends/libhdfs3/mock/MockFileSystemInter.h b/depends/libhdfs3/mock/MockFileSystemInter.h index 4d91c73..336db7e 100644 --- a/depends/libhdfs3/mock/MockFileSystemInter.h +++ b/depends/libhdfs3/mock/MockFileSystemInter.h @@ -101,6 +101,11 @@ public: MOCK_METHOD3(getFileBlockLocations, std::vector (const char * path, int64_t start, int64_t len)); MOCK_METHOD2(listAllDirectoryItems, std::vector (const char * path, bool needLocation)); MOCK_METHOD0(getPeerCache, Hdfs::Internal::PeerCache &()); + MOCK_METHOD2(createEncryptionZone, bool(const char * path, const char * keyName)); + MOCK_METHOD1(getEZForPath, Hdfs::EncryptionZoneInfo(const char * path)); + MOCK_METHOD2(listEncryptionZones, bool(const int64_t id, std::vector &)); + MOCK_METHOD0(listEncryptionZone, Hdfs::EncryptionZoneIterator()); + MOCK_METHOD0(listAllEncryptionZoneItems, std::vector()); }; #endif /* _HDFS_LIBHDFS3_MOCK_MOCKSOCKET_H_ */ http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/517e6d26/depends/libhdfs3/src/client/EncryptionZoneInfo.h -- diff --git a/depends/libhdfs3/src/client/EncryptionZoneInfo.h b/depends/libhdfs3/src/client/EncryptionZoneInfo.h new file mode 100644 index 000..d436ae7 --- /dev/null +++ b/depends/libhdfs3/src/client/EncryptionZoneInfo.h @@ -0,0 +1,80 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef _HDFS_LIBHDFS3_CLIENT_ENCRYPTIONZON
[29/50] [abbrv] incubator-hawq git commit: HAWQ-1203. Ranger Plugin Service Implementation. (with contributions by Lav Jain and Leslie Chang) (close #1092)
http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/7f36b35b/ranger-plugin/service/src/test/java/org/apache/hawq/ranger/authorization/RangerHawqAuthorizerTest.java -- diff --git a/ranger-plugin/service/src/test/java/org/apache/hawq/ranger/authorization/RangerHawqAuthorizerTest.java b/ranger-plugin/service/src/test/java/org/apache/hawq/ranger/authorization/RangerHawqAuthorizerTest.java new file mode 100644 index 000..0a439db --- /dev/null +++ b/ranger-plugin/service/src/test/java/org/apache/hawq/ranger/authorization/RangerHawqAuthorizerTest.java @@ -0,0 +1,325 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hawq.ranger.authorization; + +import org.apache.hawq.ranger.authorization.model.AuthorizationRequest; +import org.apache.hawq.ranger.authorization.model.AuthorizationResponse; +import org.apache.hawq.ranger.authorization.model.HawqPrivilege; +import org.apache.hawq.ranger.authorization.model.HawqResource; +import org.apache.hawq.ranger.authorization.model.ResourceAccess; +import org.apache.ranger.plugin.policyengine.RangerAccessRequest; +import org.apache.ranger.plugin.policyengine.RangerAccessResult; +import org.apache.ranger.plugin.service.RangerBasePlugin; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.ArgumentMatcher; +import org.mockito.Mock; +import org.mockito.internal.util.collections.Sets; +import org.mockito.runners.MockitoJUnitRunner; + +import java.util.HashMap; +import java.util.HashSet; +import java.util.Map; +import java.util.Set; + +import static junit.framework.TestCase.assertEquals; +import static junit.framework.TestCase.assertNotNull; +import static org.mockito.Matchers.any; +import static org.mockito.Matchers.argThat; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + + +@RunWith(MockitoJUnitRunner.class) +public class RangerHawqAuthorizerTest { + +private static final Integer TEST_REQUEST_ID = 1; +private static final String TEST_USER = "alex"; +private static final String TEST_CLIENT = "1.2.3.4"; +private static final String TEST_CONTEXT = "SELECT * FROM sales"; +private static final Set TEST_PRIVILEGES = Sets.newSet(HawqPrivilege.select, HawqPrivilege.update); + +private static final String TEST_RESOURCE_REQUEST = +"finance:us:sales>select,update#finance:emea:sales>create"; +private static final String TEST_RESOURCE_RESPONSE_ALL_FALSE = + "finance:us:sales>select,update>false#finance:emea:sales>create>false"; +private static final String TEST_RESOURCE_RESPONSE_ALL_TRUE = + "finance:us:sales>select,update>true#finance:emea:sales>create>true"; +private static final String TEST_RESOURCE_RESPONSE_US_ALLOWED_EMEA_DENIED = + "finance:us:sales>select,update>true#finance:emea:sales>create>false"; +private static final String TEST_RESOURCE_RESPONSE_UPDATE_DENIED = + "finance:us:sales>select,update>false#finance:emea:sales>create>true"; + +private static final String TEST_RESOURCE_REQUEST_CREATE_SCHEMA = "finance>create"; +private static final String TEST_RESOURCE_RESPONSE_CREATE_SCHEMA = "finance>create>true"; +private static final String TEST_RESOURCE_REQUEST_USAGE_SCHEMA = "finance:us>usage"; +private static final String TEST_RESOURCE_RESPONSE_USAGE_SCHEMA = "finance:us>usage>true"; + +private RangerHawqAuthorizer authorizer; + +@Mock +private RangerBasePlugin mockRangerPlugin; +@Mock +private RangerAccessResult mockRangerAccessResult; + +@Before +public void setup() throws Exception { +authorizer = RangerHawqAuthorizer.getInstance(); +authorizer.setRangerPlugin(mockRangerPlugin); +} + +@Test +public void testAuthorize_allAllowed() throws Exception { + when(mockRangerPlugin.isAccessAllowed(any(RangerAccessRequest.class))).thenReturn(mockRangerAccessResult); +when(mockRangerAccessResult.getIsAllowed()).thenReturn(true); +testRequest(TEST_RESOURCE_REQUEST, TEST_RESOURCE_RESPONSE_ALL_TRUE); +} + +@Test +public void testAuthorize_
[15/50] [abbrv] incubator-hawq git commit: HAWQ-1267. Update NOTICE file to reflect the right year for copyright
HAWQ-1267. Update NOTICE file to reflect the right year for copyright Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/a8177153 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/a8177153 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/a8177153 Branch: refs/heads/2.1.0.0-incubating Commit: a8177153b25123759d15411146c094dc350e4bab Parents: bf4742c Author: Ruilong Huo Authored: Fri Jan 13 10:00:41 2017 +0800 Committer: Ruilong Huo Committed: Fri Jan 13 10:00:41 2017 +0800 -- NOTICE | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/a8177153/NOTICE -- diff --git a/NOTICE b/NOTICE index 6589c8f..1b1b2e7 100644 --- a/NOTICE +++ b/NOTICE @@ -1,5 +1,5 @@ Apache HAWQ (incubating) -Copyright 2016 The Apache Software Foundation. +Copyright 2017 The Apache Software Foundation. This product includes software developed at The Apache Software Foundation (http://www.apache.org/).
[05/50] [abbrv] incubator-hawq git commit: HAWQ-1242. hawq-site.xml default content has wrong guc variable names
HAWQ-1242. hawq-site.xml default content has wrong guc variable names Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/61646cd5 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/61646cd5 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/61646cd5 Branch: refs/heads/2.1.0.0-incubating Commit: 61646cd55fddb4ef3feafe9fd125763796d518c0 Parents: e25fe8b Author: Yi Authored: Fri Dec 30 13:15:07 2016 +1100 Committer: Yi Committed: Fri Dec 30 13:15:07 2016 +1100 -- src/backend/utils/misc/etc/hawq-site.xml | 8 src/backend/utils/misc/etc/template-hawq-site.xml | 8 2 files changed, 8 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/61646cd5/src/backend/utils/misc/etc/hawq-site.xml -- diff --git a/src/backend/utils/misc/etc/hawq-site.xml b/src/backend/utils/misc/etc/hawq-site.xml index b327ab4..012ccbf 100644 --- a/src/backend/utils/misc/etc/hawq-site.xml +++ b/src/backend/utils/misc/etc/hawq-site.xml @@ -33,7 +33,7 @@ under the License. - hawq_standby_address_host + standby_address_host none The host name of hawq standby master. @@ -45,19 +45,19 @@ under the License. - hawq_dfs_url + dfs_url localhost:8020/hawq_default URL for accessing HDFS. - hawq_master_directory + master_directory ~/hawq-data-directory/masterdd The directory of hawq master. - hawq_segment_directory + segment_directory ~/hawq-data-directory/segmentdd The directory of hawq segment. http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/61646cd5/src/backend/utils/misc/etc/template-hawq-site.xml -- diff --git a/src/backend/utils/misc/etc/template-hawq-site.xml b/src/backend/utils/misc/etc/template-hawq-site.xml index cf6273a..1901db3 100644 --- a/src/backend/utils/misc/etc/template-hawq-site.xml +++ b/src/backend/utils/misc/etc/template-hawq-site.xml @@ -33,7 +33,7 @@ under the License. - hawq_standby_address_host + standby_address_host %standby.host% The host name of hawq standby master. @@ -45,19 +45,19 @@ under the License. - hawq_dfs_url + dfs_url %namenode.host%:%namenode.port%/%hawq.file.space% URL for accessing HDFS. - hawq_master_directory + master_directory %master.directory% The directory of hawq master. - hawq_segment_directory + segment_directory %segment.directory% The directory of hawq segment.
[48/50] [abbrv] incubator-hawq git commit: HAWQ-1309. Use default value for pxf user/port
HAWQ-1309. Use default value for pxf user/port Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/f6452d27 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/f6452d27 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/f6452d27 Branch: refs/heads/2.1.0.0-incubating Commit: f6452d273885619673f9e7c754d133e17252a38c Parents: 42c1cc1 Author: Shivram Mani Authored: Thu Feb 2 14:08:55 2017 -0800 Committer: Kavinder Dhaliwal Committed: Thu Feb 2 14:26:50 2017 -0800 -- pxf/pxf-service/src/scripts/pxf-service | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/f6452d27/pxf/pxf-service/src/scripts/pxf-service -- diff --git a/pxf/pxf-service/src/scripts/pxf-service b/pxf/pxf-service/src/scripts/pxf-service index 2d63310..39d167e 100644 --- a/pxf/pxf-service/src/scripts/pxf-service +++ b/pxf/pxf-service/src/scripts/pxf-service @@ -34,8 +34,8 @@ else source $env_script fi -pxf_user=$PXF_USER -instance_port=$PXF_PORT +pxf_user=${PXF_USER:-pxf} +instance_port=${PXF_PORT:-51200} instance_name=pxf-service if [ -z $PXF_HOME ]; then
[34/50] [abbrv] incubator-hawq git commit: HAWQ-1279. Force to recompute namespace_path when enable_ranger
HAWQ-1279. Force to recompute namespace_path when enable_ranger Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/21f1e29a Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/21f1e29a Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/21f1e29a Branch: refs/heads/2.1.0.0-incubating Commit: 21f1e29a43aee003e250e41375bf226fabbe2ce3 Parents: 8261c13 Author: interma Authored: Wed Jan 18 13:13:52 2017 +0800 Committer: hubertzhang Committed: Thu Jan 19 10:56:01 2017 +0800 -- src/backend/catalog/namespace.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/21f1e29a/src/backend/catalog/namespace.c -- diff --git a/src/backend/catalog/namespace.c b/src/backend/catalog/namespace.c index 4685bfa..7049d32 100644 --- a/src/backend/catalog/namespace.c +++ b/src/backend/catalog/namespace.c @@ -1933,7 +1933,12 @@ recomputeNamespacePath(void) * Do nothing if path is already valid. */ if (namespaceSearchPathValid && namespaceUser == roleid) - return; + { + if (!enable_ranger) + return; + else + elog(DEBUG3, "recompute search_path[%s] when enable_ranger", namespace_search_path); + } /* Need a modifiable copy of namespace_search_path string */ rawname = pstrdup(namespace_search_path);
[03/50] [abbrv] incubator-hawq git commit: HAWQ-1239. Fail to call pg_rangercheck_batch() when when 'rte->rtekind != RTE_RELATION' or 'requiredPerms == 0'
HAWQ-1239. Fail to call pg_rangercheck_batch() when when 'rte->rtekind != RTE_RELATION' or 'requiredPerms == 0' Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/cee573ad Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/cee573ad Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/cee573ad Branch: refs/heads/2.1.0.0-incubating Commit: cee573ad418a6e27a1cb13c66958a661af822e74 Parents: 4ca1587 Author: Chunling Wang Authored: Tue Dec 27 17:52:56 2016 +0800 Committer: hzhang2 Committed: Wed Dec 28 17:22:20 2016 +0800 -- src/backend/parser/parse_relation.c | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/cee573ad/src/backend/parser/parse_relation.c -- diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c index 6839207..7dbe496 100644 --- a/src/backend/parser/parse_relation.c +++ b/src/backend/parser/parse_relation.c @@ -2745,10 +2745,10 @@ ExecCheckRTPermsWithRanger(List *rangeTable) RangeTblEntry *rte = (RangeTblEntry *) lfirst(l); if (rte->rtekind != RTE_RELATION) - return; + continue; requiredPerms = rte->requiredPerms; if (requiredPerms == 0) - return; + continue; relOid = rte->relid; userid = rte->checkAsUser ? rte->checkAsUser : GetUserId(); @@ -2763,6 +2763,9 @@ ExecCheckRTPermsWithRanger(List *rangeTable) } // foreach + if (ranger_check_args == NIL) +return; + // ranger ACL check with package Oids List *aclresults = NIL; aclresults = pg_rangercheck_batch(ranger_check_args);
[28/50] [abbrv] incubator-hawq git commit: HAWQ-1276. The error message is not friendly when ranger plugin service is unavailable.
HAWQ-1276. The error message is not friendly when ranger plugin service is unavailable. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/e46f06cc Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/e46f06cc Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/e46f06cc Branch: refs/heads/2.1.0.0-incubating Commit: e46f06cc95d5bd8212cb1edf8331856461891dc6 Parents: afac2df Author: stanlyxiang Authored: Fri Jan 13 11:33:40 2017 +0800 Committer: Wen Lin Committed: Tue Jan 17 16:34:04 2017 +0800 -- src/backend/catalog/aclchk.c | 13 ++--- src/backend/libpq/rangerrest.c | 28 +--- src/backend/tcop/postgres.c| 5 +++-- 3 files changed, 22 insertions(+), 24 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/e46f06cc/src/backend/catalog/aclchk.c -- diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c index 200d9cb..ed36330 100644 --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -2739,7 +2739,7 @@ List *pg_rangercheck_batch(List *arg_list) List *aclresults = NIL; List *requestargs = NIL; ListCell *arg; - elog(LOG, "rangeracl batch check, acl list length:%d\n", arg_list->length); + elog(DEBUG3, "ranger acl batch check, acl list length: %d\n", arg_list->length); foreach(arg, arg_list) { RangerPrivilegeArgs *arg_ptr = (RangerPrivilegeArgs *) lfirst(arg); @@ -2753,7 +2753,7 @@ List *pg_rangercheck_batch(List *arg_list) RangerPrivilegeResults *aclresult = (RangerPrivilegeResults *) palloc(sizeof(RangerPrivilegeResults)); aclresult->result = RANGERCHECK_NO_PRIV; aclresult->relOid = object_oid; -// this two sign fields will be set in create_ranger_request_json() +/* this two sign fields will be set in function create_ranger_request_json */ aclresult->resource_sign = 0; aclresult->privilege_sign = 0; aclresults = lappend(aclresults, aclresult); @@ -2771,7 +2771,6 @@ List *pg_rangercheck_batch(List *arg_list) int ret = check_privilege_from_ranger(requestargs, aclresults); if (ret < 0) { - elog(WARNING, "ranger service unavailable or unexpected error\n"); ListCell *result; foreach(result, aclresults) { RangerPrivilegeResults *result_ptr = (RangerPrivilegeResults *) lfirst(result); @@ -2808,13 +2807,13 @@ pg_rangercheck(AclObjectKind objkind, Oid object_oid, Oid roleid, List* actions = getActionName(mask); bool isAll = (how == ACLMASK_ALL) ? true: false; - elog(LOG, "rangeraclcheck kind:%d,objectname:%s,role:%s,mask:%u\n",objkind,objectname,rolename,mask); + elog(DEBUG3, "ranger acl check kind: %d, object name: %s, role: %s, mask: %u\n", objkind, objectname, rolename, mask); List *resultargs = NIL; RangerPrivilegeResults *aclresult = (RangerPrivilegeResults *) palloc(sizeof(RangerPrivilegeResults)); aclresult->result = RANGERCHECK_NO_PRIV; aclresult->relOid = object_oid; - // this two sign fields will be set in create_ranger_request_json() + /* this two sign fields will be set in function create_ranger_request_json */ aclresult->resource_sign = 0; aclresult->privilege_sign = 0; resultargs = lappend(resultargs, aclresult); @@ -2834,7 +2833,7 @@ pg_rangercheck(AclObjectKind objkind, Oid object_oid, Oid roleid, { ListCell *arg; foreach(arg, resultargs) { - // only one element + /* only one element */ RangerPrivilegeResults *arg_ptr = (RangerPrivilegeResults *) lfirst(arg); if (arg_ptr->result == RANGERCHECK_OK) result = ACLCHECK_OK; @@ -2893,7 +2892,7 @@ pg_aclmask(AclObjectKind objkind, Oid table_oid, Oid roleid, case ACL_KIND_EXTPROTOCOL: return pg_extprotocol_aclmask(table_oid, roleid, mask, how); default: - elog(ERROR, "unrecognized objkind: %d", + elog(ERROR, "unrecognized object kind : %d", (int) objkind); /* not reached, but keep compiler quiet */ return ACL_NO_RIGHTS; http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/e46f06cc/src/backend/libpq/rangerrest.c -- diff --git a/src/backend/libpq/rangerrest.c b/src/backend/libpq/rangerrest.c index 74777dc..dc5d193 100644 --- a/src/backend/libpq/rangerrest.c +++ b/src/backend/libpq/rangerrest.c @@ -85,14 +85,14 @@ static int pars