[GitHub] incubator-hawq issue #1208: HAWQ-1421. PXF rpm updates to make it oss friend...
Github user shivzone commented on the issue: https://github.com/apache/incubator-hawq/pull/1208 Can you please provide me the output of sudo yum list installed | grep hadoop If this was hdp distribution, hadoop-hdfs rpms should have already been installed or been available in some repo [gpadmin@test1 ~]$ rpm -qR hadoop-hdfs.noarch hadoop_2_5_0_0_1245-hdfs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1208: HAWQ-1421. PXF rpm updates to make it oss...
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1208#discussion_r109065444 --- Diff: pxf/build.gradle --- @@ -41,17 +41,22 @@ buildscript { } } -// Hadoop distribution (can be phd or hdp) - needed to set correct classpaths +// Hadoop distribution (can be hdp or null which would default to apache) - needed to set correct classpaths // can be set with -Dhd= -def hddist = System.properties['hd'] ?: 'hdp'; -if (hddist != 'phd' && hddist != 'hdp') { +def hddist = System.properties['hd'] ?: ''; --- End diff -- System.properties returns null and not ''. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1208: HAWQ-1421. PXF rpm updates to make it oss friend...
Github user radarwave commented on the issue: https://github.com/apache/incubator-hawq/pull/1208 After set "export HD=hdp", the rpm package names changed back with the version string "3_2_1_0". Maybe this is expected. But then I still seeing it requires Apache Hadoop. See: `rpm -ivh pxf-service_3_2_1_0-3.2.1.0-1.el6.noarch.rpm error: Failed dependencies: hadoop >= 2.7.1 is needed by pxf-service_3_2_1_0-0:3.2.1.0-1.el6.noarch hadoop-hdfs >= 2.7.1 is needed by pxf-service_3_2_1_0-0:3.2.1.0-1.el6.noarch` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1208: HAWQ-1421. PXF rpm updates to make it oss...
Github user sansanichfb commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1208#discussion_r109062506 --- Diff: pxf/build.gradle --- @@ -41,17 +41,22 @@ buildscript { } } -// Hadoop distribution (can be phd or hdp) - needed to set correct classpaths +// Hadoop distribution (can be hdp or null which would default to apache) - needed to set correct classpaths // can be set with -Dhd= -def hddist = System.properties['hd'] ?: 'hdp'; -if (hddist != 'phd' && hddist != 'hdp') { +def hddist = System.properties['hd'] ?: ''; --- End diff -- As long as we don't have default value, shouldn't it be just: def hddist = System.properties['hd']? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1209: HAWQ-1404. PXF to leverage file-level sta...
GitHub user sansanichfb opened a pull request: https://github.com/apache/incubator-hawq/pull/1209 HAWQ-1404. PXF to leverage file-level stats of ORC file and emit records for COUNT(*). You can merge this pull request into a Git repository by running: $ git pull https://github.com/sansanichfb/incubator-hawq HAWQ-1404 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1209.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1209 commit 9268625b4b6222892d52a28a1abfad4db5a02f3c Author: Oleksandr Diachenko Date: 2017-03-28T19:02:32Z HAWQ-1404. PXF to leverage file-level stats of ORC file and emit records for COUNT(*). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (HAWQ-1421) Improve PXF rpm package name format and dependencies
[ https://issues.apache.org/jira/browse/HAWQ-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949834#comment-15949834 ] Shivram Mani commented on HAWQ-1421: [~rlei] if you need to build the RPM to work with hdp distribution, you will need to set the following variable {code} HD=hdp make rpm {code} The default behavior when you run make rpm would be to produce apache hadoop compliant rpms. If the user plans to run on hadoop without rpm, he can simply resort to building from PXF source using make install as mentioned in https://cwiki.apache.org/confluence/display/HAWQ/PXF+Build+and+Install Are you suggesting another approach ? > Improve PXF rpm package name format and dependencies > > > Key: HAWQ-1421 > URL: https://issues.apache.org/jira/browse/HAWQ-1421 > Project: Apache HAWQ > Issue Type: Improvement > Components: Build, PXF >Reporter: Radar Lei >Assignee: Shivram Mani > Fix For: 2.2.0.0-incubating > > > If we build pxf rpm package by 'make rpm', we will get below pxf packages: > {quote} > apache-tomcat-7.0.62-el6.noarch.rpm > pxf-3.2.1.0-root.el6.noarch.rpm > pxf-hbase_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > pxf-hdfs_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > pxf-hive_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > pxf-jdbc_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > pxf-json_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > pxf-service_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > {quote} > These rpm packages have dependencies on Apache Hadoop components only, some > other Hadoop distributes can't satisfy it. E.g. : > {quote} > rpm -ivh pxf-hdfs_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > error: Failed dependencies: > pxf-service_3_2_1_0 >= 3.2.1.0 is needed by > pxf-hdfs_3_2_1_0-0:3.2.1.0-root.el6.noarch > hadoop >= 2.7.1 is needed by pxf-hdfs_3_2_1_0-0:3.2.1.0-root.el6.noarch > hadoop-mapreduce >= 2.7.1 is needed by > pxf-hdfs_3_2_1_0-0:3.2.1.0-root.el6.noarch > {quote} > We'd better make the rpm package name format and dependencies better. > 1. Remove the version string like '3_2_1_0'. > 2. Remove the user name from the build environment. > 3. Consider do we need to include the apache-tomcat rpm package into HAWQ > rpm release tarball. > 4. Improve the hard code 'el6' string. (This might be optinal) > 5. Improve the dependencies, including the dependencies between these pxf > rpm packages. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (HAWQ-1421) Improve PXF rpm package name format and dependencies
[ https://issues.apache.org/jira/browse/HAWQ-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949834#comment-15949834 ] Shivram Mani edited comment on HAWQ-1421 at 3/30/17 9:11 PM: - [~rlei] if you need to build the RPM to work with hdp distribution, you will need to set the following variable {code} export HD=hdp make rpm {code} The default behavior when you run make rpm would be to produce apache hadoop compliant rpms. If the user plans to run on hadoop without rpm, he can simply resort to building from PXF source using make install as mentioned in https://cwiki.apache.org/confluence/display/HAWQ/PXF+Build+and+Install Are you suggesting another approach ? was (Author: shivram): [~rlei] if you need to build the RPM to work with hdp distribution, you will need to set the following variable {code} HD=hdp make rpm {code} The default behavior when you run make rpm would be to produce apache hadoop compliant rpms. If the user plans to run on hadoop without rpm, he can simply resort to building from PXF source using make install as mentioned in https://cwiki.apache.org/confluence/display/HAWQ/PXF+Build+and+Install Are you suggesting another approach ? > Improve PXF rpm package name format and dependencies > > > Key: HAWQ-1421 > URL: https://issues.apache.org/jira/browse/HAWQ-1421 > Project: Apache HAWQ > Issue Type: Improvement > Components: Build, PXF >Reporter: Radar Lei >Assignee: Shivram Mani > Fix For: 2.2.0.0-incubating > > > If we build pxf rpm package by 'make rpm', we will get below pxf packages: > {quote} > apache-tomcat-7.0.62-el6.noarch.rpm > pxf-3.2.1.0-root.el6.noarch.rpm > pxf-hbase_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > pxf-hdfs_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > pxf-hive_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > pxf-jdbc_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > pxf-json_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > pxf-service_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > {quote} > These rpm packages have dependencies on Apache Hadoop components only, some > other Hadoop distributes can't satisfy it. E.g. : > {quote} > rpm -ivh pxf-hdfs_3_2_1_0-3.2.1.0-root.el6.noarch.rpm > error: Failed dependencies: > pxf-service_3_2_1_0 >= 3.2.1.0 is needed by > pxf-hdfs_3_2_1_0-0:3.2.1.0-root.el6.noarch > hadoop >= 2.7.1 is needed by pxf-hdfs_3_2_1_0-0:3.2.1.0-root.el6.noarch > hadoop-mapreduce >= 2.7.1 is needed by > pxf-hdfs_3_2_1_0-0:3.2.1.0-root.el6.noarch > {quote} > We'd better make the rpm package name format and dependencies better. > 1. Remove the version string like '3_2_1_0'. > 2. Remove the user name from the build environment. > 3. Consider do we need to include the apache-tomcat rpm package into HAWQ > rpm release tarball. > 4. Improve the hard code 'el6' string. (This might be optinal) > 5. Improve the dependencies, including the dependencies between these pxf > rpm packages. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HAWQ-1394) docs - PXF ORC support no longer beta
[ https://issues.apache.org/jira/browse/HAWQ-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949309#comment-15949309 ] ASF GitHub Bot commented on HAWQ-1394: -- Github user lisakowen closed the pull request at: https://github.com/apache/incubator-hawq-docs/pull/104 > docs - PXF ORC support no longer beta > - > > Key: HAWQ-1394 > URL: https://issues.apache.org/jira/browse/HAWQ-1394 > Project: Apache HAWQ > Issue Type: New Feature > Components: Documentation >Reporter: Lisa Owen >Assignee: David Yozie > Fix For: 2.2.0.0-incubating > > > pxf orc file format support is no longer beta. also integrate into the > mainline docs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)