[jira] [Deleted] (HAWQ-1519) Ambiguous or conflicting information about max_segments/max_prepared_transactions
[ https://issues.apache.org/jira/browse/HAWQ-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik deleted HAWQ-1519: --- > Ambiguous or conflicting information about > max_segments/max_prepared_transactions > - > > Key: HAWQ-1519 > URL: https://issues.apache.org/jira/browse/HAWQ-1519 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Jane Beckman >Assignee: David Yozie >Priority: Minor > > The documents currently state: "When you set max_connections, you must also > set the dependent parameter max_prepared_transactions. This value must be at > least as large as the value of max_connections, and all HAWQ instances should > be set to the same value." However, Radar has stated "HAWQ updated > ‘max_connections’ value to 1280 since HDB2.0. For ‘max_prepared_transactions’ > value, we’d better keep the default value as 250 since now it’s working fine. > " This will need clarification, since the default values now differ and > either the first statement is no longer true, and should be removed, or the > user must reset one of the defaults to match the other. > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HAWQ-1484) Spin PXF into a Separate Project for Data Access
[ https://issues.apache.org/jira/browse/HAWQ-1484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052622#comment-16052622 ] Roman Shaposhnik commented on HAWQ-1484: This seems to be very much related to HAWQ-1233 [~sirinath] do you think we can close this one as a duplicate? > Spin PXF into a Separate Project for Data Access > > > Key: HAWQ-1484 > URL: https://issues.apache.org/jira/browse/HAWQ-1484 > Project: Apache HAWQ > Issue Type: New Feature >Reporter: Suminda Dharmasena >Assignee: Radar Lei > > Can the PXF be spinned into a seperate projects here they can be used as a > basis for other data access projects. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HAWQ-1351) Need to call out the fact that tweets.tar.gz is synthetic
[ https://issues.apache.org/jira/browse/HAWQ-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879103#comment-15879103 ] Roman Shaposhnik commented on HAWQ-1351: Any chance you'd be able to add a comment to tweets-pp.json with the usual AL license header? > Need to call out the fact that tweets.tar.gz is synthetic > - > > Key: HAWQ-1351 > URL: https://issues.apache.org/jira/browse/HAWQ-1351 > Project: Apache HAWQ > Issue Type: Improvement > Components: Build >Reporter: Roman Shaposhnik >Assignee: Ed Espino > > The content of ./pxf/pxf-json/src/test/resources/tweets.tar.gz is synthetic > in nature (and tiny) and thus OK to ship as part of the testsuite. However, > it would be great to call out this fact in pom.xml exclusion list. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HAWQ-1351) Need to call out the fact that tweets.tar.gz is synthetic
Roman Shaposhnik created HAWQ-1351: -- Summary: Need to call out the fact that tweets.tar.gz is synthetic Key: HAWQ-1351 URL: https://issues.apache.org/jira/browse/HAWQ-1351 Project: Apache HAWQ Issue Type: Improvement Components: Build Reporter: Roman Shaposhnik Assignee: Ed Espino The content of ./pxf/pxf-json/src/test/resources/tweets.tar.gz is synthetic in nature (and tiny) and thus OK to ship as part of the testsuite. However, it would be great to call out this fact in pom.xml exclusion list. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HAWQ-1288) Create a standalone PXF command line client
Roman Shaposhnik created HAWQ-1288: -- Summary: Create a standalone PXF command line client Key: HAWQ-1288 URL: https://issues.apache.org/jira/browse/HAWQ-1288 Project: Apache HAWQ Issue Type: Sub-task Components: PXF Reporter: Roman Shaposhnik Assignee: Ed Espino In order for PXF to start feeling like a standalone component it would be great if we could create a command line client along the lines of hadoop fs ... CLI. A related benefit here is a much crisper articulation of PXF APIs in code (something that is currently pretty difficult to untangle from the only PXF client that exists -- HAWQ's own C stub) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1287) Document PXF Setup/Build Instructions
[ https://issues.apache.org/jira/browse/HAWQ-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik updated HAWQ-1287: --- Issue Type: Sub-task (was: Improvement) Parent: HAWQ-1233 > Document PXF Setup/Build Instructions > - > > Key: HAWQ-1287 > URL: https://issues.apache.org/jira/browse/HAWQ-1287 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Documentation, PXF >Reporter: Shivram Mani >Assignee: David Yozie > > Currently HAWQ build instructions > https://cwiki.apache.org/confluence/display/HAWQ/Build+and+Install include > information to setup Hadoop apart from compiling and installing HAWQ from > source code. This provides the means to try and test HAWQ as well. > We would need to extend this to also include instructions to setup PXF along > with instructions to setup Hive/HBase and other services that PXF can > integrate with. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1235) Create a prototype for Calcite running with PXF backend
[ https://issues.apache.org/jira/browse/HAWQ-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik updated HAWQ-1235: --- Summary: Create a prototype for Calcite running with PXF backend (was: Created a prototype for Calcite running with PXF backend) > Create a prototype for Calcite running with PXF backend > --- > > Key: HAWQ-1235 > URL: https://issues.apache.org/jira/browse/HAWQ-1235 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > > A possibility of using PXF outside of its original HAWQ use case is pretty > exciting. Calcite is also pretty exciting. This JIRA tracks my efforts of > making the prototype of the two working together. > The first goal here could be make PXF available to Apache Calcite > as a backend (since Calcite itself doesn't deal with storage of data, > algorithms to process data, and a repository for storing metadata). > Calcite comes with a demo that allows you to treat a directory > full of CSV files as a DB (with individual files being tables) and > I wanted to extend that demo to use PXF reading CSV files from HDFS > instead: > http://calcite.apache.org/docs/tutorial.html > https://github.com/apache/calcite/tree/master/example/csv/src/main/java/org/apache/calcite/adapter/csv > Once that is done -- we can grow from there and use Calcite to validate > assumptions that exist between HAWQ and PXF -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-1235) Created a prototype for Calcite running with PXF backend
[ https://issues.apache.org/jira/browse/HAWQ-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik reassigned HAWQ-1235: -- Assignee: Roman Shaposhnik (was: Ed Espino) > Created a prototype for Calcite running with PXF backend > > > Key: HAWQ-1235 > URL: https://issues.apache.org/jira/browse/HAWQ-1235 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > > A possibility of using PXF outside of its original HAWQ use case is pretty > exciting. Calcite is also pretty exciting. This JIRA tracks my efforts of > making the prototype of the two working together. > The first goal here could be make PXF available to Apache Calcite > as a backend (since Calcite itself doesn't deal with storage of data, > algorithms to process data, and a repository for storing metadata). > Calcite comes with a demo that allows you to treat a directory > full of CSV files as a DB (with individual files being tables) and > I wanted to extend that demo to use PXF reading CSV files from HDFS > instead: > http://calcite.apache.org/docs/tutorial.html > https://github.com/apache/calcite/tree/master/example/csv/src/main/java/org/apache/calcite/adapter/csv > Once that is done -- we can grow from there and use Calcite to validate > assumptions that exist between HAWQ and PXF -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1235) Created a prototype for Calcite running with PXF backend
Roman Shaposhnik created HAWQ-1235: -- Summary: Created a prototype for Calcite running with PXF backend Key: HAWQ-1235 URL: https://issues.apache.org/jira/browse/HAWQ-1235 Project: Apache HAWQ Issue Type: Sub-task Components: PXF Reporter: Roman Shaposhnik Assignee: Ed Espino A possibility of using PXF outside of its original HAWQ use case is pretty exciting. Calcite is also pretty exciting. This JIRA tracks my efforts of making the prototype of the two working together. The first goal here could be make PXF available to Apache Calcite as a backend (since Calcite itself doesn't deal with storage of data, algorithms to process data, and a repository for storing metadata). Calcite comes with a demo that allows you to treat a directory full of CSV files as a DB (with individual files being tables) and I wanted to extend that demo to use PXF reading CSV files from HDFS instead: http://calcite.apache.org/docs/tutorial.html https://github.com/apache/calcite/tree/master/example/csv/src/main/java/org/apache/calcite/adapter/csv Once that is done -- we can grow from there and use Calcite to validate assumptions that exist between HAWQ and PXF -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-1234) Document HAWQ to PXF APIs
[ https://issues.apache.org/jira/browse/HAWQ-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik reassigned HAWQ-1234: -- Assignee: Roman Shaposhnik (was: Ed Espino) > Document HAWQ to PXF APIs > - > > Key: HAWQ-1234 > URL: https://issues.apache.org/jira/browse/HAWQ-1234 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > Attachments: PXFAdvancedStatsplan.pdf > > > It would be very useful to start documenting HAWQ to PXF APIs. The right > places to start are: >* libcurl (a thin wrapper for making HAWQ C code be able to do REST calls): > https://github.com/apache/incubator-hawq/blob/master/src/include/access/libchurl.h > https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/libchurl.c >* pxfmasterapi (mostly metadata calls that master is doing): > https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/pxfmasterapi.c > Here you will find how HAWQ via PXF pulls using a REST API to get external > metadata and some logic to parse the JSON response. >* gpbridgeapi (segment calls to PXF): > https://github.com/apache/incubator-hawq/blob/master/src/bin/gpfusion/gpbridgeapi.c > Here you will find other examples of (read and write calls) used to fetch > external data. > Design doc on PXF's support for analyze (pxf's analyzer) is attached -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-1224) It would be useful to have a gradle task that runs a PXF instance
[ https://issues.apache.org/jira/browse/HAWQ-1224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik reassigned HAWQ-1224: -- Assignee: Roman Shaposhnik (was: Ed Espino) > It would be useful to have a gradle task that runs a PXF instance > - > > Key: HAWQ-1224 > URL: https://issues.apache.org/jira/browse/HAWQ-1224 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > Fix For: 2.2.0.0-incubating > > Attachments: HAWQ-1224.patch.txt > > > For testing and tinkering it is very useful to be able to just say > $ gradle appRun > and have a working instance of PXF running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1224) It would be useful to have a gradle task that runs a PXF instance
[ https://issues.apache.org/jira/browse/HAWQ-1224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik updated HAWQ-1224: --- Issue Type: Sub-task (was: Bug) Parent: HAWQ-1233 > It would be useful to have a gradle task that runs a PXF instance > - > > Key: HAWQ-1224 > URL: https://issues.apache.org/jira/browse/HAWQ-1224 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Roman Shaposhnik >Assignee: Ed Espino > Fix For: 2.2.0.0-incubating > > Attachments: HAWQ-1224.patch.txt > > > For testing and tinkering it is very useful to be able to just say > $ gradle appRun > and have a working instance of PXF running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1234) Document HAWQ to PXF APIs
[ https://issues.apache.org/jira/browse/HAWQ-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik updated HAWQ-1234: --- Description: It would be very useful to start documenting HAWQ to PXF APIs. The right places to start are: * libcurl (a thin wrapper for making HAWQ C code be able to do REST calls): https://github.com/apache/incubator-hawq/blob/master/src/include/access/libchurl.h https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/libchurl.c * pxfmasterapi (mostly metadata calls that master is doing): https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/pxfmasterapi.c Here you will find how HAWQ via PXF pulls using a REST API to get external metadata and some logic to parse the JSON response. * gpbridgeapi (segment calls to PXF): https://github.com/apache/incubator-hawq/blob/master/src/bin/gpfusion/gpbridgeapi.c Here you will find other examples of (read and write calls) used to fetch external data. Design doc on PXF's support for analyze (pxf's analyzer) is attached was: It would be very useful to start documenting HAWQ to PXF APIs. The right places to start are: * libcurl (a thin wrapper for making HAWQ C code be able to do REST calls): https://github.com/apache/incubator-hawq/blob/master/src/include/access/libchurl.h https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/libchurl.c * pxfmasterapi (mostly metadata calls that master is doing): https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/pxfmasterapi.c Here you will find how HAWQ via PXF pulls using a REST API to get external metadata and some logic to parse the JSON response. * gpbridgeapi (segment calls to PXF): https://github.com/apache/incubator-hawq/blob/master/src/bin/gpfusion/gpbridgeapi.c Here you will find other examples of (read and write calls) used to fetch external data. > Document HAWQ to PXF APIs > - > > Key: HAWQ-1234 > URL: https://issues.apache.org/jira/browse/HAWQ-1234 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Roman Shaposhnik >Assignee: Ed Espino > Attachments: PXFAdvancedStatsplan.pdf > > > It would be very useful to start documenting HAWQ to PXF APIs. The right > places to start are: >* libcurl (a thin wrapper for making HAWQ C code be able to do REST calls): > https://github.com/apache/incubator-hawq/blob/master/src/include/access/libchurl.h > https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/libchurl.c >* pxfmasterapi (mostly metadata calls that master is doing): > https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/pxfmasterapi.c > Here you will find how HAWQ via PXF pulls using a REST API to get external > metadata and some logic to parse the JSON response. >* gpbridgeapi (segment calls to PXF): > https://github.com/apache/incubator-hawq/blob/master/src/bin/gpfusion/gpbridgeapi.c > Here you will find other examples of (read and write calls) used to fetch > external data. > Design doc on PXF's support for analyze (pxf's analyzer) is attached -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1234) Document HAWQ to PXF APIs
[ https://issues.apache.org/jira/browse/HAWQ-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik updated HAWQ-1234: --- Attachment: PXFAdvancedStatsplan.pdf > Document HAWQ to PXF APIs > - > > Key: HAWQ-1234 > URL: https://issues.apache.org/jira/browse/HAWQ-1234 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Roman Shaposhnik >Assignee: Ed Espino > Attachments: PXFAdvancedStatsplan.pdf > > > It would be very useful to start documenting HAWQ to PXF APIs. The right > places to start are: >* libcurl (a thin wrapper for making HAWQ C code be able to do REST calls): > https://github.com/apache/incubator-hawq/blob/master/src/include/access/libchurl.h > https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/libchurl.c >* pxfmasterapi (mostly metadata calls that master is doing): > https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/pxfmasterapi.c > Here you will find how HAWQ via PXF pulls using a REST API to get external > metadata and some logic to parse the JSON response. >* gpbridgeapi (segment calls to PXF): > https://github.com/apache/incubator-hawq/blob/master/src/bin/gpfusion/gpbridgeapi.c > Here you will find other examples of (read and write calls) used to fetch > external data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1234) Document HAWQ to PXF APIs
Roman Shaposhnik created HAWQ-1234: -- Summary: Document HAWQ to PXF APIs Key: HAWQ-1234 URL: https://issues.apache.org/jira/browse/HAWQ-1234 Project: Apache HAWQ Issue Type: Sub-task Components: PXF Reporter: Roman Shaposhnik Assignee: Ed Espino It would be very useful to start documenting HAWQ to PXF APIs. The right places to start are: * libcurl (a thin wrapper for making HAWQ C code be able to do REST calls): https://github.com/apache/incubator-hawq/blob/master/src/include/access/libchurl.h https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/libchurl.c * pxfmasterapi (mostly metadata calls that master is doing): https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/pxfmasterapi.c Here you will find how HAWQ via PXF pulls using a REST API to get external metadata and some logic to parse the JSON response. * gpbridgeapi (segment calls to PXF): https://github.com/apache/incubator-hawq/blob/master/src/bin/gpfusion/gpbridgeapi.c Here you will find other examples of (read and write calls) used to fetch external data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-1233) Turn PXF into Postgres eXtension Framework
[ https://issues.apache.org/jira/browse/HAWQ-1233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik reassigned HAWQ-1233: -- Assignee: Roman Shaposhnik (was: Ed Espino) > Turn PXF into Postgres eXtension Framework > -- > > Key: HAWQ-1233 > URL: https://issues.apache.org/jira/browse/HAWQ-1233 > Project: Apache HAWQ > Issue Type: Wish > Components: PXF >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > > Even though PXF got developed as a external tables framework for HAWQ it > seems that it could be an extremely useful standalone service connecting > Postgres ecosystem to Hadoop ecosystem. > This is an umbrella JIRA for exploring a possibility of turning PXF into a > Postgres eXtension Framework -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1233) Turn PXF into Postgres eXtension Framework
Roman Shaposhnik created HAWQ-1233: -- Summary: Turn PXF into Postgres eXtension Framework Key: HAWQ-1233 URL: https://issues.apache.org/jira/browse/HAWQ-1233 Project: Apache HAWQ Issue Type: Wish Components: PXF Reporter: Roman Shaposhnik Assignee: Ed Espino Even though PXF got developed as a external tables framework for HAWQ it seems that it could be an extremely useful standalone service connecting Postgres ecosystem to Hadoop ecosystem. This is an umbrella JIRA for exploring a possibility of turning PXF into a Postgres eXtension Framework -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1224) It would be useful to have a gradle task that runs a PXF instance
[ https://issues.apache.org/jira/browse/HAWQ-1224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik updated HAWQ-1224: --- Attachment: HAWQ-1224.patch.txt Attached patch gives an idea of what I have in mind > It would be useful to have a gradle task that runs a PXF instance > - > > Key: HAWQ-1224 > URL: https://issues.apache.org/jira/browse/HAWQ-1224 > Project: Apache HAWQ > Issue Type: Bug > Components: PXF >Reporter: Roman Shaposhnik >Assignee: Ed Espino > Attachments: HAWQ-1224.patch.txt > > > For testing and tinkering it is very useful to be able to just say > $ gradle appRun > and have a working instance of PXF running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1224) It would be useful to have a gradle task that runs a PXF instance
Roman Shaposhnik created HAWQ-1224: -- Summary: It would be useful to have a gradle task that runs a PXF instance Key: HAWQ-1224 URL: https://issues.apache.org/jira/browse/HAWQ-1224 Project: Apache HAWQ Issue Type: Bug Components: PXF Reporter: Roman Shaposhnik Assignee: Ed Espino For testing and tinkering it is very useful to be able to just say $ gradle appRun and have a working instance of PXF running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1212) Upgrade libhdfs3 with upstream hadoop/hdp
[ https://issues.apache.org/jira/browse/HAWQ-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743612#comment-15743612 ] Roman Shaposhnik commented on HAWQ-1212: My plan is to help you with mentorship required to merge it with Apache Hadoop codebase -- somebody in HAWQ will still have to commit to doing all the actual work. > Upgrade libhdfs3 with upstream hadoop/hdp > - > > Key: HAWQ-1212 > URL: https://issues.apache.org/jira/browse/HAWQ-1212 > Project: Apache HAWQ > Issue Type: New Feature > Components: libhdfs >Reporter: hongwu >Assignee: Lei Chang > > Since current libhdfs3 implementation is based on old version of HDFS, it is > necessary to upgrade libhdfs3 with latest HDFS of lots of new features and > configuration parameters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1212) Upgrade libhdfs3 with upstream hadoop/hdp
[ https://issues.apache.org/jira/browse/HAWQ-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15740635#comment-15740635 ] Roman Shaposhnik commented on HAWQ-1212: Can you please provide more details here? In general, now is a good time to start thinking about working with upstream Hadoop community to merge libhdfs3 into the Apache Hadoop. > Upgrade libhdfs3 with upstream hadoop/hdp > - > > Key: HAWQ-1212 > URL: https://issues.apache.org/jira/browse/HAWQ-1212 > Project: Apache HAWQ > Issue Type: New Feature > Components: libhdfs >Reporter: hongwu >Assignee: Lei Chang > > Since current libhdfs3 implementation is based on old version of HDFS, it is > necessary to upgrade libhdfs3 with latest HDFS of lots of new features and > configuration parameters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1134) Add Bigtop layout specific pxf-private classpath
[ https://issues.apache.org/jira/browse/HAWQ-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637866#comment-15637866 ] Roman Shaposhnik commented on HAWQ-1134: Would it be ok to commit it now and refactor later? > Add Bigtop layout specific pxf-private classpath > > > Key: HAWQ-1134 > URL: https://issues.apache.org/jira/browse/HAWQ-1134 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > > Currently PXF ships with HDP and PHD specific classpath files. It would be > great to have Bigtop specific one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-1134) Add Bigtop layout specific pxf-private classpath
[ https://issues.apache.org/jira/browse/HAWQ-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik reassigned HAWQ-1134: -- Assignee: Roman Shaposhnik (was: Lei Chang) > Add Bigtop layout specific pxf-private classpath > > > Key: HAWQ-1134 > URL: https://issues.apache.org/jira/browse/HAWQ-1134 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > > Currently PXF ships with HDP and PHD specific classpath files. It would be > great to have Bigtop specific one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1134) Add Bigtop layout specific pxf-private classpath
Roman Shaposhnik created HAWQ-1134: -- Summary: Add Bigtop layout specific pxf-private classpath Key: HAWQ-1134 URL: https://issues.apache.org/jira/browse/HAWQ-1134 Project: Apache HAWQ Issue Type: Improvement Components: PXF Reporter: Roman Shaposhnik Assignee: Lei Chang Currently PXF ships with HDP and PHD specific classpath files. It would be great to have Bigtop specific one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1046) Document migration of LibHDFS3 library to HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15518093#comment-15518093 ] Roman Shaposhnik commented on HAWQ-1046: [~bolke] first of all lets address the proverbial elephant in the room -- if we're talking about long-term viable community efforts around C/C++ direct interface with HDFS than there's HADOOP-10388. The ultimate effort really can't be separated from the Apache Hadoop community. I don't think it would be fair to assume that what we do for HAWQ is somehow a replacement for that. In fact, long term, I really see the two efforts merging. See HDFS-6994 for example. Thus, having yet another project on GitHub seems counter-productive. > Document migration of LibHDFS3 library to HAWQ > -- > > Key: HAWQ-1046 > URL: https://issues.apache.org/jira/browse/HAWQ-1046 > Project: Apache HAWQ > Issue Type: Wish > Components: libhdfs >Reporter: Matthew Rocklin >Assignee: hongwu > Fix For: backlog > > > Some people used to depend on the libhdfs3 library maintained alongside HAWQ. > This library was merged into the HAWQ codebase, making the situation a bit > more confusing. > Is independent use of libhdfs3 still supported by this community? If so what > is the best way for packagers to reason about versions and releases of this > component? It would be convenient to see documentation on how people can > best depend on libhdfs3 separately from HAWQ if this is an intention. > It looks like people have actually submitted work to the old version > See: https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/pull/28 > It looks like the warning that the library had moved has been removed: > https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/commit/ddcb2404a5a67e0f39fe49ed20591545c48ff426 > This removal may lead to some frustration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1046) Document migration of LibHDFS3 library to HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15491901#comment-15491901 ] Roman Shaposhnik commented on HAWQ-1046: Guys, the following is with my Apache HAWQ (incubating) mentor hat on: what you're describing makes me uncomfortable. It sounds like the development of a fundamental component of Apache HAWQ (incubating) is being done outside of ASF. This is not acceptable. I thought we had an understanding that the Pivotal-DataFabric was only there for historical interest. Lets discuss this more on a mailing list, but at this point, I'd like to have a plan of how we shut down Pivotal-DataFabric/pivotalrd-libhdfs3 > Document migration of LibHDFS3 library to HAWQ > -- > > Key: HAWQ-1046 > URL: https://issues.apache.org/jira/browse/HAWQ-1046 > Project: Apache HAWQ > Issue Type: Wish > Components: libhdfs >Reporter: Matthew Rocklin >Assignee: hongwu > > Some people used to depend on the libhdfs3 library maintained alongside HAWQ. > This library was merged into the HAWQ codebase, making the situation a bit > more confusing. > Is independent use of libhdfs3 still supported by this community? If so what > is the best way for packagers to reason about versions and releases of this > component? It would be convenient to see documentation on how people can > best depend on libhdfs3 separately from HAWQ if this is an intention. > It looks like people have actually submitted work to the old version > See: https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/pull/28 > It looks like the warning that the library had moved has been removed: > https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/commit/ddcb2404a5a67e0f39fe49ed20591545c48ff426 > This removal may lead to some frustration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-914) Improve user experience of HAWQ's build infrastructure
[
https://issues.apache.org/jira/browse/HAWQ-914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Shaposhnik updated HAWQ-914:
--
Affects Version/s: 2.0.0.0-incubating
> Improve user experience of HAWQ's build infrastructure
> --
>
> Key: HAWQ-914
> URL: https://issues.apache.org/jira/browse/HAWQ-914
> Project: Apache HAWQ
> Issue Type: Improvement
> Components: Build
>Affects Versions: 2.0.0.0-incubating
>Reporter: Roman Shaposhnik
>Assignee: Lei Chang
>
> This is likely to end up being an umbrella JIRA so feel free to fork off
> sub-tasks whenever it makes sense.
> As an end-user of HAWQ's build system, I'd like to see the default of the
> build system (running configure/etc. with no arguments) to be:
> # treating optional missing dependencies with a WARNING similar to what
> PostrgreSQL configure does in the following example:
> {noformat}
> checking for bison... no
> configure: WARNING:
> *** Without Bison you will not be able to build PostgreSQL from CVS nor
> *** change any of the parser definition files. You can obtain Bison from
> *** a GNU mirror site. (If you are using the official distribution of
> *** PostgreSQL then you do not need to worry about this, because the Bison
> *** output is pre-generated.) To use a different yacc program (possible,
> *** but not recommended), set the environment variable YACC before running
> *** 'configure'.
> {noformat}
> # treating all the missing suggested dependencies by failing the build and
> suggesting how to point at binary copies of these missing dependencies
> similar to what PostrgreSQL configure does in the following example:
> {noformat}
> checking for -ledit... no
> configure: error: readline library not found
> If you have readline already installed, see config.log for details on the
> failure. It is possible the compiler isn't looking in the proper directory.
> Use --without-readline to disable readline support.
> {noformat}
> # treating the core dependencies same as suggested dependencies, but
> obviously about the option of continuing the build without them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HAWQ-915) RAT is failing on the latest master
[ https://issues.apache.org/jira/browse/HAWQ-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik updated HAWQ-915: -- Attachment: rat.txt > RAT is failing on the latest master > --- > > Key: HAWQ-915 > URL: https://issues.apache.org/jira/browse/HAWQ-915 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Affects Versions: 2.0.0.0-incubating >Reporter: Roman Shaposhnik >Assignee: Lei Chang > Fix For: 2.0.0.0-incubating > > Attachments: rat.txt > > > Running RAT (via the usual mvn verify) on the current tree produces quite a > bit of hits on the source code added lately to the HAWQ's repo. The full > report is attached. This is a blocking JIRA for 2.0.0.0-incubating. PMC > members should very carefully go over the RAT report and either get rid of > offending code, update the licensing information or include it into the RAT > exclusion list. The later has to have explicit explanation of why the code > has to reside in the HAWQ repo as opposed to being a dependency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-915) RAT is failing on the latest master
[ https://issues.apache.org/jira/browse/HAWQ-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik updated HAWQ-915: -- Affects Version/s: 2.0.0.0-incubating > RAT is failing on the latest master > --- > > Key: HAWQ-915 > URL: https://issues.apache.org/jira/browse/HAWQ-915 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Affects Versions: 2.0.0.0-incubating >Reporter: Roman Shaposhnik >Assignee: Lei Chang >Priority: Blocker > Fix For: 2.0.0.0-incubating > > Attachments: rat.txt > > > Running RAT (via the usual mvn verify) on the current tree produces quite a > bit of hits on the source code added lately to the HAWQ's repo. The full > report is attached. This is a blocking JIRA for 2.0.0.0-incubating. PMC > members should very carefully go over the RAT report and either get rid of > offending code, update the licensing information or include it into the RAT > exclusion list. The later has to have explicit explanation of why the code > has to reside in the HAWQ repo as opposed to being a dependency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-915) RAT is failing on the latest master
[ https://issues.apache.org/jira/browse/HAWQ-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik updated HAWQ-915: -- Priority: Blocker (was: Major) > RAT is failing on the latest master > --- > > Key: HAWQ-915 > URL: https://issues.apache.org/jira/browse/HAWQ-915 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Affects Versions: 2.0.0.0-incubating >Reporter: Roman Shaposhnik >Assignee: Lei Chang >Priority: Blocker > Fix For: 2.0.0.0-incubating > > Attachments: rat.txt > > > Running RAT (via the usual mvn verify) on the current tree produces quite a > bit of hits on the source code added lately to the HAWQ's repo. The full > report is attached. This is a blocking JIRA for 2.0.0.0-incubating. PMC > members should very carefully go over the RAT report and either get rid of > offending code, update the licensing information or include it into the RAT > exclusion list. The later has to have explicit explanation of why the code > has to reside in the HAWQ repo as opposed to being a dependency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-915) RAT is failing on the latest master
Roman Shaposhnik created HAWQ-915: - Summary: RAT is failing on the latest master Key: HAWQ-915 URL: https://issues.apache.org/jira/browse/HAWQ-915 Project: Apache HAWQ Issue Type: Bug Components: Build Reporter: Roman Shaposhnik Assignee: Lei Chang Running RAT (via the usual mvn verify) on the current tree produces quite a bit of hits on the source code added lately to the HAWQ's repo. The full report is attached. This is a blocking JIRA for 2.0.0.0-incubating. PMC members should very carefully go over the RAT report and either get rid of offending code, update the licensing information or include it into the RAT exclusion list. The later has to have explicit explanation of why the code has to reside in the HAWQ repo as opposed to being a dependency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-914) Improve user experience of HAWQ's build infrastructure
Roman Shaposhnik created HAWQ-914:
-
Summary: Improve user experience of HAWQ's build infrastructure
Key: HAWQ-914
URL: https://issues.apache.org/jira/browse/HAWQ-914
Project: Apache HAWQ
Issue Type: Improvement
Components: Build
Reporter: Roman Shaposhnik
Assignee: Lei Chang
This is likely to end up being an umbrella JIRA so feel free to fork off
sub-tasks whenever it makes sense.
As an end-user of HAWQ's build system, I'd like to see the default of the build
system (running configure/etc. with no arguments) to be:
# treating optional missing dependencies with a WARNING similar to what
PostrgreSQL configure does in the following example:
{noformat}
checking for bison... no
configure: WARNING:
*** Without Bison you will not be able to build PostgreSQL from CVS nor
*** change any of the parser definition files. You can obtain Bison from
*** a GNU mirror site. (If you are using the official distribution of
*** PostgreSQL then you do not need to worry about this, because the Bison
*** output is pre-generated.) To use a different yacc program (possible,
*** but not recommended), set the environment variable YACC before running
*** 'configure'.
{noformat}
# treating all the missing suggested dependencies by failing the build and
suggesting how to point at binary copies of these missing dependencies similar
to what PostrgreSQL configure does in the following example:
{noformat}
checking for -ledit... no
configure: error: readline library not found
If you have readline already installed, see config.log for details on the
failure. It is possible the compiler isn't looking in the proper directory.
Use --without-readline to disable readline support.
{noformat}
# treating the core dependencies same as suggested dependencies, but obviously
about the option of continuing the build without them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HAWQ-761) Provide brew packaging for HAWQ
Roman Shaposhnik created HAWQ-761: - Summary: Provide brew packaging for HAWQ Key: HAWQ-761 URL: https://issues.apache.org/jira/browse/HAWQ-761 Project: Apache HAWQ Issue Type: Improvement Components: Build Reporter: Roman Shaposhnik Assignee: Lei Chang Now that HAWQ is getting really close to rolling its first official Incubator release it would be great to provide brew packaging for it so that more folks can take it for a spin. Here's what it takes to add a formula to brew (ask on this JIRA if you have further questions): https://github.com/Homebrew/brew/blob/master/share/doc/homebrew/Formula-Cookbook.md I propose that for the brew packaging HAWQ is configured to use local filesystem as HDFS (more correctly HCFS) and runs without YARN in a standalone mode. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-307) Ubuntu Support
[ https://issues.apache.org/jira/browse/HAWQ-307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15235459#comment-15235459 ] Roman Shaposhnik commented on HAWQ-307: --- [~clayb] thanks for the feedback! Have you tried building the latest master? Also, let me add you to the JIRA collaborator list so you can close the issues in the future. Looking forward to more collaboration on Ubuntu! > Ubuntu Support > -- > > Key: HAWQ-307 > URL: https://issues.apache.org/jira/browse/HAWQ-307 > Project: Apache HAWQ > Issue Type: New Feature > Components: Build >Reporter: Lei Chang >Assignee: Clay B. > Fix For: 2.0.0 > > > To support HAWQ running on Ubuntu OS 14.04.3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (HAWQ-182) Collect advanced statistics for HBase plugin
[ https://issues.apache.org/jira/browse/HAWQ-182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik updated HAWQ-182: -- Comment: was deleted (was: [~gracekaboh] Grace, awesome to see your interest in contributing to HAWQ! Please make sure to submit your formal GSoC application this week though. More details on the timeline and how to submit here: http://community.apache.org/gsoc.html) > Collect advanced statistics for HBase plugin > > > Key: HAWQ-182 > URL: https://issues.apache.org/jira/browse/HAWQ-182 > Project: Apache HAWQ > Issue Type: New Feature > Components: PXF >Reporter: Noa Horn >Assignee: Goden Yao > Labels: gsoc2016 > Fix For: backlog > > > Implement getFragmentsStats in HBase's fragmenter (HBaseDataFragmenter). > As a result when running ANALYZE on PXF tables with HBase profile, advanced > statistics will be collected for that table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-182) Collect advanced statistics for HBase plugin
[ https://issues.apache.org/jira/browse/HAWQ-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205240#comment-15205240 ] Roman Shaposhnik commented on HAWQ-182: --- [~gracekaboh] Grace, awesome to see your interest in contributing to HAWQ! Please make sure to submit your formal GSoC application this week though. More details on the timeline and how to submit here: http://community.apache.org/gsoc.html > Collect advanced statistics for HBase plugin > > > Key: HAWQ-182 > URL: https://issues.apache.org/jira/browse/HAWQ-182 > Project: Apache HAWQ > Issue Type: New Feature > Components: PXF >Reporter: Noa Horn >Assignee: Goden Yao > Labels: gsoc2016 > Fix For: backlog > > > Implement getFragmentsStats in HBase's fragmenter (HBaseDataFragmenter). > As a result when running ANALYZE on PXF tables with HBase profile, advanced > statistics will be collected for that table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-182) Collect advanced statistics for HBase plugin
[ https://issues.apache.org/jira/browse/HAWQ-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205241#comment-15205241 ] Roman Shaposhnik commented on HAWQ-182: --- [~gracekaboh] Grace, awesome to see your interest in contributing to HAWQ! Please make sure to submit your formal GSoC application this week though. More details on the timeline and how to submit here: http://community.apache.org/gsoc.html > Collect advanced statistics for HBase plugin > > > Key: HAWQ-182 > URL: https://issues.apache.org/jira/browse/HAWQ-182 > Project: Apache HAWQ > Issue Type: New Feature > Components: PXF >Reporter: Noa Horn >Assignee: Goden Yao > Labels: gsoc2016 > Fix For: backlog > > > Implement getFragmentsStats in HBase's fragmenter (HBaseDataFragmenter). > As a result when running ANALYZE on PXF tables with HBase profile, advanced > statistics will be collected for that table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-490) Add version and compatibility detection for HAWQ + GPORCA
[
https://issues.apache.org/jira/browse/HAWQ-490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184233#comment-15184233
]
Roman Shaposhnik commented on HAWQ-490:
---
[~xzhang.pivotal] the standard way is what I linked to. There's no other
standard way for C and C++ components.
> Add version and compatibility detection for HAWQ + GPORCA
> -
>
> Key: HAWQ-490
> URL: https://issues.apache.org/jira/browse/HAWQ-490
> Project: Apache HAWQ
> Issue Type: Improvement
> Components: Build
>Reporter: Jacob Max Frank
>Assignee: Lei Chang
>Priority: Minor
>
> Autoconf should be able to detect GPORCA version information and check for
> compatibility with the version of HAWQ being built. Additionally, running
> {{select gp_opt_version();}} on HAWQ compiled with GPORCA should output
> correct version information for its components (GPOPT, GPOS, and GPXerces).
> [~rvs] suggested using {{pkgutil}}. Alternate potential strategies include
> using a {{git submodule}} or pulling values from a {{version.h}} file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HAWQ-490) Add version and compatibility detection for HAWQ + GPORCA
[
https://issues.apache.org/jira/browse/HAWQ-490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180432#comment-15180432
]
Roman Shaposhnik commented on HAWQ-490:
---
To expand on pgkutil option -- this is how most of the native library binding
are done in the Linux/UNIX world these days. There's an integration with auto
tools and CMake as well. https://autotools.io/pkgconfig/pkg_check_modules.html
and https://cmake.org/cmake/help/v3.0/module/FindPkgConfig.html should be good
starting points.
Now, of course, this will require GPORCA side to actually deposit pkg_config
registry. Hence this is not just on the HAWQ side.
> Add version and compatibility detection for HAWQ + GPORCA
> -
>
> Key: HAWQ-490
> URL: https://issues.apache.org/jira/browse/HAWQ-490
> Project: Apache HAWQ
> Issue Type: Improvement
> Components: Build
>Reporter: Jacob Max Frank
>Assignee: Lei Chang
>Priority: Minor
>
> Autoconf should be able to detect GPORCA version information and check for
> compatibility with the version of HAWQ being built. Additionally, running
> {{select gp_opt_version();}} on HAWQ compiled with GPORCA should output
> correct version information for its components (GPOPT, GPOS, and GPXerces).
> [~rvs] suggested using {{pkgutil}}. Alternate potential strategies include
> using a {{git submodule}} or pulling values from a {{version.h}} file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (HAWQ-388) Remove unnecessary entries in NOTICE file
[ https://issues.apache.org/jira/browse/HAWQ-388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik closed HAWQ-388. - Resolution: Fixed > Remove unnecessary entries in NOTICE file > - > > Key: HAWQ-388 > URL: https://issues.apache.org/jira/browse/HAWQ-388 > Project: Apache HAWQ > Issue Type: Task >Reporter: Goden Yao >Assignee: Roman Shaposhnik > Labels: OSS > Fix For: 2.0.0-beta-incubating > > > We found things in NOTICE file that we only link against but not distribute > for the code base. We should clean them up in the NOTICE file. > 1. Jersey is not used in the code base, its license is not compatible with > ASLv2. > 2. protobuf > 3. apache.commons.codec > 4. hbase > 5. Mockito -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HAWQ-207) Enable "make RAT" for build process
[
https://issues.apache.org/jira/browse/HAWQ-207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Shaposhnik resolved HAWQ-207.
---
Resolution: Cannot Reproduce
Assignee: Roman Shaposhnik (was: Lei Chang)
Fix Version/s: 2.0.0-beta-incubating
We went a different route and implemented: mvn verify instead. Thus I consider
this issue resolved.
> Enable "make RAT" for build process
> ---
>
> Key: HAWQ-207
> URL: https://issues.apache.org/jira/browse/HAWQ-207
> Project: Apache HAWQ
> Issue Type: Task
> Components: Build
>Reporter: Goden Yao
>Assignee: Roman Shaposhnik
> Fix For: 2.0.0-beta-incubating
>
>
> * RAT gets checked into the workspace (with exceptions
> clearly documented: e.g. :
> https://github.com/apache/bigtop/blob/master/pom.xml#L297
> * Integrate RAT execution into the build, so
> {code}
> $ make rat
> {code}
> can display the license info
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HAWQ-349) Add public KEYS to Apache HAWQ repo.
[ https://issues.apache.org/jira/browse/HAWQ-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15105647#comment-15105647 ] Roman Shaposhnik commented on HAWQ-349: --- [~lei_chang] [~cwelton] please create a CODE SIGNING KEY subkey and associate it with your ASF ID (@apache.org) The way to do it is: $ gpg --edit-key gpg> addkey // answer the prompts gpg> adduid // answer the prompts > Add public KEYS to Apache HAWQ repo. > > > Key: HAWQ-349 > URL: https://issues.apache.org/jira/browse/HAWQ-349 > Project: Apache HAWQ > Issue Type: Task >Reporter: Goden Yao >Assignee: Lei Chang > Fix For: 2.0.0-beta-incubating > > Attachments: cwelton-key.txt, godenyao-KEYS.txt, leichang.txt, > rvs-key.txt > > > As per release requirement, please attach your public keys so that a KEYS > file can be created in the repo. > Few references below: > http://www.apache.org/dist/ant/KEYS > https://dist.apache.org/repos/dist/release/bigtop/KEYS > You'll need gpg command to manipulate your keys. Here's how you can do it: > $ pgp -kxa and append it to the file to be attached > (pgpk -ll && pgpk -xa ) >> the file to be > attached > (gpg --list-sigs > && gpg --armor --export ) >> the file to be attached -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-349) Add public KEYS to Apache HAWQ repo.
[ https://issues.apache.org/jira/browse/HAWQ-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik updated HAWQ-349: -- Description: As per release requirement, please attach your public keys so that a KEYS file can be created in the repo. Few references below: http://www.apache.org/dist/ant/KEYS https://dist.apache.org/repos/dist/release/bigtop/KEYS You'll need gpg command to manipulate your keys. Here's how you can do it: $ pgp -kxa and append it to the file to be attached (pgpk -ll && pgpk -xa ) >> the file to be attached (gpg --list-sigs && gpg --armor --export ) >> the file to be attached was: As per release requirement, please attach your public keys so that a KEYS file can be created in the repo. Few references below: http://www.apache.org/dist/ant/KEYS https://dist.apache.org/repos/dist/release/bigtop/KEYS You'll need gpg command to manipulate your keys > Add public KEYS to Apache HAWQ repo. > > > Key: HAWQ-349 > URL: https://issues.apache.org/jira/browse/HAWQ-349 > Project: Apache HAWQ > Issue Type: Task >Reporter: Goden Yao >Assignee: Lei Chang > Fix For: 2.0.0-beta-incubating > > Attachments: godenyao-KEYS.txt, rvs-key.txt > > > As per release requirement, please attach your public keys so that a KEYS > file can be created in the repo. > Few references below: > http://www.apache.org/dist/ant/KEYS > https://dist.apache.org/repos/dist/release/bigtop/KEYS > You'll need gpg command to manipulate your keys. Here's how you can do it: > $ pgp -kxa and append it to the file to be attached > (pgpk -ll && pgpk -xa ) >> the file to be > attached > (gpg --list-sigs > && gpg --armor --export ) >> the file to be attached -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-349) Add public KEYS to Apache HAWQ repo.
[ https://issues.apache.org/jira/browse/HAWQ-349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15103594#comment-15103594 ] Roman Shaposhnik commented on HAWQ-349: --- Attached my key > Add public KEYS to Apache HAWQ repo. > > > Key: HAWQ-349 > URL: https://issues.apache.org/jira/browse/HAWQ-349 > Project: Apache HAWQ > Issue Type: Task >Reporter: Goden Yao >Assignee: Lei Chang > Fix For: 2.0.0-beta-incubating > > Attachments: godenyao-KEYS.txt, rvs-key.txt > > > As per release requirement, please attach your public keys so that a KEYS > file can be created in the repo. > Few references below: > http://www.apache.org/dist/ant/KEYS > https://dist.apache.org/repos/dist/release/bigtop/KEYS > You'll need gpg command to manipulate your keys -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-349) Add public KEYS to Apache HAWQ repo.
[ https://issues.apache.org/jira/browse/HAWQ-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shaposhnik updated HAWQ-349: -- Attachment: rvs-key.txt > Add public KEYS to Apache HAWQ repo. > > > Key: HAWQ-349 > URL: https://issues.apache.org/jira/browse/HAWQ-349 > Project: Apache HAWQ > Issue Type: Task >Reporter: Goden Yao >Assignee: Lei Chang > Fix For: 2.0.0-beta-incubating > > Attachments: godenyao-KEYS.txt, rvs-key.txt > > > As per release requirement, please attach your public keys so that a KEYS > file can be created in the repo. > Few references below: > http://www.apache.org/dist/ant/KEYS > https://dist.apache.org/repos/dist/release/bigtop/KEYS > You'll need gpg command to manipulate your keys -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-185) clean pxf jars in source code
[ https://issues.apache.org/jira/browse/HAWQ-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15033102#comment-15033102 ] Roman Shaposhnik commented on HAWQ-185: --- [~nhorn] I am confused: are you saying gradlew can function without the jar checked into the repo? > clean pxf jars in source code > - > > Key: HAWQ-185 > URL: https://issues.apache.org/jira/browse/HAWQ-185 > Project: Apache HAWQ > Issue Type: Bug > Components: PXF >Reporter: Lei Chang >Assignee: Noa Horn > Fix For: 2.0.0-beta-incubating > > > From Roman's scan, some jar and .classes should be cleaned. >* anythings that's *.tar.gz >* anything that's a dependency jar >* ./pxf/gradle/wrapper/gradle-wrapper.jar >* ./src/test/regress/data/pxf/*.class >* ./src/test/regress/data/pxf/*.class -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1) Remove .p4ignore files from source tree
[ https://issues.apache.org/jira/browse/HAWQ-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936592#comment-14936592 ] Roman Shaposhnik commented on HAWQ-1: - Thanks for demoing ReviewBoard. Worked like a charm! > Remove .p4ignore files from source tree > --- > > Key: HAWQ-1 > URL: https://issues.apache.org/jira/browse/HAWQ-1 > Project: Apache HAWQ > Issue Type: Bug > Environment: N/A >Reporter: Gavin Sherry >Assignee: Caleb Welton >Priority: Minor > > I notice that there are .p4ignore files in the source tree. These are > irrelevant since HAWQ is hosted in git. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-7) Hawq init fail without plpython
[ https://issues.apache.org/jira/browse/HAWQ-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14933217#comment-14933217 ] Roman Shaposhnik commented on HAWQ-7: - Very much agree with [~cwelton] Guys, there's nothing wrong with lazy consensus, but initially I'd suggest staying with RTC model. Thus +1 from somebody else should be explicitly required. > Hawq init fail without plpython > > > Key: HAWQ-7 > URL: https://issues.apache.org/jira/browse/HAWQ-7 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Lei Chang >Assignee: Roman Shaposhnik > > After following steps here: https://hub.docker.com/r/mayjojo/hawq-dev/. build > succeeded. > But "hawq init master" failed. Here is the log: > 2015-09-26 05:59:04.842990 > EDT,,,p3458,th-3822651200,,cmd1,seg-1,,,x714,sx1,"FATAL","58P01","could > not access file ""$libdir/plpython"": No such file or directory",,"-- > > > ",,"internal_load_library","dfmgr.c",208, -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-3) Missing files during initial upload
[ https://issues.apache.org/jira/browse/HAWQ-3?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903261#comment-14903261 ] Roman Shaposhnik commented on HAWQ-3: - [~schubert] I totally agree with you. The kind of refactoring you're talking about is absolutely a must if you ask me. It will be, however a different JIRA. This one just tracks migration of the codebase we promised in the SGA into the ASF Git repo. > Missing files during initial upload > --- > > Key: HAWQ-3 > URL: https://issues.apache.org/jira/browse/HAWQ-3 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Radar Lei > > As my check, there are about 60 files didn't been check in to apache hawq > repo. That's should because they are in '.gitignore' file, so they got > ignored while the initial upload. > We should have apache hawq git repo get all the files in the first time, even > they are in '.gitignore'. After that, they can be ignored. > The most important missing ones which will cause 'make install' fails: > tools/bin: ext > tools/bin/pythonSrc: PyGreSQL-4.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-3) Missing files during initial upload
[ https://issues.apache.org/jira/browse/HAWQ-3?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903253#comment-14903253 ] Roman Shaposhnik commented on HAWQ-3: - This should be fixed now. Please try and let me know whether this JIRA can be closed. > Missing files during initial upload > --- > > Key: HAWQ-3 > URL: https://issues.apache.org/jira/browse/HAWQ-3 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Radar Lei > > As my check, there are about 60 files didn't been check in to apache hawq > repo. That's should because they are in '.gitignore' file, so they got > ignored while the initial upload. > We should have apache hawq git repo get all the files in the first time, even > they are in '.gitignore'. After that, they can be ignored. > The most important missing ones which will cause 'make install' fails: > tools/bin: ext > tools/bin/pythonSrc: PyGreSQL-4.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-3) Missing files during initial upload
[ https://issues.apache.org/jira/browse/HAWQ-3?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903156#comment-14903156 ] Roman Shaposhnik commented on HAWQ-3: - No wonder I couldn't build it on ASF Jenkins, but had no problem building it on my laptop! ;-) I'll look into this ASAP. > Missing files during initial upload > --- > > Key: HAWQ-3 > URL: https://issues.apache.org/jira/browse/HAWQ-3 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Radar Lei > > As my check, there are about 60 files didn't been check in to apache hawq > repo. That's should because they are in '.gitignore' file, so they got > ignored while the initial upload. > We should have apache hawq git repo get all the files in the first time, even > they are in '.gitignore'. After that, they can be ignored. > The most important missing ones which will cause 'make install' fails: > tools/bin: ext > tools/bin/pythonSrc: PyGreSQL-4.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-2) Build failed due to file missing during initial upload
[ https://issues.apache.org/jira/browse/HAWQ-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903149#comment-14903149 ] Roman Shaposhnik commented on HAWQ-2: - This seems like a duplicated of HAWQ-3. [~lei_chang] can you please close this one? > Build failed due to file missing during initial upload > -- > > Key: HAWQ-2 > URL: https://issues.apache.org/jira/browse/HAWQ-2 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Lei Chang > > Detail info on link: https://hub.docker.com/r/mayjojo/hawq-dev/ > Errors reported when doing "make install": > cd /hawq/tools/bin/pythonSrc/PyGreSQL-4.0 && CC="gcc" > CFLAGS="-I/opt/hawq-dep/include -I/opt/hawq-dep/python-2.6.2/include > -std=gnu99 -Wall -Wmissing-prototypes -Wpointer-arith -Wendif-labels > -Wformat-security -fno-strict-aliasing -fwrapv -I/opt/hawq-dep/include" > LDFLAGS="-L../../src/interfaces/libpq -L../../src/port -L../../src/port > -L/opt/hawq-dep/lib -L/opt/hawq-dep/lib64 -L/opt/hawq-dep/python-2.6.2/lib > -Wl,--as-needed -Wl,-rpath,'/opt/hawq/lib',--enable-new-dtags" python > setup.py build > /bin/sh: line 0: cd: /hawq/tools/bin/pythonSrc/PyGreSQL-4.0: No such file or > directory -- This message was sent by Atlassian JIRA (v6.3.4#6332)
