[GitHub] incubator-hawq pull request #900: HAWQ-1033. Combine update and insert into ...
GitHub user xunzhang opened a pull request: https://github.com/apache/incubator-hawq/pull/900 HAWQ-1033. Combine update and insert into one transaction, fix hadoop warning. You can merge this pull request into a Git repository by running: $ git pull https://github.com/xunzhang/incubator-hawq register Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/900.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #900 commit 682aa64190925df2be52a2d0881bd9e3bd3c2cf4 Author: xunzhang Date: 2016-09-13T06:41:45Z HAWQ-1033. Combine update and insert into one transaction, fix hadoop warning. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #900: HAWQ-1033. Combine update and insert into one tra...
Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/900 also cc @wcl14 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #900: HAWQ-1033. Combine update and insert into one tra...
Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/900 cc @ictmalili @zhangh43 @linwen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #837: HAWQ-779 support pxf filter pushdwon at th...
Github user hornn commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/837#discussion_r78508860
--- Diff: src/backend/optimizer/plan/createplan.c ---
@@ -1144,9 +1144,15 @@ static char** create_pxf_plan(char **segdb_file_map,
RelOptInfo *rel, int total_
Relation relation = RelationIdGetRelation(planner_rt_fetch(scan_relid,
ctx->root)->relid);
- segdb_work_map = map_hddata_2gp_segments(uri_str,
+ if (pxf_enable_filter_pushdown){
--- End diff --
can be refactored:
```
map_hddata_2gp_segments(uri_str,
total_segs,
segs_participating,
relation,
pxf_enable_filter_pushdown ?
ctx->root->parse->jointree->quals : NULL);
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Updated] (HAWQ-1033) add --force option for hawq register
[ https://issues.apache.org/jira/browse/HAWQ-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu updated HAWQ-1033: - Affects Version/s: 2.0.1.0-incubating > add --force option for hawq register > > > Key: HAWQ-1033 > URL: https://issues.apache.org/jira/browse/HAWQ-1033 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Command Line Tools >Affects Versions: 2.0.1.0-incubating >Reporter: Lili Ma >Assignee: hongwu > Fix For: 2.0.1.0-incubating > > > add --force option for hawq register > Will clear all the catalog contents in pg_aoseg.pg_paqseg_$relid while keep > the files on HDFS, and then re-register all the files to the table. This is > for scenario cluster Disaster Recovery: Two clusters co-exist, periodically > import data from Cluster A to Cluster B. Need Register data to Cluster B. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HAWQ-1033) add --force option for hawq register
[ https://issues.apache.org/jira/browse/HAWQ-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu resolved HAWQ-1033. -- Resolution: Incomplete > add --force option for hawq register > > > Key: HAWQ-1033 > URL: https://issues.apache.org/jira/browse/HAWQ-1033 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Command Line Tools >Affects Versions: 2.0.1.0-incubating >Reporter: Lili Ma >Assignee: hongwu > Fix For: 2.0.1.0-incubating > > > add --force option for hawq register > Will clear all the catalog contents in pg_aoseg.pg_paqseg_$relid while keep > the files on HDFS, and then re-register all the files to the table. This is > for scenario cluster Disaster Recovery: Two clusters co-exist, periodically > import data from Cluster A to Cluster B. Need Register data to Cluster B. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #900: HAWQ-1033. Combine update and insert into one tra...
Github user ictmalili commented on the issue: https://github.com/apache/incubator-hawq/pull/900 LGTM. +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (HAWQ-1034) add --repair option for hawq register
[ https://issues.apache.org/jira/browse/HAWQ-1034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu updated HAWQ-1034: - Affects Version/s: 2.0.1.0-incubating > add --repair option for hawq register > - > > Key: HAWQ-1034 > URL: https://issues.apache.org/jira/browse/HAWQ-1034 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Command Line Tools >Affects Versions: 2.0.1.0-incubating >Reporter: Lili Ma >Assignee: hongwu > Fix For: 2.0.1.0-incubating > > > add --repair option for hawq register > Will change both file folder and catalog table pg_aoseg.pg_paqseg_$relid to > the state which .yml file configures. Note may some new generated files since > the checkpoint may be deleted here. Also note the all the files in .yml file > should all under the table folder on HDFS. Limitation: Do not support cases > for hash table redistribution, table truncate and table drop. This is for > scenario rollback of table: Do checkpoints somewhere, and need to rollback to > previous checkpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #900: HAWQ-1033. Combine update and insert into ...
Github user xunzhang commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/900#discussion_r78569975 --- Diff: tools/bin/hawqregister --- @@ -460,12 +460,13 @@ if __name__ == '__main__': do_not_move, files_update, sizes_update = True, files, sizes files, sizes = [], [] else: -for k, f in enumerate(files): +files_old, sizes_old = [f for f in files], [sz for sz in sizes] --- End diff -- Notice here, alternative modification like `files_old, sizes_old = files, sizes` is wrong since python default copy is not a deep copy, it is just a pointer copy. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #900: HAWQ-1033. Combine update and insert into ...
Github user xunzhang commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/900#discussion_r78570237 --- Diff: tools/bin/hawqregister --- @@ -460,12 +460,13 @@ if __name__ == '__main__': do_not_move, files_update, sizes_update = True, files, sizes files, sizes = [], [] else: -for k, f in enumerate(files): --- End diff -- The old code here is wrong: it is risky to iterate a changing object. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (HAWQ-1046) Document migration of LibHDFS3 library to HAWQ
Matthew Rocklin created HAWQ-1046: - Summary: Document migration of LibHDFS3 library to HAWQ Key: HAWQ-1046 URL: https://issues.apache.org/jira/browse/HAWQ-1046 Project: Apache HAWQ Issue Type: Wish Components: libhdfs Reporter: Matthew Rocklin Assignee: Lei Chang Some people used to depend on the libhdfs3 library maintained alongside HAWQ. This library was merged into the HAWQ codebase, making the situation a bit more confusing. Is independent use of libhdfs3 still supported by this community? If so what is the best way for packagers to reason about versions and releases of this component? It would be convenient to see documentation on how people can best depend on libhdfs3 separately from HAWQ if this is an intention. It looks like people have actually submitted work to the old version See: https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/pull/28 It looks like the warning that the library had moved has been removed: https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/commit/ddcb2404a5a67e0f39fe49ed20591545c48ff426 This removal may lead to some frustration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1047) Push limit clause to PXF
Goden Yao created HAWQ-1047: --- Summary: Push limit clause to PXF Key: HAWQ-1047 URL: https://issues.apache.org/jira/browse/HAWQ-1047 Project: Apache HAWQ Issue Type: Improvement Components: PXF Reporter: Goden Yao Assignee: Goden Yao Fix For: backlog When user launches an external table query with "limit" clause, Hawq explicitly closes the remote connection when it has retrieved enough tuples and this raises an exception on the tomcat end. In those such queries Hawq doesn't pushdown the limit clause to Pxf so it's up to Hawq to know when it has enough tuples and end the request. **expected behavior** 1. HAWQ should push down the limit clause so PXF doesn't need to return more than the limit number of records. 2. PXF needs handle hawq close connection gracefully. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-964) Support for additional logical operators in PXF
[ https://issues.apache.org/jira/browse/HAWQ-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivram Mani updated HAWQ-964: -- Description: Currently the extension framework only allows the 'AND' logical operator across the provided predicates. Will need to support other logical operators such as 'OR' 'NOT' Have a separate enum for logical operators say PxfLogicalOperatorCode and include AND,OR,NOT in it. Other logical operators can continue existing in PxfOperatorCode enum Change the way Qualifier tree is handled to take care of precedence. in pxffilters.c. The output of this should produce a postfix notation. was: Currently the extension framework only allows the 'AND' logical operator across the provided predicates. Will need to support other logical operators such as 'OR' 'NOT' > Support for additional logical operators in PXF > --- > > Key: HAWQ-964 > URL: https://issues.apache.org/jira/browse/HAWQ-964 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Shivram Mani >Assignee: Goden Yao > Fix For: backlog > > > Currently the extension framework only allows the 'AND' logical operator > across the provided predicates. Will need to support other logical operators > such as 'OR' 'NOT' > Have a separate enum for logical operators say PxfLogicalOperatorCode and > include AND,OR,NOT in it. > Other logical operators can continue existing in PxfOperatorCode enum > Change the way Qualifier tree is handled to take care of precedence. in > pxffilters.c. > The output of this should produce a postfix notation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-963) Enhance PXF to support additional operators
[ https://issues.apache.org/jira/browse/HAWQ-963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivram Mani updated HAWQ-963: -- Description: Supported operations in PXF only include <, >, <=, >=, =, !=. Will need to add support for more operators in the PXF framework between(), in(), isNull() Add logical operator codes in PXF_LOGICAL_OPERATOR_CODE and handle these operators when the FilterString is being parsed on the pxf service side. was: Supported operations in PXF only include <, >, <=, >=, =, !=. Will need to add support for more operators in the PXF framework between(), in(), isNull(), etc > Enhance PXF to support additional operators > --- > > Key: HAWQ-963 > URL: https://issues.apache.org/jira/browse/HAWQ-963 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Shivram Mani >Assignee: Goden Yao > Fix For: backlog > > > Supported operations in PXF only include > <, >, <=, >=, =, !=. > Will need to add support for more operators in the PXF framework > between(), in(), isNull() > Add logical operator codes in PXF_LOGICAL_OPERATOR_CODE and handle these > operators when the FilterString is being parsed on the pxf service side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-964) Support for additional logical operators in PXF
[ https://issues.apache.org/jira/browse/HAWQ-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivram Mani updated HAWQ-964: -- Description: Currently the extension framework only allows the 'AND' logical operator across the provided predicates. Will need to support other logical operators such as 'OR' 'NOT' Logical operators (such as AND, OR, NOT) will be denoted by the letter 'l' Comparison operators (such as <, >, = etc) will be denoted by 'o' Constants by 'c' Variables (column index) by 'a' Have a separate enum for logical operators say PxfLogicalOperatorCode and include AND,OR,NOT in it. Other logical operators can continue existing in PxfOperatorCode enum. Change the way Qualifier tree is handled to take care of precedence. in pxffilters.c. The output of this should produce a postfix notation. was: Currently the extension framework only allows the 'AND' logical operator across the provided predicates. Will need to support other logical operators such as 'OR' 'NOT' Have a separate enum for logical operators say PxfLogicalOperatorCode and include AND,OR,NOT in it. Other logical operators can continue existing in PxfOperatorCode enum Change the way Qualifier tree is handled to take care of precedence. in pxffilters.c. The output of this should produce a postfix notation. > Support for additional logical operators in PXF > --- > > Key: HAWQ-964 > URL: https://issues.apache.org/jira/browse/HAWQ-964 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Shivram Mani >Assignee: Goden Yao > Fix For: backlog > > > Currently the extension framework only allows the 'AND' logical operator > across the provided predicates. Will need to support other logical operators > such as 'OR' 'NOT' > Logical operators (such as AND, OR, NOT) will be denoted by the letter 'l' > Comparison operators (such as <, >, = etc) will be denoted by 'o' > Constants by 'c' > Variables (column index) by 'a' > Have a separate enum for logical operators say PxfLogicalOperatorCode and > include AND,OR,NOT in it. > Other logical operators can continue existing in PxfOperatorCode enum. > Change the way Qualifier tree is handled to take care of precedence. in > pxffilters.c. > The output of this should produce a postfix notation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1048) Support OR, NOT logical operators in the HAWQ/PXF Bridge
Shivram Mani created HAWQ-1048: -- Summary: Support OR, NOT logical operators in the HAWQ/PXF Bridge Key: HAWQ-1048 URL: https://issues.apache.org/jira/browse/HAWQ-1048 Project: Apache HAWQ Issue Type: Sub-task Components: PXF Reporter: Shivram Mani Assignee: Goden Yao -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-964) Support for additional logical operators in PXF
[ https://issues.apache.org/jira/browse/HAWQ-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivram Mani reassigned HAWQ-964: - Assignee: Shivram Mani (was: Goden Yao) > Support for additional logical operators in PXF > --- > > Key: HAWQ-964 > URL: https://issues.apache.org/jira/browse/HAWQ-964 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Shivram Mani >Assignee: Shivram Mani > Fix For: backlog > > > Currently the extension framework only allows the 'AND' logical operator > across the provided predicates. Will need to support other logical operators > such as 'OR' 'NOT' > Logical operators (such as AND, OR, NOT) will be denoted by the letter 'l' > Comparison operators (such as <, >, = etc) will be denoted by 'o' > Constants by 'c' > Variables (column index) by 'a' > Have a separate enum for logical operators say PxfLogicalOperatorCode and > include AND,OR,NOT in it. > Other logical operators can continue existing in PxfOperatorCode enum. > Change the way Qualifier tree is handled to take care of precedence. in > pxffilters.c. > The output of this should produce a postfix notation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1049) Enhance PXF Service to support AND,OR,NOT logical operators in Predicate Push
Shivram Mani created HAWQ-1049: -- Summary: Enhance PXF Service to support AND,OR,NOT logical operators in Predicate Push Key: HAWQ-1049 URL: https://issues.apache.org/jira/browse/HAWQ-1049 Project: Apache HAWQ Issue Type: Sub-task Components: PXF Reporter: Shivram Mani Assignee: Goden Yao -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1049) Enhance PXF Service to support AND,OR,NOT logical operators in Predicate Push
[ https://issues.apache.org/jira/browse/HAWQ-1049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivram Mani updated HAWQ-1049: --- Assignee: Kavinder Dhaliwal (was: Goden Yao) > Enhance PXF Service to support AND,OR,NOT logical operators in Predicate Push > - > > Key: HAWQ-1049 > URL: https://issues.apache.org/jira/browse/HAWQ-1049 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Shivram Mani >Assignee: Kavinder Dhaliwal > Fix For: backlog > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1048) Support OR, NOT logical operators in the HAWQ/PXF Bridge
[ https://issues.apache.org/jira/browse/HAWQ-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivram Mani updated HAWQ-1048: --- Assignee: Oleksandr Diachenko (was: Goden Yao) > Support OR, NOT logical operators in the HAWQ/PXF Bridge > > > Key: HAWQ-1048 > URL: https://issues.apache.org/jira/browse/HAWQ-1048 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Shivram Mani >Assignee: Oleksandr Diachenko > Fix For: backlog > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1048) Support OR, NOT logical operators in the HAWQ/PXF Bridge
[ https://issues.apache.org/jira/browse/HAWQ-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivram Mani updated HAWQ-1048: --- Description: Support additional logical operators OR, NOT along with currently supported AND. Have a separate enum for logical operators say PxfLogicalOperatorCode. Other logical operators can continue existing in PxfOperatorCode enum. Change the way Qualifier tree is handled to take care of precedence. in pxffilters.c. The output of this should produce a postfix notation. > Support OR, NOT logical operators in the HAWQ/PXF Bridge > > > Key: HAWQ-1048 > URL: https://issues.apache.org/jira/browse/HAWQ-1048 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Shivram Mani >Assignee: Oleksandr Diachenko > Fix For: backlog > > > Support additional logical operators OR, NOT along with currently supported > AND. > Have a separate enum for logical operators say PxfLogicalOperatorCode. Other > logical operators can continue existing in PxfOperatorCode enum. > Change the way Qualifier tree is handled to take care of precedence. in > pxffilters.c. The output of this should produce a postfix notation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1049) Enhance PXF Service to support AND,OR,NOT logical operators in Predicate Push
[ https://issues.apache.org/jira/browse/HAWQ-1049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivram Mani updated HAWQ-1049: --- Description: Support additional logical operators OR, NOT along with currently supported AND. Update the PXF ORC Accessor to support these opearators as well. > Enhance PXF Service to support AND,OR,NOT logical operators in Predicate Push > - > > Key: HAWQ-1049 > URL: https://issues.apache.org/jira/browse/HAWQ-1049 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Shivram Mani >Assignee: Kavinder Dhaliwal > Fix For: backlog > > > Support additional logical operators OR, NOT along with currently supported > AND. > Update the PXF ORC Accessor to support these opearators as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-964) Support for additional logical operators in PXF
[ https://issues.apache.org/jira/browse/HAWQ-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivram Mani updated HAWQ-964: -- Description: Currently the extension framework only allows the 'AND' logical operator across the provided predicates. Will need to support other logical operators such as 'OR' 'NOT' Logical operators (such as AND, OR, NOT) will be denoted by the letter 'l' Comparison operators (such as <, >, = etc) will be denoted by 'o' Constants by 'c' Variables (column index) by 'a' was: Currently the extension framework only allows the 'AND' logical operator across the provided predicates. Will need to support other logical operators such as 'OR' 'NOT' Logical operators (such as AND, OR, NOT) will be denoted by the letter 'l' Comparison operators (such as <, >, = etc) will be denoted by 'o' Constants by 'c' Variables (column index) by 'a' Have a separate enum for logical operators say PxfLogicalOperatorCode and include AND,OR,NOT in it. Other logical operators can continue existing in PxfOperatorCode enum. Change the way Qualifier tree is handled to take care of precedence. in pxffilters.c. The output of this should produce a postfix notation. > Support for additional logical operators in PXF > --- > > Key: HAWQ-964 > URL: https://issues.apache.org/jira/browse/HAWQ-964 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Shivram Mani >Assignee: Shivram Mani > Fix For: backlog > > > Currently the extension framework only allows the 'AND' logical operator > across the provided predicates. Will need to support other logical operators > such as 'OR' 'NOT' > Logical operators (such as AND, OR, NOT) will be denoted by the letter 'l' > Comparison operators (such as <, >, = etc) will be denoted by 'o' > Constants by 'c' > Variables (column index) by 'a' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1049) Enhance PXF Service to support AND,OR,NOT logical operators in Predicate Pushdown
[ https://issues.apache.org/jira/browse/HAWQ-1049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Goden Yao updated HAWQ-1049: Summary: Enhance PXF Service to support AND,OR,NOT logical operators in Predicate Pushdown (was: Enhance PXF Service to support AND,OR,NOT logical operators in Predicate Push) > Enhance PXF Service to support AND,OR,NOT logical operators in Predicate > Pushdown > - > > Key: HAWQ-1049 > URL: https://issues.apache.org/jira/browse/HAWQ-1049 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Shivram Mani >Assignee: Kavinder Dhaliwal > Fix For: backlog > > > Support additional logical operators OR, NOT along with currently supported > AND. > Update the PXF ORC Accessor to support these opearators as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #896: [#126526845] Port recent changes from gpdb for ha...
Github user vraghavan78 commented on the issue: https://github.com/apache/incubator-hawq/pull/896 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #896: [#126526845] Port recent changes from gpdb for ha...
Github user paul-guo- commented on the issue: https://github.com/apache/incubator-hawq/pull/896 I'd suggest moving the test cases to src/test/feature (googletest framework) in this change or in other changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #900: HAWQ-1033. Combine update and insert into ...
Github user paul-guo- commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/900#discussion_r78670438
--- Diff: tools/bin/hawqregister ---
@@ -373,8 +402,8 @@ if __name__ == '__main__':
if len(args) != 1 or ((options.yml_config or options.force or
options.repair) and options.filepath) or (options.force and options.repair):
parser.print_help(sys.stderr)
sys.exit(1)
-if local_ssh('hadoop', logger):
-logger.error('command "hadoop" is not available.')
+if local_ssh('hdfs', logger):
+logger.error('command "hdfs" is not available.')
--- End diff --
Maybe you could be more accurate (e.g. "Maybe the hdfs path is not in the
shell environment variable PATH" or similar).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #899: HAWQ-1017. Add feature test for goh_create_type_c...
Github user ictmalili commented on the issue: https://github.com/apache/incubator-hawq/pull/899 LGTM. +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #899: HAWQ-1017. Add feature test for goh_create_type_c...
Github user paul-guo- commented on the issue: https://github.com/apache/incubator-hawq/pull/899 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #899: HAWQ-1017. Add feature test for goh_create...
Github user wengyanqing closed the pull request at: https://github.com/apache/incubator-hawq/pull/899 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #900: HAWQ-1033. Combine update and insert into one tra...
Github user wcl14 commented on the issue: https://github.com/apache/incubator-hawq/pull/900 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (HAWQ-1050) hawq register help can not return correct result indicating the help information
Lili Ma created HAWQ-1050:
-
Summary: hawq register help can not return correct result
indicating the help information
Key: HAWQ-1050
URL: https://issues.apache.org/jira/browse/HAWQ-1050
Project: Apache HAWQ
Issue Type: Bug
Reporter: Lili Ma
Assignee: Lei Chang
hawq register help can not return correct result indicating the help
information.
should keep help as a keyword and return same results as hawq register --help.
{code}
malilis-MacBook-Pro:~ malili$ hawq register help
20160914:09:56:37:007364 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Usage:
hadoop [--config confdir] COMMAND
where COMMAND is one of:
fs run a generic filesystem user client
version print the version
jar run a jar file
checknative [-a|-h] check native hadoop and compression libraries
availability
distcp copy file or directories recursively
archive -archiveName NAME -p * create a hadoop
archive
classpathprints the class path needed to get the
credential interact with credential providers
Hadoop jar and the required libraries
daemonlogget/set the log level for each daemon
traceview and modify Hadoop tracing settings
or
CLASSNAMErun the class named CLASSNAME
Most commands print help when invoked w/o parameters.
Traceback (most recent call last):
File "/usr/local/hawq/bin/hawqregister", line 398, in
check_hash_type(dburl, tablename) # Usage1 only support randomly
distributed table
File "/usr/local/hawq/bin/hawqregister", line 197, in check_hash_type
logger.error('Table not found in table gp_distribution_policy.' % tablename)
TypeError: not all arguments converted during string formatting
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HAWQ-1050) hawq register help can not return correct result indicating the help information
[
https://issues.apache.org/jira/browse/HAWQ-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lili Ma updated HAWQ-1050:
--
Issue Type: Sub-task (was: Bug)
Parent: HAWQ-991
> hawq register help can not return correct result indicating the help
> information
>
>
> Key: HAWQ-1050
> URL: https://issues.apache.org/jira/browse/HAWQ-1050
> Project: Apache HAWQ
> Issue Type: Sub-task
>Reporter: Lili Ma
>Assignee: Lei Chang
>
> hawq register help can not return correct result indicating the help
> information.
> should keep help as a keyword and return same results as hawq register --help.
> {code}
> malilis-MacBook-Pro:~ malili$ hawq register help
> 20160914:09:56:37:007364
> hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Usage: hadoop [--config
> confdir] COMMAND
>where COMMAND is one of:
> fs run a generic filesystem user client
> version print the version
> jar run a jar file
> checknative [-a|-h] check native hadoop and compression libraries
> availability
> distcp copy file or directories recursively
> archive -archiveName NAME -p * create a hadoop
> archive
> classpathprints the class path needed to get the
> credential interact with credential providers
>Hadoop jar and the required libraries
> daemonlogget/set the log level for each daemon
> traceview and modify Hadoop tracing settings
> or
> CLASSNAMErun the class named CLASSNAME
> Most commands print help when invoked w/o parameters.
> Traceback (most recent call last):
> File "/usr/local/hawq/bin/hawqregister", line 398, in
> check_hash_type(dburl, tablename) # Usage1 only support randomly
> distributed table
> File "/usr/local/hawq/bin/hawqregister", line 197, in check_hash_type
> logger.error('Table not found in table gp_distribution_policy.' %
> tablename)
> TypeError: not all arguments converted during string formatting
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] incubator-hawq issue #900: HAWQ-1033. Combine update and insert into one tra...
Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/900 Merged into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #900: HAWQ-1033. Combine update and insert into ...
Github user xunzhang closed the pull request at: https://github.com/apache/incubator-hawq/pull/900 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (HAWQ-1051) failing in reverse DNS lookup causes resource manager core dump
Yi Jin created HAWQ-1051: Summary: failing in reverse DNS lookup causes resource manager core dump Key: HAWQ-1051 URL: https://issues.apache.org/jira/browse/HAWQ-1051 Project: Apache HAWQ Issue Type: Bug Components: Resource Manager Reporter: Yi Jin Assignee: Lei Chang Fix For: 2.0.1.0-incubating This is caused due to wrongly expecting gethostbyaddr() return values. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-1051) failing in reverse DNS lookup causes resource manager core dump
[ https://issues.apache.org/jira/browse/HAWQ-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Jin reassigned HAWQ-1051: Assignee: Yi Jin (was: Lei Chang) > failing in reverse DNS lookup causes resource manager core dump > --- > > Key: HAWQ-1051 > URL: https://issues.apache.org/jira/browse/HAWQ-1051 > Project: Apache HAWQ > Issue Type: Bug > Components: Resource Manager >Reporter: Yi Jin >Assignee: Yi Jin > Fix For: 2.0.1.0-incubating > > > This is caused due to wrongly expecting gethostbyaddr() return values. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-1046) Document migration of LibHDFS3 library to HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Espino reassigned HAWQ-1046: --- Assignee: Ed Espino (was: Lei Chang) > Document migration of LibHDFS3 library to HAWQ > -- > > Key: HAWQ-1046 > URL: https://issues.apache.org/jira/browse/HAWQ-1046 > Project: Apache HAWQ > Issue Type: Wish > Components: libhdfs >Reporter: Matthew Rocklin >Assignee: Ed Espino > > Some people used to depend on the libhdfs3 library maintained alongside HAWQ. > This library was merged into the HAWQ codebase, making the situation a bit > more confusing. > Is independent use of libhdfs3 still supported by this community? If so what > is the best way for packagers to reason about versions and releases of this > component? It would be convenient to see documentation on how people can > best depend on libhdfs3 separately from HAWQ if this is an intention. > It looks like people have actually submitted work to the old version > See: https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/pull/28 > It looks like the warning that the library had moved has been removed: > https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/commit/ddcb2404a5a67e0f39fe49ed20591545c48ff426 > This removal may lead to some frustration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1046) Document migration of LibHDFS3 library to HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Espino updated HAWQ-1046: Assignee: Ruilong Huo (was: Ed Espino) > Document migration of LibHDFS3 library to HAWQ > -- > > Key: HAWQ-1046 > URL: https://issues.apache.org/jira/browse/HAWQ-1046 > Project: Apache HAWQ > Issue Type: Wish > Components: libhdfs >Reporter: Matthew Rocklin >Assignee: Ruilong Huo > > Some people used to depend on the libhdfs3 library maintained alongside HAWQ. > This library was merged into the HAWQ codebase, making the situation a bit > more confusing. > Is independent use of libhdfs3 still supported by this community? If so what > is the best way for packagers to reason about versions and releases of this > component? It would be convenient to see documentation on how people can > best depend on libhdfs3 separately from HAWQ if this is an intention. > It looks like people have actually submitted work to the old version > See: https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/pull/28 > It looks like the warning that the library had moved has been removed: > https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/commit/ddcb2404a5a67e0f39fe49ed20591545c48ff426 > This removal may lead to some frustration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1046) Document migration of LibHDFS3 library to HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489170#comment-15489170 ] Ed Espino commented on HAWQ-1046: - Ruilong and [~espino] are discussing the history and future of this project. We will provide an update soon. > Document migration of LibHDFS3 library to HAWQ > -- > > Key: HAWQ-1046 > URL: https://issues.apache.org/jira/browse/HAWQ-1046 > Project: Apache HAWQ > Issue Type: Wish > Components: libhdfs >Reporter: Matthew Rocklin >Assignee: Ruilong Huo > > Some people used to depend on the libhdfs3 library maintained alongside HAWQ. > This library was merged into the HAWQ codebase, making the situation a bit > more confusing. > Is independent use of libhdfs3 still supported by this community? If so what > is the best way for packagers to reason about versions and releases of this > component? It would be convenient to see documentation on how people can > best depend on libhdfs3 separately from HAWQ if this is an intention. > It looks like people have actually submitted work to the old version > See: https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/pull/28 > It looks like the warning that the library had moved has been removed: > https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/commit/ddcb2404a5a67e0f39fe49ed20591545c48ff426 > This removal may lead to some frustration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #807: HAWQ-897. Add feature test for create tabl...
Github user jiny2 closed the pull request at: https://github.com/apache/incubator-hawq/pull/807 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (HAWQ-1046) Document migration of LibHDFS3 library to HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Espino updated HAWQ-1046: Assignee: hongwu (was: Ruilong Huo) > Document migration of LibHDFS3 library to HAWQ > -- > > Key: HAWQ-1046 > URL: https://issues.apache.org/jira/browse/HAWQ-1046 > Project: Apache HAWQ > Issue Type: Wish > Components: libhdfs >Reporter: Matthew Rocklin >Assignee: hongwu > > Some people used to depend on the libhdfs3 library maintained alongside HAWQ. > This library was merged into the HAWQ codebase, making the situation a bit > more confusing. > Is independent use of libhdfs3 still supported by this community? If so what > is the best way for packagers to reason about versions and releases of this > component? It would be convenient to see documentation on how people can > best depend on libhdfs3 separately from HAWQ if this is an intention. > It looks like people have actually submitted work to the old version > See: https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/pull/28 > It looks like the warning that the library had moved has been removed: > https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/commit/ddcb2404a5a67e0f39fe49ed20591545c48ff426 > This removal may lead to some frustration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #900: HAWQ-1033. Combine update and insert into ...
Github user xunzhang commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/900#discussion_r78686806
--- Diff: tools/bin/hawqregister ---
@@ -373,8 +402,8 @@ if __name__ == '__main__':
if len(args) != 1 or ((options.yml_config or options.force or
options.repair) and options.filepath) or (options.force and options.repair):
parser.print_help(sys.stderr)
sys.exit(1)
-if local_ssh('hadoop', logger):
-logger.error('command "hadoop" is not available.')
+if local_ssh('hdfs', logger):
+logger.error('command "hdfs" is not available.')
--- End diff --
We do not try to predict the problem in user side,
here I only want to return to users the reason why register can not work.
Thanks anyway for the volunteer review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (HAWQ-1046) Document migration of LibHDFS3 library to HAWQ
[ https://issues.apache.org/jira/browse/HAWQ-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489516#comment-15489516 ] hongwu commented on HAWQ-1046: -- I will take care of this issue. I have recommended [~wangzw] to create another repo in github for the consideration of current libhdfs3 users from community. Then we should update README.md to give the link of the new repo. After several days(maybe 10-20days?), we will make the repo under Pivotal-Data-Attic private and old users may refer to the new link for development which has no relationship with Pivotal. For the future development of libhdfs3, I think we'd better develop in Pivotal-Data-Attic/pivotalrd-libhdfs3 as upstream since it is a dependent library(not only HAWQ can use it, but also other projects, we plan to contribute it to apache in the future). For HAWQ, we periodically sync with upstream repo's releases or commits. Make sense? > Document migration of LibHDFS3 library to HAWQ > -- > > Key: HAWQ-1046 > URL: https://issues.apache.org/jira/browse/HAWQ-1046 > Project: Apache HAWQ > Issue Type: Wish > Components: libhdfs >Reporter: Matthew Rocklin >Assignee: hongwu > > Some people used to depend on the libhdfs3 library maintained alongside HAWQ. > This library was merged into the HAWQ codebase, making the situation a bit > more confusing. > Is independent use of libhdfs3 still supported by this community? If so what > is the best way for packagers to reason about versions and releases of this > component? It would be convenient to see documentation on how people can > best depend on libhdfs3 separately from HAWQ if this is an intention. > It looks like people have actually submitted work to the old version > See: https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/pull/28 > It looks like the warning that the library had moved has been removed: > https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3/commit/ddcb2404a5a67e0f39fe49ed20591545c48ff426 > This removal may lead to some frustration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #901: HAWQ-1050. Support help without dash for r...
GitHub user xunzhang opened a pull request: https://github.com/apache/incubator-hawq/pull/901 HAWQ-1050. Support help without dash for register. You can merge this pull request into a Git repository by running: $ git pull https://github.com/xunzhang/incubator-hawq HAWQ-1050 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/901.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #901 commit cd81289bcbf646c415f97f3f3ee95e2304e8246b Author: xunzhang Date: 2016-09-14T06:35:44Z HAWQ-1050. Support help without dash for register. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #901: HAWQ-1050. Support help without dash for register...
Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/901 cc @ictmalili @radarwave --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #901: HAWQ-1050. Support help without dash for register...
Github user radarwave commented on the issue: https://github.com/apache/incubator-hawq/pull/901 LGTM +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
