[jira] [Updated] (HAWQ-1617) Incorrect processing of boolean operators in pushdown
[
https://issues.apache.org/jira/browse/HAWQ-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Leskin updated HAWQ-1617:
--
Description:
When accessing external tables, the pushdown feature sometimes works incorrect.
Consider the following query:
{{SELECT * FROM table_ex WHERE bool1=false AND id1=60003;}}
When this query is executed, we get an error "stack is not empty ...".
But what actually happens?
It turns out that such query is "translated" into a list of three items that
represents the constraints: 'BoolExpr', 'Var' and 'OpExpr' (this represents a
constraint "WHERE NOT(bool1 = true) AND (id1 = 60003)"). Note that the list
does not contain implicit AND operators.
Then, the list is processed by a piece of code in [pxffilters.c, where there is
a check of presence of
'BoolExpr's|https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/pxffilters.c#L1259].
If 'BoolExpr's are detected, no implicit AND operators are added.
In the case described, one 'BoolExpr' element is present, and no implicit AND
operators are added. This leads to the error mentioned above.
was:
When accessing external tables, the pushdown feature sometimes works incorrect.
Consider the following query:
{{SELECT * FROM table_ex WHERE bool1=false AND id1=60003;}}
When this query is executed, we get an error "stack is not empty ...".
But what actually happens?
It turns out that such query is "translated" into a list of three items that
represents the constraints: 'BoolExpr', 'Var' and 'OpExpr' (this represents a
constraint "WHERE NOT(bool1 = true) AND (id1 = 60003)"). Note that the list
does not contain implicit AND operators.
Then, the list is processed by a piece of code in [pxffilters.c, where there is
a check of presence of
'BoolExpr's|https://github.com/arenadata/incubator-hawq/blob/master/src/backend/access/external/pxffilters.c#L1259].
If 'BoolExpr's are detected, no implicit AND operators are added.
In the case described, one 'BoolExpr' element is present, and no implicit AND
operators are added. This leads to the error mentioned above.
> Incorrect processing of boolean operators in pushdown
> -
>
> Key: HAWQ-1617
> URL: https://issues.apache.org/jira/browse/HAWQ-1617
> Project: Apache HAWQ
> Issue Type: Bug
> Components: External Tables
>Reporter: Ivan Leskin
>Assignee: Radar Lei
>Priority: Major
>
> When accessing external tables, the pushdown feature sometimes works
> incorrect. Consider the following query:
> {{SELECT * FROM table_ex WHERE bool1=false AND id1=60003;}}
> When this query is executed, we get an error "stack is not empty ...".
> But what actually happens?
> It turns out that such query is "translated" into a list of three items that
> represents the constraints: 'BoolExpr', 'Var' and 'OpExpr' (this represents a
> constraint "WHERE NOT(bool1 = true) AND (id1 = 60003)"). Note that the list
> does not contain implicit AND operators.
> Then, the list is processed by a piece of code in [pxffilters.c, where there
> is a check of presence of
> 'BoolExpr's|https://github.com/apache/incubator-hawq/blob/master/src/backend/access/external/pxffilters.c#L1259].
> If 'BoolExpr's are detected, no implicit AND operators are added.
> In the case described, one 'BoolExpr' element is present, and no implicit AND
> operators are added. This leads to the error mentioned above.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HAWQ-1617) Incorrect processing of boolean operators in pushdown
[
https://issues.apache.org/jira/browse/HAWQ-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Leskin updated HAWQ-1617:
--
Description:
When accessing external tables, the pushdown feature sometimes works incorrect.
Consider the following query:
{{SELECT * FROM table_ex WHERE bool1=false AND id1=60003;}}
When this query is executed, we get an error "stack is not empty ...".
But what actually happens?
It turns out that such query is "translated" into a list of three items that
represents the constraints: 'BoolExpr', 'Var' and 'OpExpr' (this represents a
constraint "WHERE NOT(bool1 = true) AND (id1 = 60003)"). Note that the list
does not contain implicit AND operators.
Then, the list is processed by a piece of code in [pxffilters.c, where there is
a check of presence of
'BoolExpr's|[https://github.com/arenadata/incubator-hawq/blob/master/src/backend/access/external/pxffilters.c#L1259].]
If 'BoolExpr's are detected, no implicit AND operators are added.
In the case described, one 'BoolExpr' element is present, and no implicit AND
operators are added. This leads to the error mentioned above.
was:
When accessing external tables, the pushdown feature sometimes works incorrect.
Consider the following query:
{{SELECT * FROM table_ex WHERE bool1=false AND id1=60003;}}
When this query is executed, we get an error "stack is not empty ...".
But what actually happens?
It turns out that such query is "translated" into a list of three items, which
represents the constraints: 'BoolExpr', 'Var' and 'OpExpr' (this represents a
constraint "WHERE NOT(bool1 = true) AND (id1 = 60003)"). Note that the list
does not contain implicit AND operators.
Then, the list is processed by a piece of code in [pxffilters.c, where there is
a check of presence of
'BoolExpr's|[https://github.com/arenadata/incubator-hawq/blob/master/src/backend/access/external/pxffilters.c#L1259].]
If 'BoolExpr's are detected, no implicit AND operators are added.
In the case described, one 'BoolExpr' element is present, and no implicit AND
operators are added. This leads to the error mentioned above.
> Incorrect processing of boolean operators in pushdown
> -
>
> Key: HAWQ-1617
> URL: https://issues.apache.org/jira/browse/HAWQ-1617
> Project: Apache HAWQ
> Issue Type: Bug
> Components: External Tables
>Reporter: Ivan Leskin
>Assignee: Radar Lei
>Priority: Major
>
> When accessing external tables, the pushdown feature sometimes works
> incorrect. Consider the following query:
> {{SELECT * FROM table_ex WHERE bool1=false AND id1=60003;}}
> When this query is executed, we get an error "stack is not empty ...".
> But what actually happens?
> It turns out that such query is "translated" into a list of three items that
> represents the constraints: 'BoolExpr', 'Var' and 'OpExpr' (this represents a
> constraint "WHERE NOT(bool1 = true) AND (id1 = 60003)"). Note that the list
> does not contain implicit AND operators.
> Then, the list is processed by a piece of code in [pxffilters.c, where there
> is a check of presence of
> 'BoolExpr's|[https://github.com/arenadata/incubator-hawq/blob/master/src/backend/access/external/pxffilters.c#L1259].]
> If 'BoolExpr's are detected, no implicit AND operators are added.
> In the case described, one 'BoolExpr' element is present, and no implicit AND
> operators are added. This leads to the error mentioned above.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HAWQ-1617) Incorrect processing of boolean operators in pushdown
[
https://issues.apache.org/jira/browse/HAWQ-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Leskin updated HAWQ-1617:
--
Description:
When accessing external tables, the pushdown feature sometimes works incorrect.
Consider the following query:
{{SELECT * FROM table_ex WHERE bool1=false AND id1=60003;}}
When this query is executed, we get an error "stack is not empty ...".
But what actually happens?
It turns out that such query is "translated" into a list of three items, which
represents the constraints: 'BoolExpr', 'Var' and 'OpExpr' (this represents a
constraint "WHERE NOT(bool1 = true) AND (id1 = 60003)"). Note that the list
does not contain implicit AND operators.
Then, the list is processed by a piece of code in [pxffilters.c, where there is
a check of presence of
'BoolExpr's|[https://github.com/arenadata/incubator-hawq/blob/master/src/backend/access/external/pxffilters.c#L1259].]
If 'BoolExpr's are detected, no implicit AND operators are added.
In the case described, one 'BoolExpr' element is present, and no implicit AND
operators are added. This leads to the error mentioned above.
was:
When accessing external tables, the pushdown feature sometimes works incorrect.
Consider the following query:
{{SELECT * FROM table_ex WHERE bool1=false AND id1=60003;}}
When this query is executed, we get an error "stack is not empty ...".
What happens behind the curtains?
It turns out that such query is "translated" into a list of three items, which
represents the constraints: 'BoolExpr', 'Var' and 'OpExpr' (this represents a
constraint "WHERE NOT(bool1 = true) AND (id1 = 60003)"). Note that the list
does not contain implicit AND operators.
Then, the list is processed by a piece of code in [pxffilters.c, where there is
a check of presence of
'BoolExpr's|[https://github.com/arenadata/incubator-hawq/blob/master/src/backend/access/external/pxffilters.c#L1259].]
If 'BoolExpr's are detected, no implicit AND operators are added.
In the case described, one 'BoolExpr' element is present, and no implicit AND
operators are added. This leads to the error mentioned above.
> Incorrect processing of boolean operators in pushdown
> -
>
> Key: HAWQ-1617
> URL: https://issues.apache.org/jira/browse/HAWQ-1617
> Project: Apache HAWQ
> Issue Type: Bug
> Components: External Tables
>Reporter: Ivan Leskin
>Assignee: Radar Lei
>Priority: Major
>
> When accessing external tables, the pushdown feature sometimes works
> incorrect. Consider the following query:
> {{SELECT * FROM table_ex WHERE bool1=false AND id1=60003;}}
> When this query is executed, we get an error "stack is not empty ...".
> But what actually happens?
> It turns out that such query is "translated" into a list of three items,
> which represents the constraints: 'BoolExpr', 'Var' and 'OpExpr' (this
> represents a constraint "WHERE NOT(bool1 = true) AND (id1 = 60003)"). Note
> that the list does not contain implicit AND operators.
> Then, the list is processed by a piece of code in [pxffilters.c, where there
> is a check of presence of
> 'BoolExpr's|[https://github.com/arenadata/incubator-hawq/blob/master/src/backend/access/external/pxffilters.c#L1259].]
> If 'BoolExpr's are detected, no implicit AND operators are added.
> In the case described, one 'BoolExpr' element is present, and no implicit AND
> operators are added. This leads to the error mentioned above.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HAWQ-1617) Incorrect processing of boolean operators in pushdown
Ivan Leskin created HAWQ-1617:
-
Summary: Incorrect processing of boolean operators in pushdown
Key: HAWQ-1617
URL: https://issues.apache.org/jira/browse/HAWQ-1617
Project: Apache HAWQ
Issue Type: Bug
Components: External Tables
Reporter: Ivan Leskin
Assignee: Radar Lei
When accessing external tables, the pushdown feature sometimes works incorrect.
Consider the following query:
{{SELECT * FROM table_ex WHERE bool1=false AND id1=60003;}}
When this query is executed, we get an error "stack is not empty ...".
What happens behind the curtains?
It turns out that such query is "translated" into a list of three items, which
represents the constraints: 'BoolExpr', 'Var' and 'OpExpr' (this represents a
constraint "WHERE NOT(bool1 = true) AND (id1 = 60003)"). Note that the list
does not contain implicit AND operators.
Then, the list is processed by a piece of code in [pxffilters.c, where there is
a check of presence of
'BoolExpr's|[https://github.com/arenadata/incubator-hawq/blob/master/src/backend/access/external/pxffilters.c#L1259].]
If 'BoolExpr's are detected, no implicit AND operators are added.
In the case described, one 'BoolExpr' element is present, and no implicit AND
operators are added. This leads to the error mentioned above.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HAWQ-1605) Support INSERT in PXF JDBC plugin
[ https://issues.apache.org/jira/browse/HAWQ-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430237#comment-16430237 ] Ivan Leskin commented on HAWQ-1605: --- All the features mentioned have already been implemented. The code is available at github: [https://github.com/apache/incubator-hawq/pull/1353]. > Support INSERT in PXF JDBC plugin > - > > Key: HAWQ-1605 > URL: https://issues.apache.org/jira/browse/HAWQ-1605 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Ivan Leskin >Assignee: Ed Espino >Priority: Major > > Add support of INSERT queries in PXF JDBC plugin: > * Implement `WriteAccessor` and `WriteResolver` interfaces. Both are > implemented in the same classes as `ReadAccessor` and `ReadResolver`; > * Support query batching in Accessor. The size of a batch is defined by user > and may be "infinite"; > * In Accessor, use `java.sql.PreparedStatement` and built-in functions in it > to process queries; > * In `setFields()` method of Resolver, perform type conversions of the data > tuples received from PXF. > Optimize and refactor the code in PXF JDBC plugin, make some fixes: > * Make functions for building WHERE statements static where possible to > reduce the number of InputData checks; > * Organize imports; > * Check the codestyle; > * Fix the handling of TIMESTAMP values when performing SELECT requests. > Improve documentation: > * Correct or rewrite Javadoc strings for plugin functions; > * Rewrite README.md. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HAWQ-1605) Support INSERT in PXF JDBC plugin
Ivan Leskin created HAWQ-1605: - Summary: Support INSERT in PXF JDBC plugin Key: HAWQ-1605 URL: https://issues.apache.org/jira/browse/HAWQ-1605 Project: Apache HAWQ Issue Type: Improvement Components: PXF Reporter: Ivan Leskin Assignee: Ed Espino Add support of INSERT queries in PXF JDBC plugin: * Implement `WriteAccessor` and `WriteResolver` interfaces. Both are implemented in the same classes as `ReadAccessor` and `ReadResolver`; * Support query batching in Accessor. The size of a batch is defined by user and may be "infinite"; * In Accessor, use `java.sql.PreparedStatement` and built-in functions in it to process queries; * In `setFields()` method of Resolver, perform type conversions of the data tuples received from PXF. Optimize and refactor the code in PXF JDBC plugin, make some fixes: * Make functions for building WHERE statements static where possible to reduce the number of InputData checks; * Organize imports; * Check the codestyle; * Fix the handling of TIMESTAMP values when performing SELECT requests. Improve documentation: * Correct or rewrite Javadoc strings for plugin functions; * Rewrite README.md. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HAWQ-1599) PXF Ignite plugin
[ https://issues.apache.org/jira/browse/HAWQ-1599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Leskin updated HAWQ-1599: -- Priority: Trivial (was: Minor) > PXF Ignite plugin > - > > Key: HAWQ-1599 > URL: https://issues.apache.org/jira/browse/HAWQ-1599 > Project: Apache HAWQ > Issue Type: New Feature > Components: PXF >Reporter: Ivan Leskin >Assignee: Ed Espino >Priority: Trivial > > Add a new PXF plugin to provide read and wrtie access to [Ignite > database|https://ignite.apache.org/] via its native REST API. > Key features: > * Supports partitioning; > * A dedicated Ignite client with REST API enabled is used to access Ignite > database; > * Every SELECT or INSERT query splits tuples of data into groups, each of > which is sent from (to) Ignite in one response (query). The size of a group > is customizable; > * The implemented REST API can be modified (with relatively small changes) > and used to connect to any other database (or service) which supports SQL > queries via REST, and responds using JSON objects. > Plugin documentation is available at `pxf/pxf-ignite/README.md`. > h3. Dependencies > * [GSON|https://github.com/google/gson] is required to serialize and > deserialize JSON; > * [Apache commons compress > ByteUtils|https://commons.apache.org/proper/commons-compress/apidocs/org/apache/commons/compress/utils/ByteUtils.html] > is required to properly encode fragment metadata. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HAWQ-1599) PXF Ignite plugin
[ https://issues.apache.org/jira/browse/HAWQ-1599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Leskin updated HAWQ-1599: -- Priority: Major (was: Trivial) > PXF Ignite plugin > - > > Key: HAWQ-1599 > URL: https://issues.apache.org/jira/browse/HAWQ-1599 > Project: Apache HAWQ > Issue Type: New Feature > Components: PXF >Reporter: Ivan Leskin >Assignee: Ed Espino >Priority: Major > > Add a new PXF plugin to provide read and wrtie access to [Ignite > database|https://ignite.apache.org/] via its native REST API. > Key features: > * Supports partitioning; > * A dedicated Ignite client with REST API enabled is used to access Ignite > database; > * Every SELECT or INSERT query splits tuples of data into groups, each of > which is sent from (to) Ignite in one response (query). The size of a group > is customizable; > * The implemented REST API can be modified (with relatively small changes) > and used to connect to any other database (or service) which supports SQL > queries via REST, and responds using JSON objects. > Plugin documentation is available at `pxf/pxf-ignite/README.md`. > h3. Dependencies > * [GSON|https://github.com/google/gson] is required to serialize and > deserialize JSON; > * [Apache commons compress > ByteUtils|https://commons.apache.org/proper/commons-compress/apidocs/org/apache/commons/compress/utils/ByteUtils.html] > is required to properly encode fragment metadata. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HAWQ-1599) PXF Ignite plugin
[ https://issues.apache.org/jira/browse/HAWQ-1599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Leskin updated HAWQ-1599: -- Description: Add a new PXF plugin to provide read and wrtie access to [Ignite database|https://ignite.apache.org/] via its native REST API. Key features: * Supports partitioning; * A dedicated Ignite client with REST API enabled is used to access Ignite database; * Every SELECT or INSERT query splits tuples of data into groups, each of which is sent from (to) Ignite in one response (query). The size of a group is customizable; * The implemented REST API can be modified (with relatively small changes) and used to connect to any other database (or service) which supports SQL queries via REST, and responds using JSON objects. Plugin documentation is available at `pxf/pxf-ignite/README.md`. h3. Dependencies * [GSON|https://github.com/google/gson] is required to serialize and deserialize JSON; * [Apache commons compress ByteUtils|https://commons.apache.org/proper/commons-compress/apidocs/org/apache/commons/compress/utils/ByteUtils.html] is required to properly encode fragment metadata. was: This is a Jira task for the Github pull request discussed at [https://github.com/apache/incubator-hawq/pull/1344] Add a new PXF Ignite plugin to access (read & write) [Ignite database|https://ignite.apache.org/] via REST API. The complete documentation is available [here|https://github.com/arenadata/incubator-hawq/blob/pxf_ignite_plugin/pxf/pxf-ignite/README.md]. Key features: * Support partitioning (similar to one in PXF JDBC plugin); * Use a dedicated Ignite client with REST API enabled. The location of the client is a customizable PXF user property; * Every SELECT or INSERT query splits tuples of data into groups, each of which is send from (to) Ignite in one response (query). The size of a group is a customizable PXF user property; * The REST API can be modified (with relatively small changes) and used to connect to any other database (or service) which supports SQL queries via REST, and responds using JSON objects. [GSON|https://github.com/google/gson] is used to parse JSON objects. No other extra dependencies (except for those already present in HAWQ PXF) are required. > PXF Ignite plugin > - > > Key: HAWQ-1599 > URL: https://issues.apache.org/jira/browse/HAWQ-1599 > Project: Apache HAWQ > Issue Type: New Feature > Components: PXF >Reporter: Ivan Leskin >Assignee: Ed Espino >Priority: Minor > > Add a new PXF plugin to provide read and wrtie access to [Ignite > database|https://ignite.apache.org/] via its native REST API. > Key features: > * Supports partitioning; > * A dedicated Ignite client with REST API enabled is used to access Ignite > database; > * Every SELECT or INSERT query splits tuples of data into groups, each of > which is sent from (to) Ignite in one response (query). The size of a group > is customizable; > * The implemented REST API can be modified (with relatively small changes) > and used to connect to any other database (or service) which supports SQL > queries via REST, and responds using JSON objects. > Plugin documentation is available at `pxf/pxf-ignite/README.md`. > h3. Dependencies > * [GSON|https://github.com/google/gson] is required to serialize and > deserialize JSON; > * [Apache commons compress > ByteUtils|https://commons.apache.org/proper/commons-compress/apidocs/org/apache/commons/compress/utils/ByteUtils.html] > is required to properly encode fragment metadata. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HAWQ-1599) PXF Ignite plugin
[ https://issues.apache.org/jira/browse/HAWQ-1599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Leskin updated HAWQ-1599: -- Priority: Minor (was: Major) > PXF Ignite plugin > - > > Key: HAWQ-1599 > URL: https://issues.apache.org/jira/browse/HAWQ-1599 > Project: Apache HAWQ > Issue Type: New Feature > Components: PXF >Reporter: Ivan Leskin >Assignee: Ed Espino >Priority: Minor > > This is a Jira task for the Github pull request discussed at > [https://github.com/apache/incubator-hawq/pull/1344] > > Add a new PXF Ignite plugin to access (read & write) [Ignite > database|https://ignite.apache.org/] via REST API. > The complete documentation is available > [here|https://github.com/arenadata/incubator-hawq/blob/pxf_ignite_plugin/pxf/pxf-ignite/README.md]. > Key features: > * Support partitioning (similar to one in PXF JDBC plugin); > * Use a dedicated Ignite client with REST API enabled. The location of the > client is a customizable PXF user property; > * Every SELECT or INSERT query splits tuples of data into groups, each of > which is send from (to) Ignite in one response (query). The size of a group > is a customizable PXF user property; > * The REST API can be modified (with relatively small changes) and used to > connect to any other database (or service) which supports SQL queries via > REST, and responds using JSON objects. > [GSON|https://github.com/google/gson] is used to parse JSON objects. No other > extra dependencies (except for those already present in HAWQ PXF) are > required. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HAWQ-1599) PXF Ignite plugin
Ivan Leskin created HAWQ-1599: - Summary: PXF Ignite plugin Key: HAWQ-1599 URL: https://issues.apache.org/jira/browse/HAWQ-1599 Project: Apache HAWQ Issue Type: New Feature Components: PXF Reporter: Ivan Leskin Assignee: Ed Espino This is a Jira task for the Github pull request discussed at [https://github.com/apache/incubator-hawq/pull/1344] Add a new PXF Ignite plugin to access (read & write) [Ignite database|https://ignite.apache.org/] via REST API. The complete documentation is available [here|https://github.com/arenadata/incubator-hawq/blob/pxf_ignite_plugin/pxf/pxf-ignite/README.md]. Key features: * Support partitioning (similar to one in PXF JDBC plugin); * Use a dedicated Ignite client with REST API enabled. The location of the client is a customizable PXF user property; * Every SELECT or INSERT query splits tuples of data into groups, each of which is send from (to) Ignite in one response (query). The size of a group is a customizable PXF user property; * The REST API can be modified (with relatively small changes) and used to connect to any other database (or service) which supports SQL queries via REST, and responds using JSON objects. [GSON|https://github.com/google/gson] is used to parse JSON objects. No other extra dependencies (except for those already present in HAWQ PXF) are required. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
