[jira] [Commented] (PHOENIX-2373) Change ReserveNSequence Udf to take in zookeeper and tentantId as param
[ https://issues.apache.org/jira/browse/PHOENIX-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14993107#comment-14993107 ] maghamravikiran commented on PHOENIX-2373: -- The changes look good to me. I do notice the Apache license header placed incorrectly in ReserveNSequence.java . Can you please fix that . > Change ReserveNSequence Udf to take in zookeeper and tentantId as param > --- > > Key: PHOENIX-2373 > URL: https://issues.apache.org/jira/browse/PHOENIX-2373 > Project: Phoenix > Issue Type: Improvement >Reporter: Siddhi Mehta >Assignee: Siddhi Mehta >Priority: Minor > Attachments: PHOENIX-2373.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Currently the UDF reads zookeeper quorum for tuple value and tenantId is > passed in from the jobConf. > Instead wanted to make a change for the UDF to take both zookeeper quorum and > tenantId as params passed to the UDF explicitly -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-2375) Prevent write of Tephra delete markers in preDelete to ensure current data state hasn't change
[ https://issues.apache.org/jira/browse/PHOENIX-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14993050#comment-14993050 ] Thomas D'Silva commented on PHOENIX-2375: - +1 LGTM > Prevent write of Tephra delete markers in preDelete to ensure current data > state hasn't change > -- > > Key: PHOENIX-2375 > URL: https://issues.apache.org/jira/browse/PHOENIX-2375 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: James Taylor > Attachments: PHOENIX-2375.patch > > > Send Tephra family delete markers instead of real delete family markers to > bypass the Tephra coprocessor preDelete hook that writes the Tephra family > delete marker immediately. This ensures that the preBatchMutate hook sees the > previous state correctly and prevents a second unnecessary call to the > Phoenix preBatchMutate hook. > In addition, removing unnecessary call to generateDeletes for new data table > rows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-2386) Add new test cases to verify existing functionalities in Phoenix / Calcite integration
Maryann Xue created PHOENIX-2386: Summary: Add new test cases to verify existing functionalities in Phoenix / Calcite integration Key: PHOENIX-2386 URL: https://issues.apache.org/jira/browse/PHOENIX-2386 Project: Phoenix Issue Type: Task Reporter: Maryann Xue 1. Verify subqueries (decorrelation) once CALCITE-816 is fixed. 2. Verify functional index. 3. Verify local index. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-2385) User defined functions in Phoenix / Calcite integration
Maryann Xue created PHOENIX-2385: Summary: User defined functions in Phoenix / Calcite integration Key: PHOENIX-2385 URL: https://issues.apache.org/jira/browse/PHOENIX-2385 Project: Phoenix Issue Type: Task Reporter: Maryann Xue -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-2384) Investigate date/time data type and function compatibility between Phoenix and Calcite
Maryann Xue created PHOENIX-2384: Summary: Investigate date/time data type and function compatibility between Phoenix and Calcite Key: PHOENIX-2384 URL: https://issues.apache.org/jira/browse/PHOENIX-2384 Project: Phoenix Issue Type: Task Reporter: Maryann Xue Assignee: Maryann Xue -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-2383) Implement Sequence in Phoenix/Calcite integration
Maryann Xue created PHOENIX-2383: Summary: Implement Sequence in Phoenix/Calcite integration Key: PHOENIX-2383 URL: https://issues.apache.org/jira/browse/PHOENIX-2383 Project: Phoenix Issue Type: Task Reporter: Maryann Xue Assignee: Maryann Xue -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-2382) Migrate Phoenix regression test suite onto CalcitePhoenix
Maryann Xue created PHOENIX-2382: Summary: Migrate Phoenix regression test suite onto CalcitePhoenix Key: PHOENIX-2382 URL: https://issues.apache.org/jira/browse/PHOENIX-2382 Project: Phoenix Issue Type: Task Reporter: Maryann Xue -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1840) Implement server aggregate and client aggregate logic in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1840: - Labels: calcite (was: ) > Implement server aggregate and client aggregate logic in Phoenix/Calcite > Integration > > > Key: PHOENIX-1840 > URL: https://issues.apache.org/jira/browse/PHOENIX-1840 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > > To implement PhoenixAggregate with AggregatePlan when possible; otherwise use > ClientAggregatePlan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1844) Implement PhoenixReverseScanRule in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1844: - Labels: calcite (was: ) > Implement PhoenixReverseScanRule in Phoenix/Calcite Integration > --- > > Key: PHOENIX-1844 > URL: https://issues.apache.org/jira/browse/PHOENIX-1844 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 72h > Remaining Estimate: 72h > > This will act a similar way to the existing SortRemoveRule, but only that it > will push down a reverse-scan flag into PhoenixTableScan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1843) Implement getCollations() in PhoenixTable.getStatistics() and all other PhoenixRel nodes
[ https://issues.apache.org/jira/browse/PHOENIX-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1843: - Labels: calcite (was: ) > Implement getCollations() in PhoenixTable.getStatistics() and all other > PhoenixRel nodes > > > Key: PHOENIX-1843 > URL: https://issues.apache.org/jira/browse/PHOENIX-1843 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 120h > Remaining Estimate: 120h > > This would enable SortRemoveRule to work for Phoenix plans. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1841) Implement PhoenixSort in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1841: - Labels: calcite (was: ) > Implement PhoenixSort in Phoenix/Calcite Integration > > > Key: PHOENIX-1841 > URL: https://issues.apache.org/jira/browse/PHOENIX-1841 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 480h > Remaining Estimate: 480h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1838) Implement hash-join in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1838: - Labels: calcite (was: ) > Implement hash-join in Phoenix/Calcite Integration > -- > > Key: PHOENIX-1838 > URL: https://issues.apache.org/jira/browse/PHOENIX-1838 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 240h > Remaining Estimate: 240h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1829) Extend support for equi-join conditions
[ https://issues.apache.org/jira/browse/PHOENIX-1829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1829: - Labels: calcite (was: ) > Extend support for equi-join conditions > --- > > Key: PHOENIX-1829 > URL: https://issues.apache.org/jira/browse/PHOENIX-1829 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 72h > Remaining Estimate: 72h > > Currently equi-join conditions only include forms of "t1.column=t2.column". > And we are to allow other variations like "any function/expression of t1 > columns" = "any function/expression of t2 columns". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1839) Enable hash-join with RIGHT OUTER joins
[ https://issues.apache.org/jira/browse/PHOENIX-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1839: - Labels: calcite (was: ) > Enable hash-join with RIGHT OUTER joins > --- > > Key: PHOENIX-1839 > URL: https://issues.apache.org/jira/browse/PHOENIX-1839 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue >Priority: Minor > Labels: calcite > Original Estimate: 48h > Remaining Estimate: 48h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1828) Implement PhoenixJoin for Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1828: - Labels: calcite (was: ) > Implement PhoenixJoin for Phoenix/Calcite Integration > - > > Key: PHOENIX-1828 > URL: https://issues.apache.org/jira/browse/PHOENIX-1828 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 720h > Remaining Estimate: 720h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1852) Implement semi/anti joins in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1852: - Labels: calcite (was: ) > Implement semi/anti joins in Phoenix/Calcite Integration > > > Key: PHOENIX-1852 > URL: https://issues.apache.org/jira/browse/PHOENIX-1852 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 120h > Remaining Estimate: 120h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1842) Implement server sort and client sort logic in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1842: - Labels: calcite (was: ) > Implement server sort and client sort logic in Phoenix/Calcite Integration > -- > > Key: PHOENIX-1842 > URL: https://issues.apache.org/jira/browse/PHOENIX-1842 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 120h > Remaining Estimate: 120h > > Implement PhoenixSort with ScanPlan or AggregatePlan or HashJoinPlan when > possible; otherwise use ClientScanPlan or ClientAggregatePlan. > Even if the sorting cannot be done over the server, as in a ScanPlan or in a > ScanPlan wrapped by a HashJoinPlan, putting the sorting together with an > AggregatePlan or an AggregatePlan wrapped by a HashJoinPlan would still be > cheaper, since it would save an extra "spooling" process. We should make sure > that is reflected in cost calculation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1878) Implement PhoenixSchema and PhoenixTable in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1878: - Labels: calcite (was: ) > Implement PhoenixSchema and PhoenixTable in Phoenix/Calcite Integration > --- > > Key: PHOENIX-1878 > URL: https://issues.apache.org/jira/browse/PHOENIX-1878 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 240h > Remaining Estimate: 240h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1786) Implement sort-merge-join with Calcite integration
[ https://issues.apache.org/jira/browse/PHOENIX-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1786: - Labels: calcite (was: ) > Implement sort-merge-join with Calcite integration > -- > > Key: PHOENIX-1786 > URL: https://issues.apache.org/jira/browse/PHOENIX-1786 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 480h > Remaining Estimate: 480h > > This is not necessarily an optimization, but it requires similar techniques > like RelNode transform, cost adjustment, etc. > See PHOENIX-1179. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2089) Use index when skip-scan is possible or when ordering key matches
[ https://issues.apache.org/jira/browse/PHOENIX-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2089: - Labels: calcite (was: ) > Use index when skip-scan is possible or when ordering key matches > - > > Key: PHOENIX-2089 > URL: https://issues.apache.org/jira/browse/PHOENIX-2089 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 240h > Remaining Estimate: 240h > > This is very basic work for secondary index and depends on the fix for > CALCITE-761 and CALCITE-763. > Further work should be done in PhoenixTableScan.computeSelfCost() to give a > more accurate estimate for better matching among different indices. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2133) Implement PhoenixValues rel in Phoenix - Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-2133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2133: - Labels: calcite (was: ) > Implement PhoenixValues rel in Phoenix - Calcite Integration > > > Key: PHOENIX-2133 > URL: https://issues.apache.org/jira/browse/PHOENIX-2133 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 48h > Remaining Estimate: 48h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2128) Implement using local index with missing columns
[ https://issues.apache.org/jira/browse/PHOENIX-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2128: - Labels: calcite (was: ) > Implement using local index with missing columns > > > Key: PHOENIX-2128 > URL: https://issues.apache.org/jira/browse/PHOENIX-2128 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 480h > Remaining Estimate: 480h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2091) Add materialized view definition automatically based on the available indices
[ https://issues.apache.org/jira/browse/PHOENIX-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2091: - Labels: calcite (was: ) > Add materialized view definition automatically based on the available indices > - > > Key: PHOENIX-2091 > URL: https://issues.apache.org/jira/browse/PHOENIX-2091 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 120h > Remaining Estimate: 120h > > We should look for available indices in Phoenix and add them as pre-populated > materialized views in Calcite. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2047) Implement query using secondary index in Phoenix - Calcite integration
[ https://issues.apache.org/jira/browse/PHOENIX-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2047: - Labels: calcite (was: ) > Implement query using secondary index in Phoenix - Calcite integration > -- > > Key: PHOENIX-2047 > URL: https://issues.apache.org/jira/browse/PHOENIX-2047 > Project: Phoenix > Issue Type: New Feature >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 480h > Remaining Estimate: 480h > > The idea is to take advantage of the existing Materialized Views in Calcite > and define secondary index as an relational expression. > There are a couple of issues to address here: > 1. Use the index as it is, which is to fall back on full scan of data table > if any column is missing from the index table. > 2. Join missing columns with global index. > 3. Join missing columns with local index. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2127) Implement using global index with missing columns
[ https://issues.apache.org/jira/browse/PHOENIX-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2127: - Labels: calcite (was: ) > Implement using global index with missing columns > - > > Key: PHOENIX-2127 > URL: https://issues.apache.org/jira/browse/PHOENIX-2127 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 480h > Remaining Estimate: 480h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2202) Implement PhoenixUncollect
[ https://issues.apache.org/jira/browse/PHOENIX-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2202: - Labels: calcite (was: ) > Implement PhoenixUncollect > -- > > Key: PHOENIX-2202 > URL: https://issues.apache.org/jira/browse/PHOENIX-2202 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2231) Support CREATE/DROP SEQUENCE in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2231: - Labels: calcite (was: ) > Support CREATE/DROP SEQUENCE in Phoenix/Calcite Integration > --- > > Key: PHOENIX-2231 > URL: https://issues.apache.org/jira/browse/PHOENIX-2231 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: maghamravikiran > Labels: calcite > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2204) Add correlate UNNEST test cases
[ https://issues.apache.org/jira/browse/PHOENIX-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2204: - Labels: calcite (was: ) > Add correlate UNNEST test cases > --- > > Key: PHOENIX-2204 > URL: https://issues.apache.org/jira/browse/PHOENIX-2204 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2228) Support CREATE/DROP TABLE in Phoenix-Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2228: - Labels: calcite (was: ) > Support CREATE/DROP TABLE in Phoenix-Calcite Integration > > > Key: PHOENIX-2228 > URL: https://issues.apache.org/jira/browse/PHOENIX-2228 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue > Labels: calcite > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2201) Support UNNEST in Phoenix/Calcite integration
[ https://issues.apache.org/jira/browse/PHOENIX-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2201: - Labels: calcite (was: ) > Support UNNEST in Phoenix/Calcite integration > - > > Key: PHOENIX-2201 > URL: https://issues.apache.org/jira/browse/PHOENIX-2201 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2229) Support CREATE/DROP VIEW in Phoenix-Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2229: - Labels: calcite (was: ) > Support CREATE/DROP VIEW in Phoenix-Calcite Integration > --- > > Key: PHOENIX-2229 > URL: https://issues.apache.org/jira/browse/PHOENIX-2229 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue > Labels: calcite > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2203) Track Calcite support for WITH ORDINALITY
[ https://issues.apache.org/jira/browse/PHOENIX-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2203: - Labels: calcite (was: ) > Track Calcite support for WITH ORDINALITY > - > > Key: PHOENIX-2203 > URL: https://issues.apache.org/jira/browse/PHOENIX-2203 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > > Once WITH ORDINALITY is supported in Calcite, we'll be able to add the > missing part here in the integration, -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2225) Support Correlate (Nested-loop Join) in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2225: - Labels: calcite (was: ) > Support Correlate (Nested-loop Join) in Phoenix/Calcite Integration > --- > > Key: PHOENIX-2225 > URL: https://issues.apache.org/jira/browse/PHOENIX-2225 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2230) Support CREATE/DROP INDEX in Phoenix-Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2230: - Labels: calcite (was: ) > Support CREATE/DROP INDEX in Phoenix-Calcite Integration > > > Key: PHOENIX-2230 > URL: https://issues.apache.org/jira/browse/PHOENIX-2230 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue > Labels: calcite > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2226) Support Semi-Join in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2226: - Labels: calcite (was: ) > Support Semi-Join in Phoenix/Calcite Integration > > > Key: PHOENIX-2226 > URL: https://issues.apache.org/jira/browse/PHOENIX-2226 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2250) Deduct column reference for TableScan and project required columns/column-families in Scan
[ https://issues.apache.org/jira/browse/PHOENIX-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2250: - Summary: Deduct column reference for TableScan and project required columns/column-families in Scan (was: [Phoenix/Calcite] Deduct column reference for TableScan and project required columns/column-families in Scan) > Deduct column reference for TableScan and project required > columns/column-families in Scan > -- > > Key: PHOENIX-2250 > URL: https://issues.apache.org/jira/browse/PHOENIX-2250 > Project: Phoenix > Issue Type: Improvement >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2250) [Phoenix/Calcite] Deduct column reference for TableScan and project required columns/column-families in Scan
[ https://issues.apache.org/jira/browse/PHOENIX-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2250: - Labels: calcite (was: ) > [Phoenix/Calcite] Deduct column reference for TableScan and project required > columns/column-families in Scan > > > Key: PHOENIX-2250 > URL: https://issues.apache.org/jira/browse/PHOENIX-2250 > Project: Phoenix > Issue Type: Improvement >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2193) Add rules to push down Sort through Union
[ https://issues.apache.org/jira/browse/PHOENIX-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2193: - Labels: calcite (was: ) > Add rules to push down Sort through Union > - > > Key: PHOENIX-2193 > URL: https://issues.apache.org/jira/browse/PHOENIX-2193 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > > Sort will be pushed through Union to its inputs, and meanwhile the original > sort will become a merge-sort attribute in the PhoenixUnion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2192) Implement PhoenixUnion
[ https://issues.apache.org/jira/browse/PHOENIX-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2192: - Labels: calcite (was: ) > Implement PhoenixUnion > -- > > Key: PHOENIX-2192 > URL: https://issues.apache.org/jira/browse/PHOENIX-2192 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2344) Implement partial stream aggregate
[ https://issues.apache.org/jira/browse/PHOENIX-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2344: - Labels: calcite (was: ) > Implement partial stream aggregate > -- > > Key: PHOENIX-2344 > URL: https://issues.apache.org/jira/browse/PHOENIX-2344 > Project: Phoenix > Issue Type: Improvement >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > > We now have ordered group-by (stream aggregate) and unordered group-by (hash > aggregate) in Phoenix. Stream aggregate is usually much more beneficial than > hash aggregate in terms of memory usage and pipelining, but it requires that > the aggregate's input is ordered on group-by expressions, i.e. the group-by > expressions is the beginning part of the input's collation (ordering). > However, we could have something in the middle, a stream/hash hybrid > aggregate when the group-by expressions and the input collation share some > common part. For example, we group table T1 by column A, B and T1 is sorted > on column A, C, we'll have the ordered part as A, and the hash part as B. > Thus within the range of a same A, a hash table is used for collecting all > different Bs; while at the changing point of A, we can purge the intermediate > hash table and feed the result for the previous A to next operator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2191) Union Support in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2191: - Labels: calcite (was: ) > Union Support in Phoenix/Calcite Integration > > > Key: PHOENIX-2191 > URL: https://issues.apache.org/jira/browse/PHOENIX-2191 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2167) Add new interface in QueryPlan for pushing down a limit value.
[ https://issues.apache.org/jira/browse/PHOENIX-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2167: - Labels: calcite (was: ) > Add new interface in QueryPlan for pushing down a limit value. > -- > > Key: PHOENIX-2167 > URL: https://issues.apache.org/jira/browse/PHOENIX-2167 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Attachments: PHOENIX-2167.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > Calcite rel trees are compiled bottom-up into Phoenix QueryPlans, the easiest > way to implement a Limit RelNode in the tree is to create a ClientScanPlan > with the limit value and wrap it around the inner plan. However, oftentimes > this may not be efficient. > For example, select * from T limit 10, the original Phoenix compiler would > generate a ScanPlan with limit 10, apply a PageFilter and avoid using > sophisticated or expensive ResultIterators. > So it would make sense to push down the limit to a QueryPlan as low as > possible and create a new copy of the plan based on this rule. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1858) Implement PhoenixSort.getLimit() in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1858: - Labels: calcite (was: ) > Implement PhoenixSort.getLimit() in Phoenix/Calcite Integration > --- > > Key: PHOENIX-1858 > URL: https://issues.apache.org/jira/browse/PHOENIX-1858 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 24h > Remaining Estimate: 24h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1859) Implement PhoenixLimit in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1859: - Labels: calcite (was: ) > Implement PhoenixLimit in Phoenix/Calcite Integration > - > > Key: PHOENIX-1859 > URL: https://issues.apache.org/jira/browse/PHOENIX-1859 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 72h > Remaining Estimate: 72h > > Limit is represented as a Sort RelNode in Calcite. We will do sort and limit > together if sort spec is present (see PHOENIX-1858). Otherwise we will have a > standalone PhoenixLimit, for which more aggressive optimization rules can be > applied than for PhoenixSort. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1866) Implement Sort/Limit Merge Rules in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1866: - Labels: calcite (was: ) > Implement Sort/Limit Merge Rules in Phoenix/Calcite Integration > --- > > Key: PHOENIX-1866 > URL: https://issues.apache.org/jira/browse/PHOENIX-1866 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 72h > Remaining Estimate: 72h > > Sort over Sort-without-fetch --> the sort below can be removed. > Limit over Sort-without-fetch --> can be merged into Sort-with-fetch. > Limit over Sort-with-fetch/Limit --> can be merged into one Sort/Limit with > the smaller limit if both limit nodes are stateless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1836) Implement PhoenixAggregate in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1836: - Labels: calcite (was: ) > Implement PhoenixAggregate in Phoenix/Calcite Integration > - > > Key: PHOENIX-1836 > URL: https://issues.apache.org/jira/browse/PHOENIX-1836 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 480h > Remaining Estimate: 480h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1857) Implement Limit in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1857: - Labels: calcite (was: ) > Implement Limit in Phoenix/Calcite Integration > -- > > Key: PHOENIX-1857 > URL: https://issues.apache.org/jira/browse/PHOENIX-1857 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 240h > Remaining Estimate: 240h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1837) Detect ordered/unordered group-by
[ https://issues.apache.org/jira/browse/PHOENIX-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1837: - Labels: calcite (was: ) > Detect ordered/unordered group-by > - > > Key: PHOENIX-1837 > URL: https://issues.apache.org/jira/browse/PHOENIX-1837 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 120h > Remaining Estimate: 120h > > With ordered group-by, we would do much less work. The existing Phoenix code > depends on TrackOrderPreservingExpressionCompiler to calculate the flag. But > given that we would no longer use any of the Phoenix parse tree nodes and the > TrackOrderPreservingExpressionCompiler is a parse tree node visitor, we > should now implement the same logic with Calcite RexNode. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1860) Implement PhoenixLimitPushThroughJoinRule in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1860: - Labels: calcite (was: ) > Implement PhoenixLimitPushThroughJoinRule in Phoenix/Calcite Integration > > > Key: PHOENIX-1860 > URL: https://issues.apache.org/jira/browse/PHOENIX-1860 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > Original Estimate: 72h > Remaining Estimate: 72h > > The OUTER end of the join will have an additional Limit RelNode above while > the original Limit over Join node should still remain. We need to maintain a > flag or something in order to prevent the rule to fired again for the same > node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1850) Implement PhoenixTable.getStatistics() in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1850: - Labels: calcite (was: ) > Implement PhoenixTable.getStatistics() in Phoenix/Calcite Integration > - > > Key: PHOENIX-1850 > URL: https://issues.apache.org/jira/browse/PHOENIX-1850 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue > Labels: calcite > Original Estimate: 240h > Remaining Estimate: 240h > > Table statistics mainly include row count, collation and distribution. We > will try to get this information for existing Phoenix statistics and > implement the methods accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1788) Phoenix - Calcite Integration Tasks
[ https://issues.apache.org/jira/browse/PHOENIX-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1788: - Labels: calcite gsoc2015 mentor (was: gsoc2015 mentor) > Phoenix - Calcite Integration Tasks > --- > > Key: PHOENIX-1788 > URL: https://issues.apache.org/jira/browse/PHOENIX-1788 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue > Labels: calcite > Original Estimate: 1,680h > Remaining Estimate: 1,680h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1788) Phoenix - Calcite Integration Tasks
[ https://issues.apache.org/jira/browse/PHOENIX-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1788: - Labels: calcite (was: calcite gsoc2015 mentor) > Phoenix - Calcite Integration Tasks > --- > > Key: PHOENIX-1788 > URL: https://issues.apache.org/jira/browse/PHOENIX-1788 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue > Labels: calcite > Original Estimate: 1,680h > Remaining Estimate: 1,680h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1488) Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1488: - Labels: label (was: ) > Calcite Integration > --- > > Key: PHOENIX-1488 > URL: https://issues.apache.org/jira/browse/PHOENIX-1488 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor > Labels: calcite > > Grouping JIRA for improving the abstraction between the parser, compiler, and > runtime systems. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1495) Investigate using code generation instead of interpreting expressions at runtime
[ https://issues.apache.org/jira/browse/PHOENIX-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1495: - Labels: calcite (was: ) > Investigate using code generation instead of interpreting expressions at > runtime > > > Key: PHOENIX-1495 > URL: https://issues.apache.org/jira/browse/PHOENIX-1495 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor > Labels: calcite > > There may be a performance gain from using code generation for SQL > expressions instead of interpreting expressions at runtime. We should explore > various options. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1488) Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1488: - Labels: calcite (was: label) > Calcite Integration > --- > > Key: PHOENIX-1488 > URL: https://issues.apache.org/jira/browse/PHOENIX-1488 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor > Labels: calcite > > Grouping JIRA for improving the abstraction between the parser, compiler, and > runtime systems. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2090) Refine PhoenixTableScan.computeSelfCost() when scanRanges is available
[ https://issues.apache.org/jira/browse/PHOENIX-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2090: - Labels: calcite (was: ) > Refine PhoenixTableScan.computeSelfCost() when scanRanges is available > -- > > Key: PHOENIX-2090 > URL: https://issues.apache.org/jira/browse/PHOENIX-2090 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue > Labels: calcite > Original Estimate: 120h > Remaining Estimate: 120h > > We should compute a more accurate cost based on the "scanRanges" so that we > can better choose among these different indices. > For example, if we have more than one indices concerning different index > keys, for example IDX1 is indexed on column a, and IDX2 is indexed on column > b, and our query is like "select x, y, z where a between 'A1' and 'A2' and b > between 'B3' and 'B4'. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1785) Port Child/Parent Join Optimization to Phoenix/Calcite integration
[ https://issues.apache.org/jira/browse/PHOENIX-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1785: - Labels: calcite (was: ) > Port Child/Parent Join Optimization to Phoenix/Calcite integration > -- > > Key: PHOENIX-1785 > URL: https://issues.apache.org/jira/browse/PHOENIX-1785 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue > Labels: calcite > Original Estimate: 240h > Remaining Estimate: 240h > > See PHOENIX-852 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1787) Port Phoenix star-join to Phoenix/Calcite integration
[ https://issues.apache.org/jira/browse/PHOENIX-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1787: - Labels: calcite (was: ) > Port Phoenix star-join to Phoenix/Calcite integration > - > > Key: PHOENIX-1787 > URL: https://issues.apache.org/jira/browse/PHOENIX-1787 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue > Labels: calcite > Original Estimate: 480h > Remaining Estimate: 480h > > Enable parallel hash-join of multiple tables. For example A join B join C, > instead of joining two tables at a time, we can probably probe A and join > built B and built C at the same time if memory is allowed. > We can make use of Calcite Multijoin and enable Multijoin related rules. If > we find out that a star-join is not doable due to (right) outer joins or > memory limit, we can set the cost to infinite and thus discard this > optimization. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1851) Implement PhoenixRelMetadataProvider in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1851: - Priority: Minor (was: Major) > Implement PhoenixRelMetadataProvider in Phoenix/Calcite Integration > --- > > Key: PHOENIX-1851 > URL: https://issues.apache.org/jira/browse/PHOENIX-1851 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Priority: Minor > Labels: calcite > Original Estimate: 720h > Remaining Estimate: 720h > > The RelMetadataProvider is to provide information required for calculating > cost for RelNode instances. It includes row count, selectivity, distinct row > count, etc. Some of the information may be already available from existing > Phoenix statistics, and we will have to enhance Phoenix statistics to > implement the rest. > The metadata provider classes to implement include: > PhoenixRelMdCollation > PhoenixRelMdColumnOrigins > PhoenixRelMdColumnUniqueness > PhoenixRelMdDistinctRowCount > PhoenixRelMdDistribution > PhoenixRelMdExplainVisibility > PhoenixRelMdMemory > PhoenixRelMdParallelism > PhoenixRelMdPercentageOriginalRows > PhoenixRelMdPopulationSize > PhoenixRelMdPredicates > PhoenixRelMdRowCount > PhoenixRelMdSelectivity > PhoenixRelMdSize > PhoenixRelMdUniqueKeys > We have default implementations in DefaultRelMetadataProvider, so we only > need to implement those Phoenix specific metadata methods in the above > classes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1851) Implement PhoenixRelMetadataProvider in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1851: - Labels: calcite (was: ) > Implement PhoenixRelMetadataProvider in Phoenix/Calcite Integration > --- > > Key: PHOENIX-1851 > URL: https://issues.apache.org/jira/browse/PHOENIX-1851 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue > Labels: calcite > Original Estimate: 720h > Remaining Estimate: 720h > > The RelMetadataProvider is to provide information required for calculating > cost for RelNode instances. It includes row count, selectivity, distinct row > count, etc. Some of the information may be already available from existing > Phoenix statistics, and we will have to enhance Phoenix statistics to > implement the rest. > The metadata provider classes to implement include: > PhoenixRelMdCollation > PhoenixRelMdColumnOrigins > PhoenixRelMdColumnUniqueness > PhoenixRelMdDistinctRowCount > PhoenixRelMdDistribution > PhoenixRelMdExplainVisibility > PhoenixRelMdMemory > PhoenixRelMdParallelism > PhoenixRelMdPercentageOriginalRows > PhoenixRelMdPopulationSize > PhoenixRelMdPredicates > PhoenixRelMdRowCount > PhoenixRelMdSelectivity > PhoenixRelMdSize > PhoenixRelMdUniqueKeys > We have default implementations in DefaultRelMetadataProvider, so we only > need to implement those Phoenix specific metadata methods in the above > classes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1879) Provide prepare statement hooks for Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1879: - Labels: calcite (was: ) > Provide prepare statement hooks for Phoenix/Calcite Integration > --- > > Key: PHOENIX-1879 > URL: https://issues.apache.org/jira/browse/PHOENIX-1879 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue > Labels: calcite > > This is where we can setup RelOptCluster with a PhoenixRelMetadataProvider, > add/or remove RelOptRules, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-2291) Error: Could not find hash cache for joinId
[ https://issues.apache.org/jira/browse/PHOENIX-2291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992916#comment-14992916 ] Maryann Xue commented on PHOENIX-2291: -- Hi [~danmeany], is your case possibly related to PHOENIX-2381? > Error: Could not find hash cache for joinId > --- > > Key: PHOENIX-2291 > URL: https://issues.apache.org/jira/browse/PHOENIX-2291 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 4.5.0 > Environment: Centos7 cluster, HBase 1.1, Phoenix 4.5.0 >Reporter: Dan Meany >Assignee: Maryann Xue >Priority: Minor > > Intermittently get error below when joining two tables (~10k rows each). > May be load-related. > java.lang.RuntimeException: org.apache.phoenix.exception.PhoenixIOException: > org.apache.phoenix.exception.PhoenixIOException: > org.apache.hadoop.hbase.DoNotRetryIOException: Could not find hash cache for > joinId: �X�w��ZY. The cache might have expired and have been removed. > at > org.apache.phoenix.coprocessor.HashJoinRegionScanner.(HashJoinRegionScanner.java:96) > at > org.apache.phoenix.coprocessor.ScanRegionObserver.doPostScannerOpen(ScanRegionObserver.java:213) > at > org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:179) > at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$52.call(RegionCoprocessorHost.java:1316) > Query plan looks like: > | CLIENT 40-CHUNK PARALLEL 40-WAY FULL SCAN OVER TABLE1_IDX | > | SERVER FILTER BY FIRST KEY ONLY | > | PARALLEL LEFT-JOIN TABLE 0 | > | CLIENT 40-CHUNK PARALLEL 40-WAY FULL SCAN OVER TABLE2_IDX | > | AFTER-JOIN SERVER FILTER BY "MP.:ID" IS NULL | -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2373) Change ReserveNSequence Udf to take in zookeeper and tentantId as param
[ https://issues.apache.org/jira/browse/PHOENIX-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddhi Mehta updated PHOENIX-2373: -- Attachment: PHOENIX-2373.patch [~prkommireddi],[~maghamraviki...@gmail.com],[~jfernando_sfdc],[~giacomotaylor] Mind reviewing the patch? 1. I have changed the udf to take zkQuorum and tentantId in constructor og udf. 2. Optimized the udf to make only 1 connection per task instead of 1 per tuple 3. Changes to test for tentant specifc and generic connection udf. > Change ReserveNSequence Udf to take in zookeeper and tentantId as param > --- > > Key: PHOENIX-2373 > URL: https://issues.apache.org/jira/browse/PHOENIX-2373 > Project: Phoenix > Issue Type: Improvement >Reporter: Siddhi Mehta >Assignee: Siddhi Mehta >Priority: Minor > Attachments: PHOENIX-2373.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Currently the UDF reads zookeeper quorum for tuple value and tenantId is > passed in from the jobConf. > Instead wanted to make a change for the UDF to take both zookeeper quorum and > tenantId as params passed to the UDF explicitly -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PHOENIX-2291) Error: Could not find hash cache for joinId
[ https://issues.apache.org/jira/browse/PHOENIX-2291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue reassigned PHOENIX-2291: Assignee: Maryann Xue > Error: Could not find hash cache for joinId > --- > > Key: PHOENIX-2291 > URL: https://issues.apache.org/jira/browse/PHOENIX-2291 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 4.5.0 > Environment: Centos7 cluster, HBase 1.1, Phoenix 4.5.0 >Reporter: Dan Meany >Assignee: Maryann Xue >Priority: Minor > > Intermittently get error below when joining two tables (~10k rows each). > May be load-related. > java.lang.RuntimeException: org.apache.phoenix.exception.PhoenixIOException: > org.apache.phoenix.exception.PhoenixIOException: > org.apache.hadoop.hbase.DoNotRetryIOException: Could not find hash cache for > joinId: �X�w��ZY. The cache might have expired and have been removed. > at > org.apache.phoenix.coprocessor.HashJoinRegionScanner.(HashJoinRegionScanner.java:96) > at > org.apache.phoenix.coprocessor.ScanRegionObserver.doPostScannerOpen(ScanRegionObserver.java:213) > at > org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:179) > at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$52.call(RegionCoprocessorHost.java:1316) > Query plan looks like: > | CLIENT 40-CHUNK PARALLEL 40-WAY FULL SCAN OVER TABLE1_IDX | > | SERVER FILTER BY FIRST KEY ONLY | > | PARALLEL LEFT-JOIN TABLE 0 | > | CLIENT 40-CHUNK PARALLEL 40-WAY FULL SCAN OVER TABLE2_IDX | > | AFTER-JOIN SERVER FILTER BY "MP.:ID" IS NULL | -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PHOENIX-1997) Join optimization: Apply one table's where condition to the others
[
https://issues.apache.org/jira/browse/PHOENIX-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maryann Xue reassigned PHOENIX-1997:
Assignee: Maryann Xue
> Join optimization: Apply one table's where condition to the others
> --
>
> Key: PHOENIX-1997
> URL: https://issues.apache.org/jira/browse/PHOENIX-1997
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.3.1
>Reporter: Taeyun Kim
>Assignee: Maryann Xue
>Priority: Minor
> Labels: calcite
>
> Joined tables are as follows:
> {noformat}
> create table table_a
> (
> time_id integer not null,
> depth tinyint not null,
> id0 integer not null,
> id1 integer not null,
> id2 integer not null,
> id3 integer not null,
> id integer not null,
> record_type smallint not null,
> c varbinary
> constraint pk primary key(time_id, depth, id0, id1, id2, id3, id,
> record_type)
> )
> salt_buckets=4,
> compression='SNAPPY',
> create table table_b
> (
> depth tinyint not null,
> id0 integer not null,
> id1 integer not null,
> id integer not null,
> c varbinary,
> p varbinary
> constraint pk primary key(depth, id0, id1, id)
> )
> salt_buckets=2,
> compression='SNAPPY';
> create index table_b_index on table_b (id, depth, id0, id1)
> compression='SNAPPY';
> {noformat}
> The query is as follows:
> {noformat}
> select a.*, b.c
> from table_a a inner join table_b b on (a.depth = b.depth and a.id0 = b.id0
> and a.id1 = b.id1 and a.id2 = b.id)
> where a.time_id = 23796900 and a.depth = 1;
> {noformat}
> It is obvious that b.depth must also be 1 since it's on the join condition.
> And since the depth column is the first primary key column of table_b,
> table_b should be range scanned before join.
> But the query explanation is as follows:
> {noformat}
> CLIENT PARALLEL 4-WAY RANGE SCAN OVER TABLE_A [0,23796900,1]
> CLIENT MERGE SORT
> PARALLEL INNER-JOIN TABLE 0
> CLIENT PARALLEL 2-WAY FULL SCAN OVER TABLE_B
> CLIENT MERGE SORT
> {noformat}
> But when (b.depth = 1) condition is explicitly added to the query, the
> explanation is changed as the expected one:
> {noformat}
> CLIENT PARALLEL 4-WAY RANGE SCAN OVER TABLE_A [0,23796900,1]
> CLIENT MERGE SORT
> PARALLEL INNER-JOIN TABLE 0
> CLIENT PARALLEL 2-WAY RANGE SCAN OVER TABLE_B [0,1]
> CLIENT MERGE SORT
> {noformat}
> It would be nice if the optimizer could find this condition dependency and
> apply it to the query plan.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-1997) Join optimization: Apply one table's where condition to the others
[
https://issues.apache.org/jira/browse/PHOENIX-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maryann Xue updated PHOENIX-1997:
-
Labels: calcite (was: )
> Join optimization: Apply one table's where condition to the others
> --
>
> Key: PHOENIX-1997
> URL: https://issues.apache.org/jira/browse/PHOENIX-1997
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.3.1
>Reporter: Taeyun Kim
>Priority: Minor
> Labels: calcite
>
> Joined tables are as follows:
> {noformat}
> create table table_a
> (
> time_id integer not null,
> depth tinyint not null,
> id0 integer not null,
> id1 integer not null,
> id2 integer not null,
> id3 integer not null,
> id integer not null,
> record_type smallint not null,
> c varbinary
> constraint pk primary key(time_id, depth, id0, id1, id2, id3, id,
> record_type)
> )
> salt_buckets=4,
> compression='SNAPPY',
> create table table_b
> (
> depth tinyint not null,
> id0 integer not null,
> id1 integer not null,
> id integer not null,
> c varbinary,
> p varbinary
> constraint pk primary key(depth, id0, id1, id)
> )
> salt_buckets=2,
> compression='SNAPPY';
> create index table_b_index on table_b (id, depth, id0, id1)
> compression='SNAPPY';
> {noformat}
> The query is as follows:
> {noformat}
> select a.*, b.c
> from table_a a inner join table_b b on (a.depth = b.depth and a.id0 = b.id0
> and a.id1 = b.id1 and a.id2 = b.id)
> where a.time_id = 23796900 and a.depth = 1;
> {noformat}
> It is obvious that b.depth must also be 1 since it's on the join condition.
> And since the depth column is the first primary key column of table_b,
> table_b should be range scanned before join.
> But the query explanation is as follows:
> {noformat}
> CLIENT PARALLEL 4-WAY RANGE SCAN OVER TABLE_A [0,23796900,1]
> CLIENT MERGE SORT
> PARALLEL INNER-JOIN TABLE 0
> CLIENT PARALLEL 2-WAY FULL SCAN OVER TABLE_B
> CLIENT MERGE SORT
> {noformat}
> But when (b.depth = 1) condition is explicitly added to the query, the
> explanation is changed as the expected one:
> {noformat}
> CLIENT PARALLEL 4-WAY RANGE SCAN OVER TABLE_A [0,23796900,1]
> CLIENT MERGE SORT
> PARALLEL INNER-JOIN TABLE 0
> CLIENT PARALLEL 2-WAY RANGE SCAN OVER TABLE_B [0,1]
> CLIENT MERGE SORT
> {noformat}
> It would be nice if the optimizer could find this condition dependency and
> apply it to the query plan.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (PHOENIX-2381) Inner Join with any table or view with Multi_Tenant=true causes "could not find hash cache for joinId" error
[ https://issues.apache.org/jira/browse/PHOENIX-2381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue reassigned PHOENIX-2381: Assignee: Maryann Xue > Inner Join with any table or view with Multi_Tenant=true causes "could not > find hash cache for joinId" error > > > Key: PHOENIX-2381 > URL: https://issues.apache.org/jira/browse/PHOENIX-2381 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.6.0 > Environment: This is with Phoenix version 4.6.0 and HBase version > 0.98.4.2.2.6.0-2800-hadoop2. >Reporter: Don Brinn >Assignee: Maryann Xue > Labels: join, joins, multi-tenant > > I am seeing the following error when doing an INNER JOIN of a view with > MULTI_TENANT=true with any other table or view: > java.lang.RuntimeException: org.apache.phoenix.exception.PhoenixIOException: > org.apache.phoenix.exception.PhoenixIOException: > org.apache.hadoop.hbase.DoNotRetryIOException: Could not find hash cache for > joinId: Ys�0��%�. The cache might have expired and have been removed. > at > org.apache.phoenix.coprocessor.HashJoinRegionScanner.(HashJoinRegionScanner.java:95) > at > org.apache.phoenix.coprocessor.ScanRegionObserver.doPostScannerOpen(ScanRegionObserver.java:212) > at > org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:178) > at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1931) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3178) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29994) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2078) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114) > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94) > at java.lang.Thread.run(Thread.java:745) > > at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73) > at sqlline.TableOutputFormat.print(TableOutputFormat.java:33) > at sqlline.SqlLine.print(SqlLine.java:1653) > at sqlline.Commands.execute(Commands.java:833) > at sqlline.Commands.sql(Commands.java:732) > at sqlline.SqlLine.dispatch(SqlLine.java:808) > at sqlline.SqlLine.begin(SqlLine.java:681) > at sqlline.SqlLine.start(SqlLine.java:398) > at sqlline.SqlLine.main(SqlLine.java:292) > > This is with Phoenix version 4.6.0 and HBase version > 0.98.4.2.2.6.0-2800-hadoop2. > > This seems very strongly related to the MULTI_TENANT=true property on a view > or table. I see the error whenever the view has MULTI_TENANT=true and I have > a tenant-specific connection to Phoenix. I do not see the problem if the > MULTI_TENANT=true property is not set on the view or if I do not have a > tenant-specific connection to Phoenix. > > Here is an example SQL statement that has this error when the view INVENTORY > has the MULTI_TENANT=true property and I have a tenant-specific connection, > but that succeeds in other cases. (The view PRODUCT_IDS is not Multi-Tenant.) > SELECT * FROM INVENTORY INNER JOIN PRODUCT_IDS ON (PRODUCT_ID = INVENTORY.ID) > > Note: "INNER JOIN" fails under these conditions, as does "LEFT OUTER JOIN". > However, "RIGHT OUTER JOIN" and "FULL OUTER JOIN" do work. Also, if I tell > Phoenix to use a Sort Join for the Inner Join or Left Outer Join then it does > work, e.g. SELECT /*+ USE_SORT_MERGE_JOIN*/ * FROM INVENTORY INNER JOIN > PRODUCT_IDS ON (PRODUCT_ID = INVENTORY.ID); works. > > This seems to be the same problem that was discussed previously in this > mailing list: > https://mail-archives.apache.org/mod_mbox/phoenix-user/201507.mbox/%3ccaotkwx5xfbwkjf--0k-zj91tfdqwfq6rmuqw0r_lojcnj1a...@mail.gmail.com%3E > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-1787) Port Phoenix star-join to Phoenix/Calcite integration
[ https://issues.apache.org/jira/browse/PHOENIX-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue resolved PHOENIX-1787. -- Resolution: Not A Problem > Port Phoenix star-join to Phoenix/Calcite integration > - > > Key: PHOENIX-1787 > URL: https://issues.apache.org/jira/browse/PHOENIX-1787 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue > Original Estimate: 480h > Remaining Estimate: 480h > > Enable parallel hash-join of multiple tables. For example A join B join C, > instead of joining two tables at a time, we can probably probe A and join > built B and built C at the same time if memory is allowed. > We can make use of Calcite Multijoin and enable Multijoin related rules. If > we find out that a star-join is not doable due to (right) outer joins or > memory limit, we can set the cost to infinite and thus discard this > optimization. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-2221) Option to make data regions not writable when index regions are not available
[ https://issues.apache.org/jira/browse/PHOENIX-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992842#comment-14992842 ] Alicia Ying Shu commented on PHOENIX-2221: -- Working on it. Will update soon. > Option to make data regions not writable when index regions are not available > - > > Key: PHOENIX-2221 > URL: https://issues.apache.org/jira/browse/PHOENIX-2221 > Project: Phoenix > Issue Type: Improvement >Reporter: Devaraj Das >Assignee: Alicia Ying Shu > Attachments: PHOENIX-2221-v1.patch, PHOENIX-2221-v2.patch, > PHOENIX-2221.patch > > > In one usecase, it was deemed better to not accept writes when the index > regions are unavailable for any reason (as opposed to disabling the index and > the queries doing bigger data-table scans). > The idea is that the index regions are kept consistent with the data regions, > and when a query runs against the index regions, one can be reasonably sure > that the query ran with the most recent data in the data regions. When the > index regions are unavailable, the writes to the data table are rejected. > Read queries off of the index regions would have deterministic performance > (and on the other hand if the index is disabled, then the read queries would > have to go to the data regions until the indexes are rebuilt, and the queries > would suffer). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-2381) Inner Join with any table or view with Multi_Tenant=true causes "could not find hash cache for joinId" error
Don Brinn created PHOENIX-2381: -- Summary: Inner Join with any table or view with Multi_Tenant=true causes "could not find hash cache for joinId" error Key: PHOENIX-2381 URL: https://issues.apache.org/jira/browse/PHOENIX-2381 Project: Phoenix Issue Type: Bug Affects Versions: 4.6.0 Environment: This is with Phoenix version 4.6.0 and HBase version 0.98.4.2.2.6.0-2800-hadoop2. Reporter: Don Brinn I am seeing the following error when doing an INNER JOIN of a view with MULTI_TENANT=true with any other table or view: java.lang.RuntimeException: org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.DoNotRetryIOException: Could not find hash cache for joinId: Ys�0��%�. The cache might have expired and have been removed. at org.apache.phoenix.coprocessor.HashJoinRegionScanner.(HashJoinRegionScanner.java:95) at org.apache.phoenix.coprocessor.ScanRegionObserver.doPostScannerOpen(ScanRegionObserver.java:212) at org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:178) at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1931) at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3178) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2078) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94) at java.lang.Thread.run(Thread.java:745) at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73) at sqlline.TableOutputFormat.print(TableOutputFormat.java:33) at sqlline.SqlLine.print(SqlLine.java:1653) at sqlline.Commands.execute(Commands.java:833) at sqlline.Commands.sql(Commands.java:732) at sqlline.SqlLine.dispatch(SqlLine.java:808) at sqlline.SqlLine.begin(SqlLine.java:681) at sqlline.SqlLine.start(SqlLine.java:398) at sqlline.SqlLine.main(SqlLine.java:292) This is with Phoenix version 4.6.0 and HBase version 0.98.4.2.2.6.0-2800-hadoop2. This seems very strongly related to the MULTI_TENANT=true property on a view or table. I see the error whenever the view has MULTI_TENANT=true and I have a tenant-specific connection to Phoenix. I do not see the problem if the MULTI_TENANT=true property is not set on the view or if I do not have a tenant-specific connection to Phoenix. Here is an example SQL statement that has this error when the view INVENTORY has the MULTI_TENANT=true property and I have a tenant-specific connection, but that succeeds in other cases. (The view PRODUCT_IDS is not Multi-Tenant.) SELECT * FROM INVENTORY INNER JOIN PRODUCT_IDS ON (PRODUCT_ID = INVENTORY.ID) Note: "INNER JOIN" fails under these conditions, as does "LEFT OUTER JOIN". However, "RIGHT OUTER JOIN" and "FULL OUTER JOIN" do work. Also, if I tell Phoenix to use a Sort Join for the Inner Join or Left Outer Join then it does work, e.g. SELECT /*+ USE_SORT_MERGE_JOIN*/ * FROM INVENTORY INNER JOIN PRODUCT_IDS ON (PRODUCT_ID = INVENTORY.ID); works. This seems to be the same problem that was discussed previously in this mailing list: https://mail-archives.apache.org/mod_mbox/phoenix-user/201507.mbox/%3ccaotkwx5xfbwkjf--0k-zj91tfdqwfq6rmuqw0r_lojcnj1a...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2380) 'decimal' columns can only hold 'float' values
[
https://issues.apache.org/jira/browse/PHOENIX-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Liew updated PHOENIX-2380:
Summary: 'decimal' columns can only hold 'float' values (was: 'decimal'
columns cannot hold the max 'float' value)
> 'decimal' columns can only hold 'float' values
> --
>
> Key: PHOENIX-2380
> URL: https://issues.apache.org/jira/browse/PHOENIX-2380
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.5.0
>Reporter: Kevin Liew
>
> {noformat}
> CREATE TABLE test.DECIMAL_TABLE(
> KeyColumn VARCHAR(255) PRIMARY KEY,
> Column1 DECIMAL(38, 0));
> {noformat}
> Phoenix will not upsert values larger than or equal to the max float value.
> {noformat}
> upsert into test.decimal_table values ('test', 3.402823466e+38);
> java.sql.SQLException: error while executing SQL "upsert into
> test.decimal_table values ('test', 3.402823466e+38)": response code 500
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:121)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
> at sqlline.SqlLine.begin(SqlLine.java:681)
> at sqlline.SqlLine.start(SqlLine.java:398)
> at sqlline.SqlLine.main(SqlLine.java:292)
> Caused by: java.lang.RuntimeException: response code 500
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:45)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:207)
> at
> org.apache.calcite.avatica.remote.RemoteMeta.prepareAndExecute(RemoteMeta.java:169)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:477)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:109)
> ... 7 more
> 0: jdbc:phoenix:thin:url=http://localhost:876> upsert into test.decimal_table
> values ('MaxFloat', 3.402823466e+37);
> 1 row affected (0.113 seconds)
> {noformat}
> or using the bulk loader tool
> {noformat}
> 15/11/05 23:17:15 ERROR util.CSVCommonsLoader: Error upserting record
> [MaxFloat, 3.402823466e+38]: ERROR 206 (22003): The data exceeds the max
> capacity for the data type. value=34028234660
> columnName=COLUMN1
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-2380) 'decimal' columns can only hold 'float' values
[
https://issues.apache.org/jira/browse/PHOENIX-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Liew updated PHOENIX-2380:
Description:
{noformat}
CREATE TABLE test.DECIMAL_TABLE(
KeyColumn VARCHAR(255) PRIMARY KEY,
Column1 DECIMAL(38, 0));
{noformat}
Phoenix will not upsert values larger than or equal to the max float value.
{noformat}
upsert into test.decimal_table values ('test', 3.402823466e+38);
java.sql.SQLException: error while executing SQL "upsert into
test.decimal_table values ('test', 3.402823466e+38)": response code 500
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112)
at
org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:121)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
Caused by: java.lang.RuntimeException: response code 500
at
org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:45)
at
org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:207)
at
org.apache.calcite.avatica.remote.RemoteMeta.prepareAndExecute(RemoteMeta.java:169)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:477)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:109)
... 7 more
0: jdbc:phoenix:thin:url=http://localhost:876> upsert into test.decimal_table
values ('MaxFloat', 3.402823466e+37);
1 row affected (0.113 seconds)
{noformat}
or using the bulk loader tool
{noformat}
15/11/05 23:17:15 ERROR util.CSVCommonsLoader: Error upserting record
[MaxFloat, 3.402823466e+38]: ERROR 206 (22003): The data exceeds the max
capacity for the data type. value=34028234660
columnName=COLUMN1
{noformat}
This also applies for values smaller than the minimum float value
was:
{noformat}
CREATE TABLE test.DECIMAL_TABLE(
KeyColumn VARCHAR(255) PRIMARY KEY,
Column1 DECIMAL(38, 0));
{noformat}
Phoenix will not upsert values larger than or equal to the max float value.
{noformat}
upsert into test.decimal_table values ('test', 3.402823466e+38);
java.sql.SQLException: error while executing SQL "upsert into
test.decimal_table values ('test', 3.402823466e+38)": response code 500
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112)
at
org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:121)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
Caused by: java.lang.RuntimeException: response code 500
at
org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:45)
at
org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:207)
at
org.apache.calcite.avatica.remote.RemoteMeta.prepareAndExecute(RemoteMeta.java:169)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:477)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:109)
... 7 more
0: jdbc:phoenix:thin:url=http://localhost:876> upsert into test.decimal_table
values ('MaxFloat', 3.402823466e+37);
1 row affected (0.113 seconds)
{noformat}
or using the bulk loader tool
{noformat}
15/11/05 23:17:15 ERROR util.CSVCommonsLoader: Error upserting record
[MaxFloat, 3.402823466e+38]: ERROR 206 (22003): The data exceeds the max
capacity for the data type. value=34028234660
columnName=COLUMN1
{noformat}
> 'decimal' columns can only hold 'float' values
> --
>
> Key: PHOENIX-2380
> URL: https://issues.apache.org/jira/browse/PHOENIX-2380
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.5.0
>Reporter: Kevin Liew
>
> {noformat}
> CREATE TABLE test.DECIMAL_TABLE(
> KeyColumn VARCHAR(255) PRIMARY KEY,
> Column1 DECIMAL(38, 0));
> {noformat}
> Phoenix will not upsert values larger than or equal to the max float value.
> {noformat}
> upsert into test.decimal_table values ('test', 3.402823466e+38);
> java.sql.SQLException: error while executing SQL "upsert in
[jira] [Created] (PHOENIX-2380) 'decimal' columns cannot hold the max 'float' value
Kevin Liew created PHOENIX-2380:
---
Summary: 'decimal' columns cannot hold the max 'float' value
Key: PHOENIX-2380
URL: https://issues.apache.org/jira/browse/PHOENIX-2380
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.5.0
Reporter: Kevin Liew
{noformat}
CREATE TABLE test.DECIMAL_TABLE(
KeyColumn VARCHAR(255) PRIMARY KEY,
Column1 DECIMAL(38, 0));
{noformat}
Phoenix will not upsert values larger than or equal to the max float value.
{noformat}
upsert into test.decimal_table values ('test', 3.402823466e+38);
java.sql.SQLException: error while executing SQL "upsert into
test.decimal_table values ('test', 3.402823466e+38)": response code 500
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:112)
at
org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:121)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
Caused by: java.lang.RuntimeException: response code 500
at
org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:45)
at
org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:207)
at
org.apache.calcite.avatica.remote.RemoteMeta.prepareAndExecute(RemoteMeta.java:169)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:477)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:109)
... 7 more
0: jdbc:phoenix:thin:url=http://localhost:876> upsert into test.decimal_table
values ('MaxFloat', 3.402823466e+37);
1 row affected (0.113 seconds)
{noformat}
or using the bulk loader tool
{noformat}
15/11/05 23:17:15 ERROR util.CSVCommonsLoader: Error upserting record
[MaxFloat, 3.402823466e+38]: ERROR 206 (22003): The data exceeds the max
capacity for the data type. value=34028234660
columnName=COLUMN1
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2299) Support CURRENT_DATE() in Pherf data upserts
[ https://issues.apache.org/jira/browse/PHOENIX-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992625#comment-14992625 ] Cody Marcel commented on PHOENIX-2299: -- Another case where this might get unpredictable is when you have a large data load that spans across the midnight boundary for the server. [~jfernando_sfdc] We could certainly persist a specific date, but how would that work from the query side? I thought wanted to test the CURRENT_DATE() function? Is there an example query we would like to model? > Support CURRENT_DATE() in Pherf data upserts > > > Key: PHOENIX-2299 > URL: https://issues.apache.org/jira/browse/PHOENIX-2299 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Karan Singhal > > Just replace the actual date with "NOW" in the xml. Then check the string for > that value in the generator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2359) Configuration for PQS to use Protobuf serialization instead of JSON
[
https://issues.apache.org/jira/browse/PHOENIX-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated PHOENIX-2359:
Attachment: PHOENIX-2359.002.patch
.002 is a slight improvement on sqlline-thin.py. Using {{hbase}} on the
{{$PATH}}, it invokes HBaseConfTool to extract the value of
{{phoenix.queryserver.serialization}} from hbase-site.xml and automatically
updates the JDBC URL.
If {{hbase}} isn't on the PATH, the value isn't defined (or there is some
problem invoking HBaseConfTool), it defaults to JSON.
> Configuration for PQS to use Protobuf serialization instead of JSON
> ---
>
> Key: PHOENIX-2359
> URL: https://issues.apache.org/jira/browse/PHOENIX-2359
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Trivial
> Fix For: 4.7.0
>
> Attachments: PHOENIX-2359.001.patch, PHOENIX-2359.002.patch
>
>
> Calcite-1.5.0 is soon to drop. Included in that release is some work to use
> protocol buffers for the data transported over HTTP between client and server
> instead of JSON.
> I made some changes in Avatica which hopefully make Phoenix's use of the
> Avatica server easier, in addition to choosing the serialization.
> Let me stage the patch I've been using to test this. We can't apply it until
> we're confident that moving to the newer version of Calcite is safe to do.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2299) Support CURRENT_DATE() in Pherf data upserts
[ https://issues.apache.org/jira/browse/PHOENIX-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992502#comment-14992502 ] Jan Fernando commented on PHOENIX-2299: --- [~cody.mar...@gmail.com] brings up a really great point. I can see this would be very useful when using Pherf to load/seed data but could really be dangerous if calculating query perf stats that uses date filtering. What about if NOW() sets the date at the start of the load, stores it with the scenario and the queries can refer to this calculated date using a special variable? That way the tests would be repeatable and you wouldn't suddenly think your queries were blazing faster because they were returning zero data. > Support CURRENT_DATE() in Pherf data upserts > > > Key: PHOENIX-2299 > URL: https://issues.apache.org/jira/browse/PHOENIX-2299 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Karan Singhal > > Just replace the actual date with "NOW" in the xml. Then check the string for > that value in the generator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-2365) PHOENIX-2248 sub-task: Remove Pherf uber jar and create python scripts to execute from Phoenix bin directory
[
https://issues.apache.org/jira/browse/PHOENIX-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992476#comment-14992476
]
Cody Marcel commented on PHOENIX-2365:
--
This mostly looks really good Mujtaba.
Couple minor things. It looks like your path is hard coded there in the patch.
+hbasecp, stderr =
subprocess.Popen("/home/mchohan/Desktop/hbase/hbase-0.98.13-hadoop2/bin/hbase
classpath",
+ shell=True,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE).communicate()
Also, this changes the way we package and run pherf. Should we also include doc
changes to the site page? Or do you think we should have a separate Jira for
that? I noticed it's out of date in other areas too.
> PHOENIX-2248 sub-task: Remove Pherf uber jar and create python scripts to
> execute from Phoenix bin directory
>
>
> Key: PHOENIX-2365
> URL: https://issues.apache.org/jira/browse/PHOENIX-2365
> Project: Phoenix
> Issue Type: Sub-task
>Reporter: Mujtaba Chohan
>Priority: Minor
> Attachments: PHOENIX-2248.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-2299) Support CURRENT_DATE() in Pherf data upserts
[ https://issues.apache.org/jira/browse/PHOENIX-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992461#comment-14992461 ] Cody Marcel commented on PHOENIX-2299: -- [~karan.singhal] is looking into adding this support. I wanted to clarify a little as to the goal. Please correct me if I am wrong. I think what's intended here is on upsert we take the present time instead of a static or completely random time for DATE types. Then on queries where CURRENT_DATE is used, we get get back some data. If this is correct, I think the impl is pretty straight forward. It does have some potentially odd side effects though. The query results will differ based on when you run them. So if you upsert data today, it will be stale tomorrow. It will also means tests using this functionality will not be repeatable. You will have to do the "drop table ->load data -> query" process every time with new values each time. [~mujtabachohan] [~samarth.j...@gmail.com] > Support CURRENT_DATE() in Pherf data upserts > > > Key: PHOENIX-2299 > URL: https://issues.apache.org/jira/browse/PHOENIX-2299 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Karan Singhal > > Just replace the actual date with "NOW" in the xml. Then check the string for > that value in the generator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1706) Create skeleton for parsing DDL
[ https://issues.apache.org/jira/browse/PHOENIX-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1706: - Labels: calcite (was: ) > Create skeleton for parsing DDL > --- > > Key: PHOENIX-1706 > URL: https://issues.apache.org/jira/browse/PHOENIX-1706 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Julian Hyde > Labels: calcite > > Phoenix would like to leverage the Calcite parser, so would like to have the > ability to parse the following DDL statements. The current work for this is > occurring in the calcite branch. > CREATE TABLE: http://phoenix.apache.org/language/index.html#create_table > CREATE VIEW: http://phoenix.apache.org/language/index.html#create_view > CREATE INDEX: http://phoenix.apache.org/language/index.html#create_index > CREATE SEQUENCE: http://phoenix.apache.org/language/index.html#create_sequence > ALTER TABLE/VIEW: http://phoenix.apache.org/language/index.html#alter > ALTER INDEX: http://phoenix.apache.org/language/index.html#alter_index > DROP TABLE: http://phoenix.apache.org/language/index.html#drop_table > DROP VIEW: http://phoenix.apache.org/language/index.html#drop_view > DROP INDEX: http://phoenix.apache.org/language/index.html#drop_index > DROP SEQUENCE: http://phoenix.apache.org/language/index.html#drop_sequence > UPDATE STATISTICS: > http://phoenix.apache.org/language/index.html#update_statistics > TRACE ON/OFF [WITH SAMPLING ] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1789) Implement conversion from Calcite RexNode to Phoenix Expression
[ https://issues.apache.org/jira/browse/PHOENIX-1789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1789: - Labels: calcite (was: ) > Implement conversion from Calcite RexNode to Phoenix Expression > --- > > Key: PHOENIX-1789 > URL: https://issues.apache.org/jira/browse/PHOENIX-1789 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue > Labels: calcite > Attachments: failures.txt, runs.txt > > Original Estimate: 480h > Remaining Estimate: 480h > > See CalciteUtils.ExpressionFactory and ToExpressionTest (on 'calcite' > branch) for examples. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2163) Measure performance of Phoenix/Calcite querying
[ https://issues.apache.org/jira/browse/PHOENIX-2163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-2163: - Labels: calcite (was: ) > Measure performance of Phoenix/Calcite querying > --- > > Key: PHOENIX-2163 > URL: https://issues.apache.org/jira/browse/PHOENIX-2163 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Shuxiong Ye > Labels: calcite > Attachments: PHOENIX-2163.patch, PhoenixRegressor.log, > calcite-test-mac.tar.gz, hbase-logs.7167262.tar.gz, publish.7167262.tar.gz > > > The work to integrate Phoenix with Calcite has come along far enough that > queries both against the data table and through a secondary index is > functional. As a checkpoint, we should compare performance of as many queries > as possible in our regression suite for the calcite branch against the latest > Phoenix release (4.5.0). The runtime of these two systems should be the same, > so this will give us an idea of the overhead of query parsing and compilation > for Calcite. This is super important, as it'll identify outstanding work > that'll be necessary to do prior to any releases on top of this new stack. > Source code of regression suite is at > https://github.com/mujtabachohan/PhoenixRegressor > Connection string location: > https://github.com/mujtabachohan/PhoenixRegressor/blob/master/src/main/resources/settings.json > Instructions on how to compile and run: > https://github.com/mujtabachohan/PhoenixRegressor/blob/master/README.md -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-1784) Port existing Phoenix optimizations to Phoenix/Calcite integration
[ https://issues.apache.org/jira/browse/PHOENIX-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde updated PHOENIX-1784: - Labels: calcite (was: ) > Port existing Phoenix optimizations to Phoenix/Calcite integration > -- > > Key: PHOENIX-1784 > URL: https://issues.apache.org/jira/browse/PHOENIX-1784 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue > Labels: calcite > Original Estimate: 2,160h > Remaining Estimate: 2,160h > > A lot of optimizations are already included in Calcite default rules, but > there are still a few Phoenix-specific high-level optimizations that we need > to implement as rules and make sure they are reflected in the tree's cost. > A simple example can be PhoenixFilterTableScanMergeRule, which pushes a > Filter operator into a PhoenixTableScan so that the filter can be converted > to an HBase Filter or more optimally a SkipScanFilter or even a point look-up. > Please refer to sub-tasks for individual optimizations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-1850) Implement PhoenixTable.getStatistics() in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue resolved PHOENIX-1850. -- Resolution: Fixed > Implement PhoenixTable.getStatistics() in Phoenix/Calcite Integration > - > > Key: PHOENIX-1850 > URL: https://issues.apache.org/jira/browse/PHOENIX-1850 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue > Original Estimate: 240h > Remaining Estimate: 240h > > Table statistics mainly include row count, collation and distribution. We > will try to get this information for existing Phoenix statistics and > implement the methods accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-1997) Join optimization: Apply one table's where condition to the others
[
https://issues.apache.org/jira/browse/PHOENIX-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992275#comment-14992275
]
Julian Hyde commented on PHOENIX-1997:
--
Calcite already does this: JoinPushTransitivePredicatesRule
> Join optimization: Apply one table's where condition to the others
> --
>
> Key: PHOENIX-1997
> URL: https://issues.apache.org/jira/browse/PHOENIX-1997
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.3.1
>Reporter: Taeyun Kim
>Priority: Minor
>
> Joined tables are as follows:
> {noformat}
> create table table_a
> (
> time_id integer not null,
> depth tinyint not null,
> id0 integer not null,
> id1 integer not null,
> id2 integer not null,
> id3 integer not null,
> id integer not null,
> record_type smallint not null,
> c varbinary
> constraint pk primary key(time_id, depth, id0, id1, id2, id3, id,
> record_type)
> )
> salt_buckets=4,
> compression='SNAPPY',
> create table table_b
> (
> depth tinyint not null,
> id0 integer not null,
> id1 integer not null,
> id integer not null,
> c varbinary,
> p varbinary
> constraint pk primary key(depth, id0, id1, id)
> )
> salt_buckets=2,
> compression='SNAPPY';
> create index table_b_index on table_b (id, depth, id0, id1)
> compression='SNAPPY';
> {noformat}
> The query is as follows:
> {noformat}
> select a.*, b.c
> from table_a a inner join table_b b on (a.depth = b.depth and a.id0 = b.id0
> and a.id1 = b.id1 and a.id2 = b.id)
> where a.time_id = 23796900 and a.depth = 1;
> {noformat}
> It is obvious that b.depth must also be 1 since it's on the join condition.
> And since the depth column is the first primary key column of table_b,
> table_b should be range scanned before join.
> But the query explanation is as follows:
> {noformat}
> CLIENT PARALLEL 4-WAY RANGE SCAN OVER TABLE_A [0,23796900,1]
> CLIENT MERGE SORT
> PARALLEL INNER-JOIN TABLE 0
> CLIENT PARALLEL 2-WAY FULL SCAN OVER TABLE_B
> CLIENT MERGE SORT
> {noformat}
> But when (b.depth = 1) condition is explicitly added to the query, the
> explanation is changed as the expected one:
> {noformat}
> CLIENT PARALLEL 4-WAY RANGE SCAN OVER TABLE_A [0,23796900,1]
> CLIENT MERGE SORT
> PARALLEL INNER-JOIN TABLE 0
> CLIENT PARALLEL 2-WAY RANGE SCAN OVER TABLE_B [0,1]
> CLIENT MERGE SORT
> {noformat}
> It would be nice if the optimizer could find this condition dependency and
> apply it to the query plan.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-1556) Base hash join versus many-to-many decision on how many guideposts will be traversed for RHS table(s)
[ https://issues.apache.org/jira/browse/PHOENIX-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated PHOENIX-1556: - Labels: calcite (was: ) > Base hash join versus many-to-many decision on how many guideposts will be > traversed for RHS table(s) > - > > Key: PHOENIX-1556 > URL: https://issues.apache.org/jira/browse/PHOENIX-1556 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maryann Xue > Labels: calcite > > At compile time, we know how many guideposts (i.e. how many bytes) will be > scanned for the RHS table. We should, by default, base the decision of using > the hash-join verus many-to-many join on this information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-1548) Reduce number of arguments for local index join fix
[ https://issues.apache.org/jira/browse/PHOENIX-1548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue resolved PHOENIX-1548. -- Resolution: Fixed > Reduce number of arguments for local index join fix > --- > > Key: PHOENIX-1548 > URL: https://issues.apache.org/jira/browse/PHOENIX-1548 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maryann Xue >Priority: Trivial > > See comment here: > https://issues.apache.org/jira/browse/PHOENIX-1535?focusedCommentId=14253018&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14253018 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-1836) Implement PhoenixAggregate in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue resolved PHOENIX-1836. -- Resolution: Fixed > Implement PhoenixAggregate in Phoenix/Calcite Integration > - > > Key: PHOENIX-1836 > URL: https://issues.apache.org/jira/browse/PHOENIX-1836 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue >Assignee: Maryann Xue > Original Estimate: 480h > Remaining Estimate: 480h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-1837) Detect ordered/unordered group-by
[ https://issues.apache.org/jira/browse/PHOENIX-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue resolved PHOENIX-1837. -- Resolution: Fixed > Detect ordered/unordered group-by > - > > Key: PHOENIX-1837 > URL: https://issues.apache.org/jira/browse/PHOENIX-1837 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Original Estimate: 120h > Remaining Estimate: 120h > > With ordered group-by, we would do much less work. The existing Phoenix code > depends on TrackOrderPreservingExpressionCompiler to calculate the flag. But > given that we would no longer use any of the Phoenix parse tree nodes and the > TrackOrderPreservingExpressionCompiler is a parse tree node visitor, we > should now implement the same logic with Calcite RexNode. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2377) Use a priority queue in MergeSortResultIterator
[ https://issues.apache.org/jira/browse/PHOENIX-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal updated PHOENIX-2377: --- Attachment: (was: PHOENIX-2377_v1.patch) > Use a priority queue in MergeSortResultIterator > --- > > Key: PHOENIX-2377 > URL: https://issues.apache.org/jira/browse/PHOENIX-2377 > Project: Phoenix > Issue Type: Sub-task >Reporter: Ankit Singhal >Priority: Minor > Attachments: PHOENIX-2377_v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2377) Use a priority queue in MergeSortResultIterator
[ https://issues.apache.org/jira/browse/PHOENIX-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal updated PHOENIX-2377: --- Attachment: PHOENIX-2377_v1.patch > Use a priority queue in MergeSortResultIterator > --- > > Key: PHOENIX-2377 > URL: https://issues.apache.org/jira/browse/PHOENIX-2377 > Project: Phoenix > Issue Type: Sub-task >Reporter: Ankit Singhal >Priority: Minor > Attachments: PHOENIX-2377_v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-2078) Non-Phoenix convention Rel appear as a child of Phoenix Rel after application of XXXTransposeRule or other similar rules
[ https://issues.apache.org/jira/browse/PHOENIX-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue resolved PHOENIX-2078. -- Resolution: Fixed > Non-Phoenix convention Rel appear as a child of Phoenix Rel after application > of XXXTransposeRule or other similar rules > > > Key: PHOENIX-2078 > URL: https://issues.apache.org/jira/browse/PHOENIX-2078 > Project: Phoenix > Issue Type: Bug >Reporter: Maryann Xue >Assignee: Maryann Xue > Original Estimate: 120h > Remaining Estimate: 120h > > For example, in the middle of the planner opt process, we have a rel subtree > like: > LogicalFilter > PhoenixServerProject > some_phoenix_rel > And it can be matched against FilterTransposeRule, which will produce a new > rel like > PhoenixServerProject > LogicalFilter > some_phoenix_rel > This would turn out to be issue if the LogicalFilter gets converted into a > RelNode of some other convention later (for example, Enumerable), which means > an Enumerable Rel could appear as a child of a Phoenix Rel which is actually > not implementable. (In future we might be able to convert an Enumerable or > JDBC or some other Rel into a PhoenixRel, but there should always be a > converter Rel there.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-1857) Implement Limit in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue resolved PHOENIX-1857. -- Resolution: Fixed > Implement Limit in Phoenix/Calcite Integration > -- > > Key: PHOENIX-1857 > URL: https://issues.apache.org/jira/browse/PHOENIX-1857 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue >Assignee: Maryann Xue > Original Estimate: 240h > Remaining Estimate: 240h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-1860) Implement PhoenixLimitPushThroughJoinRule in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue resolved PHOENIX-1860. -- Resolution: Fixed > Implement PhoenixLimitPushThroughJoinRule in Phoenix/Calcite Integration > > > Key: PHOENIX-1860 > URL: https://issues.apache.org/jira/browse/PHOENIX-1860 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Original Estimate: 72h > Remaining Estimate: 72h > > The OUTER end of the join will have an additional Limit RelNode above while > the original Limit over Join node should still remain. We need to maintain a > flag or something in order to prevent the rule to fired again for the same > node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-1866) Implement Sort/Limit Merge Rules in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue resolved PHOENIX-1866. -- Resolution: Fixed > Implement Sort/Limit Merge Rules in Phoenix/Calcite Integration > --- > > Key: PHOENIX-1866 > URL: https://issues.apache.org/jira/browse/PHOENIX-1866 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Original Estimate: 72h > Remaining Estimate: 72h > > Sort over Sort-without-fetch --> the sort below can be removed. > Limit over Sort-without-fetch --> can be merged into Sort-with-fetch. > Limit over Sort-with-fetch/Limit --> can be merged into one Sort/Limit with > the smaller limit if both limit nodes are stateless. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-2193) Add rules to push down Sort through Union
[ https://issues.apache.org/jira/browse/PHOENIX-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue resolved PHOENIX-2193. -- Resolution: Fixed > Add rules to push down Sort through Union > - > > Key: PHOENIX-2193 > URL: https://issues.apache.org/jira/browse/PHOENIX-2193 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > > Sort will be pushed through Union to its inputs, and meanwhile the original > sort will become a merge-sort attribute in the PhoenixUnion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-2191) Union Support in Phoenix/Calcite Integration
[ https://issues.apache.org/jira/browse/PHOENIX-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue resolved PHOENIX-2191. -- Resolution: Fixed > Union Support in Phoenix/Calcite Integration > > > Key: PHOENIX-2191 > URL: https://issues.apache.org/jira/browse/PHOENIX-2191 > Project: Phoenix > Issue Type: Task >Reporter: Maryann Xue >Assignee: Maryann Xue > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-2167) Add new interface in QueryPlan for pushing down a limit value.
[ https://issues.apache.org/jira/browse/PHOENIX-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue resolved PHOENIX-2167. -- Resolution: Fixed > Add new interface in QueryPlan for pushing down a limit value. > -- > > Key: PHOENIX-2167 > URL: https://issues.apache.org/jira/browse/PHOENIX-2167 > Project: Phoenix > Issue Type: Sub-task >Reporter: Maryann Xue >Assignee: Maryann Xue > Attachments: PHOENIX-2167.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > Calcite rel trees are compiled bottom-up into Phoenix QueryPlans, the easiest > way to implement a Limit RelNode in the tree is to create a ClientScanPlan > with the limit value and wrap it around the inner plan. However, oftentimes > this may not be efficient. > For example, select * from T limit 10, the original Phoenix compiler would > generate a ScanPlan with limit 10, apply a PageFilter and avoid using > sophisticated or expensive ResultIterators. > So it would make sense to push down the limit to a QueryPlan as low as > possible and create a new copy of the plan based on this rule. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2372) PhoenixResultSet.getDate(int, Calendar) causes NPE on a null value
[
https://issues.apache.org/jira/browse/PHOENIX-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated PHOENIX-2372:
Attachment: PHOENIX-2372.003.patch
003 Fixes some minor indentation issues :)
> PhoenixResultSet.getDate(int, Calendar) causes NPE on a null value
> --
>
> Key: PHOENIX-2372
> URL: https://issues.apache.org/jira/browse/PHOENIX-2372
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.4.0, 4.5.0, 4.6.0
>Reporter: Josh Elser
>Assignee: Josh Elser
> Attachments: PHOENIX-2372.001.patch, PHOENIX-2372.002.patch,
> PHOENIX-2372.003.patch
>
>
> Ran a simple query through PQS:
> {code}
> select * from system.stats;
> {code}
> and got back a stack trace (trimmed for relevance)
> {noformat}
> java.lang.NullPointerException
> at java.util.Calendar.setTime(Calendar.java:1770)
> at
> org.apache.phoenix.jdbc.PhoenixResultSet.getDate(PhoenixResultSet.java:377)
> at
> org.apache.calcite.avatica.jdbc.JdbcResultSet.getValue(JdbcResultSet.java:172)
> at
> org.apache.calcite.avatica.jdbc.JdbcResultSet.frame(JdbcResultSet.java:142)
> {noformat}
> It looks like the {{getDate(int, Calendar)}} method on PhoenixResultSet
> doesn't check the value before passing it into the calendar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (PHOENIX-2263) Secondary index is not used in join queries in Phoenix/Calcite integration
[ https://issues.apache.org/jira/browse/PHOENIX-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue resolved PHOENIX-2263. -- Resolution: Fixed > Secondary index is not used in join queries in Phoenix/Calcite integration > -- > > Key: PHOENIX-2263 > URL: https://issues.apache.org/jira/browse/PHOENIX-2263 > Project: Phoenix > Issue Type: Bug >Reporter: Maryann Xue >Assignee: Maryann Xue > Labels: calcite > > This is not a bug in Phoenix, but is caused by CALCITE-890 and CALCITE-891. > This does not happen with all join queries. For example, problem might occur > with "select * from a join b on a.id = b.id" or "select * from a, b where > a.id = b.id and a.col0 > 100", while it might work fine with "select * from a > join b on a.id = b.id and a.col0 > 100". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2372) PhoenixResultSet.getDate(int, Calendar) causes NPE on a null value
[
https://issues.apache.org/jira/browse/PHOENIX-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated PHOENIX-2372:
Attachment: PHOENIX-2372.002.patch
.002 changes..
* getBigDecimal(int, int) avoids another NPE (despite the method being
deprecated via java.sql.ResultSet
* Adds a quick test for the above
* Sets wasNull in getDate(int,Calendar)
> PhoenixResultSet.getDate(int, Calendar) causes NPE on a null value
> --
>
> Key: PHOENIX-2372
> URL: https://issues.apache.org/jira/browse/PHOENIX-2372
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.4.0, 4.5.0, 4.6.0
>Reporter: Josh Elser
>Assignee: Josh Elser
> Attachments: PHOENIX-2372.001.patch, PHOENIX-2372.002.patch
>
>
> Ran a simple query through PQS:
> {code}
> select * from system.stats;
> {code}
> and got back a stack trace (trimmed for relevance)
> {noformat}
> java.lang.NullPointerException
> at java.util.Calendar.setTime(Calendar.java:1770)
> at
> org.apache.phoenix.jdbc.PhoenixResultSet.getDate(PhoenixResultSet.java:377)
> at
> org.apache.calcite.avatica.jdbc.JdbcResultSet.getValue(JdbcResultSet.java:172)
> at
> org.apache.calcite.avatica.jdbc.JdbcResultSet.frame(JdbcResultSet.java:142)
> {noformat}
> It looks like the {{getDate(int, Calendar)}} method on PhoenixResultSet
> doesn't check the value before passing it into the calendar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
