[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369975#comment-16369975 ] Rajeshbabu Chintaguntla commented on PHOENIX-4531: -- Committed to 5.x branch. [~jamestaylor] can we resolve this? > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4531.v1.master.patch, > PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, > PHOENIX-4531.v5.master.patch, PHOENIX-4531_5.x-HBase-2.0.patch, > PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369968#comment-16369968 ] Rajeshbabu Chintaguntla commented on PHOENIX-4531: -- [~jamestaylor] This requires PHOENIX-4386 in 5.x branch which I have committed and uploaded patch on top of 5.x branch. Going to commit it. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4531.v1.master.patch, > PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, > PHOENIX-4531.v5.master.patch, PHOENIX-4531_5.x-HBase-2.0.patch, > PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334878#comment-16334878 ] Hudson commented on PHOENIX-4531: - FAILURE: Integrated in Jenkins build Phoenix-master #1917 (See [https://builds.apache.org/job/Phoenix-master/1917/]) PHOENIX-4531 Delete on a table with a global mutable index can issue (vincentpoon: rev 195f82bae2835a6bfc9481205bad7e8f21944471) * (edit) phoenix-core/src/main/java/org/apache/phoenix/compile/DeleteCompiler.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/optimize/QueryOptimizer.java * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/BaseIndexIT.java * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/PartialIndexRebuilderIT.java * (edit) phoenix-core/src/test/java/org/apache/phoenix/compile/QueryOptimizerTest.java > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4531.v1.master.patch, > PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, > PHOENIX-4531.v5.master.patch, PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, > PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334861#comment-16334861 ] Josh Elser commented on PHOENIX-4531: - Also, I have it on my list to go back through the changes and get things synced up again between the 4.x and 5.x stuff. One more tacked onto that list isn't the end of the world. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4531.v1.master.patch, > PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, > PHOENIX-4531.v5.master.patch, PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, > PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334858#comment-16334858 ] Josh Elser commented on PHOENIX-4531: - cc/ [~sergey.soldatov] [~rajeshbabu] > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4531.v1.master.patch, > PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, > PHOENIX-4531.v5.master.patch, PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, > PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334853#comment-16334853 ] James Taylor commented on PHOENIX-4531: --- Thanks, [~vincentpoon]. Looks like 4.x-HBase-1.1 was missing some previous commits. I've caught it up and cherry-picked your patch there. We'll may need some help from [~elserj] & team for the 5.x branch. I don't have a good idea of what's missing there since they created the branch. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4531.v1.master.patch, > PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, > PHOENIX-4531.v5.master.patch, PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, > PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334817#comment-16334817 ] Vincent Poon commented on PHOENIX-4531: --- [~jamestaylor] could you create a version of your patch for 5.x-HBase-2.0, 4.x-cdh5.11.2 and 4.x-HBase-1.1 ? There are merge conflicts in DeleteCompiler. If you can resolve those, I can commit to those branches I've already committed to master, 4.x-HBase-1.3, 4.x-HBase-1.2, 4.x-HBase-0.98 Thanks! > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Attachments: PHOENIX-4531.v1.master.patch, > PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, > PHOENIX-4531.v5.master.patch, PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, > PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333101#comment-16333101 ] Vincent Poon commented on PHOENIX-4531: --- Attached a v5, where if we're comparing two index tables, we still consider the size of the tables in the comparison. This fixes a test failure in CostBasedDecisionIT Will commit on Monday unless any objections > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Attachments: PHOENIX-4531.v1.master.patch, > PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, > PHOENIX-4531.v5.master.patch, PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, > PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332856#comment-16332856 ] James Taylor commented on PHOENIX-4531: --- +1. Thanks, [~vincentpoon]. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Attachments: PHOENIX-4531.v1.master.patch, > PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, > PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332841#comment-16332841 ] Vincent Poon commented on PHOENIX-4531: --- v4 patch with a test that a DELETE statement without WHERE clause favors the data table. Including a WHERE clause favors the index. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Attachments: PHOENIX-4531.v1.master.patch, > PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, > PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331646#comment-16331646 ] James Taylor commented on PHOENIX-4531: --- Nice work, [~vincentpoon]. Thanks for tracking that down. Patch looks good. Minor nit: if it's not too difficult, would be good to have a QueryOptimizerTest that with an unhinted DELETE statement that favors the data table over the index table. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Attachments: PHOENIX-4531.v1.master.patch, > PHOENIX-4531.v3.master.patch, PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, > PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331630#comment-16331630 ] Vincent Poon commented on PHOENIX-4531: --- [~jamestaylor] The hint was not using the data table because the optimizer chooses the smaller table (fewest non-pk columns), which is generally the index. It also turns out that the logic was flipped for interpreting the hint. I attached a v3 patch that has your patch plus: * -Changes to QueryOptimizer that resolves the above. I only compare table size for queries that don't have a DATA_OVER_INDEX hint. The rest of the logic stays the same such that a WHERE clause would cause the index to still be used. * Added a test in QueryOptimizerTest * Added my test in PartialIndexRebuilderIT to test that the index gets disabled properly when there's a write failure to the index when issuing deletes. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Attachments: PHOENIX-4531.v1.master.patch, > PHOENIX-4531.v3.master.patch, PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, > PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[
https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331460#comment-16331460
]
James Taylor commented on PHOENIX-4531:
---
The {{runOnServer}} variable gets set to false if the index table is chosen as
the best table to drive the delete from. If it was kept as true, then the data
table would be used. If you have a WHERE clause in the delete that turns into a
range scan when the index is used, this is certainly going to be better than
doing a full table scan using the data table. The case I'm not sure about is if
there's no where clause (or the same number of rows would be scanned for the
index or data table). I'm not sure which is better (but probably using the data
table since as you mention the parallelization will be better).
I think It needs further investigation as to why the hint didn't cause the data
table to be chosen as the best plan.
> Delete on a table with a global mutable index can issue client-side deletes
> against the index
> -
>
> Key: PHOENIX-4531
> URL: https://issues.apache.org/jira/browse/PHOENIX-4531
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.0
> Environment:
>Reporter: Vincent Poon
>Assignee: Vincent Poon
>Priority: Major
> Attachments: PHOENIX-4531.v1.master.patch, PHOENIX-4531_v1.patch,
> PHOENIX-4531_v2.patch, PartialIndexRebuilderIT.java
>
>
> For a table with a global mutable index, I found the following result in
> client-side deletes against both the data table and index table.
> "DELETE FROM data_table"
> "DELETE FROM data_table WHERE indexed_col='v'"
> We only need the delete to be issued against the data table, because
> 1) It's redundant since a delete against the index will be issued on the
> server side when we process the delete of the data table row
> 2) Deletes issued from the client-side won't have the index failure policy
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331433#comment-16331433 ] Vincent Poon commented on PHOENIX-4531: --- [~jamestaylor] My understanding was that we would could run it on server side because we trigger a scan over the data table, which then goes through UngroupedAggregateRegionObserver, which then issues mutates against the data table, which then causes the index updates to be issued via postBatchMutateIndispensably That appears to be the case, because if you look at ServerSelectDeleteMutation, it uses the dataPlan, not the optimized queryPlan (which would use the index). When I try my patch which makes runOnServer=true, it does the right thing. I'm not sure which approach is the best, but I thought we wanted to do it server-side for better parallelization, by issuing the delete scans in parallel with each table region concurrently deleting its own rows. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Attachments: PHOENIX-4531.v1.master.patch, PHOENIX-4531_v1.patch, > PHOENIX-4531_v2.patch, PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331217#comment-16331217 ] James Taylor commented on PHOENIX-4531: --- Attached v2 patch with more comments and some cleanup to code that determines the update count returned to the client. Looks like our pre-commit is busted (or on vacation). I suspect it's due to the move to HBase 1.4 on master (but it could be coincidence). Please review, [~vincentpoon]. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Attachments: PHOENIX-4531.v1.master.patch, PHOENIX-4531_v1.patch, > PHOENIX-4531_v2.patch, PartialIndexRebuilderIT.java > > > For a table with a global mutable index, I found the following result in > client-side deletes against both the data table and index table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[
https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331138#comment-16331138
]
James Taylor commented on PHOENIX-4531:
---
If the scan run for the delete is over an index table, we can't (currently) run
that on the server side because we don't have the code that'll translate from
the index row key to the data row key on the server side. In theory, if the
index is local or global and mutable, we could run it on the server side if
this code was there.
We do attempt to favor using the data table over the index table if all things
are equal from the optimizers POV. That's what this code a little above in
DeleteCompiler should do:
{code}
if (runOnServer &&
!delete.getHint().hasHint(Hint.USE_INDEX_OVER_DATA_TABLE)) {
select = SelectStatement.create(select, HintNode.create(hint,
Hint.USE_DATA_OVER_INDEX_TABLE));
}
{code}
I'd expect that hint to cause the data table, not the index table to be used if
there's no WHERE clause. Perhaps this isn't working as expected?
It's not entirely clear what the best choice is, though. If the index table is
much smaller in size than the data table, is it better to execute the scan
against the index table, pull back over the row key and issue the deletes from
the client? Or is it better to execute the delete on the server side (though
it'd be a cross RS call)?
WDYT? Maybe if this is pursued it should be a separate JIRA, though?
> Delete on a table with a global mutable index can issue client-side deletes
> against the index
> -
>
> Key: PHOENIX-4531
> URL: https://issues.apache.org/jira/browse/PHOENIX-4531
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.0
> Environment:
>Reporter: Vincent Poon
>Assignee: Vincent Poon
>Priority: Major
> Attachments: PHOENIX-4531.v1.master.patch, PHOENIX-4531_v1.patch,
> PartialIndexRebuilderIT.java
>
>
> For a table with a global mutable index, I found the following result in
> client-side deletes against both the data table and index table.
> "DELETE FROM data_table"
> "DELETE FROM data_table WHERE indexed_col='v'"
> We only need the delete to be issued against the data table, because
> 1) It's redundant since a delete against the index will be issued on the
> server side when we process the delete of the data table row
> 2) Deletes issued from the client-side won't have the index failure policy
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[
https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331091#comment-16331091
]
Vincent Poon commented on PHOENIX-4531:
---
[~jamestaylor] your patch works in terms of preventing any mutations against
the index, but if I issue a "DELETE FROM data_table", it uses uses
ClientSelectDeleteMutationPlan, instead of ServerSelectDeleteMutationPlan. Is
that what we want?
You're right that we have the initialization of runOnServer that checks for
hasImmutableIndexes, but again, after the initialization, we run:
{code:java}
runOnServer &= queryPlans.get(0).getTableRef().getTable().getType() !=
PTableType.INDEX; {code}
So if the query optimizer came back with a plan to use the index table, even
for mutable indexes, runOnServer becomes false.
> Delete on a table with a global mutable index can issue client-side deletes
> against the index
> -
>
> Key: PHOENIX-4531
> URL: https://issues.apache.org/jira/browse/PHOENIX-4531
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.0
> Environment:
>Reporter: Vincent Poon
>Assignee: Vincent Poon
>Priority: Major
> Attachments: PHOENIX-4531.v1.master.patch, PHOENIX-4531_v1.patch,
> PartialIndexRebuilderIT.java
>
>
> For a table with a global mutable index, I found the following result in
> client-side deletes against both the data table and index table.
> "DELETE FROM data_table"
> "DELETE FROM data_table WHERE indexed_col='v'"
> We only need the delete to be issued against the data table, because
> 1) It's redundant since a delete against the index will be issued on the
> server side when we process the delete of the data table row
> 2) Deletes issued from the client-side won't have the index failure policy
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[
https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329123#comment-16329123
]
James Taylor commented on PHOENIX-4531:
---
There is an issue. If the delete is being driven from a mutable index, it looks
like we're sending the mutations for both the table and the index. Would you
mind trying my patch, [~vincentpoon]?
As far as runOnServer, we initialize it to false if the table has immutable
indexes here:
{code}
boolean runOnServer = isAutoCommit && !hasPreOrPostProcessing &&
!table.isTransactional() && !hasImmutableIndexes;
{code}
Are you seeing runOnServer as true when there are immutable indexes?
> Delete on a table with a global mutable index can issue client-side deletes
> against the index
> -
>
> Key: PHOENIX-4531
> URL: https://issues.apache.org/jira/browse/PHOENIX-4531
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.0
> Environment: For a table with a global mutable index, I found the
> following result in client-side deletes against both the data table and index
> table.
> "DELETE FROM data_table"
> "DELETE FROM data_table WHERE indexed_col='v'"
> We only need the delete to be issued against the data table, because
> 1) It's redundant since a delete against the index will be issued on the
> server side when we process the delete of the data table row
> 2) Deletes issued from the client-side won't have the index failure policy
>Reporter: Vincent Poon
>Assignee: Vincent Poon
>Priority: Major
> Attachments: PHOENIX-4531.v1.master.patch, PHOENIX-4531_v1.patch,
> PartialIndexRebuilderIT.java
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328118#comment-16328118 ] Vincent Poon commented on PHOENIX-4531: --- [~jamestaylor] Can you review the fix? I think we only don't want to runOnServer if it's an immutable index. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: For a table with a global mutable index, I found the > following result in client-side deletes against both the data table and index > table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Attachments: PHOENIX-4531.v1.master.patch, > PartialIndexRebuilderIT.java > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328086#comment-16328086 ] Vincent Poon commented on PHOENIX-4531: --- This is something more general. I just put the test in PartialIndexRebuilderIT because it's a very specific test, and I don't think we need it to run for all iterations in MutableIndexFailureIT. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: For a table with a global mutable index, I found the > following result in client-side deletes against both the data table and index > table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Attachments: PartialIndexRebuilderIT.java > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328060#comment-16328060 ] James Taylor commented on PHOENIX-4531: --- Is this specifically when an index is getting partially rebuilt after a write failure occurs? Or is it something more general? > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: For a table with a global mutable index, I found the > following result in client-side deletes against both the data table and index > table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Attachments: PartialIndexRebuilderIT.java > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328053#comment-16328053 ] Vincent Poon commented on PHOENIX-4531: --- [~jamestaylor] attached a test "testIndexFailureWithinRSDisablesIndex", can you take a look and see if it's a valid test? I think if everything's working correctly, the index should get disabled. Also when I debug, as I step through I see we don't runOnServer because of this line: runOnServer &= queryPlans.get(0).getTableRef().getTable().getType() != PTableType.INDEX; > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: For a table with a global mutable index, I found the > following result in client-side deletes against both the data table and index > table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Attachments: PartialIndexRebuilderIT.java > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328049#comment-16328049 ] James Taylor commented on PHOENIX-4531: --- Note too, that the client must be on 4.13 for this to work correctly as a number of bugs were fixed in this area in the 4.13 release. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: For a table with a global mutable index, I found the > following result in client-side deletes against both the data table and index > table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328040#comment-16328040 ] James Taylor commented on PHOENIX-4531: --- Are you sure this is the case, [~vincentpoon]? There's no declaration of the table as being IMMUTABLE? I did a quick test on this and I didn't see the issue. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: For a table with a global mutable index, I found the > following result in client-side deletes against both the data table and index > table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index
[ https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328032#comment-16328032 ] Vincent Poon commented on PHOENIX-4531: --- This is also problematic because the delete against the index table is issued first, so if there's something goes wrong with the delete, the entire statement fails with an exception, and the failure policy is never activated so the index remains active. > Delete on a table with a global mutable index can issue client-side deletes > against the index > - > > Key: PHOENIX-4531 > URL: https://issues.apache.org/jira/browse/PHOENIX-4531 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 > Environment: For a table with a global mutable index, I found the > following result in client-side deletes against both the data table and index > table. > "DELETE FROM data_table" > "DELETE FROM data_table WHERE indexed_col='v'" > We only need the delete to be issued against the data table, because > 1) It's redundant since a delete against the index will be issued on the > server side when we process the delete of the data table row > 2) Deletes issued from the client-side won't have the index failure policy >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
