IgGusev commented on code in PR #6553: URL: https://github.com/apache/ignite-3/pull/6553#discussion_r2343247014
########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + Review Comment: What are colocation columns? This is a new term and we should explain it. We use colocated columns later, are they the same thing? ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + Review Comment: ```suggestion * DML queries, which can **not** be rewritten to key-value operations, are not supported yet. + ``` ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + Review Comment: Generally its best to avoid passive voice in documentation. We can also drop mandatory requirement, as requirements are implicitly mandatory. As a suggestion: Apache Ignite can only apply partition awareness optimization if the SQL query contains an equality condition on all colocated columns. This requirement allows the client to route the query to the node where the partition with the data resides. ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. Review Comment: ```suggestion Partition awareness will only work for DML queries if it the query be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. ``` ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. +Bulk DML operations (like ```INSERT FROM SELECT``` or multi-values ```INSERT FROM VALUES```) are not supported yet. + +* Partition awareness cache on client may miss required metadata + +Client node do not parse or executed query by itself, it require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + +See ```org.apache.ignite.client.IgniteClient.Builder#sqlPartitionAwarenessMetadataCacheSize()```. + +NOTE: As for now, Partition awareness is supported for queries over single partition of a single table. + +=== How to Check if Partition Awareness is Applicable + +You can verify whether partition awareness is used for a given SQL query by inspecting the EXPLAIN command results. + +* The resulting plan should contain a `TableScan` node with equality conditions on all colocation columns of the table. Review Comment: ```suggestion * The resulting plan should contain a `TableScan` node with equality conditions on all colocation columns of the table: ``` ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. +Bulk DML operations (like ```INSERT FROM SELECT``` or multi-values ```INSERT FROM VALUES```) are not supported yet. + +* Partition awareness cache on client may miss required metadata + +Client node do not parse or executed query by itself, it require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + +See ```org.apache.ignite.client.IgniteClient.Builder#sqlPartitionAwarenessMetadataCacheSize()```. + +NOTE: As for now, Partition awareness is supported for queries over single partition of a single table. Review Comment: Why is this a note and not part of the list? ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. +Bulk DML operations (like ```INSERT FROM SELECT``` or multi-values ```INSERT FROM VALUES```) are not supported yet. + +* Partition awareness cache on client may miss required metadata + +Client node do not parse or executed query by itself, it require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + +See ```org.apache.ignite.client.IgniteClient.Builder#sqlPartitionAwarenessMetadataCacheSize()```. + +NOTE: As for now, Partition awareness is supported for queries over single partition of a single table. + +=== How to Check if Partition Awareness is Applicable + +You can verify whether partition awareness is used for a given SQL query by inspecting the EXPLAIN command results. + +* The resulting plan should contain a `TableScan` node with equality conditions on all colocation columns of the table. +---- +TableScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) +---- +* or similar `IndexScan` that search bound has a prefix of all the colocation columns. +---- +IndexScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) + searchBounds: [ExactBounds [bound=?], ExactBounds [bound=?]] +---- +* or `KeyValueGet` node +---- +KeyValueGet + table: PUBLIC.T + key: [?0] +---- +* or `KeyValueModify` node +---- +KeyValueModify + table: PUBLIC.T + key: [?0] +---- + +NOTE: SQL engine allows both literals and dynamic parameters in equality conditions for partition awareness purposes. + + +=== Examples +Assumed a table `T` is defined as: +[source, sql] +---- +CREATE TABLE T ( + id INT NOT NULL, + region_id INT NOT NULL, + customer_id INT NOT NULL, + val VARCHAR, + PRIMARY KEY (id, region_id, customer_id) +) COLOCATE BY (region_id, customer_id); +---- +Next queries can utilize partition awareness optimization: Review Comment: ```suggestion The following queries can utilize partition awareness optimization: ``` I think we also want consistent linebreaks, its better to leave an empty line before and after the example for clarity. ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. +Bulk DML operations (like ```INSERT FROM SELECT``` or multi-values ```INSERT FROM VALUES```) are not supported yet. + +* Partition awareness cache on client may miss required metadata + +Client node do not parse or executed query by itself, it require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + +See ```org.apache.ignite.client.IgniteClient.Builder#sqlPartitionAwarenessMetadataCacheSize()```. + +NOTE: As for now, Partition awareness is supported for queries over single partition of a single table. + +=== How to Check if Partition Awareness is Applicable + +You can verify whether partition awareness is used for a given SQL query by inspecting the EXPLAIN command results. + +* The resulting plan should contain a `TableScan` node with equality conditions on all colocation columns of the table. +---- +TableScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) +---- +* or similar `IndexScan` that search bound has a prefix of all the colocation columns. +---- +IndexScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) + searchBounds: [ExactBounds [bound=?], ExactBounds [bound=?]] +---- +* or `KeyValueGet` node +---- +KeyValueGet + table: PUBLIC.T + key: [?0] +---- +* or `KeyValueModify` node +---- +KeyValueModify + table: PUBLIC.T + key: [?0] +---- + +NOTE: SQL engine allows both literals and dynamic parameters in equality conditions for partition awareness purposes. + + +=== Examples +Assumed a table `T` is defined as: Review Comment: ```suggestion Assuming a table `T` is defined as: ``` ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. +Bulk DML operations (like ```INSERT FROM SELECT``` or multi-values ```INSERT FROM VALUES```) are not supported yet. + +* Partition awareness cache on client may miss required metadata + +Client node do not parse or executed query by itself, it require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + +See ```org.apache.ignite.client.IgniteClient.Builder#sqlPartitionAwarenessMetadataCacheSize()```. + +NOTE: As for now, Partition awareness is supported for queries over single partition of a single table. + +=== How to Check if Partition Awareness is Applicable + +You can verify whether partition awareness is used for a given SQL query by inspecting the EXPLAIN command results. + +* The resulting plan should contain a `TableScan` node with equality conditions on all colocation columns of the table. +---- +TableScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) +---- +* or similar `IndexScan` that search bound has a prefix of all the colocation columns. +---- +IndexScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) + searchBounds: [ExactBounds [bound=?], ExactBounds [bound=?]] +---- +* or `KeyValueGet` node Review Comment: ```suggestion * or `KeyValueGet` node: ``` ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. +Bulk DML operations (like ```INSERT FROM SELECT``` or multi-values ```INSERT FROM VALUES```) are not supported yet. + +* Partition awareness cache on client may miss required metadata + +Client node do not parse or executed query by itself, it require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + +See ```org.apache.ignite.client.IgniteClient.Builder#sqlPartitionAwarenessMetadataCacheSize()```. + +NOTE: As for now, Partition awareness is supported for queries over single partition of a single table. + +=== How to Check if Partition Awareness is Applicable + +You can verify whether partition awareness is used for a given SQL query by inspecting the EXPLAIN command results. + +* The resulting plan should contain a `TableScan` node with equality conditions on all colocation columns of the table. +---- +TableScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) +---- +* or similar `IndexScan` that search bound has a prefix of all the colocation columns. +---- +IndexScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) + searchBounds: [ExactBounds [bound=?], ExactBounds [bound=?]] +---- +* or `KeyValueGet` node +---- +KeyValueGet + table: PUBLIC.T + key: [?0] +---- +* or `KeyValueModify` node +---- +KeyValueModify + table: PUBLIC.T + key: [?0] +---- + +NOTE: SQL engine allows both literals and dynamic parameters in equality conditions for partition awareness purposes. + + +=== Examples +Assumed a table `T` is defined as: +[source, sql] +---- +CREATE TABLE T ( + id INT NOT NULL, + region_id INT NOT NULL, + customer_id INT NOT NULL, + val VARCHAR, + PRIMARY KEY (id, region_id, customer_id) +) COLOCATE BY (region_id, customer_id); +---- +Next queries can utilize partition awareness optimization: +---- +SELECT * FROM T WHERE region_id =? AND customer_id = ? + +DELETE FROM T +WHERE id = ? AND region_id = ? AND customer_id = ?; + +INSERT INTO T (id, region_id, customer_id, val) +VALUES (?, ?, ?, ?); +---- + +Next, queries can NOT utilize partition awareness. +---- +// Missed condition for colocation column `customer_id` +SELECT * FROM T WHERE region_id =? + +// Non-equality condition on colocation column Review Comment: ```suggestion // Non-equality condition on colocation column. ``` ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. Review Comment: If its stronger, we should put it first, as users can skip this very easily. Also, how is it stronger? The first one is mandatory, so is this one. Also, you cannot stack comparative forms. So grammar wise this would be correct: This statement is stronger compared to the previous one and requires equality condition on all key columns. But we should just rewrite the whole thing i think. ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. +Bulk DML operations (like ```INSERT FROM SELECT``` or multi-values ```INSERT FROM VALUES```) are not supported yet. + +* Partition awareness cache on client may miss required metadata + +Client node do not parse or executed query by itself, it require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + Review Comment: ```suggestion Client nodes do not parse or executed query by themselves, they require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + ``` Also lets link to eviction policies doc. ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. +Bulk DML operations (like ```INSERT FROM SELECT``` or multi-values ```INSERT FROM VALUES```) are not supported yet. + +* Partition awareness cache on client may miss required metadata + +Client node do not parse or executed query by itself, it require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + +See ```org.apache.ignite.client.IgniteClient.Builder#sqlPartitionAwarenessMetadataCacheSize()```. + +NOTE: As for now, Partition awareness is supported for queries over single partition of a single table. + +=== How to Check if Partition Awareness is Applicable + +You can verify whether partition awareness is used for a given SQL query by inspecting the EXPLAIN command results. + +* The resulting plan should contain a `TableScan` node with equality conditions on all colocation columns of the table. +---- +TableScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) +---- +* or similar `IndexScan` that search bound has a prefix of all the colocation columns. +---- +IndexScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) + searchBounds: [ExactBounds [bound=?], ExactBounds [bound=?]] +---- +* or `KeyValueGet` node +---- +KeyValueGet + table: PUBLIC.T + key: [?0] +---- +* or `KeyValueModify` node +---- +KeyValueModify + table: PUBLIC.T + key: [?0] +---- + +NOTE: SQL engine allows both literals and dynamic parameters in equality conditions for partition awareness purposes. Review Comment: ```suggestion NOTE: Apache Ignite supports both literals and dynamic parameters in equality conditions for partition awareness purposes. ``` ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. +Bulk DML operations (like ```INSERT FROM SELECT``` or multi-values ```INSERT FROM VALUES```) are not supported yet. + +* Partition awareness cache on client may miss required metadata + +Client node do not parse or executed query by itself, it require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + +See ```org.apache.ignite.client.IgniteClient.Builder#sqlPartitionAwarenessMetadataCacheSize()```. + +NOTE: As for now, Partition awareness is supported for queries over single partition of a single table. + +=== How to Check if Partition Awareness is Applicable + +You can verify whether partition awareness is used for a given SQL query by inspecting the EXPLAIN command results. + +* The resulting plan should contain a `TableScan` node with equality conditions on all colocation columns of the table. +---- +TableScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) +---- +* or similar `IndexScan` that search bound has a prefix of all the colocation columns. +---- +IndexScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) + searchBounds: [ExactBounds [bound=?], ExactBounds [bound=?]] +---- +* or `KeyValueGet` node +---- +KeyValueGet + table: PUBLIC.T + key: [?0] +---- +* or `KeyValueModify` node Review Comment: ```suggestion * or `KeyValueModify` node: ``` ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. +Bulk DML operations (like ```INSERT FROM SELECT``` or multi-values ```INSERT FROM VALUES```) are not supported yet. + +* Partition awareness cache on client may miss required metadata + +Client node do not parse or executed query by itself, it require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + +See ```org.apache.ignite.client.IgniteClient.Builder#sqlPartitionAwarenessMetadataCacheSize()```. + +NOTE: As for now, Partition awareness is supported for queries over single partition of a single table. + +=== How to Check if Partition Awareness is Applicable + +You can verify whether partition awareness is used for a given SQL query by inspecting the EXPLAIN command results. + +* The resulting plan should contain a `TableScan` node with equality conditions on all colocation columns of the table. +---- +TableScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) +---- +* or similar `IndexScan` that search bound has a prefix of all the colocation columns. +---- +IndexScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) + searchBounds: [ExactBounds [bound=?], ExactBounds [bound=?]] +---- +* or `KeyValueGet` node +---- +KeyValueGet + table: PUBLIC.T + key: [?0] +---- +* or `KeyValueModify` node +---- +KeyValueModify + table: PUBLIC.T + key: [?0] +---- + +NOTE: SQL engine allows both literals and dynamic parameters in equality conditions for partition awareness purposes. + + +=== Examples +Assumed a table `T` is defined as: +[source, sql] +---- +CREATE TABLE T ( + id INT NOT NULL, + region_id INT NOT NULL, + customer_id INT NOT NULL, + val VARCHAR, + PRIMARY KEY (id, region_id, customer_id) +) COLOCATE BY (region_id, customer_id); +---- +Next queries can utilize partition awareness optimization: +---- +SELECT * FROM T WHERE region_id =? AND customer_id = ? + +DELETE FROM T +WHERE id = ? AND region_id = ? AND customer_id = ?; + +INSERT INTO T (id, region_id, customer_id, val) +VALUES (?, ?, ?, ?); +---- + +Next, queries can NOT utilize partition awareness. +---- +// Missed condition for colocation column `customer_id` Review Comment: ```suggestion // Missing condition for colocation column `customer_id`. ``` Colocated column? ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. +Bulk DML operations (like ```INSERT FROM SELECT``` or multi-values ```INSERT FROM VALUES```) are not supported yet. + +* Partition awareness cache on client may miss required metadata + +Client node do not parse or executed query by itself, it require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + +See ```org.apache.ignite.client.IgniteClient.Builder#sqlPartitionAwarenessMetadataCacheSize()```. + +NOTE: As for now, Partition awareness is supported for queries over single partition of a single table. + +=== How to Check if Partition Awareness is Applicable + +You can verify whether partition awareness is used for a given SQL query by inspecting the EXPLAIN command results. + +* The resulting plan should contain a `TableScan` node with equality conditions on all colocation columns of the table. +---- +TableScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) +---- +* or similar `IndexScan` that search bound has a prefix of all the colocation columns. Review Comment: ```suggestion * or a similar `IndexScan` that search bound has a prefix of all the colocation columns: ``` ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. +Bulk DML operations (like ```INSERT FROM SELECT``` or multi-values ```INSERT FROM VALUES```) are not supported yet. + +* Partition awareness cache on client may miss required metadata + +Client node do not parse or executed query by itself, it require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + +See ```org.apache.ignite.client.IgniteClient.Builder#sqlPartitionAwarenessMetadataCacheSize()```. + +NOTE: As for now, Partition awareness is supported for queries over single partition of a single table. + +=== How to Check if Partition Awareness is Applicable + +You can verify whether partition awareness is used for a given SQL query by inspecting the EXPLAIN command results. + +* The resulting plan should contain a `TableScan` node with equality conditions on all colocation columns of the table. +---- +TableScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) +---- +* or similar `IndexScan` that search bound has a prefix of all the colocation columns. +---- +IndexScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) + searchBounds: [ExactBounds [bound=?], ExactBounds [bound=?]] +---- +* or `KeyValueGet` node +---- +KeyValueGet + table: PUBLIC.T + key: [?0] +---- +* or `KeyValueModify` node +---- +KeyValueModify + table: PUBLIC.T + key: [?0] +---- + +NOTE: SQL engine allows both literals and dynamic parameters in equality conditions for partition awareness purposes. + + +=== Examples +Assumed a table `T` is defined as: +[source, sql] +---- +CREATE TABLE T ( + id INT NOT NULL, + region_id INT NOT NULL, + customer_id INT NOT NULL, + val VARCHAR, + PRIMARY KEY (id, region_id, customer_id) +) COLOCATE BY (region_id, customer_id); +---- +Next queries can utilize partition awareness optimization: +---- +SELECT * FROM T WHERE region_id =? AND customer_id = ? + +DELETE FROM T +WHERE id = ? AND region_id = ? AND customer_id = ?; + +INSERT INTO T (id, region_id, customer_id, val) +VALUES (?, ?, ?, ?); +---- + +Next, queries can NOT utilize partition awareness. Review Comment: ```suggestion While these queries will not utilize partition awareness: ``` ########## docs/_docs/developers-guide/clients/overview.adoc: ########## @@ -105,6 +105,89 @@ For each key that needs updating, the client will get the name of the node holdi NOTE: Partition awareness assumes that the cluster is stable. Client receives information about cluster data assignments in the background, and it may be outdated by the time an update is sent. If it is, nodes will automatically redirect requests to correct nodes until data on the client is updated. +=== Limitations + +* Equality condition on colocation columns is required. + +Partition awareness optimization can be applied only if the SQL query contains an equality condition on all colocated columns. The mandatory requirement allows the client to route SQL query to the node where the partition with the data resides. + + +* DML query, which can **not** be rewritten to key-value operation, are not supported yet. + +This statement is more stronger comparing to previous one and require equality condition on all key columns. +Partition awareness work for DML query only if it could be rewritten to a single key-value operation. The explain plan will contains `KeyValueGet` or `KeyValueModify` node in that case. +Bulk DML operations (like ```INSERT FROM SELECT``` or multi-values ```INSERT FROM VALUES```) are not supported yet. + +* Partition awareness cache on client may miss required metadata + +Client node do not parse or executed query by itself, it require query metadata with colocation information from the server node to utilize partition awareness optimization. The query metadata is cached on client after the first query execution and can be used in later query runs. However, it also can be evicted due to cache eviction policies. + +See ```org.apache.ignite.client.IgniteClient.Builder#sqlPartitionAwarenessMetadataCacheSize()```. + +NOTE: As for now, Partition awareness is supported for queries over single partition of a single table. + +=== How to Check if Partition Awareness is Applicable + +You can verify whether partition awareness is used for a given SQL query by inspecting the EXPLAIN command results. + +* The resulting plan should contain a `TableScan` node with equality conditions on all colocation columns of the table. +---- +TableScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) +---- +* or similar `IndexScan` that search bound has a prefix of all the colocation columns. +---- +IndexScan + table: PUBLIC.T + predicate: AND(=(COLOCATION_COL_1, ?), =(COLOCATION_COL_2, ?)) + searchBounds: [ExactBounds [bound=?], ExactBounds [bound=?]] +---- +* or `KeyValueGet` node +---- +KeyValueGet + table: PUBLIC.T + key: [?0] +---- +* or `KeyValueModify` node +---- +KeyValueModify + table: PUBLIC.T + key: [?0] +---- + +NOTE: SQL engine allows both literals and dynamic parameters in equality conditions for partition awareness purposes. + + +=== Examples +Assumed a table `T` is defined as: +[source, sql] +---- +CREATE TABLE T ( + id INT NOT NULL, + region_id INT NOT NULL, + customer_id INT NOT NULL, + val VARCHAR, + PRIMARY KEY (id, region_id, customer_id) +) COLOCATE BY (region_id, customer_id); +---- +Next queries can utilize partition awareness optimization: +---- +SELECT * FROM T WHERE region_id =? AND customer_id = ? + +DELETE FROM T +WHERE id = ? AND region_id = ? AND customer_id = ?; + +INSERT INTO T (id, region_id, customer_id, val) +VALUES (?, ?, ?, ?); +---- + +Next, queries can NOT utilize partition awareness. +---- +// Missed condition for colocation column `customer_id` +SELECT * FROM T WHERE region_id =? + +// Non-equality condition on colocation column +SELECT * FROM T WHERE region_id = ? AND customer_id > ? +---- + +In other more complex cases (nested queries, IN condition, JOINS), partition awareness may work or may not work. It depends whether the query can be rewritten in a form, which contains sufficient information to locate the data. Review Comment: ```suggestion In more complex cases (nested queries, IN condition, JOINS), partition awareness is not guaranteed to work. It depends on whether the query can be rewritten in a form which contains sufficient information to locate the data. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
