[jira] [Updated] (PHOENIX-5872) Close Internal Phoenix Connections that were running during cancel
[ https://issues.apache.org/jira/browse/PHOENIX-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-5872: --- Affects Version/s: 4.14.3 > Close Internal Phoenix Connections that were running during cancel > -- > > Key: PHOENIX-5872 > URL: https://issues.apache.org/jira/browse/PHOENIX-5872 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.14.3 >Reporter: Daniel Wong >Assignee: Daniel Wong >Priority: Major > Attachments: PHOENIX-5872.4.x.patch > > Time Spent: 2h 50m > Remaining Estimate: 0h > > 3 part approach: > 1 don't count internal phoenix connections toward the client limit. > 2 count internal phoenix connections toward a newly defined limit > 3 track parent and child relationships between connections to close those > connections -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5872) Close Internal Phoenix Connections that were running during cancel
[ https://issues.apache.org/jira/browse/PHOENIX-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-5872: --- Affects Version/s: 4.x > Close Internal Phoenix Connections that were running during cancel > -- > > Key: PHOENIX-5872 > URL: https://issues.apache.org/jira/browse/PHOENIX-5872 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.14.3, 4.x >Reporter: Daniel Wong >Assignee: Daniel Wong >Priority: Major > Attachments: PHOENIX-5872.4.x.patch > > Time Spent: 2h 50m > Remaining Estimate: 0h > > 3 part approach: > 1 don't count internal phoenix connections toward the client limit. > 2 count internal phoenix connections toward a newly defined limit > 3 track parent and child relationships between connections to close those > connections -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (PHOENIX-4698) Tolerate orphaned views
[ https://issues.apache.org/jira/browse/PHOENIX-4698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445246#comment-16445246 ] Maddineni Sukumar commented on PHOENIX-4698: Thanks James, for creating Jira and pushing the patch. > Tolerate orphaned views > --- > > Key: PHOENIX-4698 > URL: https://issues.apache.org/jira/browse/PHOENIX-4698 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Maddineni Sukumar >Priority: Major > Fix For: 4.14.0, 5.0.0 > > Attachments: PHOENIX-4698.patch > > > It's possible that under rare circumstances that views get orphaned. We > should make sure that this situation is tolerated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-3928) Consider retrying once after any SQLException
[ https://issues.apache.org/jira/browse/PHOENIX-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049868#comment-16049868 ] Maddineni Sukumar commented on PHOENIX-3928: Yes [~jamestaylor], Index delete didnt change source table timestamp, so when we try force update it is getting same timestamp from client and server, so server is not sending latest table object when timestamps are same. One option is when we force update cache cleaning table other info like indexes/views info. > Consider retrying once after any SQLException > - > > Key: PHOENIX-3928 > URL: https://issues.apache.org/jira/browse/PHOENIX-3928 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.12.0 > > > There are more cases in which a retry would successfully execute than when a > MetaDataEntityNotFoundException. For example, certain error cases that depend > on the state of the metadata would work on retry if the metadata had changed. > We may want to retry on any SQLException and simply loop through the tables > involved (plan.getSourceRefs().iterator()), and if any meta data was updated, > go ahead and retry once. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3928) Consider retrying once after any SQLException
[ https://issues.apache.org/jira/browse/PHOENIX-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16047149#comment-16047149 ] Maddineni Sukumar commented on PHOENIX-3928: Hi [~jamestaylor] , With above scenario I am able to reproduce CommitException:Unable to update the following indexes: [I_T01],serverTimestamp=1497300602580 . I added SQLExeption catch block and did force cache update and then tried same transaction. It still failed with same error. The reason is table modified timestamp is same before and after delting index, so when we do cache update we are reusing client side table object as timestamps are same. Old object has index so we are getting same issue. I deleted table object from metadata in catch block and did cache update and then it worked fine. Is this correct approach i.e. force reloading table from metadata ? > Consider retrying once after any SQLException > - > > Key: PHOENIX-3928 > URL: https://issues.apache.org/jira/browse/PHOENIX-3928 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.12.0 > > > There are more cases in which a retry would successfully execute than when a > MetaDataEntityNotFoundException. For example, certain error cases that depend > on the state of the metadata would work on retry if the metadata had changed. > We may want to retry on any SQLException and simply loop through the tables > involved (plan.getSourceRefs().iterator()), and if any meta data was updated, > go ahead and retry once. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3928) Consider retrying once after any SQLException
[ https://issues.apache.org/jira/browse/PHOENIX-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16043737#comment-16043737 ] Maddineni Sukumar commented on PHOENIX-3928: Sure [~jamestaylor], I will look into it. > Consider retrying once after any SQLException > - > > Key: PHOENIX-3928 > URL: https://issues.apache.org/jira/browse/PHOENIX-3928 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor > Fix For: 4.12.0 > > > There are more cases in which a retry would successfully execute than when a > MetaDataEntityNotFoundException. For example, certain error cases that depend > on the state of the metadata would work on retry if the metadata had changed. > We may want to retry on any SQLException and simply loop through the tables > involved (plan.getSourceRefs().iterator()), and if any meta data was updated, > go ahead and retry once. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (PHOENIX-3928) Consider retrying once after any SQLException
[ https://issues.apache.org/jira/browse/PHOENIX-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar reassigned PHOENIX-3928: -- Assignee: Maddineni Sukumar > Consider retrying once after any SQLException > - > > Key: PHOENIX-3928 > URL: https://issues.apache.org/jira/browse/PHOENIX-3928 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.12.0 > > > There are more cases in which a retry would successfully execute than when a > MetaDataEntityNotFoundException. For example, certain error cases that depend > on the state of the metadata would work on retry if the metadata had changed. > We may want to retry on any SQLException and simply loop through the tables > involved (plan.getSourceRefs().iterator()), and if any meta data was updated, > go ahead and retry once. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3910) Tests in UpgradeIT failing after PHOENIX-3823
[ https://issues.apache.org/jira/browse/PHOENIX-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16043512#comment-16043512 ] Maddineni Sukumar commented on PHOENIX-3910: [~jamestaylor] , please review this patch when ever you have some free time. Thanks. > Tests in UpgradeIT failing after PHOENIX-3823 > - > > Key: PHOENIX-3910 > URL: https://issues.apache.org/jira/browse/PHOENIX-3910 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3910.patch, PHOENIX-3910.v2.patch > > > [~sukuna...@gmail.com], can you please take a look? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3910) Tests in UpgradeIT failing after PHOENIX-3823
[ https://issues.apache.org/jira/browse/PHOENIX-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16039837#comment-16039837 ] Maddineni Sukumar commented on PHOENIX-3910: Sure [~jamestaylor]. Added new patch with change as you suggested. > Tests in UpgradeIT failing after PHOENIX-3823 > - > > Key: PHOENIX-3910 > URL: https://issues.apache.org/jira/browse/PHOENIX-3910 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3910.patch, PHOENIX-3910.v2.patch > > > [~sukuna...@gmail.com], can you please take a look? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3910) Tests in UpgradeIT failing after PHOENIX-3823
[ https://issues.apache.org/jira/browse/PHOENIX-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3910: --- Attachment: PHOENIX-3910.v2.patch > Tests in UpgradeIT failing after PHOENIX-3823 > - > > Key: PHOENIX-3910 > URL: https://issues.apache.org/jira/browse/PHOENIX-3910 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3910.patch, PHOENIX-3910.v2.patch > > > [~sukuna...@gmail.com], can you please take a look? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3910) Tests in UpgradeIT failing after PHOENIX-3823
[ https://issues.apache.org/jira/browse/PHOENIX-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3910: --- Attachment: PHOENIX-3910.patch Patch to ignore force update cache for meta tables in case of upsert select to fix upgrade issue. > Tests in UpgradeIT failing after PHOENIX-3823 > - > > Key: PHOENIX-3910 > URL: https://issues.apache.org/jira/browse/PHOENIX-3910 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3910.patch > > > [~sukuna...@gmail.com], can you please take a look? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3910) Tests in UpgradeIT failing after PHOENIX-3823
[ https://issues.apache.org/jira/browse/PHOENIX-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16039711#comment-16039711 ] Maddineni Sukumar commented on PHOENIX-3910: Thanks [~jamestaylor] . Sorry for the delay. Injured my Knee last weekend and running around hospitals and labs. Looked into it and found that our force update cache for every upsert select on same table logic causing this error in case of upgrade i.e. When we upgrade table we are doing upsert select on SYSCAT table and our logic to force update cache coming into picture and then when we do force update cache for SYSCAT it is resulting TABLE_ALREADY_EXISTS result with result table null. I think we dont need force update cache for SYSCAT as this META table and also we cannot configure UPDATE_CACHE_FREQUECNY for SYSCAT.[~jamestaylor] / [~samarthjain] , correct me if we need above upsert select logic on SYSCAT also. If you are fine with this then I will submit patch with ignoring force update SYSCAT during upsert select . > Tests in UpgradeIT failing after PHOENIX-3823 > - > > Key: PHOENIX-3910 > URL: https://issues.apache.org/jira/browse/PHOENIX-3910 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > > [~sukuna...@gmail.com], can you please take a look? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3910) Tests in UpgradeIT failing after PHOENIX-3823
[ https://issues.apache.org/jira/browse/PHOENIX-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036156#comment-16036156 ] Maddineni Sukumar commented on PHOENIX-3910: Sure [~samarthjain] . I ran above tests locally just now and they are failing with error "org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03): Table undefined. tableName=SYSTEM.CATALOG". Is this the same error you also got? Can you point me to unit tests build report if we have any. > Tests in UpgradeIT failing after PHOENIX-3823 > - > > Key: PHOENIX-3910 > URL: https://issues.apache.org/jira/browse/PHOENIX-3910 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > > [~sukuna...@gmail.com], can you please take a look? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3908) Ensure MetaDataEntityNotFound handled for CREATE VIEW
[
https://issues.apache.org/jira/browse/PHOENIX-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maddineni Sukumar updated PHOENIX-3908:
---
Attachment: PHOENIX-3908.patch
Added unit test to verify create table and create view on diff connections.
> Ensure MetaDataEntityNotFound handled for CREATE VIEW
> -
>
> Key: PHOENIX-3908
> URL: https://issues.apache.org/jira/browse/PHOENIX-3908
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: Maddineni Sukumar
> Fix For: 4.11.0
>
> Attachments: PHOENIX-3908.patch
>
>
> We may already handle this (if so please close), but it's possible that a
> CREATE VIEW statement could reference a table that was created in a different
> connection. We should add a test for this in
> UpdateCacheAcrossDifferentClientsIT and a similar retry loop as was done in
> PhoenixStatement within CreateTableCompiler here:
> {code}
> return new BaseMutationPlan(context, operation) {
> @Override
> public MutationState execute() throws SQLException {
> try {
> return client.createTable(finalCreate, splits, parent,
> viewStatement, viewType, viewColumnConstants, isViewColumnReferenced);
> } finally {
> if (client.getConnection() != connection) {
> client.getConnection().close();
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (PHOENIX-3908) Ensure MetaDataEntityNotFound handled for CREATE VIEW
[
https://issues.apache.org/jira/browse/PHOENIX-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035292#comment-16035292
]
Maddineni Sukumar commented on PHOENIX-3908:
I tried this
Create two connections c1 and c2.
c1.createtable with UPDATE_CACHE_FREQUENCY=x
c1.upsert some rows
then
c2.create view
it worked fine.
> Ensure MetaDataEntityNotFound handled for CREATE VIEW
> -
>
> Key: PHOENIX-3908
> URL: https://issues.apache.org/jira/browse/PHOENIX-3908
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: Maddineni Sukumar
> Fix For: 4.11.0
>
>
> We may already handle this (if so please close), but it's possible that a
> CREATE VIEW statement could reference a table that was created in a different
> connection. We should add a test for this in
> UpdateCacheAcrossDifferentClientsIT and a similar retry loop as was done in
> PhoenixStatement within CreateTableCompiler here:
> {code}
> return new BaseMutationPlan(context, operation) {
> @Override
> public MutationState execute() throws SQLException {
> try {
> return client.createTable(finalCreate, splits, parent,
> viewStatement, viewType, viewColumnConstants, isViewColumnReferenced);
> } finally {
> if (client.getConnection() != connection) {
> client.getConnection().close();
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035118#comment-16035118 ] Maddineni Sukumar commented on PHOENIX-3823: Thanks a lot [~jamestaylor] for all the help/reviews and also for pushing it through. > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v10.patch, > PHOENIX-3823.v11.patch, PHOENIX-3823.v12.patch, PHOENIX-3823.v2.patch, > PHOENIX-3823.v3.patch, PHOENIX-3823.v4.patch, PHOENIX-3823.v5.patch, > PHOENIX-3823.v6.patch, PHOENIX-3823.v7.patch, PHOENIX-3823.v8.patch, > PHOENIX-3823.v9.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16034833#comment-16034833 ] Maddineni Sukumar commented on PHOENIX-3823: All above failures are unrelated to this change i.e. either operation timeouts or SYSCAT not enabled or tracing related.. > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v10.patch, > PHOENIX-3823.v2.patch, PHOENIX-3823.v3.patch, PHOENIX-3823.v4.patch, > PHOENIX-3823.v5.patch, PHOENIX-3823.v6.patch, PHOENIX-3823.v7.patch, > PHOENIX-3823.v8.patch, PHOENIX-3823.v9.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3823: --- Attachment: PHOENIX-3823.v10.patch Did all suggestions made by [~jamestaylor] except moving DROP_METADATA atrrib to individual test level as that did not work properly i.e. setting that property at driver level is working but not working If I set at individual connection level. > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v10.patch, > PHOENIX-3823.v2.patch, PHOENIX-3823.v3.patch, PHOENIX-3823.v4.patch, > PHOENIX-3823.v5.patch, PHOENIX-3823.v6.patch, PHOENIX-3823.v7.patch, > PHOENIX-3823.v8.patch, PHOENIX-3823.v9.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-2885) Set default value for UPDATE_CACHE_FREQUENCY
[ https://issues.apache.org/jira/browse/PHOENIX-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16034129#comment-16034129 ] Maddineni Sukumar commented on PHOENIX-2885: Created patch v3 against latest master. > Set default value for UPDATE_CACHE_FREQUENCY > > > Key: PHOENIX-2885 > URL: https://issues.apache.org/jira/browse/PHOENIX-2885 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-2885.4.10-HBase-0.98.patch, > PHOENIX-2885.v2.patch, PHOENIX-2885.v3.patch > > > We have the ability currently to tell Phoenix how stale we're will to have > the metadata through our UPDATE_CACHE_FREQUENCY feature[1]. By default, when > a table is created, unless explicitly specified, there is no > UPDATE_CACHE_FREQUENCY set. We should provide a new Phoenix configuration > parameter to specify a default value when a CREATE TABLE statement is > executed. In addition, when a VIEW is created, we should inherit the > UPDATE_CACHE_FREQUENCY value from the parent (unless it's explicitly > specified) and then fallback to the new default config value. > [1] https://phoenix.apache.org/language/index.html#options -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-2885) Set default value for UPDATE_CACHE_FREQUENCY
[ https://issues.apache.org/jira/browse/PHOENIX-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-2885: --- Attachment: PHOENIX-2885.v3.patch > Set default value for UPDATE_CACHE_FREQUENCY > > > Key: PHOENIX-2885 > URL: https://issues.apache.org/jira/browse/PHOENIX-2885 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-2885.4.10-HBase-0.98.patch, > PHOENIX-2885.v2.patch, PHOENIX-2885.v3.patch > > > We have the ability currently to tell Phoenix how stale we're will to have > the metadata through our UPDATE_CACHE_FREQUENCY feature[1]. By default, when > a table is created, unless explicitly specified, there is no > UPDATE_CACHE_FREQUENCY set. We should provide a new Phoenix configuration > parameter to specify a default value when a CREATE TABLE statement is > executed. In addition, when a VIEW is created, we should inherit the > UPDATE_CACHE_FREQUENCY value from the parent (unless it's explicitly > specified) and then fallback to the new default config value. > [1] https://phoenix.apache.org/language/index.html#options -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3823: --- Attachment: PHOENIX-3823.v9.patch Created new patch as old patch did not apply properly due to other latest commits. > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v2.patch, > PHOENIX-3823.v3.patch, PHOENIX-3823.v4.patch, PHOENIX-3823.v5.patch, > PHOENIX-3823.v6.patch, PHOENIX-3823.v7.patch, PHOENIX-3823.v8.patch, > PHOENIX-3823.v9.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16034067#comment-16034067 ] Maddineni Sukumar commented on PHOENIX-3823: Submitted new patch with above changes suggested by [~jamestaylor]. > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v2.patch, > PHOENIX-3823.v3.patch, PHOENIX-3823.v4.patch, PHOENIX-3823.v5.patch, > PHOENIX-3823.v6.patch, PHOENIX-3823.v7.patch, PHOENIX-3823.v8.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3823: --- Attachment: PHOENIX-3823.v8.patch > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v2.patch, > PHOENIX-3823.v3.patch, PHOENIX-3823.v4.patch, PHOENIX-3823.v5.patch, > PHOENIX-3823.v6.patch, PHOENIX-3823.v7.patch, PHOENIX-3823.v8.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033393#comment-16033393 ] Maddineni Sukumar commented on PHOENIX-3823: Hi [~jamestaylor], I will change logic to hold just tablename instead of PTable. But I cannot move above logic inside BaseColumnResolver constructor as I get to know which table I am looking for in createTableRef() method only. If you are fine with that then I will submit patch with just tablename as member variable. > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v2.patch, > PHOENIX-3823.v3.patch, PHOENIX-3823.v4.patch, PHOENIX-3823.v5.patch, > PHOENIX-3823.v6.patch, PHOENIX-3823.v7.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3823: --- Attachment: PHOENIX-3823.v7.patch Modified logic to have mutatingTable as a member variable in BaseColumnResolver and only Upsert and Delete Compilers sets this value to force update cache in case of upsertselect and delete select on same table > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v2.patch, > PHOENIX-3823.v3.patch, PHOENIX-3823.v4.patch, PHOENIX-3823.v5.patch, > PHOENIX-3823.v6.patch, PHOENIX-3823.v7.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025514#comment-16025514 ] Maddineni Sukumar commented on PHOENIX-3823: Thanks [~jamestaylor] for review. Made changes as you mentioned above()except createTableRef as it is being used many places so I did it in simple way for Upsert and Delete Compilers. > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v2.patch, > PHOENIX-3823.v3.patch, PHOENIX-3823.v4.patch, PHOENIX-3823.v5.patch, > PHOENIX-3823.v6.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3823: --- Attachment: PHOENIX-3823.v6.patch > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v2.patch, > PHOENIX-3823.v3.patch, PHOENIX-3823.v4.patch, PHOENIX-3823.v5.patch, > PHOENIX-3823.v6.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3209) Ensure scans run at specific server timestamp for UPSERT SELECT to same table
[ https://issues.apache.org/jira/browse/PHOENIX-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16014399#comment-16014399 ] Maddineni Sukumar commented on PHOENIX-3209: Hi [~jamestaylor] / [~samarthjain], I am able to reproduce scenario where if we use update_cache_frequency and upsert select on same table then we are running scan with time range 0 to Long.MAX_VALUE, so we are able to see future data or data created during query execution. Reason for this is we are getting table from cache with resolved timestamp as -1(UNSET_TIMESTAMP). So we are not setting end time for time range and it is using default value which is Long.MAX_VALUE. I added logic to do force cache update in case of upsert select on same table. Added patch in PHOENIX-3823 patch as it was easy adding unit test there. I think we need to do this logic for all upsert select scenarios irrespective of same table or not. I will run some more tests today. Please confirm. > Ensure scans run at specific server timestamp for UPSERT SELECT to same table > - > > Key: PHOENIX-3209 > URL: https://issues.apache.org/jira/browse/PHOENIX-3209 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > > This is a corner case of specifying an UPDATE_CACHE_FREQUENCY on a table and > executing an UPSERT SELECT. Without an UPDATE_CACHE_FREQUENCY, we ping the > server to ensure we have the latest version of the schema. We'll then run the > query based on the server timestamp returned as a result of checking that the > schema is up-to-date. If an UPDATE_CACHE_FREQUENCY is set, we skip this RPC > which is a potential problem in this case. This becomes more likely when we > introduce a default UPATE_CACHE_FREQUENCY with PHOENIX-2885. The fix is to > ignore the UPDATE_CACHE_FREQUENCY when an UPSERT SELECT is performed where > the source and target table are the same. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16014385#comment-16014385 ] Maddineni Sukumar edited comment on PHOENIX-3823 at 5/17/17 4:55 PM: - New patch with fix for Upsert select scenario mentioned in PHOENIX-3209. Adding in same patch as writing unit test is easy and fix is also similar(force cache update) was (Author: sukuna...@gmail.com): New patch with fix for Upsert select scenario mentioned in PHOENIX-3859. Adding in same patch as writing unit test is easy and fix is also similar(force cache update) > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v2.patch, > PHOENIX-3823.v3.patch, PHOENIX-3823.v4.patch, PHOENIX-3823.v5.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3823: --- Attachment: PHOENIX-3823.v5.patch New patch with fix for Upsert select scenario mentioned in PHOENIX-3859. Adding in same patch as writing unit test is easy and fix is also similar(force cache update) > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v2.patch, > PHOENIX-3823.v3.patch, PHOENIX-3823.v4.patch, PHOENIX-3823.v5.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3859) Upsert Select query returning updated count as double than actual value when total count is more than configured mutate batch size
[
https://issues.apache.org/jira/browse/PHOENIX-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maddineni Sukumar updated PHOENIX-3859:
---
Priority: Minor (was: Major)
> Upsert Select query returning updated count as double than actual value when
> total count is more than configured mutate batch size
> ---
>
> Key: PHOENIX-3859
> URL: https://issues.apache.org/jira/browse/PHOENIX-3859
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Maddineni Sukumar
>Assignee: Maddineni Sukumar
>Priority: Minor
>
> If I setup driver with below property
> props.put(QueryServices.MUTATE_BATCH_SIZE_ATTRIB, Integert.toString(50));
> And load more than 50 rows, for example 60 rows and then run below upsert
> select query.
> int count = conn.createStatement().executeUpdate("Upsert into TableA select *
> from TableA");
> Count comes as 120 instead of 60. If I increase batch size to more than 60
> and try then it gives proper result as 60.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (PHOENIX-3859) Upsert Select query returning updated count as double than actual value when total count is more than configured mutate batch size
[
https://issues.apache.org/jira/browse/PHOENIX-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maddineni Sukumar updated PHOENIX-3859:
---
Summary: Upsert Select query returning updated count as double than actual
value when total count is more than configured mutate batch size (was: Upsert
Select is returning updated count is wrong if overall count is more than
configured batch size)
> Upsert Select query returning updated count as double than actual value when
> total count is more than configured mutate batch size
> ---
>
> Key: PHOENIX-3859
> URL: https://issues.apache.org/jira/browse/PHOENIX-3859
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Maddineni Sukumar
>Assignee: Maddineni Sukumar
>
> If I setup driver with below property
> props.put(QueryServices.MUTATE_BATCH_SIZE_ATTRIB, Integert.toString(50));
> And load more than 50 rows, for example 60 rows and then run below upsert
> select query.
> int count = conn.createStatement().executeUpdate("Upsert into TableA select *
> from TableA");
> Count comes as 120 instead of 60. If I increase batch size to more than 60
> and try then it gives proper result as 60.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (PHOENIX-3859) Upsert Select is returning updated count is wrong if overall count is more than configured batch size
Maddineni Sukumar created PHOENIX-3859:
--
Summary: Upsert Select is returning updated count is wrong if
overall count is more than configured batch size
Key: PHOENIX-3859
URL: https://issues.apache.org/jira/browse/PHOENIX-3859
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.10.0
Reporter: Maddineni Sukumar
Assignee: Maddineni Sukumar
If I setup driver with below property
props.put(QueryServices.MUTATE_BATCH_SIZE_ATTRIB, Integert.toString(50));
And load more than 50 rows, for example 60 rows and then run below upsert
select query.
int count = conn.createStatement().executeUpdate("Upsert into TableA select *
from TableA");
Count comes as 120 instead of 60. If I increase batch size to more than 60 and
try then it gives proper result as 60.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3823: --- Attachment: PHOENIX-3823.v4.patch > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v2.patch, > PHOENIX-3823.v3.patch, PHOENIX-3823.v4.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3823: --- Attachment: PHOENIX-3823.v3.patch > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v2.patch, > PHOENIX-3823.v3.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3856) StatementContext class constructor not honouring supplied scan object
[
https://issues.apache.org/jira/browse/PHOENIX-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maddineni Sukumar updated PHOENIX-3856:
---
Attachment: PHOENIX-3856.patch
> StatementContext class constructor not honouring supplied scan object
> --
>
> Key: PHOENIX-3856
> URL: https://issues.apache.org/jira/browse/PHOENIX-3856
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Maddineni Sukumar
>Assignee: Maddineni Sukumar
>Priority: Minor
> Attachments: PHOENIX-3856.patch
>
>
> In below constructor we are creating additional scan object instead of
> supplied scan object.
> public StatementContext(PhoenixStatement statement, Scan scan) {
> this(statement, FromCompiler.EMPTY_TABLE_RESOLVER, new Scan(), new
> SequenceManager(statement));
> }
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (PHOENIX-3856) StatementContext class constructor not honouring supplied scan object
Maddineni Sukumar created PHOENIX-3856:
--
Summary: StatementContext class constructor not honouring
supplied scan object
Key: PHOENIX-3856
URL: https://issues.apache.org/jira/browse/PHOENIX-3856
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.10.0
Reporter: Maddineni Sukumar
Assignee: Maddineni Sukumar
Priority: Minor
In below constructor we are creating additional scan object instead of
supplied scan object.
public StatementContext(PhoenixStatement statement, Scan scan) {
this(statement, FromCompiler.EMPTY_TABLE_RESOLVER, new Scan(), new
SequenceManager(statement));
}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3823: --- Attachment: PHOENIX-3823.v2.patch > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch, PHOENIX-3823.v2.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16008961#comment-16008961 ] Maddineni Sukumar commented on PHOENIX-3823: Hi [~jamestaylor], Please review this patch. > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-3823.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3823: --- Attachment: PHOENIX-3823.patch > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > Attachments: PHOENIX-3823.patch > > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3841) Phoenix View creation failure with Primary table not found error when we use update_cache_frequency for primary table
[ https://issues.apache.org/jira/browse/PHOENIX-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16007121#comment-16007121 ] Maddineni Sukumar commented on PHOENIX-3841: Yes [~jamestaylor] , ran above tests multiple times and they are passing locally except MutableIndexFailureIT which is failing with and without my changes. MutableIndexFailureIT failures looks like flappers. > Phoenix View creation failure with Primary table not found error when we use > update_cache_frequency for primary table > - > > Key: PHOENIX-3841 > URL: https://issues.apache.org/jira/browse/PHOENIX-3841 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar >Priority: Minor > Fix For: 4.11.0 > > Attachments: PHOENIX-3841.patch, PHOENIX-3841.v2.patch, > PHOENIX-3841.v3.patch, PHOENIX-3841.v4.patch > > > Create VIEW command failing with actual table not found error and next retry > failed with VIEW already exists error..And its continuing like that(first > tabelnotfound and then view already exists).. > If I create table without UPDATE_CACHE_FREQUENCY then its working fine. > Create table command: > create table UpdateCacheViewTestB (k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 > VARCHAR) UPDATE_CACHE_FREQUENCY=10; > Create View command: > CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE > v1 = 'value1’; > sqlline Console output: > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> select * from > UPDATECACHEVIEWTESTB; > -- > K V1 V2 > -- > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1013 (42M04): Table already exists. tableName=MY_VIEW > (state=42M04,code=1013) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-2885) Set default value for UPDATE_CACHE_FREQUENCY
[ https://issues.apache.org/jira/browse/PHOENIX-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16007053#comment-16007053 ] Maddineni Sukumar commented on PHOENIX-2885: 1. I will look into parameterizing some tests with default update_cache_frequency param. 2. Regarding priority as of now it is something like below. long updateCacheFrequency = defaultValue!=null?defaultValue:0 if(tableProps.updateCacheFrequency!=null) updateCacheFrequency = tableProps.updateCacheFrequency else if (parentTable!=null) updateCacheFrequency = parentTable.updateCacheFrequency So first priority is to current table specific property if exists, if not then we consider parent table property, if parent table does not exists then only we consider global default configured value. If global default value does not exists then it is zero which is default behavior. So it is same as [~jamestaylor] mentioned above, so I am not changing anything there. > Set default value for UPDATE_CACHE_FREQUENCY > > > Key: PHOENIX-2885 > URL: https://issues.apache.org/jira/browse/PHOENIX-2885 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-2885.4.10-HBase-0.98.patch, PHOENIX-2885.v2.patch > > > We have the ability currently to tell Phoenix how stale we're will to have > the metadata through our UPDATE_CACHE_FREQUENCY feature[1]. By default, when > a table is created, unless explicitly specified, there is no > UPDATE_CACHE_FREQUENCY set. We should provide a new Phoenix configuration > parameter to specify a default value when a CREATE TABLE statement is > executed. In addition, when a VIEW is created, we should inherit the > UPDATE_CACHE_FREQUENCY value from the parent (unless it's explicitly > specified) and then fallback to the new default config value. > [1] https://phoenix.apache.org/language/index.html#options -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-2885) Set default value for UPDATE_CACHE_FREQUENCY
[ https://issues.apache.org/jira/browse/PHOENIX-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16007017#comment-16007017 ] Maddineni Sukumar commented on PHOENIX-2885: Thanks [~apurtell] , my eye didn't catch that. Updated new patch with variable name change. And regarding test failures, I ran them multiple times and they are passing passing locally, and I am sure this patch wont change anything related to above failed tests. This patch is to have configured default value and this unit tests covers all three scenarios related to configuring value. 1. No configured default value which means update_cache_frequency=zero, create table and assert update_cache_frequency=0 2. configure default value to some x and create table and assert table value to x 3. With default value x create a table with table specific property update_cache_frequency=y and assert table is created with value y. I didn't add test to verify behavior of update_cache_frequency as that is already tested code. And some of failures related to that are being fixed in PHOENIX-3823 . > Set default value for UPDATE_CACHE_FREQUENCY > > > Key: PHOENIX-2885 > URL: https://issues.apache.org/jira/browse/PHOENIX-2885 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-2885.4.10-HBase-0.98.patch, PHOENIX-2885.v2.patch > > > We have the ability currently to tell Phoenix how stale we're will to have > the metadata through our UPDATE_CACHE_FREQUENCY feature[1]. By default, when > a table is created, unless explicitly specified, there is no > UPDATE_CACHE_FREQUENCY set. We should provide a new Phoenix configuration > parameter to specify a default value when a CREATE TABLE statement is > executed. In addition, when a VIEW is created, we should inherit the > UPDATE_CACHE_FREQUENCY value from the parent (unless it's explicitly > specified) and then fallback to the new default config value. > [1] https://phoenix.apache.org/language/index.html#options -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-2885) Set default value for UPDATE_CACHE_FREQUENCY
[ https://issues.apache.org/jira/browse/PHOENIX-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-2885: --- Attachment: PHOENIX-2885.v2.patch > Set default value for UPDATE_CACHE_FREQUENCY > > > Key: PHOENIX-2885 > URL: https://issues.apache.org/jira/browse/PHOENIX-2885 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-2885.4.10-HBase-0.98.patch, PHOENIX-2885.v2.patch > > > We have the ability currently to tell Phoenix how stale we're will to have > the metadata through our UPDATE_CACHE_FREQUENCY feature[1]. By default, when > a table is created, unless explicitly specified, there is no > UPDATE_CACHE_FREQUENCY set. We should provide a new Phoenix configuration > parameter to specify a default value when a CREATE TABLE statement is > executed. In addition, when a VIEW is created, we should inherit the > UPDATE_CACHE_FREQUENCY value from the parent (unless it's explicitly > specified) and then fallback to the new default config value. > [1] https://phoenix.apache.org/language/index.html#options -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-2885) Set default value for UPDATE_CACHE_FREQUENCY
[ https://issues.apache.org/jira/browse/PHOENIX-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006718#comment-16006718 ] Maddineni Sukumar commented on PHOENIX-2885: [~jamestaylor] / [~mujtabachohan] / [~samarthjain] could you please review the patch. Thanks. > Set default value for UPDATE_CACHE_FREQUENCY > > > Key: PHOENIX-2885 > URL: https://issues.apache.org/jira/browse/PHOENIX-2885 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-2885.4.10-HBase-0.98.patch > > > We have the ability currently to tell Phoenix how stale we're will to have > the metadata through our UPDATE_CACHE_FREQUENCY feature[1]. By default, when > a table is created, unless explicitly specified, there is no > UPDATE_CACHE_FREQUENCY set. We should provide a new Phoenix configuration > parameter to specify a default value when a CREATE TABLE statement is > executed. In addition, when a VIEW is created, we should inherit the > UPDATE_CACHE_FREQUENCY value from the parent (unless it's explicitly > specified) and then fallback to the new default config value. > [1] https://phoenix.apache.org/language/index.html#options -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3841) Phoenix View creation failure with Primary table not found error when we use update_cache_frequency for primary table
[ https://issues.apache.org/jira/browse/PHOENIX-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16005170#comment-16005170 ] Maddineni Sukumar commented on PHOENIX-3841: Thanks [~singamteja] , attached v4 with variable name change. > Phoenix View creation failure with Primary table not found error when we use > update_cache_frequency for primary table > - > > Key: PHOENIX-3841 > URL: https://issues.apache.org/jira/browse/PHOENIX-3841 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar >Priority: Minor > Fix For: 4.11 > > Attachments: PHOENIX-3841.patch, PHOENIX-3841.v2.patch, > PHOENIX-3841.v3.patch, PHOENIX-3841.v4.patch > > > Create VIEW command failing with actual table not found error and next retry > failed with VIEW already exists error..And its continuing like that(first > tabelnotfound and then view already exists).. > If I create table without UPDATE_CACHE_FREQUENCY then its working fine. > Create table command: > create table UpdateCacheViewTestB (k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 > VARCHAR) UPDATE_CACHE_FREQUENCY=10; > Create View command: > CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE > v1 = 'value1’; > sqlline Console output: > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> select * from > UPDATECACHEVIEWTESTB; > -- > K V1 V2 > -- > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1013 (42M04): Table already exists. tableName=MY_VIEW > (state=42M04,code=1013) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3841) Phoenix View creation failure with Primary table not found error when we use update_cache_frequency for primary table
[ https://issues.apache.org/jira/browse/PHOENIX-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3841: --- Attachment: PHOENIX-3841.v4.patch > Phoenix View creation failure with Primary table not found error when we use > update_cache_frequency for primary table > - > > Key: PHOENIX-3841 > URL: https://issues.apache.org/jira/browse/PHOENIX-3841 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar >Priority: Minor > Fix For: 4.11 > > Attachments: PHOENIX-3841.patch, PHOENIX-3841.v2.patch, > PHOENIX-3841.v3.patch, PHOENIX-3841.v4.patch > > > Create VIEW command failing with actual table not found error and next retry > failed with VIEW already exists error..And its continuing like that(first > tabelnotfound and then view already exists).. > If I create table without UPDATE_CACHE_FREQUENCY then its working fine. > Create table command: > create table UpdateCacheViewTestB (k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 > VARCHAR) UPDATE_CACHE_FREQUENCY=10; > Create View command: > CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE > v1 = 'value1’; > sqlline Console output: > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> select * from > UPDATECACHEVIEWTESTB; > -- > K V1 V2 > -- > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1013 (42M04): Table already exists. tableName=MY_VIEW > (state=42M04,code=1013) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3841) Phoenix View creation failure with Primary table not found error when we use update_cache_frequency for primary table
[ https://issues.apache.org/jira/browse/PHOENIX-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3841: --- Attachment: PHOENIX-3841.v3.patch > Phoenix View creation failure with Primary table not found error when we use > update_cache_frequency for primary table > - > > Key: PHOENIX-3841 > URL: https://issues.apache.org/jira/browse/PHOENIX-3841 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar >Priority: Minor > Fix For: 4.11 > > Attachments: PHOENIX-3841.patch, PHOENIX-3841.v2.patch, > PHOENIX-3841.v3.patch > > > Create VIEW command failing with actual table not found error and next retry > failed with VIEW already exists error..And its continuing like that(first > tabelnotfound and then view already exists).. > If I create table without UPDATE_CACHE_FREQUENCY then its working fine. > Create table command: > create table UpdateCacheViewTestB (k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 > VARCHAR) UPDATE_CACHE_FREQUENCY=10; > Create View command: > CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE > v1 = 'value1’; > sqlline Console output: > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> select * from > UPDATECACHEVIEWTESTB; > -- > K V1 V2 > -- > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1013 (42M04): Table already exists. tableName=MY_VIEW > (state=42M04,code=1013) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3841) Phoenix View creation failure with Primary table not found error when we use update_cache_frequency for primary table
[ https://issues.apache.org/jira/browse/PHOENIX-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003685#comment-16003685 ] Maddineni Sukumar commented on PHOENIX-3841: Yes it fixed the issue I was facing because it got latest values for parent table from server along with timestamp, earlier when we read from cache we were getting values with timestamp -1(QueryConstants.UNSET_TIMESTAMP). Attaching v3 with comments. Thanks [~jamestaylor] > Phoenix View creation failure with Primary table not found error when we use > update_cache_frequency for primary table > - > > Key: PHOENIX-3841 > URL: https://issues.apache.org/jira/browse/PHOENIX-3841 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar >Priority: Minor > Fix For: 4.11 > > Attachments: PHOENIX-3841.patch, PHOENIX-3841.v2.patch, > PHOENIX-3841.v3.patch > > > Create VIEW command failing with actual table not found error and next retry > failed with VIEW already exists error..And its continuing like that(first > tabelnotfound and then view already exists).. > If I create table without UPDATE_CACHE_FREQUENCY then its working fine. > Create table command: > create table UpdateCacheViewTestB (k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 > VARCHAR) UPDATE_CACHE_FREQUENCY=10; > Create View command: > CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE > v1 = 'value1’; > sqlline Console output: > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> select * from > UPDATECACHEVIEWTESTB; > -- > K V1 V2 > -- > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1013 (42M04): Table already exists. tableName=MY_VIEW > (state=42M04,code=1013) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3841) Phoenix View creation failure with Primary table not found error when we use update_cache_frequency for primary table
[ https://issues.apache.org/jira/browse/PHOENIX-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3841: --- Fix Version/s: (was: 4.10.1) 4.11 > Phoenix View creation failure with Primary table not found error when we use > update_cache_frequency for primary table > - > > Key: PHOENIX-3841 > URL: https://issues.apache.org/jira/browse/PHOENIX-3841 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar >Priority: Minor > Fix For: 4.11 > > Attachments: PHOENIX-3841.patch, PHOENIX-3841.v2.patch > > > Create VIEW command failing with actual table not found error and next retry > failed with VIEW already exists error..And its continuing like that(first > tabelnotfound and then view already exists).. > If I create table without UPDATE_CACHE_FREQUENCY then its working fine. > Create table command: > create table UpdateCacheViewTestB (k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 > VARCHAR) UPDATE_CACHE_FREQUENCY=10; > Create View command: > CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE > v1 = 'value1’; > sqlline Console output: > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> select * from > UPDATECACHEVIEWTESTB; > -- > K V1 V2 > -- > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1013 (42M04): Table already exists. tableName=MY_VIEW > (state=42M04,code=1013) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3841) Phoenix View creation failure with Primary table not found error when we use update_cache_frequency for primary table
[ https://issues.apache.org/jira/browse/PHOENIX-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003656#comment-16003656 ] Maddineni Sukumar commented on PHOENIX-3841: Hmm I am newbee in this area so I would simply listen to you :) [~jamestaylor] . Fixed by forcing metadat.updatecache() to always hit server in case of table creation. Attached v2 patch, please take a look. > Phoenix View creation failure with Primary table not found error when we use > update_cache_frequency for primary table > - > > Key: PHOENIX-3841 > URL: https://issues.apache.org/jira/browse/PHOENIX-3841 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar >Priority: Minor > Fix For: 4.11 > > Attachments: PHOENIX-3841.patch, PHOENIX-3841.v2.patch > > > Create VIEW command failing with actual table not found error and next retry > failed with VIEW already exists error..And its continuing like that(first > tabelnotfound and then view already exists).. > If I create table without UPDATE_CACHE_FREQUENCY then its working fine. > Create table command: > create table UpdateCacheViewTestB (k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 > VARCHAR) UPDATE_CACHE_FREQUENCY=10; > Create View command: > CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE > v1 = 'value1’; > sqlline Console output: > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> select * from > UPDATECACHEVIEWTESTB; > -- > K V1 V2 > -- > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1013 (42M04): Table already exists. tableName=MY_VIEW > (state=42M04,code=1013) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3841) Phoenix View creation failure with Primary table not found error when we use update_cache_frequency for primary table
[ https://issues.apache.org/jira/browse/PHOENIX-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3841: --- Attachment: PHOENIX-3841.v2.patch > Phoenix View creation failure with Primary table not found error when we use > update_cache_frequency for primary table > - > > Key: PHOENIX-3841 > URL: https://issues.apache.org/jira/browse/PHOENIX-3841 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar >Priority: Minor > Fix For: 4.10.1 > > Attachments: PHOENIX-3841.patch, PHOENIX-3841.v2.patch > > > Create VIEW command failing with actual table not found error and next retry > failed with VIEW already exists error..And its continuing like that(first > tabelnotfound and then view already exists).. > If I create table without UPDATE_CACHE_FREQUENCY then its working fine. > Create table command: > create table UpdateCacheViewTestB (k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 > VARCHAR) UPDATE_CACHE_FREQUENCY=10; > Create View command: > CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE > v1 = 'value1’; > sqlline Console output: > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> select * from > UPDATECACHEVIEWTESTB; > -- > K V1 V2 > -- > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1013 (42M04): Table already exists. tableName=MY_VIEW > (state=42M04,code=1013) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3841) Phoenix View creation failure with Primary table not found error when we use update_cache_frequency for primary table
[ https://issues.apache.org/jira/browse/PHOENIX-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-3841: --- Attachment: PHOENIX-3841.patch > Phoenix View creation failure with Primary table not found error when we use > update_cache_frequency for primary table > - > > Key: PHOENIX-3841 > URL: https://issues.apache.org/jira/browse/PHOENIX-3841 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.10.0 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar >Priority: Minor > Attachments: PHOENIX-3841.patch > > > Create VIEW command failing with actual table not found error and next retry > failed with VIEW already exists error..And its continuing like that(first > tabelnotfound and then view already exists).. > If I create table without UPDATE_CACHE_FREQUENCY then its working fine. > Create table command: > create table UpdateCacheViewTestB (k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 > VARCHAR) UPDATE_CACHE_FREQUENCY=10; > Create View command: > CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE > v1 = 'value1’; > sqlline Console output: > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> select * from > UPDATECACHEVIEWTESTB; > -- > K V1 V2 > -- > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1013 (42M04): Table already exists. tableName=MY_VIEW > (state=42M04,code=1013) > 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 > VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; > Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB > (state=42M03,code=1012) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (PHOENIX-3841) Phoenix View creation failure with Primary table not found error when we use update_cache_frequency for primary table
Maddineni Sukumar created PHOENIX-3841: -- Summary: Phoenix View creation failure with Primary table not found error when we use update_cache_frequency for primary table Key: PHOENIX-3841 URL: https://issues.apache.org/jira/browse/PHOENIX-3841 Project: Phoenix Issue Type: Sub-task Affects Versions: 4.10.0 Reporter: Maddineni Sukumar Assignee: Maddineni Sukumar Priority: Minor Create VIEW command failing with actual table not found error and next retry failed with VIEW already exists error..And its continuing like that(first tabelnotfound and then view already exists).. If I create table without UPDATE_CACHE_FREQUENCY then its working fine. Create table command: create table UpdateCacheViewTestB (k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 VARCHAR) UPDATE_CACHE_FREQUENCY=10; Create View command: CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1’; sqlline Console output: 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> select * from UPDATECACHEVIEWTESTB; -- K V1 V2 -- 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB (state=42M03,code=1012) 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; Error: ERROR 1013 (42M04): Table already exists. tableName=MY_VIEW (state=42M04,code=1013) 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB (state=42M03,code=1012) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-2885) Set default value for UPDATE_CACHE_FREQUENCY
[ https://issues.apache.org/jira/browse/PHOENIX-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001248#comment-16001248 ] Maddineni Sukumar commented on PHOENIX-2885: And I don't know how to submit patch to update apache phoenix website with this config option here - "https://phoenix.apache.org/language/index.html#options";. Could you please guide me on this. > Set default value for UPDATE_CACHE_FREQUENCY > > > Key: PHOENIX-2885 > URL: https://issues.apache.org/jira/browse/PHOENIX-2885 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-2885.4.10-HBase-0.98.patch > > > We have the ability currently to tell Phoenix how stale we're will to have > the metadata through our UPDATE_CACHE_FREQUENCY feature[1]. By default, when > a table is created, unless explicitly specified, there is no > UPDATE_CACHE_FREQUENCY set. We should provide a new Phoenix configuration > parameter to specify a default value when a CREATE TABLE statement is > executed. In addition, when a VIEW is created, we should inherit the > UPDATE_CACHE_FREQUENCY value from the parent (unless it's explicitly > specified) and then fallback to the new default config value. > [1] https://phoenix.apache.org/language/index.html#options -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-2885) Set default value for UPDATE_CACHE_FREQUENCY
[ https://issues.apache.org/jira/browse/PHOENIX-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001243#comment-16001243 ] Maddineni Sukumar commented on PHOENIX-2885: Hi [~jamestaylor] , submitted patch for default config. Please review it and let me know if there is anything I have to modify. > Set default value for UPDATE_CACHE_FREQUENCY > > > Key: PHOENIX-2885 > URL: https://issues.apache.org/jira/browse/PHOENIX-2885 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-2885.4.10-HBase-0.98.patch > > > We have the ability currently to tell Phoenix how stale we're will to have > the metadata through our UPDATE_CACHE_FREQUENCY feature[1]. By default, when > a table is created, unless explicitly specified, there is no > UPDATE_CACHE_FREQUENCY set. We should provide a new Phoenix configuration > parameter to specify a default value when a CREATE TABLE statement is > executed. In addition, when a VIEW is created, we should inherit the > UPDATE_CACHE_FREQUENCY value from the parent (unless it's explicitly > specified) and then fallback to the new default config value. > [1] https://phoenix.apache.org/language/index.html#options -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-2885) Set default value for UPDATE_CACHE_FREQUENCY
[ https://issues.apache.org/jira/browse/PHOENIX-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-2885: --- Attachment: PHOENIX-2885.4.10-HBase-0.98.patch > Set default value for UPDATE_CACHE_FREQUENCY > > > Key: PHOENIX-2885 > URL: https://issues.apache.org/jira/browse/PHOENIX-2885 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > Attachments: PHOENIX-2885.4.10-HBase-0.98.patch > > > We have the ability currently to tell Phoenix how stale we're will to have > the metadata through our UPDATE_CACHE_FREQUENCY feature[1]. By default, when > a table is created, unless explicitly specified, there is no > UPDATE_CACHE_FREQUENCY set. We should provide a new Phoenix configuration > parameter to specify a default value when a CREATE TABLE statement is > executed. In addition, when a VIEW is created, we should inherit the > UPDATE_CACHE_FREQUENCY value from the parent (unless it's explicitly > specified) and then fallback to the new default config value. > [1] https://phoenix.apache.org/language/index.html#options -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-2885) Set default value for UPDATE_CACHE_FREQUENCY
[ https://issues.apache.org/jira/browse/PHOENIX-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-2885: --- Attachment: PHOENIX-2885.v1.patch > Set default value for UPDATE_CACHE_FREQUENCY > > > Key: PHOENIX-2885 > URL: https://issues.apache.org/jira/browse/PHOENIX-2885 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > > We have the ability currently to tell Phoenix how stale we're will to have > the metadata through our UPDATE_CACHE_FREQUENCY feature[1]. By default, when > a table is created, unless explicitly specified, there is no > UPDATE_CACHE_FREQUENCY set. We should provide a new Phoenix configuration > parameter to specify a default value when a CREATE TABLE statement is > executed. In addition, when a VIEW is created, we should inherit the > UPDATE_CACHE_FREQUENCY value from the parent (unless it's explicitly > specified) and then fallback to the new default config value. > [1] https://phoenix.apache.org/language/index.html#options -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-2885) Set default value for UPDATE_CACHE_FREQUENCY
[ https://issues.apache.org/jira/browse/PHOENIX-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maddineni Sukumar updated PHOENIX-2885: --- Attachment: (was: PHOENIX-2885.v1.patch) > Set default value for UPDATE_CACHE_FREQUENCY > > > Key: PHOENIX-2885 > URL: https://issues.apache.org/jira/browse/PHOENIX-2885 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > > We have the ability currently to tell Phoenix how stale we're will to have > the metadata through our UPDATE_CACHE_FREQUENCY feature[1]. By default, when > a table is created, unless explicitly specified, there is no > UPDATE_CACHE_FREQUENCY set. We should provide a new Phoenix configuration > parameter to specify a default value when a CREATE TABLE statement is > executed. In addition, when a VIEW is created, we should inherit the > UPDATE_CACHE_FREQUENCY value from the parent (unless it's explicitly > specified) and then fallback to the new default config value. > [1] https://phoenix.apache.org/language/index.html#options -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998843#comment-15998843 ] Maddineni Sukumar commented on PHOENIX-3823: FYI - [~jamestaylor] / [~mujtabachohan] Create VIEW command failing with actual table not found error and next retry failed with VIEW already exists error..And its continuing like that(first tabelnotfound and then view already exists).. If I create table without UPDATE_CACHE_FREQUENCY then its working fine. Create table command: create table UpdateCacheViewTestB (k VARCHAR PRIMARY KEY, v1 VARCHAR, v2 VARCHAR) UPDATE_CACHE_FREQUENCY=10; Create View command: CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1’; sqlline Console output: 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> select * from UPDATECACHEVIEWTESTB; +--+--+--+ |K |V1 |V2| +--+--+--+ 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB (state=42M03,code=1012) 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; Error: ERROR 1013 (42M04): Table already exists. tableName=MY_VIEW (state=42M04,code=1013) 0: jdbc:phoenix:shared-mnds1-1-sfm.ops.sfdc.n> CREATE VIEW my_view (v43 VARCHAR) AS SELECT * FROM UpdateCacheViewTestB WHERE v1 = 'value1'; Error: ERROR 1012 (42M03): Table undefined. tableName=UPDATECACHEVIEWTESTB (state=42M03,code=1012) > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15995915#comment-15995915 ] Maddineni Sukumar commented on PHOENIX-3823: Hi [~jamestaylor] , I was able to write unit test and add a catch() block in PhoenixStatement.executeQuery() for MetaDataEntityNotFoundException. But Its hard to get involved TableName of corresponding query as we are not sending table name, schema name in ColumnNotFoundException. For example in PTableImpl.getColumnForColumnName() we are returning exception with just column name but no table and schema name. I added that info and then I am able to reload cache and then query worked fine in retry. It looks like there are other places where we are not sending table details. I am trying to fix those first. > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3823) Force cache update on MetaDataEntityNotFoundException
[ https://issues.apache.org/jira/browse/PHOENIX-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993556#comment-15993556 ] Maddineni Sukumar commented on PHOENIX-3823: Sure James, I cant expect more detailed info with steps, classes to use than this :) . Thanks for making my job smooth during my first patch to Phoenix. Will work on unit test today and patch today. > Force cache update on MetaDataEntityNotFoundException > -- > > Key: PHOENIX-3823 > URL: https://issues.apache.org/jira/browse/PHOENIX-3823 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maddineni Sukumar > > When UPDATE_CACHE_FREQUENCY is used, clients will cache metadata for a period > of time which may cause the schema being used to become stale. If another > client adds a column or a new table or view, other clients won't see it. As a > result, the client will get a MetaDataEntityNotFoundException. Instead of > bubbling this up, we should retry after forcing a cache update on the tables > involved in the query. > The above works well for references to entities that don't yet exist. > However, we cannot detect when some entities are referred to which no longer > exists until the cache expires. An exception is if a physical table is > dropped which would be detected immediately, however we would allow queries > and updates to columns which have been dropped until the cache entry expires > (which seems like a reasonable tradeoff IMHO. In addition, we won't start > using indexes on tables until the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-2885) Set default value for UPDATE_CACHE_FREQUENCY
[ https://issues.apache.org/jira/browse/PHOENIX-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993363#comment-15993363 ] Maddineni Sukumar commented on PHOENIX-2885: I am thinking below are the scenarios we need to consider(correct me if below list is wrong). Will check current behavior after making default to 10 sec. 1)Table modify like add/remove column 2)Table drop 3)Index add 4)Index modify? is it possible? 5)Index drop Above for both Local index and secondary index 6)View create 7)View modify? is it possible 8)View drop Thanks > Set default value for UPDATE_CACHE_FREQUENCY > > > Key: PHOENIX-2885 > URL: https://issues.apache.org/jira/browse/PHOENIX-2885 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Maddineni Sukumar > Fix For: 4.11.0 > > > Instead of by default doing an extra RPC per query, we should have a default > value for UPDATE_CACHE_FREQUENCY. Instead of throwing when an entity > (table/column/column family, etc) is not found, we can force an update of the > client-side cache and retry the statement once. This works well for > references to entities that don't yet exist. However, we won't detect when > some entities are referred to which no longer exists until the cache expires. > An exception is if a physical table is dropped which would be detected > immediately. Worst-case, we may allow queries and updates to columns which > have been dropped. In addition, we won't start using indexes on tables until > the cache expires. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3819) Reduce Phoenix load on RS hosting SYSTEM.CATALOG region
[ https://issues.apache.org/jira/browse/PHOENIX-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15991379#comment-15991379 ] Maddineni Sukumar commented on PHOENIX-3819: FYI - [~jamestaylor] / [~lhofhansl] / [~apurtell] > Reduce Phoenix load on RS hosting SYSTEM.CATALOG region > --- > > Key: PHOENIX-3819 > URL: https://issues.apache.org/jira/browse/PHOENIX-3819 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 4.10.0 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar > > Currently Phoenix puts some load on the Region Server hosting SYSTEM.CATALOG. > Let's review that and reduce that load. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (PHOENIX-3819) Reduce Phoenix load on RS hosting SYSTEM.CATALOG region
Maddineni Sukumar created PHOENIX-3819: -- Summary: Reduce Phoenix load on RS hosting SYSTEM.CATALOG region Key: PHOENIX-3819 URL: https://issues.apache.org/jira/browse/PHOENIX-3819 Project: Phoenix Issue Type: Improvement Affects Versions: 4.10.0 Reporter: Maddineni Sukumar Assignee: Maddineni Sukumar Currently Phoenix puts some load on the Region Server hosting SYSTEM.CATALOG. Let's review that and reduce that load. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (PHOENIX-3812) Explore using HBase snapshots in async index building M/R job
Maddineni Sukumar created PHOENIX-3812: -- Summary: Explore using HBase snapshots in async index building M/R job Key: PHOENIX-3812 URL: https://issues.apache.org/jira/browse/PHOENIX-3812 Project: Phoenix Issue Type: Improvement Affects Versions: 4.10.0 Reporter: Maddineni Sukumar As per discussion with James, HBase snapshots makes it lot easier and faster to operate on existing data. So explore using HBase snapshots in index building M/R job for async index. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
