[jira] [Updated] (PHOENIX-5998) Paged server side ungrouped aggregate operations
[ https://issues.apache.org/jira/browse/PHOENIX-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-5998: -- Fix Version/s: 5.1.0 > Paged server side ungrouped aggregate operations > - > > Key: PHOENIX-5998 > URL: https://issues.apache.org/jira/browse/PHOENIX-5998 > Project: Phoenix > Issue Type: Improvement >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-5998.4.x.001.patch, PHOENIX-5998.4.x.002.patch, > PHOENIX-5998.4.x.003.patch, PHOENIX-5998.4.x.005.patch, > PHOENIX-5998.4.x.006.patch, PHOENIX-5998.master.001.patch, > PHOENIX-5998.master.002.patch > > > Phoenix provides the option of performing upsert select and delete query > operations on the client or server side. This is decided by the Phoenix > optimizer based on configuration parameters. For the server side option, the > table operation (upsert select/delete query) is parallelized such that > multiple table regions are scanned and the mutations derived from these scans > can also be executed in parallel on the server side. However, currently there > is no paging capability and the server side operation can take long enough to > lead to HBase client timeouts. When this happens, Phoenix return failure to > its applications and the rest of the parallel scans and mutations on the > server side can still continue since Phoenix has no mechanism in place to > stop these operations before returning failure to applications. This can > create unexpected race conditions between these left-over operations and the > new operations issued by applications. > Putting a limit on the time that can be spent within a single next operation > on the region scanner implemented by a Phoenix coproc using a Phoenix level > paging is highly desirable to eliminate timeouts due to long running > operations on the server side, and is a required step to prevent operation > failures and possible race conditions following these failures. > A similar paging mechanism has been already implemented for index rebuild and > verification operations and proven to be effective to prevent timeouts. This > paging can be implemented for all server side operations including > aggregates, upsert selects, delete queries and so on. This jira is for > implementing such a paging capability for the coproc > UngroupedAggregateRegionObserver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5998) Paged server side ungrouped aggregate operations
[ https://issues.apache.org/jira/browse/PHOENIX-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-5998: --- Description: Phoenix provides the option of performing upsert select and delete query operations on the client or server side. This is decided by the Phoenix optimizer based on configuration parameters. For the server side option, the table operation (upsert select/delete query) is parallelized such that multiple table regions are scanned and the mutations derived from these scans can also be executed in parallel on the server side. However, currently there is no paging capability and the server side operation can take long enough to lead to HBase client timeouts. When this happens, Phoenix return failure to its applications and the rest of the parallel scans and mutations on the server side can still continue since Phoenix has no mechanism in place to stop these operations before returning failure to applications. This can create unexpected race conditions between these left-over operations and the new operations issued by applications. Putting a limit on the time that can be spent within a single next operation on the region scanner implemented by a Phoenix coproc using a Phoenix level paging is highly desirable to eliminate timeouts due to long running operations on the server side, and is a required step to prevent operation failures and possible race conditions following these failures. A similar paging mechanism has been already implemented for index rebuild and verification operations and proven to be effective to prevent timeouts. This paging can be implemented for all server side operations including aggregates, upsert selects, delete queries and so on. This jira is for implementing such a paging capability for the coproc UngroupedAggregateRegionObserver. was:Phoenix provides the option of performing upsert select and delete query operations on the client or server side. This is decided by the Phoenix optimizer based on configuration parameters. For the server side option, the table operation (upsert select/delete query) is parallelized such that multiple table regions are scanned and the mutations derived from these scans can also be executed in parallel on the server side. However, currently there is no paging capability and the server side operation can take long enough lead to HBase client timeouts. When this happens, Phoenix can return failure to its applications and the rest of the parallel scans and mutations on the server side can still continue since Phoenix has no mechanism in place to stop these operations before returning failure to applications. This can create unexpected race conditions between these left-over operations and the new operations issued by applications. Putting a limit on the number of rows to be processed within a single RPC call (i.e., the next operation on the scanner) on the server side using a Phoenix level paging is highly desirable and a required step to prevent the possible race conditions. This paging mechanism has been already implemented for index rebuild and verification operations and proven to be effective to prevent timeouts. This paging can be implemented for all server side operations including aggregates, upsert selects, delete queries and so on. > Paged server side ungrouped aggregate operations > - > > Key: PHOENIX-5998 > URL: https://issues.apache.org/jira/browse/PHOENIX-5998 > Project: Phoenix > Issue Type: Improvement >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 4.16.0 > > Attachments: PHOENIX-5998.4.x.001.patch, PHOENIX-5998.4.x.002.patch, > PHOENIX-5998.4.x.003.patch, PHOENIX-5998.4.x.005.patch, > PHOENIX-5998.4.x.006.patch, PHOENIX-5998.master.001.patch, > PHOENIX-5998.master.002.patch > > > Phoenix provides the option of performing upsert select and delete query > operations on the client or server side. This is decided by the Phoenix > optimizer based on configuration parameters. For the server side option, the > table operation (upsert select/delete query) is parallelized such that > multiple table regions are scanned and the mutations derived from these scans > can also be executed in parallel on the server side. However, currently there > is no paging capability and the server side operation can take long enough to > lead to HBase client timeouts. When this happens, Phoenix return failure to > its applications and the rest of the parallel scans and mutations on the > server side can still continue since Phoenix has no mechanism in place to > stop these operations before returning failure to applications. This can > create unexpected race conditions between these left-over operations
[jira] [Updated] (PHOENIX-5998) Paged server side ungrouped aggregate operations
[ https://issues.apache.org/jira/browse/PHOENIX-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-5998: --- Attachment: PHOENIX-5998.4.x.006.patch > Paged server side ungrouped aggregate operations > - > > Key: PHOENIX-5998 > URL: https://issues.apache.org/jira/browse/PHOENIX-5998 > Project: Phoenix > Issue Type: Improvement >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 4.16.0 > > Attachments: PHOENIX-5998.4.x.001.patch, PHOENIX-5998.4.x.002.patch, > PHOENIX-5998.4.x.003.patch, PHOENIX-5998.4.x.005.patch, > PHOENIX-5998.4.x.006.patch, PHOENIX-5998.master.001.patch, > PHOENIX-5998.master.002.patch > > > Phoenix provides the option of performing upsert select and delete query > operations on the client or server side. This is decided by the Phoenix > optimizer based on configuration parameters. For the server side option, the > table operation (upsert select/delete query) is parallelized such that > multiple table regions are scanned and the mutations derived from these scans > can also be executed in parallel on the server side. However, currently there > is no paging capability and the server side operation can take long enough > lead to HBase client timeouts. When this happens, Phoenix can return failure > to its applications and the rest of the parallel scans and mutations on the > server side can still continue since Phoenix has no mechanism in place to > stop these operations before returning failure to applications. This can > create unexpected race conditions between these left-over operations and the > new operations issued by applications. Putting a limit on the number of rows > to be processed within a single RPC call (i.e., the next operation on the > scanner) on the server side using a Phoenix level paging is highly desirable > and a required step to prevent the possible race conditions. This paging > mechanism has been already implemented for index rebuild and verification > operations and proven to be effective to prevent timeouts. This paging can be > implemented for all server side operations including aggregates, upsert > selects, delete queries and so on. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5998) Paged server side ungrouped aggregate operations
[ https://issues.apache.org/jira/browse/PHOENIX-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-5998: --- Attachment: PHOENIX-5998.master.002.patch > Paged server side ungrouped aggregate operations > - > > Key: PHOENIX-5998 > URL: https://issues.apache.org/jira/browse/PHOENIX-5998 > Project: Phoenix > Issue Type: Improvement >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 4.16.0 > > Attachments: PHOENIX-5998.4.x.001.patch, PHOENIX-5998.4.x.002.patch, > PHOENIX-5998.4.x.003.patch, PHOENIX-5998.4.x.005.patch, > PHOENIX-5998.4.x.006.patch, PHOENIX-5998.master.001.patch, > PHOENIX-5998.master.002.patch > > > Phoenix provides the option of performing upsert select and delete query > operations on the client or server side. This is decided by the Phoenix > optimizer based on configuration parameters. For the server side option, the > table operation (upsert select/delete query) is parallelized such that > multiple table regions are scanned and the mutations derived from these scans > can also be executed in parallel on the server side. However, currently there > is no paging capability and the server side operation can take long enough > lead to HBase client timeouts. When this happens, Phoenix can return failure > to its applications and the rest of the parallel scans and mutations on the > server side can still continue since Phoenix has no mechanism in place to > stop these operations before returning failure to applications. This can > create unexpected race conditions between these left-over operations and the > new operations issued by applications. Putting a limit on the number of rows > to be processed within a single RPC call (i.e., the next operation on the > scanner) on the server side using a Phoenix level paging is highly desirable > and a required step to prevent the possible race conditions. This paging > mechanism has been already implemented for index rebuild and verification > operations and proven to be effective to prevent timeouts. This paging can be > implemented for all server side operations including aggregates, upsert > selects, delete queries and so on. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5998) Paged server side ungrouped aggregate operations
[ https://issues.apache.org/jira/browse/PHOENIX-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-5998: --- Attachment: PHOENIX-5998.4.x.005.patch > Paged server side ungrouped aggregate operations > - > > Key: PHOENIX-5998 > URL: https://issues.apache.org/jira/browse/PHOENIX-5998 > Project: Phoenix > Issue Type: Improvement >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 4.16.0 > > Attachments: PHOENIX-5998.4.x.001.patch, PHOENIX-5998.4.x.002.patch, > PHOENIX-5998.4.x.003.patch, PHOENIX-5998.4.x.005.patch, > PHOENIX-5998.master.001.patch > > > Phoenix provides the option of performing upsert select and delete query > operations on the client or server side. This is decided by the Phoenix > optimizer based on configuration parameters. For the server side option, the > table operation (upsert select/delete query) is parallelized such that > multiple table regions are scanned and the mutations derived from these scans > can also be executed in parallel on the server side. However, currently there > is no paging capability and the server side operation can take long enough > lead to HBase client timeouts. When this happens, Phoenix can return failure > to its applications and the rest of the parallel scans and mutations on the > server side can still continue since Phoenix has no mechanism in place to > stop these operations before returning failure to applications. This can > create unexpected race conditions between these left-over operations and the > new operations issued by applications. Putting a limit on the number of rows > to be processed within a single RPC call (i.e., the next operation on the > scanner) on the server side using a Phoenix level paging is highly desirable > and a required step to prevent the possible race conditions. This paging > mechanism has been already implemented for index rebuild and verification > operations and proven to be effective to prevent timeouts. This paging can be > implemented for all server side operations including aggregates, upsert > selects, delete queries and so on. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5998) Paged server side ungrouped aggregate operations
[ https://issues.apache.org/jira/browse/PHOENIX-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-5998: --- Attachment: PHOENIX-5998.master.001.patch > Paged server side ungrouped aggregate operations > - > > Key: PHOENIX-5998 > URL: https://issues.apache.org/jira/browse/PHOENIX-5998 > Project: Phoenix > Issue Type: Improvement >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 4.16.0 > > Attachments: PHOENIX-5998.4.x.001.patch, PHOENIX-5998.4.x.002.patch, > PHOENIX-5998.4.x.003.patch, PHOENIX-5998.master.001.patch > > > Phoenix provides the option of performing upsert select and delete query > operations on the client or server side. This is decided by the Phoenix > optimizer based on configuration parameters. For the server side option, the > table operation (upsert select/delete query) is parallelized such that > multiple table regions are scanned and the mutations derived from these scans > can also be executed in parallel on the server side. However, currently there > is no paging capability and the server side operation can take long enough > lead to HBase client timeouts. When this happens, Phoenix can return failure > to its applications and the rest of the parallel scans and mutations on the > server side can still continue since Phoenix has no mechanism in place to > stop these operations before returning failure to applications. This can > create unexpected race conditions between these left-over operations and the > new operations issued by applications. Putting a limit on the number of rows > to be processed within a single RPC call (i.e., the next operation on the > scanner) on the server side using a Phoenix level paging is highly desirable > and a required step to prevent the possible race conditions. This paging > mechanism has been already implemented for index rebuild and verification > operations and proven to be effective to prevent timeouts. This paging can be > implemented for all server side operations including aggregates, upsert > selects, delete queries and so on. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5998) Paged server side ungrouped aggregate operations
[ https://issues.apache.org/jira/browse/PHOENIX-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni updated PHOENIX-5998: -- Fix Version/s: (was: 4.x) 4.16.0 > Paged server side ungrouped aggregate operations > - > > Key: PHOENIX-5998 > URL: https://issues.apache.org/jira/browse/PHOENIX-5998 > Project: Phoenix > Issue Type: Improvement >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 4.16.0 > > Attachments: PHOENIX-5998.4.x.001.patch, PHOENIX-5998.4.x.002.patch, > PHOENIX-5998.4.x.003.patch > > > Phoenix provides the option of performing upsert select and delete query > operations on the client or server side. This is decided by the Phoenix > optimizer based on configuration parameters. For the server side option, the > table operation (upsert select/delete query) is parallelized such that > multiple table regions are scanned and the mutations derived from these scans > can also be executed in parallel on the server side. However, currently there > is no paging capability and the server side operation can take long enough > lead to HBase client timeouts. When this happens, Phoenix can return failure > to its applications and the rest of the parallel scans and mutations on the > server side can still continue since Phoenix has no mechanism in place to > stop these operations before returning failure to applications. This can > create unexpected race conditions between these left-over operations and the > new operations issued by applications. Putting a limit on the number of rows > to be processed within a single RPC call (i.e., the next operation on the > scanner) on the server side using a Phoenix level paging is highly desirable > and a required step to prevent the possible race conditions. This paging > mechanism has been already implemented for index rebuild and verification > operations and proven to be effective to prevent timeouts. This paging can be > implemented for all server side operations including aggregates, upsert > selects, delete queries and so on. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5998) Paged server side ungrouped aggregate operations
[ https://issues.apache.org/jira/browse/PHOENIX-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-5998: --- Attachment: PHOENIX-5998.4.x.003.patch > Paged server side ungrouped aggregate operations > - > > Key: PHOENIX-5998 > URL: https://issues.apache.org/jira/browse/PHOENIX-5998 > Project: Phoenix > Issue Type: Improvement >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 4.x > > Attachments: PHOENIX-5998.4.x.001.patch, PHOENIX-5998.4.x.002.patch, > PHOENIX-5998.4.x.003.patch > > > Phoenix provides the option of performing upsert select and delete query > operations on the client or server side. This is decided by the Phoenix > optimizer based on configuration parameters. For the server side option, the > table operation (upsert select/delete query) is parallelized such that > multiple table regions are scanned and the mutations derived from these scans > can also be executed in parallel on the server side. However, currently there > is no paging capability and the server side operation can take long enough > lead to HBase client timeouts. When this happens, Phoenix can return failure > to its applications and the rest of the parallel scans and mutations on the > server side can still continue since Phoenix has no mechanism in place to > stop these operations before returning failure to applications. This can > create unexpected race conditions between these left-over operations and the > new operations issued by applications. Putting a limit on the number of rows > to be processed within a single RPC call (i.e., the next operation on the > scanner) on the server side using a Phoenix level paging is highly desirable > and a required step to prevent the possible race conditions. This paging > mechanism has been already implemented for index rebuild and verification > operations and proven to be effective to prevent timeouts. This paging can be > implemented for all server side operations including aggregates, upsert > selects, delete queries and so on. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5998) Paged server side ungrouped aggregate operations
[ https://issues.apache.org/jira/browse/PHOENIX-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-5998: --- Attachment: PHOENIX-5998.4.x.002.patch > Paged server side ungrouped aggregate operations > - > > Key: PHOENIX-5998 > URL: https://issues.apache.org/jira/browse/PHOENIX-5998 > Project: Phoenix > Issue Type: Improvement >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 4.x > > Attachments: PHOENIX-5998.4.x.001.patch, PHOENIX-5998.4.x.002.patch > > > Phoenix provides the option of performing upsert select and delete query > operations on the client or server side. This is decided by the Phoenix > optimizer based on configuration parameters. For the server side option, the > table operation (upsert select/delete query) is parallelized such that > multiple table regions are scanned and the mutations derived from these scans > can also be executed in parallel on the server side. However, currently there > is no paging capability and the server side operation can take long enough > lead to HBase client timeouts. When this happens, Phoenix can return failure > to its applications and the rest of the parallel scans and mutations on the > server side can still continue since Phoenix has no mechanism in place to > stop these operations before returning failure to applications. This can > create unexpected race conditions between these left-over operations and the > new operations issued by applications. Putting a limit on the number of rows > to be processed within a single RPC call (i.e., the next operation on the > scanner) on the server side using a Phoenix level paging is highly desirable > and a required step to prevent the possible race conditions. This paging > mechanism has been already implemented for index rebuild and verification > operations and proven to be effective to prevent timeouts. This paging can be > implemented for all server side operations including aggregates, upsert > selects, delete queries and so on. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5998) Paged server side ungrouped aggregate operations
[ https://issues.apache.org/jira/browse/PHOENIX-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-5998: --- Summary: Paged server side ungrouped aggregate operations (was: Paged server side operations ) > Paged server side ungrouped aggregate operations > - > > Key: PHOENIX-5998 > URL: https://issues.apache.org/jira/browse/PHOENIX-5998 > Project: Phoenix > Issue Type: Improvement >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 4.x > > > Phoenix provides the option of performing upsert select and delete query > operations on the client or server side. This is decided by the Phoenix > optimizer based on configuration parameters. For the server side option, the > table operation (upsert select/delete query) is parallelized such that > multiple table regions are scanned and the mutations derived from these scans > can also be executed in parallel on the server side. However, currently there > is no paging capability and the server side operation can take long enough > lead to HBase client timeouts. When this happens, Phoenix can return failure > to its applications and the rest of the parallel scans and mutations on the > server side can still continue since Phoenix has no mechanism in place to > stop these operations before returning failure to applications. This can > create unexpected race conditions between these left-over operations and the > new operations issued by applications. Putting a limit on the number of rows > to be processed within a single RPC call (i.e., the next operation on the > scanner) on the server side using a Phoenix level paging is highly desirable > and a required step to prevent the possible race conditions. This paging > mechanism has been already implemented for index rebuild and verification > operations and proven to be effective to prevent timeouts. This paging can be > implemented for all server side operations including aggregates, upsert > selects, delete queries and so on. -- This message was sent by Atlassian Jira (v8.3.4#803005)
