[jira] [Resolved] (PHOENIX-6188) Jenkins job history uses too much storage

2020-10-20 Thread Istvan Toth (Jira) 


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Istvan Toth resolved PHOENIX-6188.
--
Resolution: Fixed

Looks good now.

> Jenkins job history uses too much storage
> -
>
> Key: PHOENIX-6188
> URL: https://issues.apache.org/jira/browse/PHOENIX-6188
> Project: Phoenix
>  Issue Type: Bug
>  Components: connectors, core
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Blocker
>
> The ASF Jenkins host gets filled up with Phoenix job data.
> Reduce the time discard times to use less pace.
> Try to use
> _buildDiscarder_(logRotator(daysToKeepStr: '15', artifactDaysToKeepStr: '5'))
> as a first iteration.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6124) Block adding/dropping a column on a parent view for clients <4.15 and for clients that have phoenix.allow.system.catalog.rollback=true

2020-10-20 Thread Viraj Jasani (Jira) 


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated PHOENIX-6124:
--
Attachment: (was: PHOENIX-6124.4.x.000.patch)

> Block adding/dropping a column on a parent view for clients <4.15 and for 
> clients that have phoenix.allow.system.catalog.rollback=true
> --
>
> Key: PHOENIX-6124
> URL: https://issues.apache.org/jira/browse/PHOENIX-6124
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Assignee: Viraj Jasani
>Priority: Blocker
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6124.4.x.000.patch, PHOENIX-6124.master.000.patch
>
>
> For pre-4.15 clients, we have to block adding/dropping a column to a table if 
> it has child views since we can’t do any checkAndPut distributed locking (see 
> [this|https://github.com/apache/phoenix/blob/6ecc66738e576a5349605c2f5b20003df03f95de/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2595-L2616]).
>  
> However, this is only prevented if the parent is a table and not if the 
> parent is a view. We should extend [the 
> condition|https://github.com/apache/phoenix/blob/6ecc66738e576a5349605c2f5b20003df03f95de/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2591]
>  to also cover views since conflicting mutations on its children can also 
> lead to inconsistencies.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6124) Block adding/dropping a column on a parent view for clients <4.15 and for clients that have phoenix.allow.system.catalog.rollback=true

2020-10-20 Thread Viraj Jasani (Jira) 


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated PHOENIX-6124:
--
Attachment: PHOENIX-6124.master.000.patch

> Block adding/dropping a column on a parent view for clients <4.15 and for 
> clients that have phoenix.allow.system.catalog.rollback=true
> --
>
> Key: PHOENIX-6124
> URL: https://issues.apache.org/jira/browse/PHOENIX-6124
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Assignee: Viraj Jasani
>Priority: Blocker
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6124.4.x.000.patch, PHOENIX-6124.master.000.patch
>
>
> For pre-4.15 clients, we have to block adding/dropping a column to a table if 
> it has child views since we can’t do any checkAndPut distributed locking (see 
> [this|https://github.com/apache/phoenix/blob/6ecc66738e576a5349605c2f5b20003df03f95de/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2595-L2616]).
>  
> However, this is only prevented if the parent is a table and not if the 
> parent is a view. We should extend [the 
> condition|https://github.com/apache/phoenix/blob/6ecc66738e576a5349605c2f5b20003df03f95de/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2591]
>  to also cover views since conflicting mutations on its children can also 
> lead to inconsistencies.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (PHOENIX-5998) Paged server side operations

2020-10-20 Thread Kadir OZDEMIR (Jira) 


 [ 
https://issues.apache.org/jira/browse/PHOENIX-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kadir OZDEMIR reassigned PHOENIX-5998:
--

Assignee: Kadir OZDEMIR

> Paged server side operations 
> -
>
> Key: PHOENIX-5998
> URL: https://issues.apache.org/jira/browse/PHOENIX-5998
> Project: Phoenix
>  Issue Type: Improvement
> Environment: Phoenix provides the option of performing upsert select 
> and delete query operations on the client or server side.  This is decided by 
> the Phoenix optimizer based on configuration parameters. For the server side 
> option, the table operation (upsert select/delete query) is parallelized such 
> that multiple table regions are scanned and the mutations derived from these 
> scans can also be executed in parallel on the server side. However, currently 
> there is no paging capability and the server side operation can take long 
> enough lead to HBase client timeouts. When this happens, Phoenix can return 
> failure to its applications and the rest of the parallel scans and mutations 
> on the server side can still continue since  Phoenix has no mechanism in 
> place to stop these operations before returning failure to applications. This 
> can create unexpected race conditions between these left-over operations and 
> the new operations issued by applications. Putting a limit on the number of 
> rows to be processed within a single RPC call (i.e., the next operation on 
> the scanner) on the server side using a Phoenix level paging is highly 
> desirable and a required step to prevent the possible race conditions. This 
> paging mechanism has been already implemented for index rebuild and 
> verification operations and proven to be effective to prevent timeouts. This 
> paging can be implemented for all server side operations including 
> aggregates, upsert selects, delete queries and so on.
>Reporter: Kadir OZDEMIR
>Assignee: Kadir OZDEMIR
>Priority: Major
> Fix For: 4.x
>
>
> Phoenix provides the option of performing upsert select and delete query 
> operations on the client or server side.  This is decided by the Phoenix 
> optimizer based on configuration parameters. For the server side option, the 
> table operation (upsert select/delete query) is parallelized such that 
> multiple table regions are scanned and the mutations derived from these scans 
> can also be executed in parallel on the server side. However, currently there 
> is no paging capability and the server side operation can take long enough 
> lead to HBase client timeouts. When this happens, Phoenix can return failure 
> to its applications and the rest of the parallel scans and mutations on the 
> server side can still continue since  Phoenix has no mechanism in place to 
> stop these operations before returning failure to applications. This can 
> create unexpected race conditions between these left-over operations and the 
> new operations issued by applications. Putting a limit on the number of rows 
> to be processed within a single RPC call (i.e., the next operation on the 
> scanner) on the server side using a Phoenix level paging is highly desirable 
> and a required step to prevent the possible race conditions. This paging 
> mechanism has been already implemented for index rebuild and verification 
> operations and proven to be effective to prevent timeouts. This paging can be 
> implemented for all server side operations including aggregates, upsert 
> selects, delete queries and so on.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-5998) Paged server side operations

2020-10-20 Thread Kadir OZDEMIR (Jira) 


 [ 
https://issues.apache.org/jira/browse/PHOENIX-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kadir OZDEMIR updated PHOENIX-5998:
---
Environment: (was: Phoenix provides the option of performing upsert 
select and delete query operations on the client or server side.  This is 
decided by the Phoenix optimizer based on configuration parameters. For the 
server side option, the table operation (upsert select/delete query) is 
parallelized such that multiple table regions are scanned and the mutations 
derived from these scans can also be executed in parallel on the server side. 
However, currently there is no paging capability and the server side operation 
can take long enough lead to HBase client timeouts. When this happens, Phoenix 
can return failure to its applications and the rest of the parallel scans and 
mutations on the server side can still continue since  Phoenix has no mechanism 
in place to stop these operations before returning failure to applications. 
This can create unexpected race conditions between these left-over operations 
and the new operations issued by applications. Putting a limit on the number of 
rows to be processed within a single RPC call (i.e., the next operation on the 
scanner) on the server side using a Phoenix level paging is highly desirable 
and a required step to prevent the possible race conditions. This paging 
mechanism has been already implemented for index rebuild and verification 
operations and proven to be effective to prevent timeouts. This paging can be 
implemented for all server side operations including aggregates, upsert 
selects, delete queries and so on.)

> Paged server side operations 
> -
>
> Key: PHOENIX-5998
> URL: https://issues.apache.org/jira/browse/PHOENIX-5998
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Kadir OZDEMIR
>Assignee: Kadir OZDEMIR
>Priority: Major
> Fix For: 4.x
>
>
> Phoenix provides the option of performing upsert select and delete query 
> operations on the client or server side.  This is decided by the Phoenix 
> optimizer based on configuration parameters. For the server side option, the 
> table operation (upsert select/delete query) is parallelized such that 
> multiple table regions are scanned and the mutations derived from these scans 
> can also be executed in parallel on the server side. However, currently there 
> is no paging capability and the server side operation can take long enough 
> lead to HBase client timeouts. When this happens, Phoenix can return failure 
> to its applications and the rest of the parallel scans and mutations on the 
> server side can still continue since  Phoenix has no mechanism in place to 
> stop these operations before returning failure to applications. This can 
> create unexpected race conditions between these left-over operations and the 
> new operations issued by applications. Putting a limit on the number of rows 
> to be processed within a single RPC call (i.e., the next operation on the 
> scanner) on the server side using a Phoenix level paging is highly desirable 
> and a required step to prevent the possible race conditions. This paging 
> mechanism has been already implemented for index rebuild and verification 
> operations and proven to be effective to prevent timeouts. This paging can be 
> implemented for all server side operations including aggregates, upsert 
> selects, delete queries and so on.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (PHOENIX-6196) Update phoenix.mutate.maxSizeBytes to accept long values

2020-10-20 Thread Ankit Singhal (Jira) 


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankit Singhal resolved PHOENIX-6196.

Fix Version/s: 4.16.0
   5.1.0
   Resolution: Fixed

> Update phoenix.mutate.maxSizeBytes to accept long values
> 
>
> Key: PHOENIX-6196
> URL: https://issues.apache.org/jira/browse/PHOENIX-6196
> Project: Phoenix
>  Issue Type: Task
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
>
> Currently, the config "phoenix.mutate.maxSizeBytes" accepts value in Int, so 
> a user can only provide up to 2GB but there are some scenarios like UPSERT 
> SELECT from temp table with large rows, where the user may want to set more 
> value to it when auto-commit is off. 
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] Release of phoenix-thirdparty 1.0.0

2020-10-20 Thread Ankit Singhal 
+1(binding)

* License and Notice - OK
* Build from the source - OK
* Used the above-created artifact 1.0.0 and build the Phoenix master - OK
* Signature/checksums - Ok
* License header in files - Ok
* Changes and ReleaseNotes - Ok

Note: Signatures are verified by using
your 825203A70405BC83AECF5F7D97351C1B794433C7

Regards,
Ankit Singhal

On Wed, Oct 14, 2020 at 12:05 AM Istvan Toth  wrote:

> Please vote on this Apache phoenix-thirdparty release candidate,
> phoenix-thirdparty-1.0.0RC0
>
> The VOTE will remain open for at least 72 hours.
>
> [ ] +1 Release this package as Apache phoenix-thirdparty 1.0.0
> [ ] -1 Do not release this package because ...
>
> The tag to be voted on is 1.0.0RC0
>
>   https://github.com/apache/phoenix-thirdparty/tree/1.0.0RC0
>
> The release files, including signatures, digests, as well as CHANGES.md
> and RELEASENOTES.md included in this RC can be found at:
>
>
> https://dist.apache.org/repos/dist/dev/phoenix/phoenix-thirdparty-1.0.0RC0/
>
> Artifacts were signed with the ${GPG_KEY} key which can be found in:
>
>   https://dist.apache.org/repos/dist/release/phoenix/KEYS
>
> To learn more about Apache Phoenix, please see
>
>   http://phoenix.apache.org/
>
> Thanks,
> Istvan
>

[jira] [Created] (PHOENIX-6197) AggregateIT and StoreNullsIT hangs

2020-10-20 Thread Istvan Toth (Jira) 
Istvan Toth created PHOENIX-6197:


 Summary: AggregateIT and StoreNullsIT hangs
 Key: PHOENIX-6197
 URL: https://issues.apache.org/jira/browse/PHOENIX-6197
 Project: Phoenix
  Issue Type: Bug
  Components: core
Affects Versions: 4.16.0
Reporter: Istvan Toth


Recent 4.x builds don't finish.

The maven output shows that AggregateIT and StoreNullsIT are started, but never 
complete.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6197) AggregateIT and StoreNullsIT hangs

2020-10-20 Thread Istvan Toth (Jira) 


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Istvan Toth updated PHOENIX-6197:
-
Affects Version/s: 5.1.0

> AggregateIT and StoreNullsIT hangs
> --
>
> Key: PHOENIX-6197
> URL: https://issues.apache.org/jira/browse/PHOENIX-6197
> Project: Phoenix
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.1.0, 4.16.0
>Reporter: Istvan Toth
>Priority: Blocker
>
> Recent 4.x builds don't finish.
> The maven output shows that AggregateIT and StoreNullsIT are started, but 
> never complete.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (PHOENIX-6197) AggregateIT and StoreNullsIT hangs

2020-10-20 Thread Istvan Toth (Jira) 


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Istvan Toth updated PHOENIX-6197:
-
Description: 
Recent 4.x builds don't finish.

The maven output shows that AggregateIT and StoreNullsIT are started, but never 
complete.

All 4.x CI builds seem to be affected.

I can see the same hang on some master builds (on HBase 2.3, though that may be 
a coincidence)

  was:
Recent 4.x builds don't finish.

The maven output shows that AggregateIT and StoreNullsIT are started, but never 
complete.

 


> AggregateIT and StoreNullsIT hangs
> --
>
> Key: PHOENIX-6197
> URL: https://issues.apache.org/jira/browse/PHOENIX-6197
> Project: Phoenix
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.1.0, 4.16.0
>Reporter: Istvan Toth
>Priority: Blocker
>
> Recent 4.x builds don't finish.
> The maven output shows that AggregateIT and StoreNullsIT are started, but 
> never complete.
> All 4.x CI builds seem to be affected.
> I can see the same hang on some master builds (on HBase 2.3, though that may 
> be a coincidence)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[DISCUSS] Phoenix bot account for GitHub

2020-10-20 Thread Istvan Toth 
Hi!

I've recently implemented the Yetus PR checks for GitHub PR, which for the
most part seem to work well.

However, it seems that none of the available GitHub credentials in Jenkins
let the job comment on the PR, so ATM the jobs are using my GitHub account.
It is not very professional looking, and I get spammed with mail on every
entry on every ticket that has a PR, which makes life difficult for me.

Looking at HBase (as ever), they have created a bot account, and are using
it for the same purpose.

I propose that we do similarly. The GitHub docs

seem
to indicate that it is OK.

One open question is how to share the credentials, so that I am not not the
only one with access. I seem to recall that we have a private SVN or git
repo for Phoenix members somewhere, that we could use to store the
login/password for it, but I cannot find it now.

Please share your opinion, and point me to the private repo, or the docs
that describes it.

regards
Istvan