[GitHub] [phoenix-tephra] stoty commented on pull request #103: Tephra 309

2020-08-10 Thread GitBox


stoty commented on pull request #103:
URL: https://github.com/apache/phoenix-tephra/pull/103#issuecomment-671733838


   removed backup files, rebased, and re-added shaded artifacts.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [phoenix-tephra] stoty commented on a change in pull request #103: Tephra 309

2020-08-10 Thread GitBox


stoty commented on a change in pull request #103:
URL: https://github.com/apache/phoenix-tephra/pull/103#discussion_r468333157



##
File path: tephra-hbase-compat-1.4-base/pom.xml
##
@@ -0,0 +1,221 @@
+
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+org.apache.tephra
+tephra
+0.16.0-incubating-SNAPSHOT
+  
+  4.0.0
+
+  tephra-hbase-compat-1.4-base
+  Apache Tephra HBase 1.4 Compatibility Base
+
+  pom
+  
+tephra-hbase-compat-1.4
+tephra-hbase-compat-1.5
+tephra-hbase-compat-1.6
+  
+
+  
+2.7.4
+  
+
+  
+
+  org.apache.tephra
+  tephra-api
+  ${project.version}
+
+
+  org.apache.tephra
+  tephra-core
+  ${project.version}
+  
+
+  org.apache.hbase
+  hbase
+
+  
+
+
+  org.apache.hbase
+  hbase-common
+  ${hbase14.version}
+  provided
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-client
+  ${hbase14.version}
+  provided
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-protocol
+  ${hbase14.version}
+  provided
+
+
+  org.apache.hbase
+  hbase-server
+  ${hbase14.version}
+  provided
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+
+
+  org.apache.tephra
+  tephra-core
+  ${project.version}
+  test-jar
+  test
+
+
+  junit
+  junit
+
+
+  org.slf4j
+  log4j-over-slf4j
+
+
+  org.slf4j
+  jcl-over-slf4j
+
+
+  org.apache.hbase
+  hbase-server
+  ${hbase14.version}
+  test-jar
+  test
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-metrics-api
+  ${hbase14.version}
+  test
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-metrics
+  ${hbase14.version}
+  test
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-testing-util
+  ${hbase14.version}
+  test
+  
+
+  asm
+  asm
+
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+  
+
+  
+
+  
+
+  org.codehaus.mojo
+  build-helper-maven-plugin
+  3.0.0
+  
+
+  add-source
+  generate-sources
+  
+add-source
+  
+  
+
+  ${project.parent.basedir}/src/main/java
+
+  
+
+
+  add-test-source
+  generate-sources
+  
+add-test-source
+  
+  
+
+  ${project.parent.basedir}/src/test/java
+
+  
+
+  
+

Review comment:
   This section copies the common java source files from -common to the 
version specific sub-sub-projects that have no source files at all. See the 
`` element. The same approach is used in all the -common based 
compatibility modules that share the java code.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [phoenix-tephra] stoty commented on pull request #103: Tephra 309

2020-08-10 Thread GitBox


stoty commented on pull request #103:
URL: https://github.com/apache/phoenix-tephra/pull/103#issuecomment-671732741


   > `tephra-examples/tephra-examples-post-1.3/hbase-1.6/pom.xml~` shouldn't be 
staged?
   
   Of course. Thanks for noticing it.
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [phoenix-tephra] stoty commented on a change in pull request #103: Tephra 309

2020-08-10 Thread GitBox


stoty commented on a change in pull request #103:
URL: https://github.com/apache/phoenix-tephra/pull/103#discussion_r468333157



##
File path: tephra-hbase-compat-1.4-base/pom.xml
##
@@ -0,0 +1,221 @@
+
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+org.apache.tephra
+tephra
+0.16.0-incubating-SNAPSHOT
+  
+  4.0.0
+
+  tephra-hbase-compat-1.4-base
+  Apache Tephra HBase 1.4 Compatibility Base
+
+  pom
+  
+tephra-hbase-compat-1.4
+tephra-hbase-compat-1.5
+tephra-hbase-compat-1.6
+  
+
+  
+2.7.4
+  
+
+  
+
+  org.apache.tephra
+  tephra-api
+  ${project.version}
+
+
+  org.apache.tephra
+  tephra-core
+  ${project.version}
+  
+
+  org.apache.hbase
+  hbase
+
+  
+
+
+  org.apache.hbase
+  hbase-common
+  ${hbase14.version}
+  provided
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-client
+  ${hbase14.version}
+  provided
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-protocol
+  ${hbase14.version}
+  provided
+
+
+  org.apache.hbase
+  hbase-server
+  ${hbase14.version}
+  provided
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+
+
+  org.apache.tephra
+  tephra-core
+  ${project.version}
+  test-jar
+  test
+
+
+  junit
+  junit
+
+
+  org.slf4j
+  log4j-over-slf4j
+
+
+  org.slf4j
+  jcl-over-slf4j
+
+
+  org.apache.hbase
+  hbase-server
+  ${hbase14.version}
+  test-jar
+  test
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-metrics-api
+  ${hbase14.version}
+  test
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-metrics
+  ${hbase14.version}
+  test
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-testing-util
+  ${hbase14.version}
+  test
+  
+
+  asm
+  asm
+
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+  
+
+  
+
+  
+
+  org.codehaus.mojo
+  build-helper-maven-plugin
+  3.0.0
+  
+
+  add-source
+  generate-sources
+  
+add-source
+  
+  
+
+  ${project.parent.basedir}/src/main/java
+
+  
+
+
+  add-test-source
+  generate-sources
+  
+add-test-source
+  
+  
+
+  ${project.parent.basedir}/src/test/java
+
+  
+
+  
+

Review comment:
   This section copies the common java source files from -common to the 
version specific sub-sub-projects that have no source files at all. See the 
 element. The same approach is used in all the -common based 
compatibility modules that share the java code.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Resolved] (PHOENIX-5829) Make it possible to build/test queryserver against all supported versions

2020-08-10 Thread Istvan Toth (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-5829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Istvan Toth resolved PHOENIX-5829.
--
Fix Version/s: queryserver-6.0.0
   Resolution: Fixed

Committed.

Thanks for the review and suggestions [~elserj].

> Make it possible to build/test queryserver against all supported versions
> -
>
> Key: PHOENIX-5829
> URL: https://issues.apache.org/jira/browse/PHOENIX-5829
> Project: Phoenix
>  Issue Type: Improvement
>  Components: queryserver
>Affects Versions: queryserver-6.0.0
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Fix For: queryserver-6.0.0
>
>
> The queryserver repo is currently building against Phoenix 4.14-2.
> Trying to build it against master doesn't work.
> While theoretically the resulting artifacts should work both with 4.x and 
> master, it is not possible to run the tests with master.
> Fix the build process so that the repo can built -and tested - with both 
> 4.15+ and master, and document how to do this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (TEPHRA-310) Add OWASP dependency check, and update the flagged direct dependencies

2020-08-10 Thread Istvan Toth (Jira)


 [ 
https://issues.apache.org/jira/browse/TEPHRA-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Istvan Toth resolved TEPHRA-310.

Fix Version/s: 0.16.0-incubating
   Resolution: Fixed

Committed.

Thanks for the review [~elserj]

> Add OWASP dependency check, and update the flagged direct dependencies
> --
>
> Key: TEPHRA-310
> URL: https://issues.apache.org/jira/browse/TEPHRA-310
> Project: Phoenix Tephra
>  Issue Type: Improvement
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Fix For: 0.16.0-incubating
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [phoenix-tephra] asfgit closed pull request #104: TEPHRA-310 Add OWASP dependency check, and update the flagged direct …

2020-08-10 Thread GitBox


asfgit closed pull request #104:
URL: https://github.com/apache/phoenix-tephra/pull/104


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Resolved] (TEPHRA-308) create artifact with shaded Guava and Twill in Tephra

2020-08-10 Thread Istvan Toth (Jira)


 [ 
https://issues.apache.org/jira/browse/TEPHRA-308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Istvan Toth resolved TEPHRA-308.

Fix Version/s: 0.16.0-incubating
   Resolution: Fixed

Committed.

Thanks for the review [~elserj]

> create artifact with shaded Guava and Twill in Tephra
> -
>
> Key: TEPHRA-308
> URL: https://issues.apache.org/jira/browse/TEPHRA-308
> Project: Phoenix Tephra
>  Issue Type: Improvement
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Fix For: 0.16.0-incubating
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [phoenix-tephra] asfgit closed pull request #102: TEPHRA-308 create artifact with shaded Guava and Twill in Tephra

2020-08-10 Thread GitBox


asfgit closed pull request #102:
URL: https://github.com/apache/phoenix-tephra/pull/102


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [phoenix-omid] joshelser commented on a change in pull request #62: OMID-156 refactor Omid to use phoenix-shaded-guava

2020-08-10 Thread GitBox


joshelser commented on a change in pull request #62:
URL: https://github.com/apache/phoenix-omid/pull/62#discussion_r468230549



##
File path: common/src/main/java/org/apache/omid/NetworkUtils.java
##
@@ -34,6 +36,14 @@
 
 public static String getDefaultNetworkInterface() {
 
+try (DatagramSocket s=new DatagramSocket()) {

Review comment:
   Making sure I get it...
   
   * Binds a socket on a random local port (ideally 127.0.0.1 for ipv4)
   * This creates an InetAddress to 1.1.1.1
   * Tries to bind to 1.1.1.1 on port 0 (what does that actually do?)
   * Pulls the network interface used to do that bind, which is what we 
actually want
   
   Any downside on this? Does it require being connected to the internet to 
work?

##
File path: common/src/main/java/org/apache/omid/NetworkUtils.java
##
@@ -34,6 +36,14 @@
 
 public static String getDefaultNetworkInterface() {
 
+try (DatagramSocket s=new DatagramSocket()) {

Review comment:
   Is there a reason we can't use `InetAddress.getLocalHost()` instead of 
something on the public internet?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (PHOENIX-6066) MetaDataEndpointImpl.doGetTable should acquire a readLock instead of an exclusive writeLock on the table header row

2020-08-10 Thread Chinmay Kulkarni (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chinmay Kulkarni updated PHOENIX-6066:
--
Description: 
Throughout MetaDataEndpointImpl, wherever we need to acquire a row lock we call 
[MetaDataEndpointImpl.acquireLock|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2377-L2386]
 which gets an exclusive writeLock on the specified row [by 
default|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2378].

Thus, even operations like doGetTable/getSchema/getFunctions which are not 
modifying the row will acquire a writeLock on these metadata rows when a 
readLock should be sufficient (see [doGetTable 
locking|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2932]
 as an example). The problem with this is, even a simple UPSERT/DELETE or 
SELECT query triggers a doGetTable (if the schema is not cached) and can 
potentially block other DDLs and more importantly other queries since these 
queries will wait until they can get a rowLock for the table header row. Even 
seemingly unrelated operations like a CREATE VIEW AS SELECT * FROM T can block 
a SELECT/UPSERT/DELETE on table T since the create view code needs to fetch the 
schema of the parent table.

Note that this is exacerbated in cases where we do server-server RPCs while 
holding rowLocks for example 
([this|https://github.com/apache/phoenix/blob/1d844950bb4ec8221873ecd2b094c20f427cd984/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2459-L2461]
 and 
[this|https://github.com/apache/phoenix/blob/1d844950bb4ec8221873ecd2b094c20f427cd984/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2479-L2484])
 which is another issue altogether.

This Jira is to discuss the possibility of acquiring a readLock in these "read 
metadata" paths to avoid blocking other "read metadata" requests stemming from 
concurrent queries. The current behavior is potentially a perf issue for 
clients that disable update-cache-frequency.

  was:
Throughout MetaDataEndpointImpl, wherever we need to acquire a row lock we call 
[MetaDataEndpointImpl.acquireLock|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2377-L2386]
 which gets an exclusive writeLock on the specified row [by 
default|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2378].

Thus, even operations like doGetTable/getSchema/getFunctions which are not 
modifying the row will acquire a writeLock on these metadata rows when a 
readLock should be sufficient (see [doGetTable 
locking|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2932]
 as an example). The problem with this is, even a simple UPSERT/DELETE or 
SELECT query triggers a doGetTable (if the schema is not cached) and can 
potentially block other DDLs and more importantly other queries since these 
queries will wait until they can get a rowLock for the table header row. Even 
seemingly unrelated operations like a CREATE VIEW AS SELECT * FROM T can block 
a SELECT/UPSERT/DELETE on table T since the create view code needs to fetch the 
schema of the parent table.

Note that this is exacerbated in cases where we do server-server RPCs while 
holding rowLocks for example 
([this|https://github.com/apache/phoenix/blob/1d844950bb4ec8221873ecd2b094c20f427cd984/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2459-L2461]
 and 
[this|https://github.com/apache/phoenix/blob/1d844950bb4ec8221873ecd2b094c20f427cd984/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2479-L2484])

This Jira is to discuss the possibility of acquiring a readLock in these "read 
metadata" paths to avoid blocking other "read metadata" requests stemming from 
concurrent queries. The current behavior is potentially a perf issue for 
clients that disable update-cache-frequency.


> MetaDataEndpointImpl.doGetTable should acquire a readLock instead of an 
> exclusive writeLock on the table header row
> ---
>
> Key: PHOENIX-6066
> URL: https://issues.apache.org/jira/browse/PHOENIX-6066
> Project: Phoenix
> 

[jira] [Updated] (PHOENIX-6066) MetaDataEndpointImpl.doGetTable should acquire a readLock instead of an exclusive writeLock on the table header row

2020-08-10 Thread Chinmay Kulkarni (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chinmay Kulkarni updated PHOENIX-6066:
--
Description: 
Throughout MetaDataEndpointImpl, wherever we need to acquire a row lock we call 
[MetaDataEndpointImpl.acquireLock|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2377-L2386]
 which gets an exclusive writeLock on the specified row [by 
default|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2378].

Thus, even operations like doGetTable/getSchema/getFunctions which are not 
modifying the row will acquire a writeLock on these metadata rows when a 
readLock should be sufficient (see [doGetTable 
locking|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2932]
 as an example). The problem with this is, even a simple UPSERT/DELETE or 
SELECT query triggers a doGetTable (if the schema is not cached) and can 
potentially block other DDLs and more importantly other queries since these 
queries will wait until they can get a rowLock for the table header row. Even 
seemingly unrelated operations like a CREATE VIEW AS SELECT * FROM T can block 
a SELECT/UPSERT/DELETE on table T since the create view code needs to fetch the 
schema of the parent table.

Note that this is exacerbated in cases where we do server-server RPCs while 
holding rowLocks for example 
([this|https://github.com/apache/phoenix/blob/1d844950bb4ec8221873ecd2b094c20f427cd984/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2459-L2461]
 and 
[this|https://github.com/apache/phoenix/blob/1d844950bb4ec8221873ecd2b094c20f427cd984/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2479-L2484])

This Jira is to discuss the possibility of acquiring a readLock in these "read 
metadata" paths to avoid blocking other "read metadata" requests stemming from 
concurrent queries. The current behavior is potentially a perf issue for 
clients that disable update-cache-frequency.

  was:
Throughout MetaDataEndpointImpl, wherever we need to acquire a row lock we call 
[MetaDataEndpointImpl.acquireLock|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2377-L2386]
 which gets an exclusive writeLock on the specified row [by 
default|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2378].
 

Thus, even operations like doGetTable/getSchema/getFunctions which are not 
modifying the row will acquire a writeLock on these metadata rows when a 
readLock should be sufficient (see [doGetTable 
locking|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2932]
 as an example). The problem with this is, even a simple UPSERT/DELETE or 
SELECT query triggers a doGetTable (if the schema is not cached) and can 
potentially block other DDLs and more importantly other queries since these 
queries will wait until they can get a rowLock for the table header row. Even 
seemingly unrelated operations like a CREATE VIEW AS SELECT * FROM T can block 
a SELECT/UPSERT/DELETE on table T since the create view code needs to fetch the 
schema of the parent table.

This Jira is to discuss the possibility of acquiring a readLock in these "read 
metadata" paths to avoid blocking other "read metadata" requests stemming from 
concurrent queries. The current behavior is potentially a perf issue for 
clients that disable update-cache-frequency.


> MetaDataEndpointImpl.doGetTable should acquire a readLock instead of an 
> exclusive writeLock on the table header row
> ---
>
> Key: PHOENIX-6066
> URL: https://issues.apache.org/jira/browse/PHOENIX-6066
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Priority: Major
>  Labels: quality-improvement
> Fix For: 5.1.0, 4.16.0
>
>
> Throughout MetaDataEndpointImpl, wherever we need to acquire a row lock we 
> call 
> [MetaDataEndpointImpl.acquireLock|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2377-L2386]
> 

[jira] [Updated] (PHOENIX-6066) MetaDataEndpointImpl.doGetTable should acquire a readLock instead of an exclusive writeLock on the table header row

2020-08-10 Thread Chinmay Kulkarni (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chinmay Kulkarni updated PHOENIX-6066:
--
Labels: quality-improvement  (was: )

> MetaDataEndpointImpl.doGetTable should acquire a readLock instead of an 
> exclusive writeLock on the table header row
> ---
>
> Key: PHOENIX-6066
> URL: https://issues.apache.org/jira/browse/PHOENIX-6066
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 5.0.0, 4.15.0
>Reporter: Chinmay Kulkarni
>Priority: Major
>  Labels: quality-improvement
> Fix For: 5.1.0, 4.16.0
>
>
> Throughout MetaDataEndpointImpl, wherever we need to acquire a row lock we 
> call 
> [MetaDataEndpointImpl.acquireLock|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2377-L2386]
>  which gets an exclusive writeLock on the specified row [by 
> default|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2378].
>  
> Thus, even operations like doGetTable/getSchema/getFunctions which are not 
> modifying the row will acquire a writeLock on these metadata rows when a 
> readLock should be sufficient (see [doGetTable 
> locking|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2932]
>  as an example). The problem with this is, even a simple UPSERT/DELETE or 
> SELECT query triggers a doGetTable (if the schema is not cached) and can 
> potentially block other DDLs and more importantly other queries since these 
> queries will wait until they can get a rowLock for the table header row. Even 
> seemingly unrelated operations like a CREATE VIEW AS SELECT * FROM T can 
> block a SELECT/UPSERT/DELETE on table T since the create view code needs to 
> fetch the schema of the parent table.
> This Jira is to discuss the possibility of acquiring a readLock in these 
> "read metadata" paths to avoid blocking other "read metadata" requests 
> stemming from concurrent queries. The current behavior is potentially a perf 
> issue for clients that disable update-cache-frequency.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [phoenix-tephra] joshelser commented on a change in pull request #103: Tephra 309

2020-08-10 Thread GitBox


joshelser commented on a change in pull request #103:
URL: https://github.com/apache/phoenix-tephra/pull/103#discussion_r468150460



##
File path: tephra-hbase-compat-1.4-base/pom.xml
##
@@ -0,0 +1,221 @@
+
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+  
+org.apache.tephra
+tephra
+0.16.0-incubating-SNAPSHOT
+  
+  4.0.0
+
+  tephra-hbase-compat-1.4-base
+  Apache Tephra HBase 1.4 Compatibility Base
+
+  pom
+  
+tephra-hbase-compat-1.4
+tephra-hbase-compat-1.5
+tephra-hbase-compat-1.6
+  
+
+  
+2.7.4
+  
+
+  
+
+  org.apache.tephra
+  tephra-api
+  ${project.version}
+
+
+  org.apache.tephra
+  tephra-core
+  ${project.version}
+  
+
+  org.apache.hbase
+  hbase
+
+  
+
+
+  org.apache.hbase
+  hbase-common
+  ${hbase14.version}
+  provided
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-client
+  ${hbase14.version}
+  provided
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-protocol
+  ${hbase14.version}
+  provided
+
+
+  org.apache.hbase
+  hbase-server
+  ${hbase14.version}
+  provided
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+
+
+  org.apache.tephra
+  tephra-core
+  ${project.version}
+  test-jar
+  test
+
+
+  junit
+  junit
+
+
+  org.slf4j
+  log4j-over-slf4j
+
+
+  org.slf4j
+  jcl-over-slf4j
+
+
+  org.apache.hbase
+  hbase-server
+  ${hbase14.version}
+  test-jar
+  test
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-metrics-api
+  ${hbase14.version}
+  test
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-metrics
+  ${hbase14.version}
+  test
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+
+  org.apache.hbase
+  hbase-testing-util
+  ${hbase14.version}
+  test
+  
+
+  asm
+  asm
+
+
+  org.slf4j
+  slf4j-log4j12
+
+  
+
+  
+
+  
+
+  
+
+  org.codehaus.mojo
+  build-helper-maven-plugin
+  3.0.0
+  
+
+  add-source
+  generate-sources
+  
+add-source
+  
+  
+
+  ${project.parent.basedir}/src/main/java
+
+  
+
+
+  add-test-source
+  generate-sources
+  
+add-test-source
+  
+  
+
+  ${project.parent.basedir}/src/test/java
+
+  
+
+  
+

Review comment:
   Shouldn't be necessary -- these are included by default.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (PHOENIX-6068) (5.x) Read repair reduces the number of rows returned for LIMIT queries

2020-08-10 Thread Josh Elser (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated PHOENIX-6068:

Priority: Blocker  (was: Major)

> (5.x) Read repair reduces the number of rows returned for LIMIT queries
> ---
>
> Key: PHOENIX-6068
> URL: https://issues.apache.org/jira/browse/PHOENIX-6068
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.0.0, 4.14.3
>Reporter: Kadir OZDEMIR
>Assignee: Kadir OZDEMIR
>Priority: Blocker
> Fix For: 4.16.0
>
>
> Phoenix uses HBase PageFilter to limit the number of rows returned by scans. 
> If a scanned index row is unverified, GlobalIndexChecker repairs this rows. 
> This repair operation leads to either skipping the unverified row or scanning 
> its repaired version. Every scanned row including unverified rows are counted 
> by the page filter. Since unverified rows are counted but not returned for 
> the query, the actual number of rows returned for a LIMIT query becomes less 
> than the set limit (i.e., page size) for the query.  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (PHOENIX-6068) (5.x) Read repair reduces the number of rows returned for LIMIT queries

2020-08-10 Thread Josh Elser (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser reassigned PHOENIX-6068:
---

Assignee: (was: Kadir OZDEMIR)

> (5.x) Read repair reduces the number of rows returned for LIMIT queries
> ---
>
> Key: PHOENIX-6068
> URL: https://issues.apache.org/jira/browse/PHOENIX-6068
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.0.0, 4.14.3
>Reporter: Kadir OZDEMIR
>Priority: Blocker
> Fix For: 5.1.0
>
>
> Phoenix uses HBase PageFilter to limit the number of rows returned by scans. 
> If a scanned index row is unverified, GlobalIndexChecker repairs this rows. 
> This repair operation leads to either skipping the unverified row or scanning 
> its repaired version. Every scanned row including unverified rows are counted 
> by the page filter. Since unverified rows are counted but not returned for 
> the query, the actual number of rows returned for a LIMIT query becomes less 
> than the set limit (i.e., page size) for the query.  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (PHOENIX-6068) (5.x) Read repair reduces the number of rows returned for LIMIT queries

2020-08-10 Thread Josh Elser (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated PHOENIX-6068:

Fix Version/s: (was: 4.16.0)
   5.1.0

> (5.x) Read repair reduces the number of rows returned for LIMIT queries
> ---
>
> Key: PHOENIX-6068
> URL: https://issues.apache.org/jira/browse/PHOENIX-6068
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.0.0, 4.14.3
>Reporter: Kadir OZDEMIR
>Assignee: Kadir OZDEMIR
>Priority: Blocker
> Fix For: 5.1.0
>
>
> Phoenix uses HBase PageFilter to limit the number of rows returned by scans. 
> If a scanned index row is unverified, GlobalIndexChecker repairs this rows. 
> This repair operation leads to either skipping the unverified row or scanning 
> its repaired version. Every scanned row including unverified rows are counted 
> by the page filter. Since unverified rows are counted but not returned for 
> the query, the actual number of rows returned for a LIMIT query becomes less 
> than the set limit (i.e., page size) for the query.  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (PHOENIX-6068) (5.x) Read repair reduces the number of rows returned for LIMIT queries

2020-08-10 Thread Josh Elser (Jira)
Josh Elser created PHOENIX-6068:
---

 Summary: (5.x) Read repair reduces the number of rows returned for 
LIMIT queries
 Key: PHOENIX-6068
 URL: https://issues.apache.org/jira/browse/PHOENIX-6068
 Project: Phoenix
  Issue Type: Bug
Affects Versions: 5.0.0, 4.14.3
Reporter: Kadir OZDEMIR
Assignee: Kadir OZDEMIR
 Fix For: 4.16.0


Phoenix uses HBase PageFilter to limit the number of rows returned by scans. If 
a scanned index row is unverified, GlobalIndexChecker repairs this rows. This 
repair operation leads to either skipping the unverified row or scanning its 
repaired version. Every scanned row including unverified rows are counted by 
the page filter. Since unverified rows are counted but not returned for the 
query, the actual number of rows returned for a LIMIT query becomes less than 
the set limit (i.e., page size) for the query.  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (PHOENIX-6067) (5.x) IndexTool's inline verification should not verify rows beyond max lookback age

2020-08-10 Thread Josh Elser (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated PHOENIX-6067:

Priority: Blocker  (was: Major)

> (5.x) IndexTool's inline verification should not verify rows beyond max 
> lookback age
> 
>
> Key: PHOENIX-6067
> URL: https://issues.apache.org/jira/browse/PHOENIX-6067
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Swaroopa Kadam
>Priority: Blocker
> Fix For: 4.15.1
>
>
> IndexTool's inline verification should not verify rows beyond max lookback age
> Similar to Phoenix-5734



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (PHOENIX-6067) (5.x) IndexTool's inline verification should not verify rows beyond max lookback age

2020-08-10 Thread Josh Elser (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated PHOENIX-6067:

Fix Version/s: (was: 4.15.1)
   5.1.0

> (5.x) IndexTool's inline verification should not verify rows beyond max 
> lookback age
> 
>
> Key: PHOENIX-6067
> URL: https://issues.apache.org/jira/browse/PHOENIX-6067
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Swaroopa Kadam
>Priority: Blocker
> Fix For: 5.1.0
>
>
> IndexTool's inline verification should not verify rows beyond max lookback age
> Similar to Phoenix-5734



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (PHOENIX-6067) (5.x) IndexTool's inline verification should not verify rows beyond max lookback age

2020-08-10 Thread Josh Elser (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser reassigned PHOENIX-6067:
---

Assignee: (was: Weiming Wang)

> (5.x) IndexTool's inline verification should not verify rows beyond max 
> lookback age
> 
>
> Key: PHOENIX-6067
> URL: https://issues.apache.org/jira/browse/PHOENIX-6067
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Swaroopa Kadam
>Priority: Major
> Fix For: 4.15.1
>
>
> IndexTool's inline verification should not verify rows beyond max lookback age
> Similar to Phoenix-5734



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: Roadmap to releasing Phoenix 5.1, PQS 6.0 and Connectors 6.0

2020-08-10 Thread Josh Elser

Agreed with Istvan on the importance of the new secondary indexing.

Thanks for the super-clear list, Geoffrey. I'll spin out clones of each 
of the BLOCKERs you mention and tag them for 5.1.0 (not that they get lost).


Thanks you, Istvan, for your very thoughtful write-up. Looks like a 
great plan.


On 8/5/20 1:11 AM, Istvan Toth wrote:

Hi!

Strongly Consistent Global Indexes is indeed the killer new feature in 5.1,
and it's important to have it working as well as possible.
I was only vaguely aware of the deficiencies in master, thanks for
explaining it (and working on fixing them)

Thanks
Istvan

On Tue, Aug 4, 2020 at 9:22 PM Geoffrey Jacoby  wrote:


Thanks, Istvan, Richard, Josh and Rajeshbabu (and anyone else I missed :-)
) for all your hard work getting master and the ancillary projects into a
releasable state.

An additional task that I think should happen before 5.1 can be released is
getting the indexing code in master up to parity with 4.x. As you may know,
between 4.14 and the upcoming 4.16, a lot of work has been done to build a
new, self-repairing consistent secondary index framework for Phoenix. Most
of that work has been ported to the master branch, but there are still some
significant gaps.

The biggest gap comes from the new indexing framework relying heavily on
Phoenix's SCN "lookback" feature to do point-in-time selects during index
creation and verification. SCN has in the past been an unreliable tool,
because HBase's flush and major compaction code cleans up expired versions
and removes chunks of prior history. To work around this, in 4.16 we've
introduced "max lookback age" in PHOENIX-5645, which allows operators to
configure a moving window during which compactions and flushes will not
purge versions for any history.

PHOENIX-5645 doesn't exist in master, because the coprocessor changes in
HBase 2.0 made implementing the needed compaction hooks impossible.
HBASE-24321, released in HBase 2.3, makes it possible again, though only
for Phoenix builds using the 2.3 profile. (Since it adds to an interface,
HBase compatibility guidelines prevent HBASE-24321 from being backported to
2.1 or 2.2.)

So, for 5.1 I believe we need forward ports for :
PHOENIX-5881 (implementing max lookback age for 2.3) - BLOCKER
PHOENIX-5735 (IndexTool verification distinguishes between inconsistencies
before or after max lookback age) - BLOCKER
PHOENIX-5928 (Simplification / perf improvement for index builds)
PHOENIX-5969 (Bug fix for querying indexes with limit clauses) - BLOCKER
PHOENIX-5951 (Configurable failure logging for past-max-lookback rows)
PHOENIX-6058 (Better behavior on verification when max lookback is disabled
-- needed for HBase 2.1 and 2.2 profiles) - BLOCKER

Kadir, Swaroopa, Abhishek, and Gokcen, please add in any that I missed. :-)
  Happy to discuss what is and isn't a blocker.

I plan to do PHOENIX-5881 next week, and the rest depends on it and will
follow afterward.

Thanks,

Geoffrey


On Mon, Aug 3, 2020 at 7:24 AM Istvan Toth  wrote:


Hi!

It's been more than two years since we've released 5.0.0, and almost as
long since Connectors and PQS have been split from the main repo.

I believe that we are now at the point where we've solved, or are close

to

solving the issues that have prevented us from releasing a useful and
relevant 5.1.0 , as well as making an actual releases of PQS and

Connectors

that are usable with both 5.x and 4.x.

The two major blockers that are still open are

- PHOENIX-6010 Create phoenix-thirdparty, and consume guava through it
- PHOENIX-5784 Phoenix-connectors doesn't work with phoenix master
branch

but I hope that we can wrap those up in the next few weeks.

This is going to be a complex process, as we'll have to release new
versions of ALL of our components. To recap, the affected projects, (and
their dependencies):

- phoenix-thirdparty
- tephra
- omid (phoenix-thirdparty)
- phoenix (tehpra ?, omid, phoenix-thirdparty)
- PQS
- Connectors

The 5.1 release is also a point where we can revisit the decision to
support Tephra. We have inherited those projects because of low developer
interest, and it hasn't increased visibly since we've adopted them.
Rajeshbabu and Josh have done some analysis and, as a part of our day

job,

are investing time first with Omid to ensure it's functional with the

rest

of Phoenix in its new home/packaging.

Tephra also carries the technical debt of being dependent on the
discontinued Twill library, which in turn is locked to oid Guava

versions.

In TEPHRA-308 I am implementing the stopgap solution of shading both

away,

so it is not a blocker for 5.1, but concentrating on one library would
probably be a smarter use of the almost non-existent developer time that
goes into maintaining our transactional solution.

I plan to add a profile to build Phoenix without Tephra, thus avoiding

the

problematic dependencies that it has. (Alternatively, the default can be
omitting Tephra, and defining a 

[jira] [Created] (PHOENIX-6067) (5.x) IndexTool's inline verification should not verify rows beyond max lookback age

2020-08-10 Thread Josh Elser (Jira)
Josh Elser created PHOENIX-6067:
---

 Summary: (5.x) IndexTool's inline verification should not verify 
rows beyond max lookback age
 Key: PHOENIX-6067
 URL: https://issues.apache.org/jira/browse/PHOENIX-6067
 Project: Phoenix
  Issue Type: Improvement
Reporter: Swaroopa Kadam
Assignee: Weiming Wang
 Fix For: 4.15.1


IndexTool's inline verification should not verify rows beyond max lookback age

Similar to Phoenix-5734



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (PHOENIX-5946) Implement SchemaExtractionTool utility to get effective DDL from cluster

2020-08-10 Thread Swaroopa Kadam (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-5946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swaroopa Kadam updated PHOENIX-5946:

Fix Version/s: 4.16.0
   5.1.0

> Implement SchemaExtractionTool utility to get effective DDL from cluster
> 
>
> Key: PHOENIX-5946
> URL: https://issues.apache.org/jira/browse/PHOENIX-5946
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Swaroopa Kadam
>Assignee: Swaroopa Kadam
>Priority: Major
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-5946.4.x.patch, PHOENIX-5946.master.patch
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> The utility will take table/view/index and schema as a parameter to generate 
> effective DDL for those entities in a cluster. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (PHOENIX-5948) Add initial support to read table and schema from CommandLine

2020-08-10 Thread Swaroopa Kadam (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swaroopa Kadam resolved PHOENIX-5948.
-
Resolution: Implemented

> Add initial support to read table and schema from CommandLine
> -
>
> Key: PHOENIX-5948
> URL: https://issues.apache.org/jira/browse/PHOENIX-5948
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Swaroopa Kadam
>Assignee: Swaroopa Kadam
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (PHOENIX-5948) Add initial support to read table and schema from CommandLine

2020-08-10 Thread Swaroopa Kadam (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swaroopa Kadam reassigned PHOENIX-5948:
---

Assignee: Swaroopa Kadam

> Add initial support to read table and schema from CommandLine
> -
>
> Key: PHOENIX-5948
> URL: https://issues.apache.org/jira/browse/PHOENIX-5948
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Swaroopa Kadam
>Assignee: Swaroopa Kadam
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (PHOENIX-5947) Add Unit tests and Integration tests for utility

2020-08-10 Thread Swaroopa Kadam (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-5947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swaroopa Kadam resolved PHOENIX-5947.
-
Resolution: Implemented

> Add Unit tests and Integration tests for utility
> 
>
> Key: PHOENIX-5947
> URL: https://issues.apache.org/jira/browse/PHOENIX-5947
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Swaroopa Kadam
>Assignee: Qinrui Li
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (PHOENIX-5949) Add support for "all" keyword in the utility

2020-08-10 Thread Swaroopa Kadam (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-5949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swaroopa Kadam resolved PHOENIX-5949.
-
Resolution: Later

> Add support for "all" keyword in the utility
> 
>
> Key: PHOENIX-5949
> URL: https://issues.apache.org/jira/browse/PHOENIX-5949
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Swaroopa Kadam
>Priority: Major
>
> when passed an -all keyword to utility, it will query the sys cat and emit 
> the effective DDL for all views, indexes, and tables on the cluster. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (PHOENIX-5946) Implement SchemaExtractionTool utility to get effective DDL from cluster

2020-08-10 Thread Swaroopa Kadam (Jira)


 [ 
https://issues.apache.org/jira/browse/PHOENIX-5946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swaroopa Kadam updated PHOENIX-5946:

Attachment: PHOENIX-5946.master.patch

> Implement SchemaExtractionTool utility to get effective DDL from cluster
> 
>
> Key: PHOENIX-5946
> URL: https://issues.apache.org/jira/browse/PHOENIX-5946
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Swaroopa Kadam
>Assignee: Swaroopa Kadam
>Priority: Major
> Attachments: PHOENIX-5946.4.x.patch, PHOENIX-5946.master.patch
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> The utility will take table/view/index and schema as a parameter to generate 
> effective DDL for those entities in a cluster. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (PHOENIX-6066) MetaDataEndpointImpl.doGetTable should acquire a readLock instead of an exclusive writeLock on the table header row

2020-08-10 Thread Chinmay Kulkarni (Jira)
Chinmay Kulkarni created PHOENIX-6066:
-

 Summary: MetaDataEndpointImpl.doGetTable should acquire a readLock 
instead of an exclusive writeLock on the table header row
 Key: PHOENIX-6066
 URL: https://issues.apache.org/jira/browse/PHOENIX-6066
 Project: Phoenix
  Issue Type: Improvement
Affects Versions: 4.15.0, 5.0.0
Reporter: Chinmay Kulkarni
 Fix For: 5.1.0, 4.16.0


Throughout MetaDataEndpointImpl, wherever we need to acquire a row lock we call 
[MetaDataEndpointImpl.acquireLock|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2377-L2386]
 which gets an exclusive writeLock on the specified row [by 
default|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2378].
 

Thus, even operations like doGetTable/getSchema/getFunctions which are not 
modifying the row will acquire a writeLock on these metadata rows when a 
readLock should be sufficient (see [doGetTable 
locking|https://github.com/apache/phoenix/blob/bba7d59f81f2b91342fa5a7ee213170739573d6a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L2932]
 as an example). The problem with this is, even a simple UPSERT/DELETE or 
SELECT query triggers a doGetTable (if the schema is not cached) and can 
potentially block other DDLs and more importantly other queries since these 
queries will wait until they can get a rowLock for the table header row. Even 
seemingly unrelated operations like a CREATE VIEW AS SELECT * FROM T can block 
a SELECT/UPSERT/DELETE on table T since the create view code needs to fetch the 
schema of the parent table.

This Jira is to discuss the possibility of acquiring a readLock in these "read 
metadata" paths to avoid blocking other "read metadata" requests stemming from 
concurrent queries. The current behavior is potentially a perf issue for 
clients that disable update-cache-frequency.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [phoenix-tephra] stoty opened a new pull request #104: TEPHRA-310 Add OWASP dependency check, and update the flagged direct …

2020-08-10 Thread GitBox


stoty opened a new pull request #104:
URL: https://github.com/apache/phoenix-tephra/pull/104


   …dependencies
   
   add check (activated with -Powasp-dependency-check)
   update logback to 1.2.0
   update thrift to 0.9.3-1



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (TEPHRA-310) Add OWASP dependency check, and update the flagged direct dependencies

2020-08-10 Thread Istvan Toth (Jira)
Istvan Toth created TEPHRA-310:
--

 Summary: Add OWASP dependency check, and update the flagged direct 
dependencies
 Key: TEPHRA-310
 URL: https://issues.apache.org/jira/browse/TEPHRA-310
 Project: Phoenix Tephra
  Issue Type: Improvement
Reporter: Istvan Toth
Assignee: Istvan Toth






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [phoenix-omid] stoty commented on pull request #63: OMID-158 Add OWASP dependency check, and update the flagged direct de…

2020-08-10 Thread GitBox


stoty commented on pull request #63:
URL: https://github.com/apache/phoenix-omid/pull/63#issuecomment-671204792


   The test failures are fixed in the pending OMID-157, I am not duplicating 
the fixes for that here.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [phoenix-omid] stoty opened a new pull request #63: OMID-158 Add OWASP dependency check, and update the flagged direct de…

2020-08-10 Thread GitBox


stoty opened a new pull request #63:
URL: https://github.com/apache/phoenix-omid/pull/63


   …pendencies
   
   add check (activated with -Powasp-dependency-check)
   remove spurious log4j-extras dependency
   update commons-beanutils to latest



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (OMID-158) Add OWASP dependency check, and update the flagged direct dependencies

2020-08-10 Thread Istvan Toth (Jira)
Istvan Toth created OMID-158:


 Summary: Add OWASP dependency check, and update the flagged direct 
dependencies
 Key: OMID-158
 URL: https://issues.apache.org/jira/browse/OMID-158
 Project: Phoenix Omid
  Issue Type: Improvement
Reporter: Istvan Toth
Assignee: Istvan Toth






--
This message was sent by Atlassian Jira
(v8.3.4#803005)