[jira] [Updated] (PHOENIX-1119) Use Zipkin to visualize Phoenix metrics data

2016-07-30 Thread Nishani (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishani  updated PHOENIX-1119:
--
Attachment: Screenshot from 2016-07-30.png

Hi,
The custom zipkin-server is created. The phoenix tracing table data needs to be 
read and converted into zipkin traces. Then they are exposed through the rest 
interface which serve the zipkin ui.

> Use Zipkin to visualize Phoenix metrics data
> 
>
> Key: PHOENIX-1119
> URL: https://issues.apache.org/jira/browse/PHOENIX-1119
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: Nishani 
>  Labels: gsoc2016, tracing
> Attachments: Screenshot from 2016-07-30.png
>
>
> Zipkin provides a nice tool for visualizing trace information: 
> http://twitter.github.io/zipkin/
> It's likely not difficult to visualize the Phoenix tracing data through this 
> tool.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (PHOENIX-3130) Ignore ASYNC and build indexes synchronously based on config property

2016-07-30 Thread James Taylor (JIRA)
James Taylor created PHOENIX-3130:
-

 Summary: Ignore ASYNC and build indexes synchronously based on 
config property
 Key: PHOENIX-3130
 URL: https://issues.apache.org/jira/browse/PHOENIX-3130
 Project: Phoenix
  Issue Type: Bug
Reporter: James Taylor


At test time, we want to ignore the ASYNC keyword on CREATE INDEX and instead 
just build the index synchronously. We can control this through a config 
parameter defaulted to true at test time and false otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (PHOENIX-3130) Ignore ASYNC and build indexes synchronously based on config property

2016-07-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-3130:
--
Fix Version/s: 4.8.1

> Ignore ASYNC and build indexes synchronously based on config property
> -
>
> Key: PHOENIX-3130
> URL: https://issues.apache.org/jira/browse/PHOENIX-3130
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
> Fix For: 4.8.1
>
>
> At test time, we want to ignore the ASYNC keyword on CREATE INDEX and instead 
> just build the index synchronously. We can control this through a config 
> parameter defaulted to true at test time and false otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (PHOENIX-3130) Ignore ASYNC and build indexes synchronously based on config property

2016-07-30 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400717#comment-15400717
 ] 

James Taylor commented on PHOENIX-3130:
---

FYI, [~rangent]. This won't make the 4.8 release, but we can do it in a patch 
release.

[~tdsilva] - can you take this one?

> Ignore ASYNC and build indexes synchronously based on config property
> -
>
> Key: PHOENIX-3130
> URL: https://issues.apache.org/jira/browse/PHOENIX-3130
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
> Fix For: 4.8.1
>
>
> At test time, we want to ignore the ASYNC keyword on CREATE INDEX and instead 
> just build the index synchronously. We can control this through a config 
> parameter defaulted to true at test time and false otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Reopened] (PHOENIX-3121) Queries with filter and reverse scan failing when limit is a multiple of scanner cache size

2016-07-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor reopened PHOENIX-3121:
---

> Queries with filter and reverse scan failing when limit is a multiple of 
> scanner cache size
> ---
>
> Key: PHOENIX-3121
> URL: https://issues.apache.org/jira/browse/PHOENIX-3121
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Samarth Jain
>Assignee: Samarth Jain
> Fix For: 4.8.0
>
> Attachments: PHOENIX-3121.patch, PHOENIX-3121_addendum1.patch, 
> PHOENIX-3121_v2.patch
>
>
> {code}
> org.apache.hadoop.hbase.UnknownScannerException: 
> org.apache.hadoop.hbase.UnknownScannerException: Unknown scanner '644116'. 
> This can happen due to any of the following reasons: a) Scanner id given is 
> wrong, b) Scanner lease expired because of long wait between consecutive 
> client checkins, c) Server may be closing down, d) RegionServer restart 
> during upgrade.
> If the issue is due to reason (b), a possible fix would be increasing the 
> value of'hbase.client.scanner.timeout.period' configuration.
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3228)
> at 
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32492)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2208)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (PHOENIX-3121) Queries with filter and reverse scan failing when limit is a multiple of scanner cache size

2016-07-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-3121:
--
Attachment: PHOENIX-3121_addendum1.patch

[~samarthjain] or [~mujtabachohan] - would you mind trying the above patch with 
your limit and offset tests please?

> Queries with filter and reverse scan failing when limit is a multiple of 
> scanner cache size
> ---
>
> Key: PHOENIX-3121
> URL: https://issues.apache.org/jira/browse/PHOENIX-3121
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Samarth Jain
>Assignee: Samarth Jain
> Fix For: 4.8.0
>
> Attachments: PHOENIX-3121.patch, PHOENIX-3121_addendum1.patch, 
> PHOENIX-3121_v2.patch
>
>
> {code}
> org.apache.hadoop.hbase.UnknownScannerException: 
> org.apache.hadoop.hbase.UnknownScannerException: Unknown scanner '644116'. 
> This can happen due to any of the following reasons: a) Scanner id given is 
> wrong, b) Scanner lease expired because of long wait between consecutive 
> client checkins, c) Server may be closing down, d) RegionServer restart 
> during upgrade.
> If the issue is due to reason (b), a possible fix would be increasing the 
> value of'hbase.client.scanner.timeout.period' configuration.
> at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3228)
> at 
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32492)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2208)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (PHOENIX-3128) Remove extraneous operations during upsert with local immutable index

2016-07-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-3128:
--
Attachment: PHOENIX-3128_v2.patch

Would you mind seeing if this improves performance, [~junegunn]?

Please review, [~rajeshbabu]. The only part I'm not sure on is if we ever want 
{{PhoenixRuntime.getUncommittedDataIterator()}} to return the mutations for 
local indexes. It currently never will.

> Remove extraneous operations during upsert with local immutable index
> -
>
> Key: PHOENIX-3128
> URL: https://issues.apache.org/jira/browse/PHOENIX-3128
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Junegunn Choi
>Assignee: Junegunn Choi
> Fix For: 4.8.0
>
> Attachments: PHOENIX-3128.patch, PHOENIX-3128_v2.patch, 
> PHOENIX-3128_wip.patch
>
>
> Upsert to a table with a local immutable index is supposed to be more 
> efficient than to a table with a local mutable index, but it's actually 
> slower (in our environment by 30%) due to extraneous operations involved.
> The problem is twofold:
> 1. Client unnecessarily prepares and sends index update.
> 2. Index cleanup is done regardless of the immutability of the table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (PHOENIX-3128) Remove extraneous operations during upsert with local immutable index

2016-07-30 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400777#comment-15400777
 ] 

Hadoop QA commented on PHOENIX-3128:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12821169/PHOENIX-3128_v2.patch
  against master branch at commit 3251ac58a6a9de890285ae82ba86d76618fa0a1c.
  ATTACHMENT ID: 12821169

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/484//console

This message is automatically generated.

> Remove extraneous operations during upsert with local immutable index
> -
>
> Key: PHOENIX-3128
> URL: https://issues.apache.org/jira/browse/PHOENIX-3128
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Junegunn Choi
>Assignee: Junegunn Choi
> Fix For: 4.8.0
>
> Attachments: PHOENIX-3128.patch, PHOENIX-3128_v2.patch, 
> PHOENIX-3128_wip.patch
>
>
> Upsert to a table with a local immutable index is supposed to be more 
> efficient than to a table with a local mutable index, but it's actually 
> slower (in our environment by 30%) due to extraneous operations involved.
> The problem is twofold:
> 1. Client unnecessarily prepares and sends index update.
> 2. Index cleanup is done regardless of the immutability of the table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (PHOENIX-3128) Remove extraneous operations during upsert with local immutable index

2016-07-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-3128:
--
Attachment: PHOENIX-3128_v3.patch

Rebased patch.

> Remove extraneous operations during upsert with local immutable index
> -
>
> Key: PHOENIX-3128
> URL: https://issues.apache.org/jira/browse/PHOENIX-3128
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Junegunn Choi
>Assignee: Junegunn Choi
> Fix For: 4.8.0
>
> Attachments: PHOENIX-3128.patch, PHOENIX-3128_v2.patch, 
> PHOENIX-3128_v3.patch, PHOENIX-3128_wip.patch
>
>
> Upsert to a table with a local immutable index is supposed to be more 
> efficient than to a table with a local mutable index, but it's actually 
> slower (in our environment by 30%) due to extraneous operations involved.
> The problem is twofold:
> 1. Client unnecessarily prepares and sends index update.
> 2. Index cleanup is done regardless of the immutability of the table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (PHOENIX-3128) Remove extraneous operations during upsert with local immutable index

2016-07-30 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400804#comment-15400804
 ] 

Hadoop QA commented on PHOENIX-3128:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12821170/PHOENIX-3128_v3.patch
  against master branch at commit 3251ac58a6a9de890285ae82ba86d76618fa0a1c.
  ATTACHMENT ID: 12821170

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:red}-1 javac{color}.  The patch appears to cause mvn compile goal to 
fail .

Compilation errors resume:
[ERROR] COMPILATION ERROR : 
[ERROR] 
/home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/IndexIT.java:[964,63]
 cannot find symbol
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.0:testCompile 
(default-testCompile) on project phoenix-core: Compilation failure
[ERROR] 
/home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/it/java/org/apache/phoenix/end2end/index/IndexIT.java:[964,63]
 cannot find symbol
[ERROR] symbol:   variable fullIndexName
[ERROR] location: class org.apache.phoenix.end2end.index.IndexIT
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :phoenix-core


Console output: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/485//console

This message is automatically generated.

> Remove extraneous operations during upsert with local immutable index
> -
>
> Key: PHOENIX-3128
> URL: https://issues.apache.org/jira/browse/PHOENIX-3128
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Junegunn Choi
>Assignee: Junegunn Choi
> Fix For: 4.8.0
>
> Attachments: PHOENIX-3128.patch, PHOENIX-3128_v2.patch, 
> PHOENIX-3128_v3.patch, PHOENIX-3128_wip.patch
>
>
> Upsert to a table with a local immutable index is supposed to be more 
> efficient than to a table with a local mutable index, but it's actually 
> slower (in our environment by 30%) due to extraneous operations involved.
> The problem is twofold:
> 1. Client unnecessarily prepares and sends index update.
> 2. Index cleanup is done regardless of the immutability of the table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (PHOENIX-3128) Remove extraneous operations during upsert with local immutable index

2016-07-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-3128:
--
Attachment: PHOENIX-3128_v4.patch

Patch that compiles...

> Remove extraneous operations during upsert with local immutable index
> -
>
> Key: PHOENIX-3128
> URL: https://issues.apache.org/jira/browse/PHOENIX-3128
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Junegunn Choi
>Assignee: Junegunn Choi
> Fix For: 4.8.0
>
> Attachments: PHOENIX-3128.patch, PHOENIX-3128_v2.patch, 
> PHOENIX-3128_v3.patch, PHOENIX-3128_v4.patch, PHOENIX-3128_wip.patch
>
>
> Upsert to a table with a local immutable index is supposed to be more 
> efficient than to a table with a local mutable index, but it's actually 
> slower (in our environment by 30%) due to extraneous operations involved.
> The problem is twofold:
> 1. Client unnecessarily prepares and sends index update.
> 2. Index cleanup is done regardless of the immutability of the table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (PHOENIX-3128) Remove extraneous operations during upsert with local immutable index

2016-07-30 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400851#comment-15400851
 ] 

Hadoop QA commented on PHOENIX-3128:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12821173/PHOENIX-3128_v4.patch
  against master branch at commit 3251ac58a6a9de890285ae82ba86d76618fa0a1c.
  ATTACHMENT ID: 12821173

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
34 warning messages.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+long ts = conn.unwrap(PhoenixConnection.class).getTable(new 
PTableKey(null,fullTableName)).getTimeStamp();
+private void assertNoIndexDeletes(Connection conn, long minTimestamp, 
String fullIndexName) throws IOException, SQLException {
+processRollback(env, indexMetaData, txRollbackAttribute, 
currentScanner, mutations, tx, upsertColumns, indexUpdates);
+processMutation(env, indexMetaData, txRollbackAttribute, 
currentScanner, mutations, tx, upsertColumns, deleteColumns, indexUpdates);
+TxTableState state = new TxTableState(env, deleteColumns, 
indexMetaData.getAttributes(), tx.getWritePointer(), m, emptyColRef, result);
+TxTableState state = new TxTableState(env, upsertColumns, 
indexMetaData.getAttributes(), tx.getWritePointer(), m);

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MutableIndexToolIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.txn.RollbackIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CsvBulkLoadToolIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.tx.TxCheckpointIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.txn.MutableRollbackIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DistinctPrefixFilterIT

Test results: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/486//testReport/
Javadoc warnings: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/486//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/486//console

This message is automatically generated.

> Remove extraneous operations during upsert with local immutable index
> -
>
> Key: PHOENIX-3128
> URL: https://issues.apache.org/jira/browse/PHOENIX-3128
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Junegunn Choi
>Assignee: Junegunn Choi
> Fix For: 4.8.0
>
> Attachments: PHOENIX-3128.patch, PHOENIX-3128_v2.patch, 
> PHOENIX-3128_v3.patch, PHOENIX-3128_v4.patch, PHOENIX-3128_wip.patch
>
>
> Upsert to a table with a local immutable index is supposed to be more 
> efficient than to a table with a local mutable index, but it's actually 
> slower (in our environment by 30%) due to extraneous operations involved.
> The problem is twofold:
> 1. Client unnecessarily prepares and sends index update.
> 2. Index cleanup is done regardless of the immutability of the table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (PHOENIX-3111) Possible Deadlock/delay while building index, upsert select, delete rows at server

2016-07-30 Thread Andrew Purtell (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-3111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400889#comment-15400889
 ] 

Andrew Purtell commented on PHOENIX-3111:
-

> My meta comment is that it appears pretty damn hard to reliably write data to 
> the same table from a coprocessor. There must be a way that HBase can make 
> that easier (I hope).

My meta comment is Phoenix developers are doing technically inappropriate 
things - writes in code designed to be read only, reads in code designed to be 
write only. This is par for the course for indexes, I think. This is not a 
native concept and will push and stretch code paths in new directions. That 
said, the underlying code as-is is not a blank canvas that can support 
arbitrary extension. Phoenix devs have to take the time to read and understand 
HBase code and what it does at this moment when injecting new logic into 
internals. As that code is a complex code base you have my empathy (and surely 
that of other committers) so please come over to dev@hbase more often with 
questions and discussions about what you are up to and want to accomplish. Only 
some of us hang out on your lists and JIRA and will catch these complaints. 
Furthermore, if you develop - perhaps with our help (re: above comment about 
dev@) - and propose patches to HBase to support what you want to do, I'm sure 
the HBase committers will seriously consider them and try to help, in the form 
of code review and, hopefully, commits.

> Possible Deadlock/delay while building index, upsert select, delete rows at 
> server
> --
>
> Key: PHOENIX-3111
> URL: https://issues.apache.org/jira/browse/PHOENIX-3111
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Sergio Peleato
>Assignee: Rajeshbabu Chintaguntla
>Priority: Critical
> Fix For: 4.8.0
>
> Attachments: PHOENIX-3111.patch
>
>
> There is a possible deadlock while building local index or running upsert 
> select, delete at server. The situation might happen in this case.
> In the above queries we scan mutations from table and write back to same 
> table in that case there is a chance of memstore might reach the threshold of 
> blocking memstore size then RegionTooBusyException might be thrown back to 
> client and queries might retry scanning.
> Let's suppose if we take a local index build index case we first scan from 
> the data table and prepare index mutations and write back to same table.
> So there is chance of memstore full as well in that case we try to flush the 
> region. But if the split happen in between then split might be waiting for 
> write lock on the region to close and flush wait for readlock because the 
> write lock in the queue until the local index build completed. Local index 
> build won't complete because we are not allowed to write until there is 
> flush. This might not be complete deadlock situation but the queries might 
> take lot of time to complete in this cases.
> {noformat}
> "regionserver//192.168.0.53:16201-splits-1469165876186" #269 prio=5 
> os_prio=31 tid=0x7f7fb2050800 nid=0x1c033 waiting on condition 
> [0x000139b68000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <0x0006ede72550> (a 
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at 
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1422)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1370)
> - locked <0x0006ede69d00> (a java.lang.Object)
> at 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:394)
> at 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:278)
> at 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:561)
> at 
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at 
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:154)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWork

JIRA email still going to d...@phoenix.incubator.apache.org

2016-07-30 Thread Andrew Purtell
See subject. 



[jira] [Updated] (PHOENIX-3128) Remove extraneous operations during upsert with local immutable index

2016-07-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-3128:
--
Fix Version/s: (was: 4.8.0)
   4.8.1

> Remove extraneous operations during upsert with local immutable index
> -
>
> Key: PHOENIX-3128
> URL: https://issues.apache.org/jira/browse/PHOENIX-3128
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Junegunn Choi
>Assignee: Junegunn Choi
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3128.patch, PHOENIX-3128_v2.patch, 
> PHOENIX-3128_v3.patch, PHOENIX-3128_v4.patch, PHOENIX-3128_wip.patch
>
>
> Upsert to a table with a local immutable index is supposed to be more 
> efficient than to a table with a local mutable index, but it's actually 
> slower (in our environment by 30%) due to extraneous operations involved.
> The problem is twofold:
> 1. Client unnecessarily prepares and sends index update.
> 2. Index cleanup is done regardless of the immutability of the table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (PHOENIX-3128) Remove extraneous operations during upsert with local immutable index

2016-07-30 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400958#comment-15400958
 ] 

James Taylor commented on PHOENIX-3128:
---

Not a blocker IMHO, but it can go in if all tests pass.

> Remove extraneous operations during upsert with local immutable index
> -
>
> Key: PHOENIX-3128
> URL: https://issues.apache.org/jira/browse/PHOENIX-3128
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Junegunn Choi
>Assignee: Junegunn Choi
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3128.patch, PHOENIX-3128_v2.patch, 
> PHOENIX-3128_v3.patch, PHOENIX-3128_v4.patch, PHOENIX-3128_wip.patch
>
>
> Upsert to a table with a local immutable index is supposed to be more 
> efficient than to a table with a local mutable index, but it's actually 
> slower (in our environment by 30%) due to extraneous operations involved.
> The problem is twofold:
> 1. Client unnecessarily prepares and sends index update.
> 2. Index cleanup is done regardless of the immutability of the table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: JIRA email still going to d...@phoenix.incubator.apache.org

2016-07-30 Thread James Taylor
How do we change it?

On Saturday, July 30, 2016, Andrew Purtell  wrote:

> See subject.
>
>