[jira] [Comment Edited] (PHOENIX-1081) CPU usage 100% With phoenix

2014-07-11 Thread yang ming (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059499#comment-14059499
 ] 

yang ming edited comment on PHOENIX-1081 at 7/11/14 11:40 PM:
--

Hbase version is 0.94.19.I had added some debug log,may be the endless loop is 
here.
{color:red}at 
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:390){color}
And the problem may be in this method.
{color:blue}at org.apache.phoenix.filter.SkipScanFilter.navigate{color}
   LOOP: while((kv = this.heap.peek()) != null) {
// Check that the heap gives us KVs in an increasing order.
assert prevKV == null || comparator == null || 
comparator.compare(prevKV, kv) <= 0 :
  "Key " + prevKV + " followed by a " + "smaller key " + kv + " in cf " 
+ store;
prevKV = kv;
{color:red}ScanQueryMatcher.MatchCode qcode = matcher.match(kv);{color}
switch(qcode) {
  case INCLUDE:
  case INCLUDE_AND_SEEK_NEXT_ROW:
  case INCLUDE_AND_SEEK_NEXT_COL:

Filter f = matcher.getFilter();
outResult.add(f == null ? kv : f.transform(kv));
count++;


was (Author: yangming860101):
Hbase version is 0.94.19.I had added some debug log,may be the endless loop is 
here.
{color:red}at 
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:390){color}
And the problem may be in this method.
{color:blue}at org.apache.phoenix.filter.SkipScanFilter.navigate{color}
   LOOP: while((kv = this.heap.peek()) != null) {
// Check that the heap gives us KVs in an increasing order.
assert prevKV == null || comparator == null || 
comparator.compare(prevKV, kv) <= 0 :
  "Key " + prevKV + " followed by a " + "smaller key " + kv + " in cf " 
+ store;
prevKV = kv;
{color:red}
ScanQueryMatcher.MatchCode qcode = matcher.match(kv);
{color}
switch(qcode) {
  case INCLUDE:
  case INCLUDE_AND_SEEK_NEXT_ROW:
  case INCLUDE_AND_SEEK_NEXT_COL:

Filter f = matcher.getFilter();
outResult.add(f == null ? kv : f.transform(kv));
count++;

> CPU usage 100% With phoenix 
> 
>
> Key: PHOENIX-1081
> URL: https://issues.apache.org/jira/browse/PHOENIX-1081
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: yang ming
>Priority: Critical
> Attachments: JMX.jpg, jstat.jpg, the jstack of all threads, the 
> jstack of thread 12725.jpg, the jstack of thread 12748.jpg, the threads of 
> regionserver process.jpg
>
>
> The concurrent of the system is not high,but CPU usage often up to 100%.
> I had stopped the system,but regionserver's CPU usage is still high.
> what can case this problem?
> table row count:6000 million
> table ddl:
> create table if not exists summary
> (
> videoid integer not null,
> date date not null,
> platform varchar not null,
> device varchar not null,
> systemgroup varchar not null,
> system varchar not null,
> vv bigint,
> ts bigint,
> up bigint,
> down bigint,
> comment bigint,
> favori bigint,
> favord bigint,
> quote bigint,
> reply bigint
> constraint pk primary key (videoid, date,platform, device, systemgroup,system)
> )salt_buckets = 30,versions=1,compression='snappy';
> query 1:
> select sum(vv) as sumvv,sum(comment) as sumcomment,sum(up) as sumup,sum(down) 
> as sumdown,sum(reply) as sumreply,count(*) as count from summary(reply 
> bigint) where videoid 
> in(137102991,151113895,171559204,171559439,171573932,171573932,171573932,171574082,171574082,171574164,171677219,171794335,171902734,172364368,172475141,172700554,172700554,172700554,172716705,172784258,172835778,173112067,173165316,173165316,173379601,173448315,173503961,173692664,173911358,174077089,174099017,174349633,174349877,174651474,174651474,174759297,174883566,174883566,174987670,174987670,175131298)
>  and date>=to_date('2013-09-01','-MM-dd') and 
> date<=to_date('2014-07-07','-MM-dd')



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Comment Edited] (PHOENIX-1081) CPU usage 100% With phoenix

2014-07-11 Thread yang ming (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059499#comment-14059499
 ] 

yang ming edited comment on PHOENIX-1081 at 7/11/14 11:39 PM:
--

Hbase version is 0.94.19.I had added some debug log,may be the endless loop is 
here.
{color:red}at 
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:390){color}
And the problem may be in this method.
{color:blue}at org.apache.phoenix.filter.SkipScanFilter.navigate{color}
   LOOP: while((kv = this.heap.peek()) != null) {
// Check that the heap gives us KVs in an increasing order.
assert prevKV == null || comparator == null || 
comparator.compare(prevKV, kv) <= 0 :
  "Key " + prevKV + " followed by a " + "smaller key " + kv + " in cf " 
+ store;
prevKV = kv;
{color:red}
ScanQueryMatcher.MatchCode qcode = matcher.match(kv);
{color}
switch(qcode) {
  case INCLUDE:
  case INCLUDE_AND_SEEK_NEXT_ROW:
  case INCLUDE_AND_SEEK_NEXT_COL:

Filter f = matcher.getFilter();
outResult.add(f == null ? kv : f.transform(kv));
count++;


was (Author: yangming860101):
Hbase version is 0.94.19.I had added some debug log,may be the endless loop is 
here.
{color:red}at 
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:390){color}
And the problem may be in this method.
{color:blue}at org.apache.phoenix.filter.SkipScanFilter.navigate{blue}
   LOOP: while((kv = this.heap.peek()) != null) {
// Check that the heap gives us KVs in an increasing order.
assert prevKV == null || comparator == null || 
comparator.compare(prevKV, kv) <= 0 :
  "Key " + prevKV + " followed by a " + "smaller key " + kv + " in cf " 
+ store;
prevKV = kv;
{color:red}
ScanQueryMatcher.MatchCode qcode = matcher.match(kv);
{color}
switch(qcode) {
  case INCLUDE:
  case INCLUDE_AND_SEEK_NEXT_ROW:
  case INCLUDE_AND_SEEK_NEXT_COL:

Filter f = matcher.getFilter();
outResult.add(f == null ? kv : f.transform(kv));
count++;

> CPU usage 100% With phoenix 
> 
>
> Key: PHOENIX-1081
> URL: https://issues.apache.org/jira/browse/PHOENIX-1081
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: yang ming
>Priority: Critical
> Attachments: JMX.jpg, jstat.jpg, the jstack of all threads, the 
> jstack of thread 12725.jpg, the jstack of thread 12748.jpg, the threads of 
> regionserver process.jpg
>
>
> The concurrent of the system is not high,but CPU usage often up to 100%.
> I had stopped the system,but regionserver's CPU usage is still high.
> what can case this problem?
> table row count:6000 million
> table ddl:
> create table if not exists summary
> (
> videoid integer not null,
> date date not null,
> platform varchar not null,
> device varchar not null,
> systemgroup varchar not null,
> system varchar not null,
> vv bigint,
> ts bigint,
> up bigint,
> down bigint,
> comment bigint,
> favori bigint,
> favord bigint,
> quote bigint,
> reply bigint
> constraint pk primary key (videoid, date,platform, device, systemgroup,system)
> )salt_buckets = 30,versions=1,compression='snappy';
> query 1:
> select sum(vv) as sumvv,sum(comment) as sumcomment,sum(up) as sumup,sum(down) 
> as sumdown,sum(reply) as sumreply,count(*) as count from summary(reply 
> bigint) where videoid 
> in(137102991,151113895,171559204,171559439,171573932,171573932,171573932,171574082,171574082,171574164,171677219,171794335,171902734,172364368,172475141,172700554,172700554,172700554,172716705,172784258,172835778,173112067,173165316,173165316,173379601,173448315,173503961,173692664,173911358,174077089,174099017,174349633,174349877,174651474,174651474,174759297,174883566,174883566,174987670,174987670,175131298)
>  and date>=to_date('2013-09-01','-MM-dd') and 
> date<=to_date('2014-07-07','-MM-dd')



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Comment Edited] (PHOENIX-1081) CPU usage 100% With phoenix

2014-07-11 Thread yang ming (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059499#comment-14059499
 ] 

yang ming edited comment on PHOENIX-1081 at 7/11/14 11:39 PM:
--

Hbase version is 0.94.19.I had added some debug log,may be the endless loop is 
here.
{color:red}at 
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:390){color}
And the problem may be in this method.
{color:blue}at org.apache.phoenix.filter.SkipScanFilter.navigate{blue}
   LOOP: while((kv = this.heap.peek()) != null) {
// Check that the heap gives us KVs in an increasing order.
assert prevKV == null || comparator == null || 
comparator.compare(prevKV, kv) <= 0 :
  "Key " + prevKV + " followed by a " + "smaller key " + kv + " in cf " 
+ store;
prevKV = kv;
{color:red}
ScanQueryMatcher.MatchCode qcode = matcher.match(kv);
{color}
switch(qcode) {
  case INCLUDE:
  case INCLUDE_AND_SEEK_NEXT_ROW:
  case INCLUDE_AND_SEEK_NEXT_COL:

Filter f = matcher.getFilter();
outResult.add(f == null ? kv : f.transform(kv));
count++;


was (Author: yangming860101):
Hbase version is 0.94.19.I had added some debug log,may be the endless loop is 
here.
{color:red}at 
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:390){color}
   LOOP: while((kv = this.heap.peek()) != null) {
// Check that the heap gives us KVs in an increasing order.
assert prevKV == null || comparator == null || 
comparator.compare(prevKV, kv) <= 0 :
  "Key " + prevKV + " followed by a " + "smaller key " + kv + " in cf " 
+ store;
prevKV = kv;
{color:red}
ScanQueryMatcher.MatchCode qcode = matcher.match(kv);
{color}
switch(qcode) {
  case INCLUDE:
  case INCLUDE_AND_SEEK_NEXT_ROW:
  case INCLUDE_AND_SEEK_NEXT_COL:

Filter f = matcher.getFilter();
outResult.add(f == null ? kv : f.transform(kv));
count++;

> CPU usage 100% With phoenix 
> 
>
> Key: PHOENIX-1081
> URL: https://issues.apache.org/jira/browse/PHOENIX-1081
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: yang ming
>Priority: Critical
> Attachments: JMX.jpg, jstat.jpg, the jstack of all threads, the 
> jstack of thread 12725.jpg, the jstack of thread 12748.jpg, the threads of 
> regionserver process.jpg
>
>
> The concurrent of the system is not high,but CPU usage often up to 100%.
> I had stopped the system,but regionserver's CPU usage is still high.
> what can case this problem?
> table row count:6000 million
> table ddl:
> create table if not exists summary
> (
> videoid integer not null,
> date date not null,
> platform varchar not null,
> device varchar not null,
> systemgroup varchar not null,
> system varchar not null,
> vv bigint,
> ts bigint,
> up bigint,
> down bigint,
> comment bigint,
> favori bigint,
> favord bigint,
> quote bigint,
> reply bigint
> constraint pk primary key (videoid, date,platform, device, systemgroup,system)
> )salt_buckets = 30,versions=1,compression='snappy';
> query 1:
> select sum(vv) as sumvv,sum(comment) as sumcomment,sum(up) as sumup,sum(down) 
> as sumdown,sum(reply) as sumreply,count(*) as count from summary(reply 
> bigint) where videoid 
> in(137102991,151113895,171559204,171559439,171573932,171573932,171573932,171574082,171574082,171574164,171677219,171794335,171902734,172364368,172475141,172700554,172700554,172700554,172716705,172784258,172835778,173112067,173165316,173165316,173379601,173448315,173503961,173692664,173911358,174077089,174099017,174349633,174349877,174651474,174651474,174759297,174883566,174883566,174987670,174987670,175131298)
>  and date>=to_date('2013-09-01','-MM-dd') and 
> date<=to_date('2014-07-07','-MM-dd')



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Comment Edited] (PHOENIX-1081) CPU usage 100% With phoenix

2014-07-11 Thread yang ming (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059499#comment-14059499
 ] 

yang ming edited comment on PHOENIX-1081 at 7/11/14 11:37 PM:
--

Hbase version is 0.94.19.I had added some debug log,may be the endless loop is 
here.
{color:red}at 
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:390){color}
   LOOP: while((kv = this.heap.peek()) != null) {
// Check that the heap gives us KVs in an increasing order.
assert prevKV == null || comparator == null || 
comparator.compare(prevKV, kv) <= 0 :
  "Key " + prevKV + " followed by a " + "smaller key " + kv + " in cf " 
+ store;
prevKV = kv;
{color:red}
ScanQueryMatcher.MatchCode qcode = matcher.match(kv);
{color}
switch(qcode) {
  case INCLUDE:
  case INCLUDE_AND_SEEK_NEXT_ROW:
  case INCLUDE_AND_SEEK_NEXT_COL:

Filter f = matcher.getFilter();
outResult.add(f == null ? kv : f.transform(kv));
count++;


was (Author: yangming860101):
Hbase version is 0.94.19.I had added some debug log,may be the endless loop is 
here.
   LOOP: while((kv = this.heap.peek()) != null) {
// Check that the heap gives us KVs in an increasing order.
assert prevKV == null || comparator == null || 
comparator.compare(prevKV, kv) <= 0 :
  "Key " + prevKV + " followed by a " + "smaller key " + kv + " in cf " 
+ store;
prevKV = kv;
{color:red}
ScanQueryMatcher.MatchCode qcode = matcher.match(kv);
{color}
switch(qcode) {
  case INCLUDE:
  case INCLUDE_AND_SEEK_NEXT_ROW:
  case INCLUDE_AND_SEEK_NEXT_COL:

Filter f = matcher.getFilter();
outResult.add(f == null ? kv : f.transform(kv));
count++;

> CPU usage 100% With phoenix 
> 
>
> Key: PHOENIX-1081
> URL: https://issues.apache.org/jira/browse/PHOENIX-1081
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: yang ming
>Priority: Critical
> Attachments: JMX.jpg, jstat.jpg, the jstack of all threads, the 
> jstack of thread 12725.jpg, the jstack of thread 12748.jpg, the threads of 
> regionserver process.jpg
>
>
> The concurrent of the system is not high,but CPU usage often up to 100%.
> I had stopped the system,but regionserver's CPU usage is still high.
> what can case this problem?
> table row count:6000 million
> table ddl:
> create table if not exists summary
> (
> videoid integer not null,
> date date not null,
> platform varchar not null,
> device varchar not null,
> systemgroup varchar not null,
> system varchar not null,
> vv bigint,
> ts bigint,
> up bigint,
> down bigint,
> comment bigint,
> favori bigint,
> favord bigint,
> quote bigint,
> reply bigint
> constraint pk primary key (videoid, date,platform, device, systemgroup,system)
> )salt_buckets = 30,versions=1,compression='snappy';
> query 1:
> select sum(vv) as sumvv,sum(comment) as sumcomment,sum(up) as sumup,sum(down) 
> as sumdown,sum(reply) as sumreply,count(*) as count from summary(reply 
> bigint) where videoid 
> in(137102991,151113895,171559204,171559439,171573932,171573932,171573932,171574082,171574082,171574164,171677219,171794335,171902734,172364368,172475141,172700554,172700554,172700554,172716705,172784258,172835778,173112067,173165316,173165316,173379601,173448315,173503961,173692664,173911358,174077089,174099017,174349633,174349877,174651474,174651474,174759297,174883566,174883566,174987670,174987670,175131298)
>  and date>=to_date('2013-09-01','-MM-dd') and 
> date<=to_date('2014-07-07','-MM-dd')



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Comment Edited] (PHOENIX-1081) CPU usage 100% With phoenix

2014-07-11 Thread yang ming (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059493#comment-14059493
 ] 

yang ming edited comment on PHOENIX-1081 at 7/11/14 11:37 PM:
--

"IPC Server handler 69 on 60020" daemon prio=10 tid=0x4def5000 
nid=0x31ed runnable [0x48c36000]
   java.lang.Thread.State: RUNNABLE
at 
org.apache.phoenix.filter.SkipScanFilter.navigate(SkipScanFilter.java:288)
at 
org.apache.phoenix.filter.SkipScanFilter.filterKeyValue(SkipScanFilter.java:112)
{color:blue}at 
org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:354){color}
{color:red}at 
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:390){color}
at 
org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:143)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:4047)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:4123)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3990)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3980)
at 
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.scanUnordered(GroupedAggregateRegionObserver.java:384)
at 
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.doPostScannerOpen(GroupedAggregateRegionObserver.java:133)
at 
org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:66)
at 
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1316)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2588)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2556)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:323)
at 
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1434)


was (Author: yangming860101):
"IPC Server handler 69 on 60020" daemon prio=10 tid=0x4def5000 
nid=0x31ed runnable [0x48c36000]
   java.lang.Thread.State: RUNNABLE
at 
org.apache.phoenix.filter.SkipScanFilter.navigate(SkipScanFilter.java:288)
at 
org.apache.phoenix.filter.SkipScanFilter.filterKeyValue(SkipScanFilter.java:112)
at 

{color:blue}org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:354){color}
{color:red}at 
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:390){color}
at 
org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:143)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:4047)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:4123)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3990)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3980)
at 
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.scanUnordered(GroupedAggregateRegionObserver.java:384)
at 
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.doPostScannerOpen(GroupedAggregateRegionObserver.java:133)
at 
org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:66)
at 
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1316)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2588)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2556)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:323)
at 
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1434)

> CPU usage 100% With phoenix 
> 
>
> Key: PHOENIX-1081
> URL: https://issues.apache.org/jira/browse/PHOENIX-1081
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: yang ming
>   

[jira] [Comment Edited] (PHOENIX-1081) CPU usage 100% With phoenix

2014-07-11 Thread yang ming (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059493#comment-14059493
 ] 

yang ming edited comment on PHOENIX-1081 at 7/11/14 11:35 PM:
--

"IPC Server handler 69 on 60020" daemon prio=10 tid=0x4def5000 
nid=0x31ed runnable [0x48c36000]
   java.lang.Thread.State: RUNNABLE
at 
org.apache.phoenix.filter.SkipScanFilter.navigate(SkipScanFilter.java:288)
at 
org.apache.phoenix.filter.SkipScanFilter.filterKeyValue(SkipScanFilter.java:112)
at 

{color:blue}org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:354){color}
{color:red}at 
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:390){color}
at 
org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:143)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:4047)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:4123)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3990)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3980)
at 
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.scanUnordered(GroupedAggregateRegionObserver.java:384)
at 
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.doPostScannerOpen(GroupedAggregateRegionObserver.java:133)
at 
org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:66)
at 
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1316)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2588)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2556)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:323)
at 
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1434)


was (Author: yangming860101):
"IPC Server handler 69 on 60020" daemon prio=10 tid=0x4def5000 
nid=0x31ed runnable [0x48c36000]
   java.lang.Thread.State: RUNNABLE
at 
org.apache.phoenix.filter.SkipScanFilter.navigate(SkipScanFilter.java:288)
at 
org.apache.phoenix.filter.SkipScanFilter.filterKeyValue(SkipScanFilter.java:112)
at 
org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:354)
{color:red}
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:390)
{color}
at 
org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:143)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:4047)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:4123)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3990)
at 
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3980)
at 
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.scanUnordered(GroupedAggregateRegionObserver.java:384)
at 
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver.doPostScannerOpen(GroupedAggregateRegionObserver.java:133)
at 
org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:66)
at 
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1316)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.internalOpenScanner(HRegionServer.java:2588)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2556)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:323)
at 
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1434)

> CPU usage 100% With phoenix 
> 
>
> Key: PHOENIX-1081
> URL: https://issues.apache.org/jira/browse/PHOENIX-1081
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: yang ming
>