[jira] [Created] (HBASE-11695) PeriodicFlusher and WakeFrequency issues

2014-08-07 Thread Lars Hofhansl (JIRA)
Lars Hofhansl created HBASE-11695:
-

 Summary: PeriodicFlusher and WakeFrequency issues
 Key: HBASE-11695
 URL: https://issues.apache.org/jira/browse/HBASE-11695
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.21
Reporter: Lars Hofhansl


We just ran into a flush storm caused by the PeriodicFlusher.
Many memstore became eligible for flushing at exactly the same time, the effect 
we've seen is that the exact same region was flushed multiple times, because 
the flusher wakes up too often (every 10s). The jitter of 20s is larger than 
that and it takes some time to actually flush the memstore.

Here's one example. We've seen 100's of these, monopolizing the flush queue and 
preventing important flushes from happening.

{code}
06-Aug-2014 20:11:56  [regionserver60020.periodicFlusher] INFO  
org.apache.hadoop.hbase.regionserver.HRegionServer[1397]-- 
regionserver60020.periodicFlusher requesting flush for region 
tsdb,\x00\x00\x0AO\xCF* 
\x00\x00\x01\x00\x01\x1F\x00\x00\x03\x00\x00\x0C,1340147003629.ef4a680b962592de910d0fdeb376dfc2.
 after a delay of 13449
06-Aug-2014 20:12:06  [regionserver60020.periodicFlusher] INFO  
org.apache.hadoop.hbase.regionserver.HRegionServer[1397]-- 
regionserver60020.periodicFlusher requesting flush for region 
tsdb,\x00\x00\x0AO\xCF* 
\x00\x00\x01\x00\x01\x1F\x00\x00\x03\x00\x00\x0C,1340147003629.ef4a680b962592de910d0fdeb376dfc2.
 after a delay of 14060
{code}

So we need to increase the period of the PeriodicFlusher to at least the random 
jitter, also increase the default random jitter (20s does not help with many 
regions).




--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Reopened] (HBASE-11527) Cluster free memory limit check should consider L2 block cache size also when L2 cache is onheap.

2014-08-07 Thread Anoop Sam John (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-11527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John reopened HBASE-11527:



Sure let me make patch for branch-1 as well. Reopening the jira.

 Cluster free memory limit check should consider L2 block cache size also when 
 L2 cache is onheap.
 -

 Key: HBASE-11527
 URL: https://issues.apache.org/jira/browse/HBASE-11527
 Project: HBase
  Issue Type: Bug
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 2.0.0

 Attachments: HBASE-11527.patch






--
This message was sent by Atlassian JIRA
(v6.2#6252)


Re: [VOTE] The 1st HBase 0.98.5 release candidate (RC0) is available, vote closing 8/11/2014

2014-08-07 Thread Ted Yu
+1

verified signatures,
ran unit tests
ran integration tests (such as IntegrationTestBigLinkedLis) against a small
cluster
inspected master log


On Tue, Aug 5, 2014 at 10:31 PM, Esteban Gutierrez este...@cloudera.com
wrote:

 +1 (non-binding)

 verified signatures, recompiled, ran PE and YCSB in a 5 node cluster and
 everything looks good so far. I will keep running additional tests and will
 report any issue.

 Thanks Andrew!
 esteban.


 --
 Cloudera, Inc.



 On Tue, Aug 5, 2014 at 12:19 AM, Andrew Purtell apurt...@apache.org
 wrote:

  ​The 1st HBase 0.98.​5 release candidate (RC0) is available for download
  ​at http://people.apache.org/~apurtell/0.98.5RC0/ and Maven artifacts
 are
  also available in the temporary repository ​
  https://repository.apache.org/content/repositories/orgapachehbase-1027/
 
  Signed with my code signing key D5365CCD.
 
  The issues resolved in this release can be found here​:​​ ​
 
 
 https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753version=12326953
 
 
  Please try out the candidate and vote +1/-1 by midnight Pacific Time
 (00:00
  -​​0800 GMT) on ​August 11 on whether or not we should release​ this as
  0.98.5. Three +1 votes from PMC will be required to
  release.​​
  ​​
 
  --
  Best regards,
 
 - Andy
 
  Problems worthy of attack prove their worth by hitting back. - Piet Hein
  (via Tom White)
 



[jira] [Created] (HBASE-11696) Make CombinedBlockCache resizable.

2014-08-07 Thread Anoop Sam John (JIRA)
Anoop Sam John created HBASE-11696:
--

 Summary: Make CombinedBlockCache resizable.
 Key: HBASE-11696
 URL: https://issues.apache.org/jira/browse/HBASE-11696
 Project: HBase
  Issue Type: Improvement
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 2.0.0


HBASE-5349 adds auto tuning of memstore heap and block cache heap. Block cache 
needs to be resizable in order for this. CombinedBlockCache is not marked 
resizable now. We can make this. On resize the L1 cache (ie. LRU cache) can get 
resized.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HBASE-11697) Improve the 'Too many blocks' message on UI blockcache status page

2014-08-07 Thread Andrew Purtell (JIRA)
Andrew Purtell created HBASE-11697:
--

 Summary: Improve the 'Too many blocks' message on UI blockcache 
status page
 Key: HBASE-11697
 URL: https://issues.apache.org/jira/browse/HBASE-11697
 Project: HBase
  Issue Type: Improvement
Reporter: Andrew Purtell
Priority: Minor
 Fix For: 0.99.0, 2.0.0, 0.98.6
 Attachments: TooManyBlocks.png

If metrics calculations over blockcache contents stopped after examining 
hbase.ui.blockcache.by.file.max items, the UI will put up a message. However, 
this notion of too many blocks / fullness refers to structures used for 
calculating blockcache metrics. See BlockCacheUtil. We should improve this 
message so it does not leave a user the impression the blockcache may be in a 
bad state.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HBASE-11698) TestFullLogReconstruction has duplicate config setting

2014-08-07 Thread Sean Busbey (JIRA)
Sean Busbey created HBASE-11698:
---

 Summary: TestFullLogReconstruction has duplicate config setting
 Key: HBASE-11698
 URL: https://issues.apache.org/jira/browse/HBASE-11698
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.98.5
Reporter: Sean Busbey
Priority: Trivial


The 0.98 branch's version of TestFullLogReconstruction has a duplicate config 
setting during setupBeforeClass

{code}
54// faster failover with cluster.shutdown();fs.close() idiom
55c.setInt(hbase.ipc.client.connect.max.retries, 1);
56c.setInt(hbase.ipc.client.connect.max.retries, 1);
{code}

the 0.98.4 release has a line for setting this config and for the HDFS version. 
the branch-1 and master versions of this set of patches have only the setting 
of the hbase one, so I think that's the correct behavior.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


Re: Failing region replica tests in trunk

2014-08-07 Thread Jimmy Xiang
HBase master jenkins is finally blue again. Let's keep it blue.

https://builds.apache.org/job/HBase-TRUNK/5380/

Thanks,
Jimmy


On Wed, Aug 6, 2014 at 3:24 PM, Stack st...@duboce.net wrote:

 On Wed, Aug 6, 2014 at 3:21 PM, Devaraj Das d...@hortonworks.com wrote:

  Yeah the replica tests issues are due to turning on the zk-less
  assignment by default.
 
  If HBASE-11611 addresses the issues to do with zk-less assignments +
  replicas, let's wait for that.
 
 
 Grand. Thanks DD,
 St.Ack



Re: Failing region replica tests in trunk

2014-08-07 Thread Stack
On Thu, Aug 7, 2014 at 1:22 PM, Jimmy Xiang jxi...@cloudera.com wrote:

 HBase master jenkins is finally blue again. Let's keep it blue.

 https://builds.apache.org/job/HBase-TRUNK/5380/

 Thanks,
 Jimmy


Hot Dog and Amen
St.Ack


[jira] [Created] (HBASE-11699) Region servers exclusion list to HMaster.

2014-08-07 Thread Gomathivinayagam Muthuvinayagam (JIRA)
Gomathivinayagam Muthuvinayagam created HBASE-11699:
---

 Summary: Region servers exclusion list to HMaster.
 Key: HBASE-11699
 URL: https://issues.apache.org/jira/browse/HBASE-11699
 Project: HBase
  Issue Type: New Feature
  Components: Admin, Client, regionserver, Zookeeper
Affects Versions: 0.98.3
Reporter: Gomathivinayagam Muthuvinayagam
Priority: Minor
 Fix For: 0.98.3


Currently HBase does not support adding set of region servers to be in the 
exclusion list. So that administrators can prevent accidental startups of some 
region servers to join the cluster. There was initially some work done, and it 
is available in https://issues.apache.org/jira/browse/HBASE-3833. It was not 
done after that. 

I am planning to contribute it as a patch, and I would like to do some 
improvements as well. Instead of storing the exclusion entries on a file, I am 
planning to store it on zookeeper. Can anyone suggest thoughts on this? 





--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HBASE-11700) Fix typo of hbase.store.compaction.ratio (should be hbase.hstore.compaction.ratio) in Ref Guide

2014-08-07 Thread Misty Stanley-Jones (JIRA)
Misty Stanley-Jones created HBASE-11700:
---

 Summary: Fix typo of hbase.store.compaction.ratio (should be 
hbase.hstore.compaction.ratio) in Ref Guide
 Key: HBASE-11700
 URL: https://issues.apache.org/jira/browse/HBASE-11700
 Project: HBase
  Issue Type: Bug
  Components: documentation
Reporter: Misty Stanley-Jones


Hey, noticed that in the doc it lists out the following option: 
hbase.store.compaction.ratio but it should be hbase.hstore.compaction.ratio. 
Doc looks really good though! I will hopefully have more feedback for you soon. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Resolved] (HBASE-11697) Improve the 'Too many blocks' message on UI blockcache status page

2014-08-07 Thread Andrew Purtell (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-11697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell resolved HBASE-11697.


  Resolution: Fixed
Hadoop Flags: Reviewed

Committed to 0.98+

 Improve the 'Too many blocks' message on UI blockcache status page
 --

 Key: HBASE-11697
 URL: https://issues.apache.org/jira/browse/HBASE-11697
 Project: HBase
  Issue Type: Improvement
Reporter: Andrew Purtell
Assignee: Mikhail Antonov
Priority: Minor
 Fix For: 0.99.0, 2.0.0, 0.98.6

 Attachments: HBASE-11697.patch, TooManyBlocks.png


 If metrics calculations over blockcache contents stopped after examining 
 hbase.ui.blockcache.by.file.max items, the UI will put up a message. However, 
 this notion of too many blocks / fullness refers to structures used for 
 calculating blockcache metrics. See BlockCacheUtil. We should improve this 
 message so it does not leave a user the impression the blockcache may be in a 
 bad state.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Resolved] (HBASE-11691) Backport HBASE-7156 (Add Data Block Encoding and -D opts to Performance Evaluation) to 0.94

2014-08-07 Thread Andrew Purtell (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-11691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell resolved HBASE-11691.


  Resolution: Fixed
Hadoop Flags: Reviewed

Pushed to 0.94

 Backport HBASE-7156 (Add Data Block Encoding and -D opts to Performance 
 Evaluation) to 0.94
 ---

 Key: HBASE-11691
 URL: https://issues.apache.org/jira/browse/HBASE-11691
 Project: HBase
  Issue Type: Task
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Minor
 Fix For: 0.94.23

 Attachments: HBASE-11691.patch


 Backport HBASE-7156 (Add Data Block Encoding and -D opts to Performance 
 Evaluation) to 0.94 branch.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HBASE-11701) Start and end of memstore flush log should be on the same level

2014-08-07 Thread Esteban Gutierrez (JIRA)
Esteban Gutierrez created HBASE-11701:
-

 Summary: Start and end of memstore flush log should be on the same 
level
 Key: HBASE-11701
 URL: https://issues.apache.org/jira/browse/HBASE-11701
 Project: HBase
  Issue Type: Improvement
  Components: Usability
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Trivial


Currently the start of the memstore flush is done in DEBUG level and when it 
finishes is done in INFO. Both should be done in the same level.





--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HBASE-11702) Better introspection of long running compactions

2014-08-07 Thread Andrew Purtell (JIRA)
Andrew Purtell created HBASE-11702:
--

 Summary: Better introspection of long running compactions
 Key: HBASE-11702
 URL: https://issues.apache.org/jira/browse/HBASE-11702
 Project: HBase
  Issue Type: Improvement
Reporter: Andrew Purtell
Assignee: Andrew Purtell
 Fix For: 0.99.0, 2.0.0, 0.98.6


For better introspection of long running compactions, periodically print 
compaction progress for a file at DEBUG level (thread name, file path, total 
compacted KVs, total compacted bytes, completion percent, rate) and expose 
regionserver wide compaction metrics (KVs compacted and bytes compacted) as 
Hadoop value metrics.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HBASE-11703) Meta region state could be corrupted

2014-08-07 Thread Jimmy Xiang (JIRA)
Jimmy Xiang created HBASE-11703:
---

 Summary: Meta region state could be corrupted
 Key: HBASE-11703
 URL: https://issues.apache.org/jira/browse/HBASE-11703
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


Internal meta region state could be corrupted if the meta is not on master:

1. the meta region server (not master) shuts down,
2. meta SSH offlines it without updating the dead server's region list,
3. meta is transitioned to pending_open and the previous server (the dead 
server) of meta is lost,
4. meta is assigned somewhere else without updating its previous server,
5. normal SSH processes the dead server and offlines all of it's dead regions 
including the meta, so the meta internal state is corrupted



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (HBASE-11704) Include set-up instructions for building with IDEs besides Eclipse

2014-08-07 Thread Misty Stanley-Jones (JIRA)
Misty Stanley-Jones created HBASE-11704:
---

 Summary: Include set-up instructions for building with IDEs 
besides Eclipse
 Key: HBASE-11704
 URL: https://issues.apache.org/jira/browse/HBASE-11704
 Project: HBase
  Issue Type: Bug
  Components: documentation
Reporter: Misty Stanley-Jones


We have pretty detailed instructions for setting up Eclipse to build HBase at 
http://hbase.apache.org/book/ides.html#eclipse. It would be nice to have this 
for other popular IDEs such as IntelliJ.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Resolved] (HBASE-11693) Backport HBASE-11026 (Provide option to filter out all rows in PerformanceEvaluation tool) to 0.94

2014-08-07 Thread Andrew Purtell (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-11693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell resolved HBASE-11693.


  Resolution: Fixed
Hadoop Flags: Reviewed

Pushed to 0.94

 Backport HBASE-11026 (Provide option to filter out all rows in 
 PerformanceEvaluation tool) to 0.94
 --

 Key: HBASE-11693
 URL: https://issues.apache.org/jira/browse/HBASE-11693
 Project: HBase
  Issue Type: Task
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Minor
 Fix For: 0.94.23

 Attachments: HBASE-11693.patch


 Backport HBASE-11026 (Provide option to filter out all rows in 
 PerformanceEvaluation tool) to 0.94 branch.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Resolved] (HBASE-11700) Fix typo of hbase.store.compaction.ratio (should be hbase.hstore.compaction.ratio) in Ref Guide

2014-08-07 Thread Misty Stanley-Jones (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones resolved HBASE-11700.
-

Resolution: Duplicate

 Fix typo of hbase.store.compaction.ratio (should be 
 hbase.hstore.compaction.ratio) in Ref Guide
 ---

 Key: HBASE-11700
 URL: https://issues.apache.org/jira/browse/HBASE-11700
 Project: HBase
  Issue Type: Bug
  Components: documentation
Reporter: Misty Stanley-Jones
  Labels: beginner

 Hey, noticed that in the doc it lists out the following option: 
 hbase.store.compaction.ratio but it should be hbase.hstore.compaction.ratio. 
 Doc looks really good though! I will hopefully have more feedback for you 
 soon. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[VOTE] The 1st hbase 0.94.22 release candidate is available for download

2014-08-07 Thread Nick Dimiduk
- verified md5 [✓]
- verified asc [✓]
- build and test default profile with Oracle 1.6.0_31 [✓]
Tests run: 926, Failures: 0, Errors: 0, Skipped: 2
- build and test default profile with Oracle 1.7.0_21 [✓]
Tests run: 926, Failures: 0, Errors: 0, Skipped: 2
- Builds vs hadoop-2.4.0 completes but I get test failure. I see that
version is Not Tested in our support matrix, so [✓]

This machine is kind if a mess and I can't get hadoop-1 working to test the
RC properly. I'm headed out if town and I won't have more time before the
deadline, but I thought I'd report my findings. No issues from my side but
I can't really sign off on it either, so +0 (non-binding anyway, we require
PMC, right?) from me.

-n

On Wed, Aug 6, 2014 at 11:25 AM, Andrew Purtell apurt...@apache.org
javascript:_e(%7B%7D,'cvml','apurt...@apache.org'); wrote:

 +1

 - Hash matches tarball
 - Signature validates
 - Launched an all-localhost cluster and ran LoadTestTool up to 1M rows, no
 errors or items of concerns observed in the logs, using 7u65
 - Built from source using the Hadoop 2 profile
 - RAT checks pass
 - Unit tests pass locally, using 7u65




 On Sat, Aug 2, 2014 at 9:28 AM, lars hofhansl la...@apache.org
 javascript:_e(%7B%7D,'cvml','la...@apache.org'); wrote:

  The 1st 0.94.22 RC is available for download at
  http://people.apache.org/~larsh/hbase-0.94.22-rc0/
  Signed with my code signing key: C7CFE328
 
  HBase 0.94.22 is a small bug fix release with 13 fixes:
  [HBASE-10645] - Fix wrapping of Requests Counts Regionserver level
  metrics
  [HBASE-11360] - SnapshotFileCache causes too many cache refreshes
  [HBASE-11479] - SecureConnection can't be closed when SecureClient is
  stopping because InterruptedException won't be caught in
  SecureClient#setupIOstreams()
  [HBASE-11496] - HBASE-9745 broke cygwin CLASSPATH translation
  [HBASE-11552] - Read/Write requests count metric value is too short
  [HBASE-11565] - Stale connection could stay for a while
  [HBASE-11633] - [0.94] port HBASE-11217 Race between SplitLogManager
  task creation + TimeoutMonitor
  [HBASE-2217] - VM OPTS for shell only
  [HBASE-7910] - Dont use reflection for security
  [HBASE-11444] - Remove use of reflection for User#getShortName
  [HBASE-11450] - Improve file size info in SnapshotInfo tool
  [HBASE-11480] - ClientScanner might not close the HConnection created
  in construction
  [HBASE-11623] - mutateRowsWithLocks might require
 updatesLock.readLock
  with waitTime=0
 
  TThe list of changes is also available here:
 
 https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753version=12327117
 
  Here's the test run for this RC:
  https://builds.apache.org/job/HBase-0.94.22/34/
 
  Please try out the RC, check out the doc, take it for a spin, etc, and
  vote +1/-1 by EOD August 8th on whether we should release this as
 0.94.22.
 
  Thanks.
 
  -- Lars




 --
 Best regards,

- Andy

 Problems worthy of attack prove their worth by hitting back. - Piet Hein
 (via Tom White)



Re: [VOTE] The 1st hbase 0.94.22 release candidate is available for download

2014-08-07 Thread lars hofhansl
Thanks Nick (and Andrew, and JM). Will release tomorrow.
Will also set time aside tomorrow to test 0.98.5.

-- Lars




 From: Nick Dimiduk ndimi...@gmail.com
To: hbase-dev dev@hbase.apache.org 
Cc: lars hofhansl la...@apache.org 
Sent: Thursday, August 7, 2014 9:39 PM
Subject: [VOTE] The 1st hbase 0.94.22 release candidate is available for 
download
 


- verified md5 [✓]
- verified asc [✓]
- build and test default profile with Oracle 1.6.0_31 [✓]
Tests run: 926, Failures: 0, Errors: 0, Skipped: 2
- build and test default profile with Oracle 1.7.0_21 [✓]
Tests run: 926, Failures: 0, Errors: 0, Skipped: 2
- Builds vs hadoop-2.4.0 completes but I get test failure. I see that version 
is Not Tested in our support matrix, so [✓]

This machine is kind if a mess and I can't get hadoop-1 working to test the RC 
properly. I'm headed out if town and I won't have more time before the 
deadline, but I thought I'd report my findings. No issues from my side but I 
can't really sign off on it either, so +0 (non-binding anyway, we require PMC, 
right?) from me.

-n




On Wed, Aug 6, 2014 at 11:25 AM, Andrew Purtell apurt...@apache.org wrote:

+1

- Hash matches tarball
- Signature validates
- Launched an all-localhost cluster and ran LoadTestTool up to 1M rows, no
errors or items of concerns observed in the logs, using 7u65
- Built from source using the Hadoop 2 profile
- RAT checks pass
- Unit tests pass locally, using 7u65





On Sat, Aug 2, 2014 at 9:28 AM, lars hofhansl la...@apache.org wrote:

 The 1st 0.94.22 RC is available for download at
 http://people.apache.org/~larsh/hbase-0.94.22-rc0/
 Signed with my code signing key: C7CFE328

 HBase 0.94.22 is a small bug fix release with 13 fixes:
     [HBASE-10645] - Fix wrapping of Requests Counts Regionserver level
 metrics
     [HBASE-11360] - SnapshotFileCache causes too many cache refreshes
     [HBASE-11479] - SecureConnection can't be closed when SecureClient is
 stopping because InterruptedException won't be caught in
 SecureClient#setupIOstreams()
     [HBASE-11496] - HBASE-9745 broke cygwin CLASSPATH translation
     [HBASE-11552] - Read/Write requests count metric value is too short
     [HBASE-11565] - Stale connection could stay for a while
     [HBASE-11633] - [0.94] port HBASE-11217 Race between SplitLogManager
 task creation + TimeoutMonitor
     [HBASE-2217] - VM OPTS for shell only
     [HBASE-7910] - Dont use reflection for security
     [HBASE-11444] - Remove use of reflection for User#getShortName
     [HBASE-11450] - Improve file size info in SnapshotInfo tool
     [HBASE-11480] - ClientScanner might not close the HConnection created
 in construction
     [HBASE-11623] - mutateRowsWithLocks might require updatesLock.readLock
 with waitTime=0

 TThe list of changes is also available here:
 https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753version=12327117

 Here's the test run for this RC:
 https://builds.apache.org/job/HBase-0.94.22/34/

 Please try out the RC, check out the doc, take it for a spin, etc, and
 vote +1/-1 by EOD August 8th on whether we should release this as 0.94.22.

 Thanks.

 -- Lars




--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)


[jira] [Resolved] (HBASE-11323) BucketCache all the time!

2014-08-07 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-11323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-11323.
---

  Resolution: Fixed
Release Note: Use the LruBlockCache default if your data fits the 
blockcache.  If block cache churn or you want a block cache that is immune to 
the vagaries of BC, deploy the offheap bucketcache.  See 
http://people.apache.org/~stack/bc/

 BucketCache all the time!
 -

 Key: HBASE-11323
 URL: https://issues.apache.org/jira/browse/HBASE-11323
 Project: HBase
  Issue Type: Sub-task
  Components: io
Reporter: stack
Assignee: stack
 Fix For: 2.0.0

 Attachments: 
 BlockCacheReportLruBlockCachevsOffHeapCombinedBlockCacheSmall4G (1).pdf, 
 ReportBlockCache.pdf


 One way to realize the parent issue is to just enable bucket cache all the 
 time; i.e. always have offheap enabled.  Would have to do some work to make 
 it drop-dead simple on initial setup (I think it doable).
 So, upside would be the offheap upsides (less GC, less likely to go away and 
 never come back because of full GC when heap is large, etc.).
 Downside is higher latency.   In Nick's BlockCache 101 there is little to no 
 difference between onheap and offheap.  In a basic compare doing scans and 
 gets -- details to follow -- I have BucketCache deploy about 20% less ops 
 than LRUBC when all incache and maybe 10% less ops when falling out of cache. 
   I can't tell difference in means and 95th and 99th are roughly same (more 
 stable with BucketCache).  GC profile is much better with BucketCache -- way 
 less.  BucketCache uses about 7% more user CPU.
 More detail on comparison to follow.
 I think the numbers disagree enough we should probably do the [~lhofhansl] 
 suggestion, that we allow you to have a table sit in LRUBC, something the 
 current bucket cache layout does not do.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


Re: [VOTE] The 1st hbase 0.94.22 release candidate is available for download

2014-08-07 Thread Anoop John
+1
Verified signature
Built from src
Deployed in a small cluster and doing some random shell commands.
Ran tests.
All looks good.

-Anoop-

On Fri, Aug 8, 2014 at 10:13 AM, lars hofhansl la...@apache.org wrote:

 Thanks Nick (and Andrew, and JM). Will release tomorrow.
 Will also set time aside tomorrow to test 0.98.5.

 -- Lars



 
  From: Nick Dimiduk ndimi...@gmail.com
 To: hbase-dev dev@hbase.apache.org
 Cc: lars hofhansl la...@apache.org
 Sent: Thursday, August 7, 2014 9:39 PM
 Subject: [VOTE] The 1st hbase 0.94.22 release candidate is available for
 download



 - verified md5 [✓]
 - verified asc [✓]
 - build and test default profile with Oracle 1.6.0_31 [✓]
 Tests run: 926, Failures: 0, Errors: 0, Skipped: 2
 - build and test default profile with Oracle 1.7.0_21 [✓]
 Tests run: 926, Failures: 0, Errors: 0, Skipped: 2
 - Builds vs hadoop-2.4.0 completes but I get test failure. I see that
 version is Not Tested in our support matrix, so [✓]

 This machine is kind if a mess and I can't get hadoop-1 working to test
 the RC properly. I'm headed out if town and I won't have more time before
 the deadline, but I thought I'd report my findings. No issues from my side
 but I can't really sign off on it either, so +0 (non-binding anyway, we
 require PMC, right?) from me.

 -n




 On Wed, Aug 6, 2014 at 11:25 AM, Andrew Purtell apurt...@apache.org
 wrote:

 +1
 
 - Hash matches tarball
 - Signature validates
 - Launched an all-localhost cluster and ran LoadTestTool up to 1M rows, no
 errors or items of concerns observed in the logs, using 7u65
 - Built from source using the Hadoop 2 profile
 - RAT checks pass
 - Unit tests pass locally, using 7u65
 
 
 
 
 
  On Sat, Aug 2, 2014 at 9:28 AM, lars hofhansl la...@apache.org wrote:
 
  The 1st 0.94.22 RC is available for download at
  http://people.apache.org/~larsh/hbase-0.94.22-rc0/
  Signed with my code signing key: C7CFE328
 
  HBase 0.94.22 is a small bug fix release with 13 fixes:
  [HBASE-10645] - Fix wrapping of Requests Counts Regionserver level
  metrics
  [HBASE-11360] - SnapshotFileCache causes too many cache refreshes
  [HBASE-11479] - SecureConnection can't be closed when SecureClient
 is
  stopping because InterruptedException won't be caught in
  SecureClient#setupIOstreams()
  [HBASE-11496] - HBASE-9745 broke cygwin CLASSPATH translation
  [HBASE-11552] - Read/Write requests count metric value is too short
  [HBASE-11565] - Stale connection could stay for a while
  [HBASE-11633] - [0.94] port HBASE-11217 Race between SplitLogManager
  task creation + TimeoutMonitor
  [HBASE-2217] - VM OPTS for shell only
  [HBASE-7910] - Dont use reflection for security
  [HBASE-11444] - Remove use of reflection for User#getShortName
  [HBASE-11450] - Improve file size info in SnapshotInfo tool
  [HBASE-11480] - ClientScanner might not close the HConnection
 created
  in construction
  [HBASE-11623] - mutateRowsWithLocks might require
 updatesLock.readLock
  with waitTime=0
 
  TThe list of changes is also available here:
 
 https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753version=12327117
 
  Here's the test run for this RC:
  https://builds.apache.org/job/HBase-0.94.22/34/
 
  Please try out the RC, check out the doc, take it for a spin, etc, and
  vote +1/-1 by EOD August 8th on whether we should release this as
 0.94.22.
 
  Thanks.
 
  -- Lars
 
 
 
 
 --
 Best regards,
 
- Andy
 
 Problems worthy of attack prove their worth by hitting back. - Piet Hein
 (via Tom White)