[jira] [Resolved] (HBASE-11963) Synchronize peer cluster replication connection attempts

2014-09-12 Thread Lars Hofhansl (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl resolved HBASE-11963.
---
  Resolution: Fixed
Hadoop Flags: Reviewed

Fixed. Thanks [~sukuna...@gmail.com].

> Synchronize peer cluster replication connection attempts
> 
>
> Key: HBASE-11963
> URL: https://issues.apache.org/jira/browse/HBASE-11963
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Andrew Purtell
>Assignee: Maddineni Sukumar
> Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1
>
> Attachments: 11963-0.94.txt, HBASE-11963-0.98.patch, HBASE-11963.patch
>
>
> Synchronize peer cluster connection attempts to avoid races and rate limit 
> connections when multiple replication sources try to connect to the peer 
> cluster. If the peer cluster is down we can get out of control over time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: First release candidate for HBase 0.99.0 (RC0) is available. Please vote by 09/17/2014

2014-09-12 Thread Andrew Purtell
I agree it would be surprising to have masters running RegionServers and 
hosting regions. Maybe we can take that kind of departure for 2.0? (Or even 
1.1?) It's not clear what state that will end up in. Default-on features in 1.0 
should carry forward and promote stability and familiarity? 


> On Sep 11, 2014, at 10:02 AM, Stack  wrote:
> 
> Thanks for doing the helpful writeup Enis. It helps doing evaluation.
> 
> I have one question below.
> 
> On Thu, Sep 11, 2014 at 1:56 AM, Enis Söztutar  wrote:
> ...
> 
>> 
>> Starting with 0.99.0, the HBase master server and backup master servers
>> will
>> also act as a region server. RPC port and info port for web UI is shared
>> for
>> the master and region server roles. Active master will be hosting the meta
>> table (and other hbase system tables, acl and namespace) by default
>> (unless configured otherwise). The master and backup masters will not be
>> hosting user level regions. See HBASE-10569 and HBASE-11604 for more
>> details.
> 
> I think we should change this so this is NOT the default in 1.0. What do
> folks think?  The new deploy topology will surprise going from earlier
> version. Better folks enable it explicitly*?
> St.Ack
> 
> * I used to be in favor of this feature being on by default but I have
> since changed my mind given how I see meta hosting evolving in the
> near-future.


Re: build YCSB for hbase 0.98.5-hadoop2

2014-09-12 Thread 牛兆捷
I add HBase 0.98.5-hadooo2 dependency into pom.xml, but it says can't find
it in maven repo.
On Sep 13, 2014 10:29 AM, "牛兆捷"  wrote:

> If I want to rebuild YCSB specific for hbase 0.98.5-hadoop2, what should I
> do?
>
> --
> *Regards,*
> *Zhaojie*
>
>


[jira] [Created] (HBASE-11971) [0.98] Deprecate KeyValue based APIs in WALEdit

2014-09-12 Thread Anoop Sam John (JIRA)
Anoop Sam John created HBASE-11971:
--

 Summary: [0.98] Deprecate KeyValue based APIs in WALEdit
 Key: HBASE-11971
 URL: https://issues.apache.org/jira/browse/HBASE-11971
 Project: HBase
  Issue Type: Improvement
  Components: wal
Affects Versions: 0.98.6
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Priority: Critical
 Fix For: 0.98.7


HBASE-11805 replaces KeyValue based APIs in WALEdit with Cell based in 0.99+ 
versions. We need to deprecate the old APIs in 0.98 branch and add extra Cell 
based APIs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (HBASE-11805) KeyValue to Cell Convert in WALEdit APIs

2014-09-12 Thread Anoop Sam John (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-11805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John resolved HBASE-11805.

   Resolution: Fixed
Fix Version/s: (was: 0.98.7)
 Release Note: 
The KeyValue based APIs in WALEdit is been removed and replaced with Cell based 
APIs
void add(KeyValue)->   void add(Cell)
ArrayList getKeyValues()->ArrayList getCells()

  was:
In 0.98 WALEdit below APIs are deprecated
void add(KeyValue)
ArrayList getKeyValues()
And added new APIs
void add(Cell)
ArrayList getCells()

in 0.99+ the deprecated APIs are removed and replaced with new mentioned.

 Hadoop Flags: Incompatible change,Reviewed  (was: Reviewed)

> KeyValue to Cell Convert in WALEdit APIs
> 
>
> Key: HBASE-11805
> URL: https://issues.apache.org/jira/browse/HBASE-11805
> Project: HBase
>  Issue Type: Improvement
>  Components: wal
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0, 0.99.0
>
> Attachments: HBASE-11805.patch, HBASE-11805_0.98.patch, 
> HBASE-11805_0.98_V2.patch, HBASE-11805_0.98_addendum.patch, 
> HBASE-11805_0.99.patch, HBASE-11805_V2.patch, HBASE-11805_V3.patch
>
>
> In almost all other main interface class/APIs we have changed KeyValue to 
> Cell. But missing in WALEdit. This is public marked for Replication (Well it 
> should be for CP also) 
> These 2 APIs deal with KVs
> add(KeyValue kv)
> ArrayList getKeyValues()
> Suggest deprecate them and add for 0.98
> add(Cell kv) 
> List getCells()
> And just replace from 1.0



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


build YCSB for hbase 0.98.5-hadoop2

2014-09-12 Thread 牛兆捷
If I want to rebuild YCSB specific for hbase 0.98.5-hadoop2, what should I
do?

-- 
*Regards,*
*Zhaojie*


[jira] [Created] (HBASE-11970) BlockCache would be instantiated when Master doesn't host regions

2014-09-12 Thread Ted Yu (JIRA)
Ted Yu created HBASE-11970:
--

 Summary: BlockCache would be instantiated when Master doesn't host 
regions
 Key: HBASE-11970
 URL: https://issues.apache.org/jira/browse/HBASE-11970
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Ted Yu


HMaster ctor calls HRegionServer ctor which calls CacheConfig ctor where 
CacheConfig.instantiateBlockCache() would create BlockCache.

This happens even if Master is configured not to host regions.

This was observed when I tried to answer question from the thread titled:
'The default hbase.regionserver.info.port doesn't work'



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HBASE-11969) 3 Thrift annotated filters for TFilterV2

2014-09-12 Thread Hyeonjong (JIRA)
Hyeonjong created HBASE-11969:
-

 Summary: 3 Thrift annotated filters for TFilterV2
 Key: HBASE-11969
 URL: https://issues.apache.org/jira/browse/HBASE-11969
 Project: HBase
  Issue Type: Sub-task
  Components: Filters, Thrift
Affects Versions: 0.89-fb
Reporter: Hyeonjong
Priority: Minor
 Fix For: 0.89-fb


- FilterList, SingleColumnValueFilter, SingleColumnValueExcludeFilter were 
annotated as ThriftStruct
- Latest thrift IDL was applied
- Small modifications for code cleanup



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HBASE-11968) If MOB is enabled, it should make sure hfile v3 is being used.

2014-09-12 Thread Jonathan Hsieh (JIRA)
Jonathan Hsieh created HBASE-11968:
--

 Summary: If MOB is enabled, it should make sure hfile v3 is being 
used.
 Key: HBASE-11968
 URL: https://issues.apache.org/jira/browse/HBASE-11968
 Project: HBase
  Issue Type: Bug
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
 Fix For: hbase-11339


There are currently no warnings if HFile V2 is used while the mob feature is 
enabled.  Ideally the mob code path should check if hfile v3 is used and if it 
is not default to the normal read/write paths.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: First release candidate for HBase 0.99.0 (RC0) is available. Please vote by 09/17/2014

2014-09-12 Thread Stack
+1

md5 is good.  Signature checked out but said this:

gpg: Can't check signature: public key not found

... so maybe put yours up in public place?

Thanks for filling out CHANGES.txt with included fixes.

Tried to start up master but failed because old dir in place (HMaster
wouldn't go down on FATAL error... HBASE-11967).

I wish we could turn down the zk logging especially its IOE dumps though
all is working 'normally'.

Docs look good (did you copy in the latest?)

Loaded up some data.  Verified it there.  Seems slower.  More MR going on?
 Logs look ok.

I've been running close to tip on a little cluster here and it looks
basically fine.

Will continue testing...

(I don't think you need a week-long VOTE for a developer release we know
has issues in it)

St.Ack


On Thu, Sep 11, 2014 at 1:56 AM, Enis Söztutar  wrote:

> I am very happy to say that the first HBase 0.99.0 release candidate (RC0)
> is
> available for download at https://people.apache.org/~enis/hbase-0.99.0RC0/
>
> Maven artifacts are also available in the temporary repository
> https://repository.apache.org/content/repositories/orgapachehbase-1037/
>
> Signed with my code signing key E964B5FF.
>
> Signed tag in the repository can be found here:
>
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=tag;h=336a63b5f58bb1c4859e3c2b6a2f5c0d68a7d9e1
>
> NOTE IN GIGANTIC LETTERS THAT THIS IS A DEVELOPER RELEASE.
> DO NOT USE THIS RELEASE IN PRODUCTION.
>
> HBase 0.99.0 is a "developer preview" release, and an odd-numbered release
> as
> defined in  https://hbase.apache.org/book/upgrading.html#hbase.versioning.
> This release IS NOT intended for production use, and does not contain any
> backwards or forwards compatibility guarantees (even within minor versions
> 0.99.x). Please refrain from deploying this over important data.
>
> 0.99.0 release is provided from branch-1, which will be the basis for
> HBase-1.0
> release. A few 0.99.x releases are planned before 1.0. The reason for
> doing a developer preview release is to get more testing for the branch-1
> code
> that will be released soon as HBase-1.0.0. Thus, all contribution in terms
> of
> testing, benchmarking, checking API / source /wire compatibility, checking
> out
> documentation and further code contribution is highly appreciated. 1.0 will
> be
> the first series in the 1.x line of releases which are expected to keep
> compatibility with previous 1.x releases. Thus it is very important to
> check
> the client side and server side APIs for compatibility and maintainability
> concerns for future releases.
>
> 0.99.0 contains slightly more than 1K issues resolved with many
> improvements
> and bug fixes. The theme of (eventual) 1.0 release is to become a stable
> base
> for future 1.x series of releases. 1.0 release will aim to achieve at least
> the same level of stability of 0.98 releases without introducing too many
> new
> features.
>
> Some work has been under way to clearly mark and differentiate client
> facing
> APIs, and redefine some of the client interfaces for improving semantics,
> easy
> of use and maintainability. 0.99.0 contains some of the completed items,
> and
> the rest can be found HBASE-10602. Marking/remarking of interfaces with
> InterfaceAudience has also been going on, which will identify areas for
> compatibility (with clients, coprocessors and dependent projects like
> Phoenix)
> for future releases.
>
> Starting with 0.99.0, the HBase master server and backup master servers
> will
> also act as a region server. RPC port and info port for web UI is shared
> for
> the master and region server roles. Active master will be hosting the meta
> table (and other hbase system tables, acl and namespace) by default
> (unless configured otherwise). The master and backup masters will not be
> hosting user level regions. See HBASE-10569 and HBASE-11604 for more
> details.
>
> This release contains Phase 1 items for experimental "Read availability
> using
> timeline consistent region replicas" feature. A region can be hosted in
> multiple region servers in read-only mode. One of the replicas for the
> region
> will be primary, accepting writes, and other replicas will be sharing the
> same
> data files. Read requests can be done against any replica for the region
> with
> backup RPCs for high availability with timeline consistency guarantees.
> More
> information can be found at HBASE-10070.
>
> Other notable improvements in this release are
>  - Automatic tuning of global memstore and block cache sizes
>  - Various security, tags and visibility labels improvements
>  - Bucket cache improvements (usability and compressed data blocks)
>  - A new pluggable replication endpoint to plug in to HBase's inter-cluster
>replication to replicate to a custom data store
>  - A Dockerfile to easily build and run HBase from source
>  - Internal refactoring for abstracting away zookeeper usage
>  - Truncate table command
>  - Region assignment to use hbase:meta table instead of zo

[jira] [Created] (HBASE-11967) HMaster in standalone won't go down if it gets 'Unhandled exception'

2014-09-12 Thread stack (JIRA)
stack created HBASE-11967:
-

 Summary: HMaster in standalone won't go down if it gets 'Unhandled 
exception'
 Key: HBASE-11967
 URL: https://issues.apache.org/jira/browse/HBASE-11967
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0
Reporter: stack
 Fix For: 0.99.1


Got this testing 0.99.0 RC.

I started it up after unbundling and got this:

{code}
 99 2014-09-12 14:50:42,761 FATAL [ActiveMasterManager] master.HMaster: 
Unhandled exception. Starting shutdown.
100 org.apache.hadoop.hbase.util.FileSystemVersionException: HBase file layout 
needs to be upgraded. You have version null and I want version 8. Consult 
http://hbase.apache.org/book.html for further information about upgrading 
HBase. Is your hbase.rootdir valid? If so, you may need to run 'hbase hbck 
-fixVersionFile'.
101   at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:587)

{code}

But now my master is stuck trying to become master in here:

{code}
 46 "main" prio=5 tid=0x7fe825001000 nid=0x1303 waiting on condition 
[0x000107383000]
 47java.lang.Thread.State: TIMED_WAITING (sleeping)
 48   at java.lang.Thread.sleep(Native Method)
 49   at 
org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:218)
 50   at 
org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:438)
 51   at 
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:188)
 52   at 
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139)
 53   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
 54   at 
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
 55   at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1794)
...

{code}

Any ideas [~jxiang]?  Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: HBase reads, isolation levels and RegionScanner internal locking

2014-09-12 Thread Stack
On Fri, Sep 12, 2014 at 1:21 PM, Vladimir Rodionov 
wrote:

> All row mutate operations in HBase are atomic:
> puts/deletes/increments/appends.
> Atomicity in HBase has the same meaning as in RDBMs exactly - operations
> completes as a whole or does not at all.  There is an additional guarantee
> in HBase that all reads are ROW - atomic as well - one will never read
> partial result of atomic mutate operation (on a ROW). READ_UNCOMMITTED will
> weaken this read atomicity guarantee and it will be possible to read
> partial results of a row mutation operation,
>
> so we discard row read atomicity but still have cell atomicity
> (hopefully?).
>
> This is my understanding of what READ_UNCOMMITTED means in HBase.
>
> PS
>
> I created :
> https://issues.apache.org/jira/browse/HBASE-11965


Thanks Vladimir,
St.Ack


Re: HBase reads, isolation levels and RegionScanner internal locking

2014-09-12 Thread Vladimir Rodionov
All row mutate operations in HBase are atomic: puts/deletes/increments/appends.
Atomicity in HBase has the same meaning as in RDBMs exactly - operations
completes as a whole or does not at all.  There is an additional guarantee
in HBase that all reads are ROW - atomic as well - one will never read
partial result of atomic mutate operation (on a ROW). READ_UNCOMMITTED will
weaken this read atomicity guarantee and it will be possible to read
partial results of a row mutation operation,

so we discard row read atomicity but still have cell atomicity (hopefully?).

This is my understanding of what READ_UNCOMMITTED means in HBase.

PS

I created :
https://issues.apache.org/jira/browse/HBASE-11965

-Vladimir


On Fri, Sep 12, 2014 at 12:28 PM, Stack  wrote:

> On Fri, Sep 12, 2014 at 11:25 AM, Vladimir Rodionov <
> vladrodio...@gmail.com>
> wrote:
>
> > Michael,
> >
> > This is not a row-level locking - it is region-wide lock. This is a major
> > reason of  the following performance problems:
> >
> >
> Pardon my misreading as row-scoping (I'd just come off reading Michael
> Segel's note).
>
>
>
> > 1) Multi gets are bad if inside the same region
> > 2) Multiple scanners over the same region are bad
> > 3) Scan during compaction are bad.
> >
>
>
>
> > I need some input from HBase folks here:
> >
> > 1) READ_UNCOMMITTED safe if lock free?
> >
>
>
> And rely on MVCC only? That'd be cool Vladimir.  When you say
> READ_UNCOMMITTED, are you row or cell-scoped?  Are you thinking that you'd
> make it so the row lock was also optional?
>
>
>
> > 2) Confirmation that region-wide lock is for read consistency only.
> >
> >
> >
> The region lock, IIRC, was added so we can close the region cleanly.  All
> gets/puts/etc. take the lock and hold it while operating to prevent the
> region being closed out from under them.  It was put in place long ago and
> not revisited since.
>
> Hope this helps.
>
>
> Related, there is Liang Xie's effort over in HDFS:
> https://issues.apache.org/jira/browse/HDFS-6735
> St.Ack.
>
>
>
> > On Fri, Sep 12, 2014 at 11:04 AM, Stack  wrote:
> >
> > > On Thu, Sep 11, 2014 at 3:58 PM, Vladimir Rodionov <
> > vladrodio...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hi, all
> > > >
> > > > We have two isolation levels in (used to be in Scan) in Query now.
> See:
> > > > https://issues.apache.org/jira/browse/HBASE-11936
> > > >
> > > > I moved isolation levels API from Scan upward to Query class. The
> > reason:
> > > > this API was not available for Get operations. The rationals? Improve
> > > > performance of get and multi-gets over the same region.
> > > >
> > > > As many of you aware, RegionScannerImpl is heavily synchronized on
> > > internal
> > > > region's lock.  Now some questions:
> > > >
> > > > 1. Is it safe to bypass this locking (in next() call) in
> > READ_UNCOMMITTED
> > > > mode?
> > >
> > > We will do all necessary checks, of course, before calling nextRaw().
> > > > 2. What was the reason of this locking in a first place for reads in
> > > > READ_COMMITTED mode? Except obvious - no-dirty-reads allowed? Can
> > someone
> > > > tell me what else bad can happen?
> > > >
> > >
> > >
> > > There is only the obvious (that I know of) Vladimir.  We've been so
> > fixated
> > > on ensuring consistent view on a row, we've not done the work to allow
> > > other read types. I'm not sure what would happen if you were to skirt
> row
> > > lock.  Try hacking on TestAtomicOperation to undo lock and see what
> > > happens?
> > >
> > > St.Ack
> > >
> >
>


[jira] [Created] (HBASE-11966) Minor error in TestHRegion.testCheckAndMutate_WithCorrectValue()

2014-09-12 Thread Srikanth Srungarapu (JIRA)
Srikanth Srungarapu created HBASE-11966:
---

 Summary: Minor error in 
TestHRegion.testCheckAndMutate_WithCorrectValue()
 Key: HBASE-11966
 URL: https://issues.apache.org/jira/browse/HBASE-11966
 Project: HBase
  Issue Type: Test
  Components: regionserver
Reporter: Srikanth Srungarapu
Assignee: Srikanth Srungarapu
Priority: Minor


In testCheckAndMutate_WithCorrectValue(), put is being used where delete is 
intended.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: HBase reads, isolation levels and RegionScanner internal locking

2014-09-12 Thread Stack
On Fri, Sep 12, 2014 at 11:25 AM, Vladimir Rodionov 
wrote:

> Michael,
>
> This is not a row-level locking - it is region-wide lock. This is a major
> reason of  the following performance problems:
>
>
Pardon my misreading as row-scoping (I'd just come off reading Michael
Segel's note).



> 1) Multi gets are bad if inside the same region
> 2) Multiple scanners over the same region are bad
> 3) Scan during compaction are bad.
>



> I need some input from HBase folks here:
>
> 1) READ_UNCOMMITTED safe if lock free?
>


And rely on MVCC only? That'd be cool Vladimir.  When you say
READ_UNCOMMITTED, are you row or cell-scoped?  Are you thinking that you'd
make it so the row lock was also optional?



> 2) Confirmation that region-wide lock is for read consistency only.
>
>
>
The region lock, IIRC, was added so we can close the region cleanly.  All
gets/puts/etc. take the lock and hold it while operating to prevent the
region being closed out from under them.  It was put in place long ago and
not revisited since.

Hope this helps.


Related, there is Liang Xie's effort over in HDFS:
https://issues.apache.org/jira/browse/HDFS-6735
St.Ack.



> On Fri, Sep 12, 2014 at 11:04 AM, Stack  wrote:
>
> > On Thu, Sep 11, 2014 at 3:58 PM, Vladimir Rodionov <
> vladrodio...@gmail.com
> > >
> > wrote:
> >
> > > Hi, all
> > >
> > > We have two isolation levels in (used to be in Scan) in Query now. See:
> > > https://issues.apache.org/jira/browse/HBASE-11936
> > >
> > > I moved isolation levels API from Scan upward to Query class. The
> reason:
> > > this API was not available for Get operations. The rationals? Improve
> > > performance of get and multi-gets over the same region.
> > >
> > > As many of you aware, RegionScannerImpl is heavily synchronized on
> > internal
> > > region's lock.  Now some questions:
> > >
> > > 1. Is it safe to bypass this locking (in next() call) in
> READ_UNCOMMITTED
> > > mode?
> >
> > We will do all necessary checks, of course, before calling nextRaw().
> > > 2. What was the reason of this locking in a first place for reads in
> > > READ_COMMITTED mode? Except obvious - no-dirty-reads allowed? Can
> someone
> > > tell me what else bad can happen?
> > >
> >
> >
> > There is only the obvious (that I know of) Vladimir.  We've been so
> fixated
> > on ensuring consistent view on a row, we've not done the work to allow
> > other read types. I'm not sure what would happen if you were to skirt row
> > lock.  Try hacking on TestAtomicOperation to undo lock and see what
> > happens?
> >
> > St.Ack
> >
>


[jira] [Created] (HBASE-11965) Optimize locking in HRegion

2014-09-12 Thread Vladimir Rodionov (JIRA)
Vladimir Rodionov created HBASE-11965:
-

 Summary: Optimize locking in HRegion
 Key: HBASE-11965
 URL: https://issues.apache.org/jira/browse/HBASE-11965
 Project: HBase
  Issue Type: Bug
  Components: Performance
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov


Described in this thread:

http://mail-archives.apache.org/mod_mbox/hbase-dev/201409.mbox/%3CCAAg3a2qmWJtQbdAk7PrX%2BW8SZWxsYhNggM5f2RNkGTXUri34YQ%40mail.gmail.com%3E

*startRegionOperation* and *closeRegionOperation* in HRegion acquires and 
releases region-wide lock. The locking happens for every mutation, read and 
scan next. This is a serious bottleneck if we do:

* Mutli - get on a same region.
* Run multiple scanners on same region.
* Do scan during compaction.
* Access region simultaneously from different threads (most generic case)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: HBase reads, isolation levels and RegionScanner internal locking

2014-09-12 Thread Vladimir Rodionov
There is no RLL (row level locking) in HBase , all locks- are region-wide.
This is the issue I am going to address
in a separate JIRA.

Example from HRegion.append

// Lock row

startRegionOperation(Operation.APPEND);

Regards,
-Vladimir


On Fri, Sep 12, 2014 at 11:57 AM, Vladimir Rodionov 
wrote:

> According to Bible:
> http://hbase.apache.org/acid-semantics.html
>
> HBase declares that all row reads are consistent - partial reads for rows
> are not possible.
>
> [
> Consistency and Isolation
>
>1. All rows returned via any access API will consist of a complete row
>that existed at some point in the table's history.
>2. This is true across column families - i.e a get of a full row that
>occurs concurrent with some mutations 1,2,3,4,5 will return a complete row
>that existed at some point in time between mutation i and i+1 for some i
>between 1 and 5.
>3. The state of a row will only move forward through the history of
>edits to it.
>
> ]
>
> In this case, there is no much sense in READ_UNCOMMITTED per se.
>
> -Vladimir
>
>
> On Fri, Sep 12, 2014 at 11:04 AM, Stack  wrote:
>
>> On Thu, Sep 11, 2014 at 3:58 PM, Vladimir Rodionov <
>> vladrodio...@gmail.com>
>> wrote:
>>
>> > Hi, all
>> >
>> > We have two isolation levels in (used to be in Scan) in Query now. See:
>> > https://issues.apache.org/jira/browse/HBASE-11936
>> >
>> > I moved isolation levels API from Scan upward to Query class. The
>> reason:
>> > this API was not available for Get operations. The rationals? Improve
>> > performance of get and multi-gets over the same region.
>> >
>> > As many of you aware, RegionScannerImpl is heavily synchronized on
>> internal
>> > region's lock.  Now some questions:
>> >
>> > 1. Is it safe to bypass this locking (in next() call) in
>> READ_UNCOMMITTED
>> > mode?
>>
>> We will do all necessary checks, of course, before calling nextRaw().
>> > 2. What was the reason of this locking in a first place for reads in
>> > READ_COMMITTED mode? Except obvious - no-dirty-reads allowed? Can
>> someone
>> > tell me what else bad can happen?
>> >
>>
>>
>> There is only the obvious (that I know of) Vladimir.  We've been so
>> fixated
>> on ensuring consistent view on a row, we've not done the work to allow
>> other read types. I'm not sure what would happen if you were to skirt row
>> lock.  Try hacking on TestAtomicOperation to undo lock and see what
>> happens?
>>
>> St.Ack
>>
>
>


Re: HBase reads, isolation levels and RegionScanner internal locking

2014-09-12 Thread Vladimir Rodionov
According to Bible:
http://hbase.apache.org/acid-semantics.html

HBase declares that all row reads are consistent - partial reads for rows
are not possible.

[
Consistency and Isolation

   1. All rows returned via any access API will consist of a complete row
   that existed at some point in the table's history.
   2. This is true across column families - i.e a get of a full row that
   occurs concurrent with some mutations 1,2,3,4,5 will return a complete row
   that existed at some point in time between mutation i and i+1 for some i
   between 1 and 5.
   3. The state of a row will only move forward through the history of
   edits to it.

]

In this case, there is no much sense in READ_UNCOMMITTED per se.

-Vladimir


On Fri, Sep 12, 2014 at 11:04 AM, Stack  wrote:

> On Thu, Sep 11, 2014 at 3:58 PM, Vladimir Rodionov  >
> wrote:
>
> > Hi, all
> >
> > We have two isolation levels in (used to be in Scan) in Query now. See:
> > https://issues.apache.org/jira/browse/HBASE-11936
> >
> > I moved isolation levels API from Scan upward to Query class. The reason:
> > this API was not available for Get operations. The rationals? Improve
> > performance of get and multi-gets over the same region.
> >
> > As many of you aware, RegionScannerImpl is heavily synchronized on
> internal
> > region's lock.  Now some questions:
> >
> > 1. Is it safe to bypass this locking (in next() call) in READ_UNCOMMITTED
> > mode?
>
> We will do all necessary checks, of course, before calling nextRaw().
> > 2. What was the reason of this locking in a first place for reads in
> > READ_COMMITTED mode? Except obvious - no-dirty-reads allowed? Can someone
> > tell me what else bad can happen?
> >
>
>
> There is only the obvious (that I know of) Vladimir.  We've been so fixated
> on ensuring consistent view on a row, we've not done the work to allow
> other read types. I'm not sure what would happen if you were to skirt row
> lock.  Try hacking on TestAtomicOperation to undo lock and see what
> happens?
>
> St.Ack
>


Re: HBase reads, isolation levels and RegionScanner internal locking

2014-09-12 Thread Vladimir Rodionov
Michael,

This is not a row-level locking - it is region-wide lock. This is a major
reason of  the following performance problems:

1) Multi gets are bad if inside the same region
2) Multiple scanners over the same region are bad
3) Scan during compaction are bad.

I need some input from HBase folks here:

1) READ_UNCOMMITTED safe if lock free?
2) Confirmation that region-wide lock is for read consistency only.


On Fri, Sep 12, 2014 at 11:04 AM, Stack  wrote:

> On Thu, Sep 11, 2014 at 3:58 PM, Vladimir Rodionov  >
> wrote:
>
> > Hi, all
> >
> > We have two isolation levels in (used to be in Scan) in Query now. See:
> > https://issues.apache.org/jira/browse/HBASE-11936
> >
> > I moved isolation levels API from Scan upward to Query class. The reason:
> > this API was not available for Get operations. The rationals? Improve
> > performance of get and multi-gets over the same region.
> >
> > As many of you aware, RegionScannerImpl is heavily synchronized on
> internal
> > region's lock.  Now some questions:
> >
> > 1. Is it safe to bypass this locking (in next() call) in READ_UNCOMMITTED
> > mode?
>
> We will do all necessary checks, of course, before calling nextRaw().
> > 2. What was the reason of this locking in a first place for reads in
> > READ_COMMITTED mode? Except obvious - no-dirty-reads allowed? Can someone
> > tell me what else bad can happen?
> >
>
>
> There is only the obvious (that I know of) Vladimir.  We've been so fixated
> on ensuring consistent view on a row, we've not done the work to allow
> other read types. I'm not sure what would happen if you were to skirt row
> lock.  Try hacking on TestAtomicOperation to undo lock and see what
> happens?
>
> St.Ack
>


[jira] [Resolved] (HBASE-11959) TestAssignmentManagerOnCluster is flaky

2014-09-12 Thread Jimmy Xiang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-11959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-11959.
-
   Resolution: Fixed
Fix Version/s: 0.99.1
 Hadoop Flags: Reviewed

I tested related tests 10 times locally and they are good with this patch. 
Integrated into branch 1. Thanks.

> TestAssignmentManagerOnCluster is flaky
> ---
>
> Key: HBASE-11959
> URL: https://issues.apache.org/jira/browse/HBASE-11959
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.99.0
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Fix For: 0.99.1
>
> Attachments: hbase-11959.patch
>
>
> https://builds.apache.org/job/HBase-1.0/178/testReport/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: HBase reads, isolation levels and RegionScanner internal locking

2014-09-12 Thread Stack
On Thu, Sep 11, 2014 at 3:58 PM, Vladimir Rodionov 
wrote:

> Hi, all
>
> We have two isolation levels in (used to be in Scan) in Query now. See:
> https://issues.apache.org/jira/browse/HBASE-11936
>
> I moved isolation levels API from Scan upward to Query class. The reason:
> this API was not available for Get operations. The rationals? Improve
> performance of get and multi-gets over the same region.
>
> As many of you aware, RegionScannerImpl is heavily synchronized on internal
> region's lock.  Now some questions:
>
> 1. Is it safe to bypass this locking (in next() call) in READ_UNCOMMITTED
> mode?

We will do all necessary checks, of course, before calling nextRaw().
> 2. What was the reason of this locking in a first place for reads in
> READ_COMMITTED mode? Except obvious - no-dirty-reads allowed? Can someone
> tell me what else bad can happen?
>


There is only the obvious (that I know of) Vladimir.  We've been so fixated
on ensuring consistent view on a row, we've not done the work to allow
other read types. I'm not sure what would happen if you were to skirt row
lock.  Try hacking on TestAtomicOperation to undo lock and see what happens?

St.Ack


[jira] [Created] (HBASE-11964) Improve replication source thread handling

2014-09-12 Thread Andrew Purtell (JIRA)
Andrew Purtell created HBASE-11964:
--

 Summary: Improve replication source thread handling
 Key: HBASE-11964
 URL: https://issues.apache.org/jira/browse/HBASE-11964
 Project: HBase
  Issue Type: Sub-task
Reporter: Andrew Purtell
 Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1


Improve replication source thread handling. Limit parallelism when transferring 
queues. Ensure replication sources terminate properly.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HBASE-11963) Synchronize peer cluster replication connection attempts

2014-09-12 Thread Andrew Purtell (JIRA)
Andrew Purtell created HBASE-11963:
--

 Summary: Synchronize peer cluster replication connection attempts
 Key: HBASE-11963
 URL: https://issues.apache.org/jira/browse/HBASE-11963
 Project: HBase
  Issue Type: Sub-task
Reporter: Andrew Purtell
 Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1


Synchronize peer cluster connection attempts to avoid races and rate limit 
connections when multiple replication sources try to connect to the peer 
cluster. If the peer cluster is down we can get out of control over time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: HBase reads, isolation levels and RegionScanner internal locking

2014-09-12 Thread Michael Segel
Vlad, 

I understand. 
However several of the HBase committers aren’t really schooled in RDBMS design.

And again, the older (going back to 0.23 ) use of the term RLL isn’t relational 
RLL and when you start to talk about isolation you’re getting in to the RDBMS 
RLL 

So you really need to define what you mean when you say RLL. I don’t want to 
assume one thing when you meant another. 

Just like talking about salts.  ;-) 


On Sep 12, 2014, at 5:53 PM, Vladimir Rodionov  wrote:

> Michael, this is HBase developers mailing list.
> 
> -Vladimir
> 
> 
> 
> 
> On Fri, Sep 12, 2014 at 12:08 AM, Michael Segel 
> wrote:
> 
>> Silly question…
>> 
>> HBase uses the term RLL (row level locking) to make the writes to a row
>> atomic.
>> 
>> When you start to get in to isolation, RLL takes on a different meaning.
>> 
>> So now you have to better define what do you mean by locking. Are you
>> taking about HBase RLL,
>> or are you talking about Transactional RLL ( RDBMS RLL) ?
>> 
>> 
>> On Sep 11, 2014, at 11:58 PM, Vladimir Rodionov 
>> wrote:
>> 
>>> Hi, all
>>> 
>>> We have two isolation levels in (used to be in Scan) in Query now. See:
>>> https://issues.apache.org/jira/browse/HBASE-11936
>>> 
>>> I moved isolation levels API from Scan upward to Query class. The reason:
>>> this API was not available for Get operations. The rationals? Improve
>>> performance of get and multi-gets over the same region.
>>> 
>>> As many of you aware, RegionScannerImpl is heavily synchronized on
>> internal
>>> region's lock.  Now some questions:
>>> 
>>> 1. Is it safe to bypass this locking (in next() call) in READ_UNCOMMITTED
>>> mode?
>>> We will do all necessary checks, of course, before calling nextRaw().
>>> 2. What was the reason of this locking in a first place for reads in
>>> READ_COMMITTED mode? Except obvious - no-dirty-reads allowed? Can someone
>>> tell me what else bad can happen?
>>> 
>>> -Vladimir
>> 
>> 



[jira] [Created] (HBASE-11962) Port HBASE-11897 "Add append and remove peer table-cfs cmds for replication" to 0.98

2014-09-12 Thread Ted Yu (JIRA)
Ted Yu created HBASE-11962:
--

 Summary: Port HBASE-11897 "Add append and remove peer table-cfs 
cmds for replication" to 0.98
 Key: HBASE-11962
 URL: https://issues.apache.org/jira/browse/HBASE-11962
 Project: HBase
  Issue Type: Improvement
Reporter: Ted Yu
Priority: Minor
 Fix For: 0.98.7


This issue is to backport the commands for appending and removing peer 
table-cfs for replication to 0.98



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: HBase reads, isolation levels and RegionScanner internal locking

2014-09-12 Thread Vladimir Rodionov
Michael, this is HBase developers mailing list.

-Vladimir




On Fri, Sep 12, 2014 at 12:08 AM, Michael Segel 
wrote:

> Silly question…
>
> HBase uses the term RLL (row level locking) to make the writes to a row
> atomic.
>
> When you start to get in to isolation, RLL takes on a different meaning.
>
> So now you have to better define what do you mean by locking. Are you
> taking about HBase RLL,
> or are you talking about Transactional RLL ( RDBMS RLL) ?
>
>
> On Sep 11, 2014, at 11:58 PM, Vladimir Rodionov 
> wrote:
>
> > Hi, all
> >
> > We have two isolation levels in (used to be in Scan) in Query now. See:
> > https://issues.apache.org/jira/browse/HBASE-11936
> >
> > I moved isolation levels API from Scan upward to Query class. The reason:
> > this API was not available for Get operations. The rationals? Improve
> > performance of get and multi-gets over the same region.
> >
> > As many of you aware, RegionScannerImpl is heavily synchronized on
> internal
> > region's lock.  Now some questions:
> >
> > 1. Is it safe to bypass this locking (in next() call) in READ_UNCOMMITTED
> > mode?
> > We will do all necessary checks, of course, before calling nextRaw().
> > 2. What was the reason of this locking in a first place for reads in
> > READ_COMMITTED mode? Except obvious - no-dirty-reads allowed? Can someone
> > tell me what else bad can happen?
> >
> > -Vladimir
>
>


[jira] [Created] (HBASE-11961) Document region state transitions

2014-09-12 Thread Jimmy Xiang (JIRA)
Jimmy Xiang created HBASE-11961:
---

 Summary: Document region state transitions
 Key: HBASE-11961
 URL: https://issues.apache.org/jira/browse/HBASE-11961
 Project: HBase
  Issue Type: Task
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


Document the region state transitions in the refguide.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HBASE-11960) Provide a sample to show how to use Thrift client authentication

2014-09-12 Thread Jimmy Xiang (JIRA)
Jimmy Xiang created HBASE-11960:
---

 Summary: Provide a sample to show how to use Thrift client 
authentication
 Key: HBASE-11960
 URL: https://issues.apache.org/jira/browse/HBASE-11960
 Project: HBase
  Issue Type: Task
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


Now Thrift server can authenticate clients. However, many people asked how to 
use it. Although we have some info in the refguide, it doesn't mention how to 
change the client to turn it on. A sample should help.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HBASE-11959) TestAssignmentManagerOnCluster is falky

2014-09-12 Thread Jimmy Xiang (JIRA)
Jimmy Xiang created HBASE-11959:
---

 Summary: TestAssignmentManagerOnCluster is falky
 Key: HBASE-11959
 URL: https://issues.apache.org/jira/browse/HBASE-11959
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


https://builds.apache.org/job/HBase-1.0/178/testReport/





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: Building the site - interesting problem

2014-09-12 Thread Ted Yu
+1 on inclusion in dev-support. 

On Sep 12, 2014, at 6:26 AM, Sean Busbey  wrote:

> Yeah +1 for inclusion in dev-support. Be sure to add a ASL header to it
> before pushing.
> 
> -- 
> Sean
> On Sep 12, 2014 12:35 AM, "Misty Stanley-Jones" 
> wrote:
> 
>> I wrote the script and I think the website is all fixed now. You can follow
>> my progress and see the script at
>> https://issues.apache.org/jira/browse/HBASE-11951. Do you guys think it's
>> something that should go in the dev-support/ directory?
>> 
>> 
>> On Thu, Sep 11, 2014 at 6:05 PM, Misty Stanley-Jones <
>> mstanleyjo...@cloudera.com> wrote:
>> 
>>> All these files are stale or there in error and will also be deleted from
>>> SVN. Like the directories above, they are vestigial or badly-copied files
>>> belonging to the book, for the most part.
>>> 
>>> ls: /Users/misty/git/hbase/target/staging/./acid.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./apas02.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./apas03.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./apas04.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./apbs02.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./apbs03.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./apbs04.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./apd.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./apes02.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./apes03.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./apjs02.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./apjs03.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./apjs04.html: No such file or
>>> directory
>>> ls:
>> /Users/misty/git/hbase/target/staging/./appendix_contributing_to_documentation.html:
>>> No such file or directory
>>> ls: /Users/misty/git/hbase/target/staging/./appendix_hfile_format.html:
>> No
>>> such file or directory
>>> ls: /Users/misty/git/hbase/target/staging/./arch.bulk.load.html: No such
>>> file or directory
>>> ls: /Users/misty/git/hbase/target/staging/./arch.catalog.html: No such
>>> file or directory
>>> ls: /Users/misty/git/hbase/target/staging/./arch.hdfs.html: No such file
>>> or directory
>>> ls: /Users/misty/git/hbase/target/staging/./architecture.html: No such
>>> file or directory
>>> ls: /Users/misty/git/hbase/target/staging/./asf.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./asf.reporting.html: No such
>>> file or directory
>>> ls: /Users/misty/git/hbase/target/staging/./build.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./c.html: No such file or
>>> directory
>>> ls:
>>> /Users/misty/git/hbase/target/staging/./case_studies/case_studies.html:
>> No
>>> such file or directory
>>> ls:
>> /Users/misty/git/hbase/target/staging/./case_studies/casestudies.perftroub.html:
>>> No such file or directory
>>> ls:
>> /Users/misty/git/hbase/target/staging/./case_studies/casestudies.schema.html:
>>> No such file or directory
>>> ls: /Users/misty/git/hbase/target/staging/./case_studies.html: No such
>>> file or directory
>>> ls: /Users/misty/git/hbase/target/staging/./casestudies.html: No such
>> file
>>> or directory
>>> ls: /Users/misty/git/hbase/target/staging/./casestudies.perftroub.html:
>> No
>>> such file or directory
>>> ls: /Users/misty/git/hbase/target/staging/./casestudies.schema.html: No
>>> such file or directory
>>> ls: /Users/misty/git/hbase/target/staging/./cells.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./cf.keep.deleted.html: No such
>>> file or directory
>>> ls: /Users/misty/git/hbase/target/staging/./ch01s03.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./ch01s17.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./ch03s03.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./ch07s02.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./ch07s03.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./ch07s04.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./ch07s05.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./ch10s03.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./ch14s17.html: No such file or
>>> directory
>>> ls: /Users/misty/git/hbase/target/staging/./changing.compression.html: No
>>> such file or directory
>>> ls: /Users/misty/git/hbase/target/staging/./client.filter.html: No such
>>> file or directory
>>> ls: /Users/misty/git/hba

Re: Building the site - interesting problem

2014-09-12 Thread Sean Busbey
Yeah +1 for inclusion in dev-support. Be sure to add a ASL header to it
before pushing.

-- 
Sean
On Sep 12, 2014 12:35 AM, "Misty Stanley-Jones" 
wrote:

> I wrote the script and I think the website is all fixed now. You can follow
> my progress and see the script at
> https://issues.apache.org/jira/browse/HBASE-11951. Do you guys think it's
> something that should go in the dev-support/ directory?
>
>
> On Thu, Sep 11, 2014 at 6:05 PM, Misty Stanley-Jones <
> mstanleyjo...@cloudera.com> wrote:
>
> > All these files are stale or there in error and will also be deleted from
> > SVN. Like the directories above, they are vestigial or badly-copied files
> > belonging to the book, for the most part.
> >
> > ls: /Users/misty/git/hbase/target/staging/./acid.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./apas02.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./apas03.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./apas04.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./apbs02.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./apbs03.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./apbs04.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./apd.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./apes02.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./apes03.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./apjs02.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./apjs03.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./apjs04.html: No such file or
> > directory
> > ls:
> >
> /Users/misty/git/hbase/target/staging/./appendix_contributing_to_documentation.html:
> > No such file or directory
> > ls: /Users/misty/git/hbase/target/staging/./appendix_hfile_format.html:
> No
> > such file or directory
> > ls: /Users/misty/git/hbase/target/staging/./arch.bulk.load.html: No such
> > file or directory
> > ls: /Users/misty/git/hbase/target/staging/./arch.catalog.html: No such
> > file or directory
> > ls: /Users/misty/git/hbase/target/staging/./arch.hdfs.html: No such file
> > or directory
> > ls: /Users/misty/git/hbase/target/staging/./architecture.html: No such
> > file or directory
> > ls: /Users/misty/git/hbase/target/staging/./asf.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./asf.reporting.html: No such
> > file or directory
> > ls: /Users/misty/git/hbase/target/staging/./build.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./c.html: No such file or
> > directory
> > ls:
> > /Users/misty/git/hbase/target/staging/./case_studies/case_studies.html:
> No
> > such file or directory
> > ls:
> >
> /Users/misty/git/hbase/target/staging/./case_studies/casestudies.perftroub.html:
> > No such file or directory
> > ls:
> >
> /Users/misty/git/hbase/target/staging/./case_studies/casestudies.schema.html:
> > No such file or directory
> > ls: /Users/misty/git/hbase/target/staging/./case_studies.html: No such
> > file or directory
> > ls: /Users/misty/git/hbase/target/staging/./casestudies.html: No such
> file
> > or directory
> > ls: /Users/misty/git/hbase/target/staging/./casestudies.perftroub.html:
> No
> > such file or directory
> > ls: /Users/misty/git/hbase/target/staging/./casestudies.schema.html: No
> > such file or directory
> > ls: /Users/misty/git/hbase/target/staging/./cells.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./cf.keep.deleted.html: No such
> > file or directory
> > ls: /Users/misty/git/hbase/target/staging/./ch01s03.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./ch01s17.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./ch03s03.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./ch07s02.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./ch07s03.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./ch07s04.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./ch07s05.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./ch10s03.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./ch14s17.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging/./changing.compression.html: No
> > such file or directory
> > ls: /Users/misty/git/hbase/target/staging/./client.filter.html: No such
> > file or directory
> > ls: /Users/misty/git/hbase/target/staging/./client.html: No such file or
> > directory
> > ls: /Users/misty/git/hbase/target/staging

[jira] [Created] (HBASE-11958) Add documents about snapshot owner

2014-09-12 Thread Liu Shaohui (JIRA)
Liu Shaohui created HBASE-11958:
---

 Summary: Add documents about snapshot owner
 Key: HBASE-11958
 URL: https://issues.apache.org/jira/browse/HBASE-11958
 Project: HBase
  Issue Type: Sub-task
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Minor


HBASE-11869 introduct snapshot owner feature. We need to add documents about it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HBASE-11957) Backport HBASE-5974 to 0.94

2014-09-12 Thread Liu Shaohui (JIRA)
Liu Shaohui created HBASE-11957:
---

 Summary: Backport HBASE-5974 to 0.94
 Key: HBASE-11957
 URL: https://issues.apache.org/jira/browse/HBASE-11957
 Project: HBase
  Issue Type: Bug
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Critical
 Fix For: 0.94.24


HBASE-5974:Scanner retry behavior with RPC timeout on next() seems incorrect, 
which cause data missing in hbase scan.

I think we should fix it in 0.94.
[~lhofhansl]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: HBase reads, isolation levels and RegionScanner internal locking

2014-09-12 Thread Michael Segel
Silly question… 

HBase uses the term RLL (row level locking) to make the writes to a row atomic. 

When you start to get in to isolation, RLL takes on a different meaning. 

So now you have to better define what do you mean by locking. Are you taking 
about HBase RLL, 
or are you talking about Transactional RLL ( RDBMS RLL) ? 


On Sep 11, 2014, at 11:58 PM, Vladimir Rodionov  wrote:

> Hi, all
> 
> We have two isolation levels in (used to be in Scan) in Query now. See:
> https://issues.apache.org/jira/browse/HBASE-11936
> 
> I moved isolation levels API from Scan upward to Query class. The reason:
> this API was not available for Get operations. The rationals? Improve
> performance of get and multi-gets over the same region.
> 
> As many of you aware, RegionScannerImpl is heavily synchronized on internal
> region's lock.  Now some questions:
> 
> 1. Is it safe to bypass this locking (in next() call) in READ_UNCOMMITTED
> mode?
> We will do all necessary checks, of course, before calling nextRaw().
> 2. What was the reason of this locking in a first place for reads in
> READ_COMMITTED mode? Except obvious - no-dirty-reads allowed? Can someone
> tell me what else bad can happen?
> 
> -Vladimir