Re: [In Memory] Questions about Make HBase Table In Memory

[Proust/Feng Guizhou]

Thanks Ted for your reply, the idea actually based on the HBase doc about
Block Cache Design, so with IN_MEMORY=true and proactively read(GET/SCAN)
the table, we can get the table served from memory for best and stable read
performance.


On 24 October 2016 at 11:19, Ted Yu  wrote:

> Puts are stored in memstore. Get / Scan would include such data in memstore
> when row keys match.
>
> For #2, see http://hbase.apache.org/book.html#block.cache.design
>
> For #3, one indication is latency of the result. Latency would be longer
> when disk is involved.
>
>
> On Sun, Oct 23, 2016 at 8:01 PM, Proust(Proust/Feng Guizhou) [Saipan] ­ <
> pf...@coupang.com> wrote:
>
> > Hi, HBase Experts
> >
> > I have several questions related to HBase BlockCache and IN_MEMORY
> setting
> > 1. Does BlockCache cache the data when do PUT request, so next time
> > GET/SCAN request against the same data will be served from Memory only?
> > 2. If HBase Table ColumnFamily have *IN_MEMORY=true*, and every time I do
> > PUT  followed immediately with a GET/SCAN against the same data, assume I
> > have memory larger than the specific table, is that this way I can make
> > sure the whole table is in memory and all read requests against this
> table
> > will be served from memory only?
> > 3. How can I determine whether a GET served from Disk Reading or Memory?
> > 4. Compared to Item 2 above against "Prefetch Option for Blockcache",
> which
> > one is best for read performance?
> >
> > The purpose I try to achieve is best and stable read performance even
> when
> > have high HDFS IO like re-balance when new servers added. we want the
> > performance could be comparable against memory solution like Redis.
> >
> > Thanks
> > Proust
> >
> > --
> >
> > 地址: 上海市浦东新区金科路2889弄长泰广场C座7楼
> > Address: 7th floor, No#3, Jinke road 2889, Pudong district, Shanghai,
> > China.
> > Mobile: +86 13621672634
> >
>



-- 

地址: 上海市浦东新区金科路2889弄长泰广场C座7楼
Address: 7th floor, No#3, Jinke road 2889, Pudong district, Shanghai, China.
Mobile: +86 13621672634

Re: [In Memory] Questions about Make HBase Table In Memory

[Ted Yu]

Puts are stored in memstore. Get / Scan would include such data in memstore
when row keys match.

For #2, see http://hbase.apache.org/book.html#block.cache.design

For #3, one indication is latency of the result. Latency would be longer
when disk is involved.


On Sun, Oct 23, 2016 at 8:01 PM, Proust(Proust/Feng Guizhou) [Saipan] ­ <
pf...@coupang.com> wrote:

> Hi, HBase Experts
>
> I have several questions related to HBase BlockCache and IN_MEMORY setting
> 1. Does BlockCache cache the data when do PUT request, so next time
> GET/SCAN request against the same data will be served from Memory only?
> 2. If HBase Table ColumnFamily have *IN_MEMORY=true*, and every time I do
> PUT  followed immediately with a GET/SCAN against the same data, assume I
> have memory larger than the specific table, is that this way I can make
> sure the whole table is in memory and all read requests against this table
> will be served from memory only?
> 3. How can I determine whether a GET served from Disk Reading or Memory?
> 4. Compared to Item 2 above against "Prefetch Option for Blockcache", which
> one is best for read performance?
>
> The purpose I try to achieve is best and stable read performance even when
> have high HDFS IO like re-balance when new servers added. we want the
> performance could be comparable against memory solution like Redis.
>
> Thanks
> Proust
>
> --
>
> 地址: 上海市浦东新区金科路2889弄长泰广场C座7楼
> Address: 7th floor, No#3, Jinke road 2889, Pudong district, Shanghai,
> China.
> Mobile: +86 13621672634
>

Re: Hbase Row key lock

[anil gupta]

As per my experience, in normal case lock wont be held for 60 seconds.
How many writes/sec per node you are doing?
Seems like there is some hotspotting in your use case or cluster might need
some tuning/tweaking. Have you verified that your writes/reads are evenly
spread out. Do u have a time component as prefix of your rowkey?

On Sun, Oct 23, 2016 at 7:01 PM, Manjeet Singh 
wrote:

> Anil its written it can hold lock upto 60 second. In my case my job get
> stuck and many update for same rowkey cause fir bead health of hbase in cdh
> 5.8
>
> On 24 Oct 2016 06:26, "anil gupta"  wrote:
>
> Writes/Updates usually takes few milliseconds in HBase. So, in normal cases
> lock wont be held for seconds.
>
> On Sun, Oct 23, 2016 at 12:57 PM, Manjeet Singh <
> manjeet.chand...@gmail.com>
> wrote:
>
> > Anil all information are correct I am talking about suppose I didn't set
> > any version and I have very simple requirement to update if I found xyz
> > record and if I hv few ETL process which are responsible for aggregate
> the
> > data which is very common. ... why my hbase stuck if I try to update same
> > rowkey... its mean its hold the lock for few second
> >
> > On 24 Oct 2016 00:46, "anil gupta"  wrote:
> >
> > > Writes within a HBase row are atomic. Now, whichever write becomes the
> > > latest write(with the help of timestamp value) will prevail as the
> > default
> > > value. If you set versions to more than 1 in column family, then you
> will
> > > be able to see both the values if you query for multiple versions.
> > >
> > > HTH,
> > > Anil Gupta
> > >
> > > On Sun, Oct 23, 2016 at 12:02 PM, Manjeet Singh <
> > > manjeet.chand...@gmail.com>
> > > wrote:
> > >
> > > > Till now what i understand their is no update
> > > >
> > > > if two different thread try to update same record what happen
> > > >
> > > > first record insert with some version
> > > > second thread comes and change the version and its like a new insert
> > with
> > > > some version
> > > > this process called MVCC
> > > >
> > > > If I am correct how hbase support MVCC mean any configuration for
> > > handlling
> > > > multiple thread at same time?
> > > >
> > > > On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh <
> > > > manjeet.chand...@gmail.com>
> > > > wrote:
> > > >
> > > > > No I don't have 50 clients? I want to understand internal working
> of
> > > > Hbase
> > > > > in my usecase I have bulk update operation from spark job we have 7
> > > > > different kafka pipeline and 7 spark job
> > > > > it might happen that my 2 0r 3 spark job have same rowkey for
> update
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak <
> dimaspi...@apache.org
> > >
> > > > > wrote:
> > > > >
> > > > >> If your typical use case sees 50 clients simultaneously trying to
> > > update
> > > > >> the same row, then a strongly consistent data store that writes to
> > > disk
> > > > >> for
> > > > >> fault tolerance may not be for you. That said, such a use case
> seems
> > > > >> extremely unusual to me and I'd ask why you're trying to update
> the
> > > same
> > > > >> row in such a manner.
> > > > >>
> > > > >> On Sunday, October 23, 2016, Manjeet Singh <
> > > manjeet.chand...@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Hi Dima,
> > > > >> >
> > > > >> > I didn't get ? point is assume I have 50 different client all
> > having
> > > > >> same
> > > > >> > rowkey all want to update on same rowkey at same time now just
> > tell
> > > > what
> > > > >> > will happen? who will get what value?
> > > > >> >
> > > > >> > Thanks
> > > > >> > Manjeet
> > > > >> >
> > > > >> > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak <
> > > dimaspi...@apache.org
> > > > >> > > wrote:
> > > > >> >
> > > > >> > > Unless told not to, HBase will always write to memory and
> append
> > > to
> > > > >> the
> > > > >> > WAL
> > > > >> > > on disk before returning and saying the write succeeded.
> That's
> > by
> > > > >> design
> > > > >> > > and the same write pattern that companies like Apple and
> > Facebook
> > > > have
> > > > >> > > found works for them at scale. So what's there to solve?
> > > > >> > >
> > > > >> > > On Sunday, October 23, 2016, Manjeet Singh <
> > > > >> manjeet.chand...@gmail.com
> > > > >> > >
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Hi All,
> > > > >> > > >
> > > > >> > > > I have read below mention blog and it also said Hbase holds
> > the
> > > > >> lock on
> > > > >> > > > rowkey level
> > > > >> > > > https://blogs.apache.org/hbase/entry/apache_hbase_
> > > > >> > internals_locking_and
> > > > >> > > > (0) Obtain Row Lock
> > > > >> > > > (1) Write to Write-Ahead-Log (WAL)
> > > > >> > > > (2) Update MemStore: write each cell to the memstore
> > > > >> > > > (3) Release Row Lock
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > SO question is how to solve this if I have very frequent
> > update

Re: Hbase Row key lock

[Ted Yu]

Manjeet:
Did you have a chance to get jstack during the lock contention period ?

Cheers 

> On Oct 23, 2016, at 12:57 PM, Manjeet Singh  
> wrote:
> 
> Anil all information are correct I am talking about suppose I didn't set
> any version and I have very simple requirement to update if I found xyz
> record and if I hv few ETL process which are responsible for aggregate the
> data which is very common. ... why my hbase stuck if I try to update same
> rowkey... its mean its hold the lock for few second
> 
>> On 24 Oct 2016 00:46, "anil gupta"  wrote:
>> 
>> Writes within a HBase row are atomic. Now, whichever write becomes the
>> latest write(with the help of timestamp value) will prevail as the default
>> value. If you set versions to more than 1 in column family, then you will
>> be able to see both the values if you query for multiple versions.
>> 
>> HTH,
>> Anil Gupta
>> 
>> On Sun, Oct 23, 2016 at 12:02 PM, Manjeet Singh <
>> manjeet.chand...@gmail.com>
>> wrote:
>> 
>>> Till now what i understand their is no update
>>> 
>>> if two different thread try to update same record what happen
>>> 
>>> first record insert with some version
>>> second thread comes and change the version and its like a new insert with
>>> some version
>>> this process called MVCC
>>> 
>>> If I am correct how hbase support MVCC mean any configuration for
>> handlling
>>> multiple thread at same time?
>>> 
>>> On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh <
>>> manjeet.chand...@gmail.com>
>>> wrote:
>>> 
 No I don't have 50 clients? I want to understand internal working of
>>> Hbase
 in my usecase I have bulk update operation from spark job we have 7
 different kafka pipeline and 7 spark job
 it might happen that my 2 0r 3 spark job have same rowkey for update
 
 
 
 On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak 
 wrote:
 
> If your typical use case sees 50 clients simultaneously trying to
>> update
> the same row, then a strongly consistent data store that writes to
>> disk
> for
> fault tolerance may not be for you. That said, such a use case seems
> extremely unusual to me and I'd ask why you're trying to update the
>> same
> row in such a manner.
> 
> On Sunday, October 23, 2016, Manjeet Singh <
>> manjeet.chand...@gmail.com>
> wrote:
> 
>> Hi Dima,
>> 
>> I didn't get ? point is assume I have 50 different client all having
> same
>> rowkey all want to update on same rowkey at same time now just tell
>>> what
>> will happen? who will get what value?
>> 
>> Thanks
>> Manjeet
>> 
>> On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak <
>> dimaspi...@apache.org
>> > wrote:
>> 
>>> Unless told not to, HBase will always write to memory and append
>> to
> the
>> WAL
>>> on disk before returning and saying the write succeeded. That's by
> design
>>> and the same write pattern that companies like Apple and Facebook
>>> have
>>> found works for them at scale. So what's there to solve?
>>> 
>>> On Sunday, October 23, 2016, Manjeet Singh <
> manjeet.chand...@gmail.com
>> >
>>> wrote:
>>> 
 Hi All,
 
 I have read below mention blog and it also said Hbase holds the
> lock on
 rowkey level
 https://blogs.apache.org/hbase/entry/apache_hbase_
>> internals_locking_and
 (0) Obtain Row Lock
 (1) Write to Write-Ahead-Log (WAL)
 (2) Update MemStore: write each cell to the memstore
 (3) Release Row Lock
 
 
 SO question is how to solve this if I have very frequent update
>> on
>> Hbase
 
 Thanks
 Manjeet
 
 On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh <
>>> manjeet.chand...@gmail.com 
 >
 wrote:
 
> Hi All
> 
> Can anyone help me about how and in which version of Hbase
>>> support
>>> Rowkey
> lock ?
> I have seen article about rowkey lock but it was about .94
> version it
 said
> that if row key not exist and any update request come and that
> rowkey
>>> not
> exist then in this case Hbase hold the lock for 60 sec.
> 
> currently I am using Hbase 1.2.2 version
> 
> Thanks
> Manjeet
> 
> 
> 
> --
> luv all
 
 
 
 --
 luv all
>>> 
>>> 
>>> --
>>> -Dima
>> 
>> 
>> 
>> --
>> luv all
> 
> 
> --
> -Dima
 
 
 
 --
 luv all
>>> 
>>> 
>>> 
>>> --
>>> luv all
>> 
>> 
>> 
>> --
>> Thanks & Regards,
>> Anil Gupta
>> 

Re: Hbase Row key lock

[anil gupta]

Writes/Updates usually takes few milliseconds in HBase. So, in normal cases
lock wont be held for seconds.

On Sun, Oct 23, 2016 at 12:57 PM, Manjeet Singh 
wrote:

> Anil all information are correct I am talking about suppose I didn't set
> any version and I have very simple requirement to update if I found xyz
> record and if I hv few ETL process which are responsible for aggregate the
> data which is very common. ... why my hbase stuck if I try to update same
> rowkey... its mean its hold the lock for few second
>
> On 24 Oct 2016 00:46, "anil gupta"  wrote:
>
> > Writes within a HBase row are atomic. Now, whichever write becomes the
> > latest write(with the help of timestamp value) will prevail as the
> default
> > value. If you set versions to more than 1 in column family, then you will
> > be able to see both the values if you query for multiple versions.
> >
> > HTH,
> > Anil Gupta
> >
> > On Sun, Oct 23, 2016 at 12:02 PM, Manjeet Singh <
> > manjeet.chand...@gmail.com>
> > wrote:
> >
> > > Till now what i understand their is no update
> > >
> > > if two different thread try to update same record what happen
> > >
> > > first record insert with some version
> > > second thread comes and change the version and its like a new insert
> with
> > > some version
> > > this process called MVCC
> > >
> > > If I am correct how hbase support MVCC mean any configuration for
> > handlling
> > > multiple thread at same time?
> > >
> > > On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh <
> > > manjeet.chand...@gmail.com>
> > > wrote:
> > >
> > > > No I don't have 50 clients? I want to understand internal working of
> > > Hbase
> > > > in my usecase I have bulk update operation from spark job we have 7
> > > > different kafka pipeline and 7 spark job
> > > > it might happen that my 2 0r 3 spark job have same rowkey for update
> > > >
> > > >
> > > >
> > > > On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak  >
> > > > wrote:
> > > >
> > > >> If your typical use case sees 50 clients simultaneously trying to
> > update
> > > >> the same row, then a strongly consistent data store that writes to
> > disk
> > > >> for
> > > >> fault tolerance may not be for you. That said, such a use case seems
> > > >> extremely unusual to me and I'd ask why you're trying to update the
> > same
> > > >> row in such a manner.
> > > >>
> > > >> On Sunday, October 23, 2016, Manjeet Singh <
> > manjeet.chand...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hi Dima,
> > > >> >
> > > >> > I didn't get ? point is assume I have 50 different client all
> having
> > > >> same
> > > >> > rowkey all want to update on same rowkey at same time now just
> tell
> > > what
> > > >> > will happen? who will get what value?
> > > >> >
> > > >> > Thanks
> > > >> > Manjeet
> > > >> >
> > > >> > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak <
> > dimaspi...@apache.org
> > > >> > > wrote:
> > > >> >
> > > >> > > Unless told not to, HBase will always write to memory and append
> > to
> > > >> the
> > > >> > WAL
> > > >> > > on disk before returning and saying the write succeeded. That's
> by
> > > >> design
> > > >> > > and the same write pattern that companies like Apple and
> Facebook
> > > have
> > > >> > > found works for them at scale. So what's there to solve?
> > > >> > >
> > > >> > > On Sunday, October 23, 2016, Manjeet Singh <
> > > >> manjeet.chand...@gmail.com
> > > >> > >
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hi All,
> > > >> > > >
> > > >> > > > I have read below mention blog and it also said Hbase holds
> the
> > > >> lock on
> > > >> > > > rowkey level
> > > >> > > > https://blogs.apache.org/hbase/entry/apache_hbase_
> > > >> > internals_locking_and
> > > >> > > > (0) Obtain Row Lock
> > > >> > > > (1) Write to Write-Ahead-Log (WAL)
> > > >> > > > (2) Update MemStore: write each cell to the memstore
> > > >> > > > (3) Release Row Lock
> > > >> > > >
> > > >> > > >
> > > >> > > > SO question is how to solve this if I have very frequent
> update
> > on
> > > >> > Hbase
> > > >> > > >
> > > >> > > > Thanks
> > > >> > > > Manjeet
> > > >> > > >
> > > >> > > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh <
> > > >> > > manjeet.chand...@gmail.com 
> > > >> > > > >
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > Hi All
> > > >> > > > >
> > > >> > > > > Can anyone help me about how and in which version of Hbase
> > > support
> > > >> > > Rowkey
> > > >> > > > > lock ?
> > > >> > > > > I have seen article about rowkey lock but it was about .94
> > > >> version it
> > > >> > > > said
> > > >> > > > > that if row key not exist and any update request come and
> that
> > > >> rowkey
> > > >> > > not
> > > >> > > > > exist then in this case Hbase hold the lock for 60 sec.
> > > >> > > > >
> > > >> > > > > currently I am using Hbase 1.2.2 version
> > > >> > > > >
> > > >> > > > > Thanks
> > > >> > > > > Manjeet

[ANNOUNCE] Apache HBase 1.1.7 is now available for download

[Andrew Purtell]

Apache HBase 1.1.7 is now available for download. Get it from an Apache
mirror [1] or Maven repository. The list of changes in this release can be
found in the release notes [2] or at the bottom of this announcement.

Thanks to all who contributed to this release.

Best,
The HBase Dev Team

1. http://www.apache.org/dyn/closer.lua/hbase/
2. https://s.apache.org/vGTs

---

HBASE-13587
TestSnapshotCloneIndependence.testOnlineSnapshotDeleteIndependent is flakey
HBASE-13603 Write test asserting desired priority of RS->Master RPCs
HBASE-14223 Meta WALs are not cleared if meta region was closed and RS
aborts
HBASE-14391 Empty regionserver WAL will never be deleted although the
coresponding regionserver has been stale
HBASE-14610 IntegrationTestRpcClient from HBASE-14535 is failing with Async
RPC client
HBASE-15033 Backport test-patch.sh and zombie-detector.sh from master to
branch-1.1
HBASE-15168 Zombie stomping branch-1.1 edition
HBASE-15185 Fix jdk8 javadoc warnings for branch-1.1
HBASE-15308 Flakey TestSplitWalDataLoss on branch-1.1
HBASE-15580 Tag coprocessor limitedprivate scope to StoreFile.Reader
HBASE-15984 Given failure to parse a given WAL that was closed cleanly,
replay the WAL.
HBASE-16165 Decrease RpcServer.callQueueSize before writeResponse causes OOM
HBASE-16260 Audit dependencies for Category-X
HBASE-16294 hbck reporting "No HDFS region dir found" for replicas
HBASE-16340 ensure no Xerces jars included
HBASE-16373 precommit needs a dockerfile with hbase prereqs
HBASE-16375 Mapreduce mini cluster using HBaseTestingUtility not setting
correct resourcemanager and jobhistory webapp address of
MapReduceTestingShim
HBASE-16460 Can't rebuild the BucketAllocator's data structures when
BucketCache uses FileIOEngine
HBASE-16518 Remove old .arcconfig file
HBASE-16522 Procedure v2 - Cache system user and avoid IOException
HBASE-16527 IOExceptions from DFS client still can cause CatalogJanitor to
delete referenced files
HBASE-16535 Use regex to exclude generated classes for findbugs
HBASE-16538 Version mismatch in HBaseConfiguration.checkDefaultsVersion
HBASE-16552 MiniHBaseCluster#getServerWith() does not ignore stopped RSs
HBASE-16562 ITBLL should fail to start if misconfigured
HBASE-16589 Adjust log level for FATAL messages from
HBaseReplicationEndpoint that are not fatal
HBASE-16604 Scanner retries on IOException can cause the scans to miss data
HBASE-16649 Truncate table with splits preserved can cause both data loss
and truncated data appeared again
HBASE-16662 Fix open POODLE vulnerabilities

[ANNOUNCE] Apache HBase 0.98.23 is now available for download

[Andrew Purtell]

Apache HBase 0.98.23 is now available for download. Get it from an Apache
mirror [1] or Maven repository. The list of changes in this release can be
found in the release notes [2] or at the bottom of this announcement.

Thanks to all who contributed to this release.

Best,
The HBase Dev Team

1. http://www.apache.org/dyn/closer.lua/hbase/
2. https://s.apache.org/u6n7

---

HBASE-11354 HConnectionImplementation#DelayedClosing does not start
HBASE-15983 Replication improperly discards data from end-of-wal in some
cases.
HBASE-15984 Given failure to parse a given WAL that was closed cleanly,
replay the WAL.
HBASE-16165 Decrease RpcServer.callQueueSize before writeResponse causes OOM
HBASE-16373 precommit needs a dockerfile with hbase prereqs
HBASE-16562 ITBLL should fail to start if misconfigured
HBASE-16576 Shell add_peer doesn't allow setting cluster_key for custom
endpoints
HBASE-16589 Adjust log level for FATAL messages from
HBaseReplicationEndpoint that are not fatal
HBASE-16649 Truncate table with splits preserved can cause both data loss
and truncated data appeared again
HBASE-16660 ArrayIndexOutOfBounds during the majorCompactionCheck in
DateTieredCompaction
HBASE-16662 Fix open POODLE vulnerabilities
HBASE-16694 Reduce garbage for onDiskChecksum in HFileBlock
HBASE-16705 Eliminate long to Long auto boxing in LongComparator

Re: Hbase Row key lock

[Manjeet Singh]

Anil all information are correct I am talking about suppose I didn't set
any version and I have very simple requirement to update if I found xyz
record and if I hv few ETL process which are responsible for aggregate the
data which is very common. ... why my hbase stuck if I try to update same
rowkey... its mean its hold the lock for few second

On 24 Oct 2016 00:46, "anil gupta"  wrote:

> Writes within a HBase row are atomic. Now, whichever write becomes the
> latest write(with the help of timestamp value) will prevail as the default
> value. If you set versions to more than 1 in column family, then you will
> be able to see both the values if you query for multiple versions.
>
> HTH,
> Anil Gupta
>
> On Sun, Oct 23, 2016 at 12:02 PM, Manjeet Singh <
> manjeet.chand...@gmail.com>
> wrote:
>
> > Till now what i understand their is no update
> >
> > if two different thread try to update same record what happen
> >
> > first record insert with some version
> > second thread comes and change the version and its like a new insert with
> > some version
> > this process called MVCC
> >
> > If I am correct how hbase support MVCC mean any configuration for
> handlling
> > multiple thread at same time?
> >
> > On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh <
> > manjeet.chand...@gmail.com>
> > wrote:
> >
> > > No I don't have 50 clients? I want to understand internal working of
> > Hbase
> > > in my usecase I have bulk update operation from spark job we have 7
> > > different kafka pipeline and 7 spark job
> > > it might happen that my 2 0r 3 spark job have same rowkey for update
> > >
> > >
> > >
> > > On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak 
> > > wrote:
> > >
> > >> If your typical use case sees 50 clients simultaneously trying to
> update
> > >> the same row, then a strongly consistent data store that writes to
> disk
> > >> for
> > >> fault tolerance may not be for you. That said, such a use case seems
> > >> extremely unusual to me and I'd ask why you're trying to update the
> same
> > >> row in such a manner.
> > >>
> > >> On Sunday, October 23, 2016, Manjeet Singh <
> manjeet.chand...@gmail.com>
> > >> wrote:
> > >>
> > >> > Hi Dima,
> > >> >
> > >> > I didn't get ? point is assume I have 50 different client all having
> > >> same
> > >> > rowkey all want to update on same rowkey at same time now just tell
> > what
> > >> > will happen? who will get what value?
> > >> >
> > >> > Thanks
> > >> > Manjeet
> > >> >
> > >> > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak <
> dimaspi...@apache.org
> > >> > > wrote:
> > >> >
> > >> > > Unless told not to, HBase will always write to memory and append
> to
> > >> the
> > >> > WAL
> > >> > > on disk before returning and saying the write succeeded. That's by
> > >> design
> > >> > > and the same write pattern that companies like Apple and Facebook
> > have
> > >> > > found works for them at scale. So what's there to solve?
> > >> > >
> > >> > > On Sunday, October 23, 2016, Manjeet Singh <
> > >> manjeet.chand...@gmail.com
> > >> > >
> > >> > > wrote:
> > >> > >
> > >> > > > Hi All,
> > >> > > >
> > >> > > > I have read below mention blog and it also said Hbase holds the
> > >> lock on
> > >> > > > rowkey level
> > >> > > > https://blogs.apache.org/hbase/entry/apache_hbase_
> > >> > internals_locking_and
> > >> > > > (0) Obtain Row Lock
> > >> > > > (1) Write to Write-Ahead-Log (WAL)
> > >> > > > (2) Update MemStore: write each cell to the memstore
> > >> > > > (3) Release Row Lock
> > >> > > >
> > >> > > >
> > >> > > > SO question is how to solve this if I have very frequent update
> on
> > >> > Hbase
> > >> > > >
> > >> > > > Thanks
> > >> > > > Manjeet
> > >> > > >
> > >> > > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh <
> > >> > > manjeet.chand...@gmail.com 
> > >> > > > >
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi All
> > >> > > > >
> > >> > > > > Can anyone help me about how and in which version of Hbase
> > support
> > >> > > Rowkey
> > >> > > > > lock ?
> > >> > > > > I have seen article about rowkey lock but it was about .94
> > >> version it
> > >> > > > said
> > >> > > > > that if row key not exist and any update request come and that
> > >> rowkey
> > >> > > not
> > >> > > > > exist then in this case Hbase hold the lock for 60 sec.
> > >> > > > >
> > >> > > > > currently I am using Hbase 1.2.2 version
> > >> > > > >
> > >> > > > > Thanks
> > >> > > > > Manjeet
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > luv all
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > > luv all
> > >> > > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > -Dima
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > luv all
> > >> >
> > >>
> > >>
> > >> --
> > >> -Dima
> > >>
> > >
> > >
> > >
> > > --
> > > luv all
> > >
> >
> >
> >
> > --
> > luv all
> >
>
>
>
> --
> Thanks & Regards,
> Anil 

Re: Hbase Row key lock

[anil gupta]

Writes within a HBase row are atomic. Now, whichever write becomes the
latest write(with the help of timestamp value) will prevail as the default
value. If you set versions to more than 1 in column family, then you will
be able to see both the values if you query for multiple versions.

HTH,
Anil Gupta

On Sun, Oct 23, 2016 at 12:02 PM, Manjeet Singh 
wrote:

> Till now what i understand their is no update
>
> if two different thread try to update same record what happen
>
> first record insert with some version
> second thread comes and change the version and its like a new insert with
> some version
> this process called MVCC
>
> If I am correct how hbase support MVCC mean any configuration for handlling
> multiple thread at same time?
>
> On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh <
> manjeet.chand...@gmail.com>
> wrote:
>
> > No I don't have 50 clients? I want to understand internal working of
> Hbase
> > in my usecase I have bulk update operation from spark job we have 7
> > different kafka pipeline and 7 spark job
> > it might happen that my 2 0r 3 spark job have same rowkey for update
> >
> >
> >
> > On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak 
> > wrote:
> >
> >> If your typical use case sees 50 clients simultaneously trying to update
> >> the same row, then a strongly consistent data store that writes to disk
> >> for
> >> fault tolerance may not be for you. That said, such a use case seems
> >> extremely unusual to me and I'd ask why you're trying to update the same
> >> row in such a manner.
> >>
> >> On Sunday, October 23, 2016, Manjeet Singh 
> >> wrote:
> >>
> >> > Hi Dima,
> >> >
> >> > I didn't get ? point is assume I have 50 different client all having
> >> same
> >> > rowkey all want to update on same rowkey at same time now just tell
> what
> >> > will happen? who will get what value?
> >> >
> >> > Thanks
> >> > Manjeet
> >> >
> >> > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak  >> > > wrote:
> >> >
> >> > > Unless told not to, HBase will always write to memory and append to
> >> the
> >> > WAL
> >> > > on disk before returning and saying the write succeeded. That's by
> >> design
> >> > > and the same write pattern that companies like Apple and Facebook
> have
> >> > > found works for them at scale. So what's there to solve?
> >> > >
> >> > > On Sunday, October 23, 2016, Manjeet Singh <
> >> manjeet.chand...@gmail.com
> >> > >
> >> > > wrote:
> >> > >
> >> > > > Hi All,
> >> > > >
> >> > > > I have read below mention blog and it also said Hbase holds the
> >> lock on
> >> > > > rowkey level
> >> > > > https://blogs.apache.org/hbase/entry/apache_hbase_
> >> > internals_locking_and
> >> > > > (0) Obtain Row Lock
> >> > > > (1) Write to Write-Ahead-Log (WAL)
> >> > > > (2) Update MemStore: write each cell to the memstore
> >> > > > (3) Release Row Lock
> >> > > >
> >> > > >
> >> > > > SO question is how to solve this if I have very frequent update on
> >> > Hbase
> >> > > >
> >> > > > Thanks
> >> > > > Manjeet
> >> > > >
> >> > > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh <
> >> > > manjeet.chand...@gmail.com 
> >> > > > >
> >> > > > wrote:
> >> > > >
> >> > > > > Hi All
> >> > > > >
> >> > > > > Can anyone help me about how and in which version of Hbase
> support
> >> > > Rowkey
> >> > > > > lock ?
> >> > > > > I have seen article about rowkey lock but it was about .94
> >> version it
> >> > > > said
> >> > > > > that if row key not exist and any update request come and that
> >> rowkey
> >> > > not
> >> > > > > exist then in this case Hbase hold the lock for 60 sec.
> >> > > > >
> >> > > > > currently I am using Hbase 1.2.2 version
> >> > > > >
> >> > > > > Thanks
> >> > > > > Manjeet
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > luv all
> >> > > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > luv all
> >> > > >
> >> > >
> >> > >
> >> > > --
> >> > > -Dima
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > luv all
> >> >
> >>
> >>
> >> --
> >> -Dima
> >>
> >
> >
> >
> > --
> > luv all
> >
>
>
>
> --
> luv all
>



-- 
Thanks & Regards,
Anil Gupta

Re: Hbase Row key lock

[Manjeet Singh]

Till now what i understand their is no update

if two different thread try to update same record what happen

first record insert with some version
second thread comes and change the version and its like a new insert with
some version
this process called MVCC

If I am correct how hbase support MVCC mean any configuration for handlling
multiple thread at same time?

On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh 
wrote:

> No I don't have 50 clients? I want to understand internal working of Hbase
> in my usecase I have bulk update operation from spark job we have 7
> different kafka pipeline and 7 spark job
> it might happen that my 2 0r 3 spark job have same rowkey for update
>
>
>
> On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak 
> wrote:
>
>> If your typical use case sees 50 clients simultaneously trying to update
>> the same row, then a strongly consistent data store that writes to disk
>> for
>> fault tolerance may not be for you. That said, such a use case seems
>> extremely unusual to me and I'd ask why you're trying to update the same
>> row in such a manner.
>>
>> On Sunday, October 23, 2016, Manjeet Singh 
>> wrote:
>>
>> > Hi Dima,
>> >
>> > I didn't get ? point is assume I have 50 different client all having
>> same
>> > rowkey all want to update on same rowkey at same time now just tell what
>> > will happen? who will get what value?
>> >
>> > Thanks
>> > Manjeet
>> >
>> > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak > > > wrote:
>> >
>> > > Unless told not to, HBase will always write to memory and append to
>> the
>> > WAL
>> > > on disk before returning and saying the write succeeded. That's by
>> design
>> > > and the same write pattern that companies like Apple and Facebook have
>> > > found works for them at scale. So what's there to solve?
>> > >
>> > > On Sunday, October 23, 2016, Manjeet Singh <
>> manjeet.chand...@gmail.com
>> > >
>> > > wrote:
>> > >
>> > > > Hi All,
>> > > >
>> > > > I have read below mention blog and it also said Hbase holds the
>> lock on
>> > > > rowkey level
>> > > > https://blogs.apache.org/hbase/entry/apache_hbase_
>> > internals_locking_and
>> > > > (0) Obtain Row Lock
>> > > > (1) Write to Write-Ahead-Log (WAL)
>> > > > (2) Update MemStore: write each cell to the memstore
>> > > > (3) Release Row Lock
>> > > >
>> > > >
>> > > > SO question is how to solve this if I have very frequent update on
>> > Hbase
>> > > >
>> > > > Thanks
>> > > > Manjeet
>> > > >
>> > > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh <
>> > > manjeet.chand...@gmail.com 
>> > > > >
>> > > > wrote:
>> > > >
>> > > > > Hi All
>> > > > >
>> > > > > Can anyone help me about how and in which version of Hbase support
>> > > Rowkey
>> > > > > lock ?
>> > > > > I have seen article about rowkey lock but it was about .94
>> version it
>> > > > said
>> > > > > that if row key not exist and any update request come and that
>> rowkey
>> > > not
>> > > > > exist then in this case Hbase hold the lock for 60 sec.
>> > > > >
>> > > > > currently I am using Hbase 1.2.2 version
>> > > > >
>> > > > > Thanks
>> > > > > Manjeet
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > luv all
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > luv all
>> > > >
>> > >
>> > >
>> > > --
>> > > -Dima
>> > >
>> >
>> >
>> >
>> > --
>> > luv all
>> >
>>
>>
>> --
>> -Dima
>>
>
>
>
> --
> luv all
>



-- 
luv all

Re: Hbase Row key lock

[Manjeet Singh]

No I don't have 50 clients? I want to understand internal working of Hbase
in my usecase I have bulk update operation from spark job we have 7
different kafka pipeline and 7 spark job
it might happen that my 2 0r 3 spark job have same rowkey for update



On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak  wrote:

> If your typical use case sees 50 clients simultaneously trying to update
> the same row, then a strongly consistent data store that writes to disk for
> fault tolerance may not be for you. That said, such a use case seems
> extremely unusual to me and I'd ask why you're trying to update the same
> row in such a manner.
>
> On Sunday, October 23, 2016, Manjeet Singh 
> wrote:
>
> > Hi Dima,
> >
> > I didn't get ? point is assume I have 50 different client all having same
> > rowkey all want to update on same rowkey at same time now just tell what
> > will happen? who will get what value?
> >
> > Thanks
> > Manjeet
> >
> > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak  > > wrote:
> >
> > > Unless told not to, HBase will always write to memory and append to the
> > WAL
> > > on disk before returning and saying the write succeeded. That's by
> design
> > > and the same write pattern that companies like Apple and Facebook have
> > > found works for them at scale. So what's there to solve?
> > >
> > > On Sunday, October 23, 2016, Manjeet Singh  > >
> > > wrote:
> > >
> > > > Hi All,
> > > >
> > > > I have read below mention blog and it also said Hbase holds the lock
> on
> > > > rowkey level
> > > > https://blogs.apache.org/hbase/entry/apache_hbase_
> > internals_locking_and
> > > > (0) Obtain Row Lock
> > > > (1) Write to Write-Ahead-Log (WAL)
> > > > (2) Update MemStore: write each cell to the memstore
> > > > (3) Release Row Lock
> > > >
> > > >
> > > > SO question is how to solve this if I have very frequent update on
> > Hbase
> > > >
> > > > Thanks
> > > > Manjeet
> > > >
> > > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh <
> > > manjeet.chand...@gmail.com 
> > > > >
> > > > wrote:
> > > >
> > > > > Hi All
> > > > >
> > > > > Can anyone help me about how and in which version of Hbase support
> > > Rowkey
> > > > > lock ?
> > > > > I have seen article about rowkey lock but it was about .94 version
> it
> > > > said
> > > > > that if row key not exist and any update request come and that
> rowkey
> > > not
> > > > > exist then in this case Hbase hold the lock for 60 sec.
> > > > >
> > > > > currently I am using Hbase 1.2.2 version
> > > > >
> > > > > Thanks
> > > > > Manjeet
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > luv all
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > luv all
> > > >
> > >
> > >
> > > --
> > > -Dima
> > >
> >
> >
> >
> > --
> > luv all
> >
>
>
> --
> -Dima
>



-- 
luv all

Re: Hbase Row key lock

[Dima Spivak]

If your typical use case sees 50 clients simultaneously trying to update
the same row, then a strongly consistent data store that writes to disk for
fault tolerance may not be for you. That said, such a use case seems
extremely unusual to me and I'd ask why you're trying to update the same
row in such a manner.

On Sunday, October 23, 2016, Manjeet Singh 
wrote:

> Hi Dima,
>
> I didn't get ? point is assume I have 50 different client all having same
> rowkey all want to update on same rowkey at same time now just tell what
> will happen? who will get what value?
>
> Thanks
> Manjeet
>
> On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak  > wrote:
>
> > Unless told not to, HBase will always write to memory and append to the
> WAL
> > on disk before returning and saying the write succeeded. That's by design
> > and the same write pattern that companies like Apple and Facebook have
> > found works for them at scale. So what's there to solve?
> >
> > On Sunday, October 23, 2016, Manjeet Singh  >
> > wrote:
> >
> > > Hi All,
> > >
> > > I have read below mention blog and it also said Hbase holds the lock on
> > > rowkey level
> > > https://blogs.apache.org/hbase/entry/apache_hbase_
> internals_locking_and
> > > (0) Obtain Row Lock
> > > (1) Write to Write-Ahead-Log (WAL)
> > > (2) Update MemStore: write each cell to the memstore
> > > (3) Release Row Lock
> > >
> > >
> > > SO question is how to solve this if I have very frequent update on
> Hbase
> > >
> > > Thanks
> > > Manjeet
> > >
> > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh <
> > manjeet.chand...@gmail.com 
> > > >
> > > wrote:
> > >
> > > > Hi All
> > > >
> > > > Can anyone help me about how and in which version of Hbase support
> > Rowkey
> > > > lock ?
> > > > I have seen article about rowkey lock but it was about .94 version it
> > > said
> > > > that if row key not exist and any update request come and that rowkey
> > not
> > > > exist then in this case Hbase hold the lock for 60 sec.
> > > >
> > > > currently I am using Hbase 1.2.2 version
> > > >
> > > > Thanks
> > > > Manjeet
> > > >
> > > >
> > > >
> > > > --
> > > > luv all
> > > >
> > >
> > >
> > >
> > > --
> > > luv all
> > >
> >
> >
> > --
> > -Dima
> >
>
>
>
> --
> luv all
>


-- 
-Dima

Re: Hbase Row key lock

[Manjeet Singh]

Hi Dima,

I didn't get ? point is assume I have 50 different client all having same
rowkey all want to update on same rowkey at same time now just tell what
will happen? who will get what value?

Thanks
Manjeet

On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak  wrote:

> Unless told not to, HBase will always write to memory and append to the WAL
> on disk before returning and saying the write succeeded. That's by design
> and the same write pattern that companies like Apple and Facebook have
> found works for them at scale. So what's there to solve?
>
> On Sunday, October 23, 2016, Manjeet Singh 
> wrote:
>
> > Hi All,
> >
> > I have read below mention blog and it also said Hbase holds the lock on
> > rowkey level
> > https://blogs.apache.org/hbase/entry/apache_hbase_internals_locking_and
> > (0) Obtain Row Lock
> > (1) Write to Write-Ahead-Log (WAL)
> > (2) Update MemStore: write each cell to the memstore
> > (3) Release Row Lock
> >
> >
> > SO question is how to solve this if I have very frequent update on Hbase
> >
> > Thanks
> > Manjeet
> >
> > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh <
> manjeet.chand...@gmail.com
> > >
> > wrote:
> >
> > > Hi All
> > >
> > > Can anyone help me about how and in which version of Hbase support
> Rowkey
> > > lock ?
> > > I have seen article about rowkey lock but it was about .94 version it
> > said
> > > that if row key not exist and any update request come and that rowkey
> not
> > > exist then in this case Hbase hold the lock for 60 sec.
> > >
> > > currently I am using Hbase 1.2.2 version
> > >
> > > Thanks
> > > Manjeet
> > >
> > >
> > >
> > > --
> > > luv all
> > >
> >
> >
> >
> > --
> > luv all
> >
>
>
> --
> -Dima
>



-- 
luv all

Re: Hbase Row key lock

[Dima Spivak]

Unless told not to, HBase will always write to memory and append to the WAL
on disk before returning and saying the write succeeded. That's by design
and the same write pattern that companies like Apple and Facebook have
found works for them at scale. So what's there to solve?

On Sunday, October 23, 2016, Manjeet Singh 
wrote:

> Hi All,
>
> I have read below mention blog and it also said Hbase holds the lock on
> rowkey level
> https://blogs.apache.org/hbase/entry/apache_hbase_internals_locking_and
> (0) Obtain Row Lock
> (1) Write to Write-Ahead-Log (WAL)
> (2) Update MemStore: write each cell to the memstore
> (3) Release Row Lock
>
>
> SO question is how to solve this if I have very frequent update on Hbase
>
> Thanks
> Manjeet
>
> On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh  >
> wrote:
>
> > Hi All
> >
> > Can anyone help me about how and in which version of Hbase support Rowkey
> > lock ?
> > I have seen article about rowkey lock but it was about .94 version it
> said
> > that if row key not exist and any update request come and that rowkey not
> > exist then in this case Hbase hold the lock for 60 sec.
> >
> > currently I am using Hbase 1.2.2 version
> >
> > Thanks
> > Manjeet
> >
> >
> >
> > --
> > luv all
> >
>
>
>
> --
> luv all
>


-- 
-Dima

Re: Hbase Row key lock

[Manjeet Singh]

Hi All,

I have read below mention blog and it also said Hbase holds the lock on
rowkey level
https://blogs.apache.org/hbase/entry/apache_hbase_internals_locking_and
(0) Obtain Row Lock
(1) Write to Write-Ahead-Log (WAL)
(2) Update MemStore: write each cell to the memstore
(3) Release Row Lock


SO question is how to solve this if I have very frequent update on Hbase

Thanks
Manjeet

On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh 
wrote:

> Hi All
>
> Can anyone help me about how and in which version of Hbase support Rowkey
> lock ?
> I have seen article about rowkey lock but it was about .94 version it said
> that if row key not exist and any update request come and that rowkey not
> exist then in this case Hbase hold the lock for 60 sec.
>
> currently I am using Hbase 1.2.2 version
>
> Thanks
> Manjeet
>
>
>
> --
> luv all
>



-- 
luv all