Re: Hbase Row key lock
As per my experience, in normal case lock wont be held for 60 seconds. How many writes/sec per node you are doing? Seems like there is some hotspotting in your use case or cluster might need some tuning/tweaking. Have you verified that your writes/reads are evenly spread out. Do u have a time component as prefix of your rowkey? On Sun, Oct 23, 2016 at 7:01 PM, Manjeet Singh wrote: > Anil its written it can hold lock upto 60 second. In my case my job get > stuck and many update for same rowkey cause fir bead health of hbase in cdh > 5.8 > > On 24 Oct 2016 06:26, "anil gupta" wrote: > > Writes/Updates usually takes few milliseconds in HBase. So, in normal cases > lock wont be held for seconds. > > On Sun, Oct 23, 2016 at 12:57 PM, Manjeet Singh < > manjeet.chand...@gmail.com> > wrote: > > > Anil all information are correct I am talking about suppose I didn't set > > any version and I have very simple requirement to update if I found xyz > > record and if I hv few ETL process which are responsible for aggregate > the > > data which is very common. ... why my hbase stuck if I try to update same > > rowkey... its mean its hold the lock for few second > > > > On 24 Oct 2016 00:46, "anil gupta" wrote: > > > > > Writes within a HBase row are atomic. Now, whichever write becomes the > > > latest write(with the help of timestamp value) will prevail as the > > default > > > value. If you set versions to more than 1 in column family, then you > will > > > be able to see both the values if you query for multiple versions. > > > > > > HTH, > > > Anil Gupta > > > > > > On Sun, Oct 23, 2016 at 12:02 PM, Manjeet Singh < > > > manjeet.chand...@gmail.com> > > > wrote: > > > > > > > Till now what i understand their is no update > > > > > > > > if two different thread try to update same record what happen > > > > > > > > first record insert with some version > > > > second thread comes and change the version and its like a new insert > > with > > > > some version > > > > this process called MVCC > > > > > > > > If I am correct how hbase support MVCC mean any configuration for > > > handlling > > > > multiple thread at same time? > > > > > > > > On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh < > > > > manjeet.chand...@gmail.com> > > > > wrote: > > > > > > > > > No I don't have 50 clients? I want to understand internal working > of > > > > Hbase > > > > > in my usecase I have bulk update operation from spark job we have 7 > > > > > different kafka pipeline and 7 spark job > > > > > it might happen that my 2 0r 3 spark job have same rowkey for > update > > > > > > > > > > > > > > > > > > > > On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak < > dimaspi...@apache.org > > > > > > > > wrote: > > > > > > > > > >> If your typical use case sees 50 clients simultaneously trying to > > > update > > > > >> the same row, then a strongly consistent data store that writes to > > > disk > > > > >> for > > > > >> fault tolerance may not be for you. That said, such a use case > seems > > > > >> extremely unusual to me and I'd ask why you're trying to update > the > > > same > > > > >> row in such a manner. > > > > >> > > > > >> On Sunday, October 23, 2016, Manjeet Singh < > > > manjeet.chand...@gmail.com> > > > > >> wrote: > > > > >> > > > > >> > Hi Dima, > > > > >> > > > > > >> > I didn't get ? point is assume I have 50 different client all > > having > > > > >> same > > > > >> > rowkey all want to update on same rowkey at same time now just > > tell > > > > what > > > > >> > will happen? who will get what value? > > > > >> > > > > > >> > Thanks > > > > >> > Manjeet > > > > >> > > > > > >> > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak < > > > dimaspi...@apache.org > > > > >> > > wrote: > > > > >> > > > > > >> > > Unless told not to, HBase will always write to memory and > append > > > to > > > > >> the > > > > >> > WAL > > > > >> > > on disk before returning and saying the write succeeded. > That's > > by > > > > >> design > > > > >> > > and the same write pattern that companies like Apple and > > Facebook > > > > have > > > > >> > > found works for them at scale. So what's there to solve? > > > > >> > > > > > > >> > > On Sunday, October 23, 2016, Manjeet Singh < > > > > >> manjeet.chand...@gmail.com > > > > >> > > > > > > >> > > wrote: > > > > >> > > > > > > >> > > > Hi All, > > > > >> > > > > > > > >> > > > I have read below mention blog and it also said Hbase holds > > the > > > > >> lock on > > > > >> > > > rowkey level > > > > >> > > > https://blogs.apache.org/hbase/entry/apache_hbase_ > > > > >> > internals_locking_and > > > > >> > > > (0) Obtain Row Lock > > > > >> > > > (1) Write to Write-Ahead-Log (WAL) > > > > >> > > > (2) Update MemStore: write each cell to the memstore > > > > >> > > > (3) Release Row Lock > > > > >> > > > > > > > >> > > > > > > > >> > > > SO question is how to solve this if I have very frequent > > update > > > on > > > > >> > Hbase > > > > >> > > > > > > > >> > > > Thanks > > > > >> > > > Manjeet > > > >
Re: Hbase Row key lock
Anil its written it can hold lock upto 60 second. In my case my job get stuck and many update for same rowkey cause fir bead health of hbase in cdh 5.8 On 24 Oct 2016 06:26, "anil gupta" wrote: Writes/Updates usually takes few milliseconds in HBase. So, in normal cases lock wont be held for seconds. On Sun, Oct 23, 2016 at 12:57 PM, Manjeet Singh wrote: > Anil all information are correct I am talking about suppose I didn't set > any version and I have very simple requirement to update if I found xyz > record and if I hv few ETL process which are responsible for aggregate the > data which is very common. ... why my hbase stuck if I try to update same > rowkey... its mean its hold the lock for few second > > On 24 Oct 2016 00:46, "anil gupta" wrote: > > > Writes within a HBase row are atomic. Now, whichever write becomes the > > latest write(with the help of timestamp value) will prevail as the > default > > value. If you set versions to more than 1 in column family, then you will > > be able to see both the values if you query for multiple versions. > > > > HTH, > > Anil Gupta > > > > On Sun, Oct 23, 2016 at 12:02 PM, Manjeet Singh < > > manjeet.chand...@gmail.com> > > wrote: > > > > > Till now what i understand their is no update > > > > > > if two different thread try to update same record what happen > > > > > > first record insert with some version > > > second thread comes and change the version and its like a new insert > with > > > some version > > > this process called MVCC > > > > > > If I am correct how hbase support MVCC mean any configuration for > > handlling > > > multiple thread at same time? > > > > > > On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh < > > > manjeet.chand...@gmail.com> > > > wrote: > > > > > > > No I don't have 50 clients? I want to understand internal working of > > > Hbase > > > > in my usecase I have bulk update operation from spark job we have 7 > > > > different kafka pipeline and 7 spark job > > > > it might happen that my 2 0r 3 spark job have same rowkey for update > > > > > > > > > > > > > > > > On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak > > > > > wrote: > > > > > > > >> If your typical use case sees 50 clients simultaneously trying to > > update > > > >> the same row, then a strongly consistent data store that writes to > > disk > > > >> for > > > >> fault tolerance may not be for you. That said, such a use case seems > > > >> extremely unusual to me and I'd ask why you're trying to update the > > same > > > >> row in such a manner. > > > >> > > > >> On Sunday, October 23, 2016, Manjeet Singh < > > manjeet.chand...@gmail.com> > > > >> wrote: > > > >> > > > >> > Hi Dima, > > > >> > > > > >> > I didn't get ? point is assume I have 50 different client all > having > > > >> same > > > >> > rowkey all want to update on same rowkey at same time now just > tell > > > what > > > >> > will happen? who will get what value? > > > >> > > > > >> > Thanks > > > >> > Manjeet > > > >> > > > > >> > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak < > > dimaspi...@apache.org > > > >> > > wrote: > > > >> > > > > >> > > Unless told not to, HBase will always write to memory and append > > to > > > >> the > > > >> > WAL > > > >> > > on disk before returning and saying the write succeeded. That's > by > > > >> design > > > >> > > and the same write pattern that companies like Apple and > Facebook > > > have > > > >> > > found works for them at scale. So what's there to solve? > > > >> > > > > > >> > > On Sunday, October 23, 2016, Manjeet Singh < > > > >> manjeet.chand...@gmail.com > > > >> > > > > > >> > > wrote: > > > >> > > > > > >> > > > Hi All, > > > >> > > > > > > >> > > > I have read below mention blog and it also said Hbase holds > the > > > >> lock on > > > >> > > > rowkey level > > > >> > > > https://blogs.apache.org/hbase/entry/apache_hbase_ > > > >> > internals_locking_and > > > >> > > > (0) Obtain Row Lock > > > >> > > > (1) Write to Write-Ahead-Log (WAL) > > > >> > > > (2) Update MemStore: write each cell to the memstore > > > >> > > > (3) Release Row Lock > > > >> > > > > > > >> > > > > > > >> > > > SO question is how to solve this if I have very frequent > update > > on > > > >> > Hbase > > > >> > > > > > > >> > > > Thanks > > > >> > > > Manjeet > > > >> > > > > > > >> > > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh < > > > >> > > manjeet.chand...@gmail.com > > > >> > > > > > > > >> > > > wrote: > > > >> > > > > > > >> > > > > Hi All > > > >> > > > > > > > >> > > > > Can anyone help me about how and in which version of Hbase > > > support > > > >> > > Rowkey > > > >> > > > > lock ? > > > >> > > > > I have seen article about rowkey lock but it was about .94 > > > >> version it > > > >> > > > said > > > >> > > > > that if row key not exist and any update request come and > that > > > >> rowkey > > > >> > > not > > > >> > > > > exist then in this case Hbase hold the lock for 60 sec. > > > >> > > > > > > > >> > > > > currently I am using Hbase 1.2.2 version >
Re: Hbase Row key lock
Manjeet: Did you have a chance to get jstack during the lock contention period ? Cheers > On Oct 23, 2016, at 12:57 PM, Manjeet Singh > wrote: > > Anil all information are correct I am talking about suppose I didn't set > any version and I have very simple requirement to update if I found xyz > record and if I hv few ETL process which are responsible for aggregate the > data which is very common. ... why my hbase stuck if I try to update same > rowkey... its mean its hold the lock for few second > >> On 24 Oct 2016 00:46, "anil gupta" wrote: >> >> Writes within a HBase row are atomic. Now, whichever write becomes the >> latest write(with the help of timestamp value) will prevail as the default >> value. If you set versions to more than 1 in column family, then you will >> be able to see both the values if you query for multiple versions. >> >> HTH, >> Anil Gupta >> >> On Sun, Oct 23, 2016 at 12:02 PM, Manjeet Singh < >> manjeet.chand...@gmail.com> >> wrote: >> >>> Till now what i understand their is no update >>> >>> if two different thread try to update same record what happen >>> >>> first record insert with some version >>> second thread comes and change the version and its like a new insert with >>> some version >>> this process called MVCC >>> >>> If I am correct how hbase support MVCC mean any configuration for >> handlling >>> multiple thread at same time? >>> >>> On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh < >>> manjeet.chand...@gmail.com> >>> wrote: >>> No I don't have 50 clients? I want to understand internal working of >>> Hbase in my usecase I have bulk update operation from spark job we have 7 different kafka pipeline and 7 spark job it might happen that my 2 0r 3 spark job have same rowkey for update On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak wrote: > If your typical use case sees 50 clients simultaneously trying to >> update > the same row, then a strongly consistent data store that writes to >> disk > for > fault tolerance may not be for you. That said, such a use case seems > extremely unusual to me and I'd ask why you're trying to update the >> same > row in such a manner. > > On Sunday, October 23, 2016, Manjeet Singh < >> manjeet.chand...@gmail.com> > wrote: > >> Hi Dima, >> >> I didn't get ? point is assume I have 50 different client all having > same >> rowkey all want to update on same rowkey at same time now just tell >>> what >> will happen? who will get what value? >> >> Thanks >> Manjeet >> >> On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak < >> dimaspi...@apache.org >> > wrote: >> >>> Unless told not to, HBase will always write to memory and append >> to > the >> WAL >>> on disk before returning and saying the write succeeded. That's by > design >>> and the same write pattern that companies like Apple and Facebook >>> have >>> found works for them at scale. So what's there to solve? >>> >>> On Sunday, October 23, 2016, Manjeet Singh < > manjeet.chand...@gmail.com >> > >>> wrote: >>> Hi All, I have read below mention blog and it also said Hbase holds the > lock on rowkey level https://blogs.apache.org/hbase/entry/apache_hbase_ >> internals_locking_and (0) Obtain Row Lock (1) Write to Write-Ahead-Log (WAL) (2) Update MemStore: write each cell to the memstore (3) Release Row Lock SO question is how to solve this if I have very frequent update >> on >> Hbase Thanks Manjeet On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh < >>> manjeet.chand...@gmail.com > wrote: > Hi All > > Can anyone help me about how and in which version of Hbase >>> support >>> Rowkey > lock ? > I have seen article about rowkey lock but it was about .94 > version it said > that if row key not exist and any update request come and that > rowkey >>> not > exist then in this case Hbase hold the lock for 60 sec. > > currently I am using Hbase 1.2.2 version > > Thanks > Manjeet > > > > -- > luv all -- luv all >>> >>> >>> -- >>> -Dima >> >> >> >> -- >> luv all > > > -- > -Dima -- luv all >>> >>> >>> >>> -- >>> luv all >> >> >> >> -- >> Thanks & Regards, >> Anil Gupta >>
Re: Hbase Row key lock
Writes/Updates usually takes few milliseconds in HBase. So, in normal cases lock wont be held for seconds. On Sun, Oct 23, 2016 at 12:57 PM, Manjeet Singh wrote: > Anil all information are correct I am talking about suppose I didn't set > any version and I have very simple requirement to update if I found xyz > record and if I hv few ETL process which are responsible for aggregate the > data which is very common. ... why my hbase stuck if I try to update same > rowkey... its mean its hold the lock for few second > > On 24 Oct 2016 00:46, "anil gupta" wrote: > > > Writes within a HBase row are atomic. Now, whichever write becomes the > > latest write(with the help of timestamp value) will prevail as the > default > > value. If you set versions to more than 1 in column family, then you will > > be able to see both the values if you query for multiple versions. > > > > HTH, > > Anil Gupta > > > > On Sun, Oct 23, 2016 at 12:02 PM, Manjeet Singh < > > manjeet.chand...@gmail.com> > > wrote: > > > > > Till now what i understand their is no update > > > > > > if two different thread try to update same record what happen > > > > > > first record insert with some version > > > second thread comes and change the version and its like a new insert > with > > > some version > > > this process called MVCC > > > > > > If I am correct how hbase support MVCC mean any configuration for > > handlling > > > multiple thread at same time? > > > > > > On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh < > > > manjeet.chand...@gmail.com> > > > wrote: > > > > > > > No I don't have 50 clients? I want to understand internal working of > > > Hbase > > > > in my usecase I have bulk update operation from spark job we have 7 > > > > different kafka pipeline and 7 spark job > > > > it might happen that my 2 0r 3 spark job have same rowkey for update > > > > > > > > > > > > > > > > On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak > > > > > wrote: > > > > > > > >> If your typical use case sees 50 clients simultaneously trying to > > update > > > >> the same row, then a strongly consistent data store that writes to > > disk > > > >> for > > > >> fault tolerance may not be for you. That said, such a use case seems > > > >> extremely unusual to me and I'd ask why you're trying to update the > > same > > > >> row in such a manner. > > > >> > > > >> On Sunday, October 23, 2016, Manjeet Singh < > > manjeet.chand...@gmail.com> > > > >> wrote: > > > >> > > > >> > Hi Dima, > > > >> > > > > >> > I didn't get ? point is assume I have 50 different client all > having > > > >> same > > > >> > rowkey all want to update on same rowkey at same time now just > tell > > > what > > > >> > will happen? who will get what value? > > > >> > > > > >> > Thanks > > > >> > Manjeet > > > >> > > > > >> > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak < > > dimaspi...@apache.org > > > >> > > wrote: > > > >> > > > > >> > > Unless told not to, HBase will always write to memory and append > > to > > > >> the > > > >> > WAL > > > >> > > on disk before returning and saying the write succeeded. That's > by > > > >> design > > > >> > > and the same write pattern that companies like Apple and > Facebook > > > have > > > >> > > found works for them at scale. So what's there to solve? > > > >> > > > > > >> > > On Sunday, October 23, 2016, Manjeet Singh < > > > >> manjeet.chand...@gmail.com > > > >> > > > > > >> > > wrote: > > > >> > > > > > >> > > > Hi All, > > > >> > > > > > > >> > > > I have read below mention blog and it also said Hbase holds > the > > > >> lock on > > > >> > > > rowkey level > > > >> > > > https://blogs.apache.org/hbase/entry/apache_hbase_ > > > >> > internals_locking_and > > > >> > > > (0) Obtain Row Lock > > > >> > > > (1) Write to Write-Ahead-Log (WAL) > > > >> > > > (2) Update MemStore: write each cell to the memstore > > > >> > > > (3) Release Row Lock > > > >> > > > > > > >> > > > > > > >> > > > SO question is how to solve this if I have very frequent > update > > on > > > >> > Hbase > > > >> > > > > > > >> > > > Thanks > > > >> > > > Manjeet > > > >> > > > > > > >> > > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh < > > > >> > > manjeet.chand...@gmail.com > > > >> > > > > > > > >> > > > wrote: > > > >> > > > > > > >> > > > > Hi All > > > >> > > > > > > > >> > > > > Can anyone help me about how and in which version of Hbase > > > support > > > >> > > Rowkey > > > >> > > > > lock ? > > > >> > > > > I have seen article about rowkey lock but it was about .94 > > > >> version it > > > >> > > > said > > > >> > > > > that if row key not exist and any update request come and > that > > > >> rowkey > > > >> > > not > > > >> > > > > exist then in this case Hbase hold the lock for 60 sec. > > > >> > > > > > > > >> > > > > currently I am using Hbase 1.2.2 version > > > >> > > > > > > > >> > > > > Thanks > > > >> > > > > Manjeet > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > -- > > > >> > > > > luv all > > > >> > > > > > > > >> > > > > >
Re: Hbase Row key lock
Anil all information are correct I am talking about suppose I didn't set any version and I have very simple requirement to update if I found xyz record and if I hv few ETL process which are responsible for aggregate the data which is very common. ... why my hbase stuck if I try to update same rowkey... its mean its hold the lock for few second On 24 Oct 2016 00:46, "anil gupta" wrote: > Writes within a HBase row are atomic. Now, whichever write becomes the > latest write(with the help of timestamp value) will prevail as the default > value. If you set versions to more than 1 in column family, then you will > be able to see both the values if you query for multiple versions. > > HTH, > Anil Gupta > > On Sun, Oct 23, 2016 at 12:02 PM, Manjeet Singh < > manjeet.chand...@gmail.com> > wrote: > > > Till now what i understand their is no update > > > > if two different thread try to update same record what happen > > > > first record insert with some version > > second thread comes and change the version and its like a new insert with > > some version > > this process called MVCC > > > > If I am correct how hbase support MVCC mean any configuration for > handlling > > multiple thread at same time? > > > > On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh < > > manjeet.chand...@gmail.com> > > wrote: > > > > > No I don't have 50 clients? I want to understand internal working of > > Hbase > > > in my usecase I have bulk update operation from spark job we have 7 > > > different kafka pipeline and 7 spark job > > > it might happen that my 2 0r 3 spark job have same rowkey for update > > > > > > > > > > > > On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak > > > wrote: > > > > > >> If your typical use case sees 50 clients simultaneously trying to > update > > >> the same row, then a strongly consistent data store that writes to > disk > > >> for > > >> fault tolerance may not be for you. That said, such a use case seems > > >> extremely unusual to me and I'd ask why you're trying to update the > same > > >> row in such a manner. > > >> > > >> On Sunday, October 23, 2016, Manjeet Singh < > manjeet.chand...@gmail.com> > > >> wrote: > > >> > > >> > Hi Dima, > > >> > > > >> > I didn't get ? point is assume I have 50 different client all having > > >> same > > >> > rowkey all want to update on same rowkey at same time now just tell > > what > > >> > will happen? who will get what value? > > >> > > > >> > Thanks > > >> > Manjeet > > >> > > > >> > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak < > dimaspi...@apache.org > > >> > > wrote: > > >> > > > >> > > Unless told not to, HBase will always write to memory and append > to > > >> the > > >> > WAL > > >> > > on disk before returning and saying the write succeeded. That's by > > >> design > > >> > > and the same write pattern that companies like Apple and Facebook > > have > > >> > > found works for them at scale. So what's there to solve? > > >> > > > > >> > > On Sunday, October 23, 2016, Manjeet Singh < > > >> manjeet.chand...@gmail.com > > >> > > > > >> > > wrote: > > >> > > > > >> > > > Hi All, > > >> > > > > > >> > > > I have read below mention blog and it also said Hbase holds the > > >> lock on > > >> > > > rowkey level > > >> > > > https://blogs.apache.org/hbase/entry/apache_hbase_ > > >> > internals_locking_and > > >> > > > (0) Obtain Row Lock > > >> > > > (1) Write to Write-Ahead-Log (WAL) > > >> > > > (2) Update MemStore: write each cell to the memstore > > >> > > > (3) Release Row Lock > > >> > > > > > >> > > > > > >> > > > SO question is how to solve this if I have very frequent update > on > > >> > Hbase > > >> > > > > > >> > > > Thanks > > >> > > > Manjeet > > >> > > > > > >> > > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh < > > >> > > manjeet.chand...@gmail.com > > >> > > > > > > >> > > > wrote: > > >> > > > > > >> > > > > Hi All > > >> > > > > > > >> > > > > Can anyone help me about how and in which version of Hbase > > support > > >> > > Rowkey > > >> > > > > lock ? > > >> > > > > I have seen article about rowkey lock but it was about .94 > > >> version it > > >> > > > said > > >> > > > > that if row key not exist and any update request come and that > > >> rowkey > > >> > > not > > >> > > > > exist then in this case Hbase hold the lock for 60 sec. > > >> > > > > > > >> > > > > currently I am using Hbase 1.2.2 version > > >> > > > > > > >> > > > > Thanks > > >> > > > > Manjeet > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > -- > > >> > > > > luv all > > >> > > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > -- > > >> > > > luv all > > >> > > > > > >> > > > > >> > > > > >> > > -- > > >> > > -Dima > > >> > > > > >> > > > >> > > > >> > > > >> > -- > > >> > luv all > > >> > > > >> > > >> > > >> -- > > >> -Dima > > >> > > > > > > > > > > > > -- > > > luv all > > > > > > > > > > > -- > > luv all > > > > > > -- > Thanks & Regards, > Anil Gupta >
Re: Hbase Row key lock
Writes within a HBase row are atomic. Now, whichever write becomes the latest write(with the help of timestamp value) will prevail as the default value. If you set versions to more than 1 in column family, then you will be able to see both the values if you query for multiple versions. HTH, Anil Gupta On Sun, Oct 23, 2016 at 12:02 PM, Manjeet Singh wrote: > Till now what i understand their is no update > > if two different thread try to update same record what happen > > first record insert with some version > second thread comes and change the version and its like a new insert with > some version > this process called MVCC > > If I am correct how hbase support MVCC mean any configuration for handlling > multiple thread at same time? > > On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh < > manjeet.chand...@gmail.com> > wrote: > > > No I don't have 50 clients? I want to understand internal working of > Hbase > > in my usecase I have bulk update operation from spark job we have 7 > > different kafka pipeline and 7 spark job > > it might happen that my 2 0r 3 spark job have same rowkey for update > > > > > > > > On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak > > wrote: > > > >> If your typical use case sees 50 clients simultaneously trying to update > >> the same row, then a strongly consistent data store that writes to disk > >> for > >> fault tolerance may not be for you. That said, such a use case seems > >> extremely unusual to me and I'd ask why you're trying to update the same > >> row in such a manner. > >> > >> On Sunday, October 23, 2016, Manjeet Singh > >> wrote: > >> > >> > Hi Dima, > >> > > >> > I didn't get ? point is assume I have 50 different client all having > >> same > >> > rowkey all want to update on same rowkey at same time now just tell > what > >> > will happen? who will get what value? > >> > > >> > Thanks > >> > Manjeet > >> > > >> > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak >> > > wrote: > >> > > >> > > Unless told not to, HBase will always write to memory and append to > >> the > >> > WAL > >> > > on disk before returning and saying the write succeeded. That's by > >> design > >> > > and the same write pattern that companies like Apple and Facebook > have > >> > > found works for them at scale. So what's there to solve? > >> > > > >> > > On Sunday, October 23, 2016, Manjeet Singh < > >> manjeet.chand...@gmail.com > >> > > > >> > > wrote: > >> > > > >> > > > Hi All, > >> > > > > >> > > > I have read below mention blog and it also said Hbase holds the > >> lock on > >> > > > rowkey level > >> > > > https://blogs.apache.org/hbase/entry/apache_hbase_ > >> > internals_locking_and > >> > > > (0) Obtain Row Lock > >> > > > (1) Write to Write-Ahead-Log (WAL) > >> > > > (2) Update MemStore: write each cell to the memstore > >> > > > (3) Release Row Lock > >> > > > > >> > > > > >> > > > SO question is how to solve this if I have very frequent update on > >> > Hbase > >> > > > > >> > > > Thanks > >> > > > Manjeet > >> > > > > >> > > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh < > >> > > manjeet.chand...@gmail.com > >> > > > > > >> > > > wrote: > >> > > > > >> > > > > Hi All > >> > > > > > >> > > > > Can anyone help me about how and in which version of Hbase > support > >> > > Rowkey > >> > > > > lock ? > >> > > > > I have seen article about rowkey lock but it was about .94 > >> version it > >> > > > said > >> > > > > that if row key not exist and any update request come and that > >> rowkey > >> > > not > >> > > > > exist then in this case Hbase hold the lock for 60 sec. > >> > > > > > >> > > > > currently I am using Hbase 1.2.2 version > >> > > > > > >> > > > > Thanks > >> > > > > Manjeet > >> > > > > > >> > > > > > >> > > > > > >> > > > > -- > >> > > > > luv all > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > -- > >> > > > luv all > >> > > > > >> > > > >> > > > >> > > -- > >> > > -Dima > >> > > > >> > > >> > > >> > > >> > -- > >> > luv all > >> > > >> > >> > >> -- > >> -Dima > >> > > > > > > > > -- > > luv all > > > > > > -- > luv all > -- Thanks & Regards, Anil Gupta
Re: Hbase Row key lock
Till now what i understand their is no update if two different thread try to update same record what happen first record insert with some version second thread comes and change the version and its like a new insert with some version this process called MVCC If I am correct how hbase support MVCC mean any configuration for handlling multiple thread at same time? On Mon, Oct 24, 2016 at 12:24 AM, Manjeet Singh wrote: > No I don't have 50 clients? I want to understand internal working of Hbase > in my usecase I have bulk update operation from spark job we have 7 > different kafka pipeline and 7 spark job > it might happen that my 2 0r 3 spark job have same rowkey for update > > > > On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak > wrote: > >> If your typical use case sees 50 clients simultaneously trying to update >> the same row, then a strongly consistent data store that writes to disk >> for >> fault tolerance may not be for you. That said, such a use case seems >> extremely unusual to me and I'd ask why you're trying to update the same >> row in such a manner. >> >> On Sunday, October 23, 2016, Manjeet Singh >> wrote: >> >> > Hi Dima, >> > >> > I didn't get ? point is assume I have 50 different client all having >> same >> > rowkey all want to update on same rowkey at same time now just tell what >> > will happen? who will get what value? >> > >> > Thanks >> > Manjeet >> > >> > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak > > > wrote: >> > >> > > Unless told not to, HBase will always write to memory and append to >> the >> > WAL >> > > on disk before returning and saying the write succeeded. That's by >> design >> > > and the same write pattern that companies like Apple and Facebook have >> > > found works for them at scale. So what's there to solve? >> > > >> > > On Sunday, October 23, 2016, Manjeet Singh < >> manjeet.chand...@gmail.com >> > > >> > > wrote: >> > > >> > > > Hi All, >> > > > >> > > > I have read below mention blog and it also said Hbase holds the >> lock on >> > > > rowkey level >> > > > https://blogs.apache.org/hbase/entry/apache_hbase_ >> > internals_locking_and >> > > > (0) Obtain Row Lock >> > > > (1) Write to Write-Ahead-Log (WAL) >> > > > (2) Update MemStore: write each cell to the memstore >> > > > (3) Release Row Lock >> > > > >> > > > >> > > > SO question is how to solve this if I have very frequent update on >> > Hbase >> > > > >> > > > Thanks >> > > > Manjeet >> > > > >> > > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh < >> > > manjeet.chand...@gmail.com >> > > > > >> > > > wrote: >> > > > >> > > > > Hi All >> > > > > >> > > > > Can anyone help me about how and in which version of Hbase support >> > > Rowkey >> > > > > lock ? >> > > > > I have seen article about rowkey lock but it was about .94 >> version it >> > > > said >> > > > > that if row key not exist and any update request come and that >> rowkey >> > > not >> > > > > exist then in this case Hbase hold the lock for 60 sec. >> > > > > >> > > > > currently I am using Hbase 1.2.2 version >> > > > > >> > > > > Thanks >> > > > > Manjeet >> > > > > >> > > > > >> > > > > >> > > > > -- >> > > > > luv all >> > > > > >> > > > >> > > > >> > > > >> > > > -- >> > > > luv all >> > > > >> > > >> > > >> > > -- >> > > -Dima >> > > >> > >> > >> > >> > -- >> > luv all >> > >> >> >> -- >> -Dima >> > > > > -- > luv all > -- luv all
Re: Hbase Row key lock
No I don't have 50 clients? I want to understand internal working of Hbase in my usecase I have bulk update operation from spark job we have 7 different kafka pipeline and 7 spark job it might happen that my 2 0r 3 spark job have same rowkey for update On Mon, Oct 24, 2016 at 12:20 AM, Dima Spivak wrote: > If your typical use case sees 50 clients simultaneously trying to update > the same row, then a strongly consistent data store that writes to disk for > fault tolerance may not be for you. That said, such a use case seems > extremely unusual to me and I'd ask why you're trying to update the same > row in such a manner. > > On Sunday, October 23, 2016, Manjeet Singh > wrote: > > > Hi Dima, > > > > I didn't get ? point is assume I have 50 different client all having same > > rowkey all want to update on same rowkey at same time now just tell what > > will happen? who will get what value? > > > > Thanks > > Manjeet > > > > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak > > wrote: > > > > > Unless told not to, HBase will always write to memory and append to the > > WAL > > > on disk before returning and saying the write succeeded. That's by > design > > > and the same write pattern that companies like Apple and Facebook have > > > found works for them at scale. So what's there to solve? > > > > > > On Sunday, October 23, 2016, Manjeet Singh > > > > > wrote: > > > > > > > Hi All, > > > > > > > > I have read below mention blog and it also said Hbase holds the lock > on > > > > rowkey level > > > > https://blogs.apache.org/hbase/entry/apache_hbase_ > > internals_locking_and > > > > (0) Obtain Row Lock > > > > (1) Write to Write-Ahead-Log (WAL) > > > > (2) Update MemStore: write each cell to the memstore > > > > (3) Release Row Lock > > > > > > > > > > > > SO question is how to solve this if I have very frequent update on > > Hbase > > > > > > > > Thanks > > > > Manjeet > > > > > > > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh < > > > manjeet.chand...@gmail.com > > > > > > > > > wrote: > > > > > > > > > Hi All > > > > > > > > > > Can anyone help me about how and in which version of Hbase support > > > Rowkey > > > > > lock ? > > > > > I have seen article about rowkey lock but it was about .94 version > it > > > > said > > > > > that if row key not exist and any update request come and that > rowkey > > > not > > > > > exist then in this case Hbase hold the lock for 60 sec. > > > > > > > > > > currently I am using Hbase 1.2.2 version > > > > > > > > > > Thanks > > > > > Manjeet > > > > > > > > > > > > > > > > > > > > -- > > > > > luv all > > > > > > > > > > > > > > > > > > > > > -- > > > > luv all > > > > > > > > > > > > > -- > > > -Dima > > > > > > > > > > > -- > > luv all > > > > > -- > -Dima > -- luv all
Re: Hbase Row key lock
If your typical use case sees 50 clients simultaneously trying to update the same row, then a strongly consistent data store that writes to disk for fault tolerance may not be for you. That said, such a use case seems extremely unusual to me and I'd ask why you're trying to update the same row in such a manner. On Sunday, October 23, 2016, Manjeet Singh wrote: > Hi Dima, > > I didn't get ? point is assume I have 50 different client all having same > rowkey all want to update on same rowkey at same time now just tell what > will happen? who will get what value? > > Thanks > Manjeet > > On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak > wrote: > > > Unless told not to, HBase will always write to memory and append to the > WAL > > on disk before returning and saying the write succeeded. That's by design > > and the same write pattern that companies like Apple and Facebook have > > found works for them at scale. So what's there to solve? > > > > On Sunday, October 23, 2016, Manjeet Singh > > > wrote: > > > > > Hi All, > > > > > > I have read below mention blog and it also said Hbase holds the lock on > > > rowkey level > > > https://blogs.apache.org/hbase/entry/apache_hbase_ > internals_locking_and > > > (0) Obtain Row Lock > > > (1) Write to Write-Ahead-Log (WAL) > > > (2) Update MemStore: write each cell to the memstore > > > (3) Release Row Lock > > > > > > > > > SO question is how to solve this if I have very frequent update on > Hbase > > > > > > Thanks > > > Manjeet > > > > > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh < > > manjeet.chand...@gmail.com > > > > > > > wrote: > > > > > > > Hi All > > > > > > > > Can anyone help me about how and in which version of Hbase support > > Rowkey > > > > lock ? > > > > I have seen article about rowkey lock but it was about .94 version it > > > said > > > > that if row key not exist and any update request come and that rowkey > > not > > > > exist then in this case Hbase hold the lock for 60 sec. > > > > > > > > currently I am using Hbase 1.2.2 version > > > > > > > > Thanks > > > > Manjeet > > > > > > > > > > > > > > > > -- > > > > luv all > > > > > > > > > > > > > > > > -- > > > luv all > > > > > > > > > -- > > -Dima > > > > > > -- > luv all > -- -Dima
Re: Hbase Row key lock
Hi Dima, I didn't get ? point is assume I have 50 different client all having same rowkey all want to update on same rowkey at same time now just tell what will happen? who will get what value? Thanks Manjeet On Mon, Oct 24, 2016 at 12:12 AM, Dima Spivak wrote: > Unless told not to, HBase will always write to memory and append to the WAL > on disk before returning and saying the write succeeded. That's by design > and the same write pattern that companies like Apple and Facebook have > found works for them at scale. So what's there to solve? > > On Sunday, October 23, 2016, Manjeet Singh > wrote: > > > Hi All, > > > > I have read below mention blog and it also said Hbase holds the lock on > > rowkey level > > https://blogs.apache.org/hbase/entry/apache_hbase_internals_locking_and > > (0) Obtain Row Lock > > (1) Write to Write-Ahead-Log (WAL) > > (2) Update MemStore: write each cell to the memstore > > (3) Release Row Lock > > > > > > SO question is how to solve this if I have very frequent update on Hbase > > > > Thanks > > Manjeet > > > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh < > manjeet.chand...@gmail.com > > > > > wrote: > > > > > Hi All > > > > > > Can anyone help me about how and in which version of Hbase support > Rowkey > > > lock ? > > > I have seen article about rowkey lock but it was about .94 version it > > said > > > that if row key not exist and any update request come and that rowkey > not > > > exist then in this case Hbase hold the lock for 60 sec. > > > > > > currently I am using Hbase 1.2.2 version > > > > > > Thanks > > > Manjeet > > > > > > > > > > > > -- > > > luv all > > > > > > > > > > > -- > > luv all > > > > > -- > -Dima > -- luv all
Re: Hbase Row key lock
Unless told not to, HBase will always write to memory and append to the WAL on disk before returning and saying the write succeeded. That's by design and the same write pattern that companies like Apple and Facebook have found works for them at scale. So what's there to solve? On Sunday, October 23, 2016, Manjeet Singh wrote: > Hi All, > > I have read below mention blog and it also said Hbase holds the lock on > rowkey level > https://blogs.apache.org/hbase/entry/apache_hbase_internals_locking_and > (0) Obtain Row Lock > (1) Write to Write-Ahead-Log (WAL) > (2) Update MemStore: write each cell to the memstore > (3) Release Row Lock > > > SO question is how to solve this if I have very frequent update on Hbase > > Thanks > Manjeet > > On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh > > wrote: > > > Hi All > > > > Can anyone help me about how and in which version of Hbase support Rowkey > > lock ? > > I have seen article about rowkey lock but it was about .94 version it > said > > that if row key not exist and any update request come and that rowkey not > > exist then in this case Hbase hold the lock for 60 sec. > > > > currently I am using Hbase 1.2.2 version > > > > Thanks > > Manjeet > > > > > > > > -- > > luv all > > > > > > -- > luv all > -- -Dima
Re: Hbase Row key lock
Hi All, I have read below mention blog and it also said Hbase holds the lock on rowkey level https://blogs.apache.org/hbase/entry/apache_hbase_internals_locking_and (0) Obtain Row Lock (1) Write to Write-Ahead-Log (WAL) (2) Update MemStore: write each cell to the memstore (3) Release Row Lock SO question is how to solve this if I have very frequent update on Hbase Thanks Manjeet On Wed, Aug 17, 2016 at 9:54 AM, Manjeet Singh wrote: > Hi All > > Can anyone help me about how and in which version of Hbase support Rowkey > lock ? > I have seen article about rowkey lock but it was about .94 version it said > that if row key not exist and any update request come and that rowkey not > exist then in this case Hbase hold the lock for 60 sec. > > currently I am using Hbase 1.2.2 version > > Thanks > Manjeet > > > > -- > luv all > -- luv all
Re: Hbase Row key lock
HBase uses MVCC to deal with concurrent writes, this is a good explanation: https://blogs.apache.org/hbase/entry/apache_hbase_internals_locking_and -- Iain Wright This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged, exempt from disclosure under applicable law. If you are not the intended recipient, do not disclose or disseminate the message to anyone except the intended recipient. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender by return email, and delete all copies of this message. On Tue, Aug 23, 2016 at 9:59 AM, Manjeet Singh wrote: > Hi All, > In Hbase current version what happen if two different client write the same > rowkey at same time in hbase > Assume if its kind of update. > Does Rowkey lock Hbase server automatically deal with this? > > > Thanks > Manjeet > > On Wed, Aug 17, 2016 at 1:04 PM, Dima Spivak wrote: > > > Row locks on the client side were deprecated in 0.94 (see HBASE-7341) and > > removed in 0.96 (see HBASE-7315). As you note, they could lead to > deadlocks > > and also had problems when region moves or splits occurred. > > > > Is there a specific reason you're looking for this functionality, > Manjeet? > > > > -Dima > > > > On Tuesday, August 16, 2016, Manjeet Singh > > wrote: > > > > > Hi All > > > > > > Can anyone help me about how and in which version of Hbase support > Rowkey > > > lock ? > > > I have seen article about rowkey lock but it was about .94 version it > > said > > > that if row key not exist and any update request come and that rowkey > not > > > exist then in this case Hbase hold the lock for 60 sec. > > > > > > currently I am using Hbase 1.2.2 version > > > > > > Thanks > > > Manjeet > > > > > > > > > > > > -- > > > luv all > > > > > > > > > -- > > -Dima > > > > > > -- > luv all >
Re: Hbase Row key lock
Hi All, In Hbase current version what happen if two different client write the same rowkey at same time in hbase Assume if its kind of update. Does Rowkey lock Hbase server automatically deal with this? Thanks Manjeet On Wed, Aug 17, 2016 at 1:04 PM, Dima Spivak wrote: > Row locks on the client side were deprecated in 0.94 (see HBASE-7341) and > removed in 0.96 (see HBASE-7315). As you note, they could lead to deadlocks > and also had problems when region moves or splits occurred. > > Is there a specific reason you're looking for this functionality, Manjeet? > > -Dima > > On Tuesday, August 16, 2016, Manjeet Singh > wrote: > > > Hi All > > > > Can anyone help me about how and in which version of Hbase support Rowkey > > lock ? > > I have seen article about rowkey lock but it was about .94 version it > said > > that if row key not exist and any update request come and that rowkey not > > exist then in this case Hbase hold the lock for 60 sec. > > > > currently I am using Hbase 1.2.2 version > > > > Thanks > > Manjeet > > > > > > > > -- > > luv all > > > > > -- > -Dima > -- luv all
Re: Hbase Row key lock
Hi Manjeet, Then have you looked at CheckandPut and the others ChecAnd* calls? JMS 2016-08-17 5:08 GMT-04:00 Manjeet Singh : > Hi Dima > > Thanks for your reply its useful for me, My concern is that I have very > frequent get/put opration using spark Hbase connector so i wan tto make > sure if > my any rowkey was in progress for update and another client try to update > it will not impact, mean I want to know if hbase deal on serverside rowkey > level locking > > Thanks > Manjeet > > On Wed, Aug 17, 2016 at 1:04 PM, Dima Spivak wrote: > > > Row locks on the client side were deprecated in 0.94 (see HBASE-7341) and > > removed in 0.96 (see HBASE-7315). As you note, they could lead to > deadlocks > > and also had problems when region moves or splits occurred. > > > > Is there a specific reason you're looking for this functionality, > Manjeet? > > > > -Dima > > > > On Tuesday, August 16, 2016, Manjeet Singh > > wrote: > > > > > Hi All > > > > > > Can anyone help me about how and in which version of Hbase support > Rowkey > > > lock ? > > > I have seen article about rowkey lock but it was about .94 version it > > said > > > that if row key not exist and any update request come and that rowkey > not > > > exist then in this case Hbase hold the lock for 60 sec. > > > > > > currently I am using Hbase 1.2.2 version > > > > > > Thanks > > > Manjeet > > > > > > > > > > > > -- > > > luv all > > > > > > > > > -- > > -Dima > > > > > > -- > luv all >
Re: Hbase Row key lock
Hi Dima Thanks for your reply its useful for me, My concern is that I have very frequent get/put opration using spark Hbase connector so i wan tto make sure if my any rowkey was in progress for update and another client try to update it will not impact, mean I want to know if hbase deal on serverside rowkey level locking Thanks Manjeet On Wed, Aug 17, 2016 at 1:04 PM, Dima Spivak wrote: > Row locks on the client side were deprecated in 0.94 (see HBASE-7341) and > removed in 0.96 (see HBASE-7315). As you note, they could lead to deadlocks > and also had problems when region moves or splits occurred. > > Is there a specific reason you're looking for this functionality, Manjeet? > > -Dima > > On Tuesday, August 16, 2016, Manjeet Singh > wrote: > > > Hi All > > > > Can anyone help me about how and in which version of Hbase support Rowkey > > lock ? > > I have seen article about rowkey lock but it was about .94 version it > said > > that if row key not exist and any update request come and that rowkey not > > exist then in this case Hbase hold the lock for 60 sec. > > > > currently I am using Hbase 1.2.2 version > > > > Thanks > > Manjeet > > > > > > > > -- > > luv all > > > > > -- > -Dima > -- luv all
Re: Hbase Row key lock
Row locks on the client side were deprecated in 0.94 (see HBASE-7341) and removed in 0.96 (see HBASE-7315). As you note, they could lead to deadlocks and also had problems when region moves or splits occurred. Is there a specific reason you're looking for this functionality, Manjeet? -Dima On Tuesday, August 16, 2016, Manjeet Singh wrote: > Hi All > > Can anyone help me about how and in which version of Hbase support Rowkey > lock ? > I have seen article about rowkey lock but it was about .94 version it said > that if row key not exist and any update request come and that rowkey not > exist then in this case Hbase hold the lock for 60 sec. > > currently I am using Hbase 1.2.2 version > > Thanks > Manjeet > > > > -- > luv all > -- -Dima
Hbase Row key lock
Hi All Can anyone help me about how and in which version of Hbase support Rowkey lock ? I have seen article about rowkey lock but it was about .94 version it said that if row key not exist and any update request come and that rowkey not exist then in this case Hbase hold the lock for 60 sec. currently I am using Hbase 1.2.2 version Thanks Manjeet -- luv all
