having to
> contact each region (or at least each region server).
>
>
> -- Lars
>
>
>
>
> From: Michael Segel
> To: user@hbase.apache.org
> Sent: Tuesday, January 8, 2013 6:33 AM
> Subject: Re: HBase - Secondary Index
>
&g
changes needed at client side. :)
>>
>> As Anil said there will be pros and cons for every way and which one suits
>> your usage, needs to be adopted. :)
>>
>> -Anoop-
>>
>> From: anil gupta [anilgupt...@gmail.com
be pros and cons for every way and which one suits
> your usage, needs to be adopted. :)
>
> -Anoop-
>
> From: anil gupta [anilgupt...@gmail.com]
> Sent: Wednesday, January 09, 2013 6:58 AM
> To: user@hbase.apache.org; lars hofhansl
> Su
d there will be pros and cons for every way and which one suits
> your usage, needs to be adopted. :)
>
> -Anoop-
>
> From: anil gupta [anilgupt...@gmail.com]
> Sent: Wednesday, January 09, 2013 6:58 AM
> To: user@hbase.apache.org; lars
t; > Hi,
> > It is inverted index based on column(s) value(s)
> > It will be region wise indexing. Can work when some one knows the rowkey
> range or NOT.
> >
> > -Anoop-
> >
> > From: Mohit Anchlia [mohitanch..
t; >
> > ________
> > From: Michael Segel
> > To: user@hbase.apache.org
> > Sent: Tuesday, January 8, 2013 6:33 AM
> > Subject: Re: HBase - Secondary Index
> >
> > So if you're using an inverted table / index why on earth are
Michael Segel
> To: user@hbase.apache.org
> Sent: Tuesday, January 8, 2013 6:33 AM
> Subject: Re: HBase - Secondary Index
>
> So if you're using an inverted table / index why on earth are you doing it at
> the region level?
>
> I've tried to explain this to ot
Michael Segel
> To: user@hbase.apache.org
> Sent: Tuesday, January 8, 2013 6:33 AM
> Subject: Re: HBase - Secondary Index
>
> So if you're using an inverted table / index why on earth are you doing it at
> the region level?
>
> I've tried to explain this to ot
Hi,
> > It is inverted index based on column(s) value(s)
> > It will be region wise indexing. Can work when some one knows the rowkey
> range or NOT.
> >
> > -Anoop-
> >
> > From: Mohit Anchlia [mohitanch...@gmail.com]
&g
egion server).
-- Lars
From: Michael Segel
To: user@hbase.apache.org
Sent: Tuesday, January 8, 2013 6:33 AM
Subject: Re: HBase - Secondary Index
So if you're using an inverted table / index why on earth are you doing it at
the region level?
I've t
ote:
>
> > Hi,
> > It is inverted index based on column(s) value(s)
> > It will be region wise indexing. Can work when some one knows the rowkey
> range or NOT.
> >
> > -Anoop-
> > ____
> > From: Mohit Anchlia [mohit
nchlia [mohitanch...@gmail.com]
> Sent: Monday, January 07, 2013 9:47 AM
> To: user@hbase.apache.org
> Subject: Re: HBase - Secondary Index
>
> Hi Anoop,
>
> Am I correct in understanding that this indexing mechanism is only
> applicable when you know the row key? It's
@hbase.apache.org
Subject: Re: HBase - Secondary Index
Hi Anoop,
Am I correct in understanding that this indexing mechanism is only
applicable when you know the row key? It's not an inverted index truly
based on the column value.
Mohit
On Sun, Jan 6, 2013 at 7:48 PM, Anoop Sam John wrote:
> H
e to explain in details..
>
> -Anoop-
>
> From: Adrien Mogenet [adrien.moge...@gmail.com]
> Sent: Monday, January 07, 2013 2:00 AM
> To: user@hbase.apache.org
> Subject: Re: HBase - Secondary Index
>
> Nice topic, perhaps one of the m
all are done I will
be able to explain in details..
-Anoop-
From: Adrien Mogenet [adrien.moge...@gmail.com]
Sent: Monday, January 07, 2013 2:00 AM
To: user@hbase.apache.org
Subject: Re: HBase - Secondary Index
Nice topic, perhaps one of the most important
; performing as per my study :)
> > > >
> > > > yes, you are right, I guess it's just a drawback of any index
> approach.
> > > > Thanks for the explanation.
> > > >
> > > > Shengjie
> > > >
> > > > On 28 December 2012 04:14, Anoop Sam John
> wrote:
> > &
gt; > >
> > > On 28 December 2012 04:14, Anoop Sam John wrote:
> > >
> > > > > Do you have link to that presentation?
> > > >
> > > > http://hbtc2012.hadooper.cn/subject/track4TedYu4.pdf
> > > >
> > > > -Anoop-
> &g
;
> > On 28 December 2012 04:14, Anoop Sam John wrote:
> >
> > > > Do you have link to that presentation?
> > >
> > > http://hbtc2012.hadooper.cn/subject/track4TedYu4.pdf
> > >
> > > -Anoop-
> > >
> > >
> > > From: Mohit Anchl
e link to that presentation?
> >
> > http://hbtc2012.hadooper.cn/subject/track4TedYu4.pdf
> >
> > -Anoop-
> >
> > ____
> > From: Mohit Anchlia [mohitanch...@gmail.com]
> > Sent: Friday, December 28, 2012 9:12 AM
> > To: us
rack4TedYu4.pdf
>
> -Anoop-
>
>
> From: Mohit Anchlia [mohitanch...@gmail.com]
> Sent: Friday, December 28, 2012 9:12 AM
> To: user@hbase.apache.org
> Subject: Re: HBase - Secondary Index
>
> On Thu, Dec 27, 2012 at 7:33 PM, Anoop Sam John
> Do you have link to that presentation?
http://hbtc2012.hadooper.cn/subject/track4TedYu4.pdf
-Anoop-
From: Mohit Anchlia [mohitanch...@gmail.com]
Sent: Friday, December 28, 2012 9:12 AM
To: user@hbase.apache.org
Subject: Re: HBase - Secondary Index
rom: Shengjie Min [kelvin@gmail.com]
Sent: Thursday, December 27, 2012 9:59 PM
To: user@hbase.apache.org
Subject: Re: HBase - Secondary Index
>Didnt follow u completely here. There wont be any get() happening.. As the
>exact rowkey in a region we get from the index table, we can seek to t
Dont consider the exact values. What is the % of increase
> in latency is important :) Those were not high end machines.
>
> -Anoop-
>
> From: Shengjie Min [kelvin@gmail.com]
> Sent: Thursday, December 27, 2012 9:59 PM
> To: user@hbase.a
>Didnt follow u completely here. There wont be any get() happening.. As the
>exact rowkey in a region we get from the index table, we can seek to the
>exact position and return that row.
Sorry, When I misused "get()" here, I meant seeking. Yes, if it's just
small number of rows returned, this work
As per the design there is no get() operation at all. Incase of equals
query nothing is cached in memory.
For Range may be we need to cache some intermediate result.
Regards
Ram
On Thu, Dec 27, 2012 at 9:24 PM, Anoop John wrote:
> >how the massive number of get() is going to
> perform againt t
>how the massive number of get() is going to
perform againt the main table
Didnt follow u completely here. There wont be any get() happening.. As the
exact rowkey in a region we get from the index table, we can seek to the
exact position and return that row.
-Anoop-
On Thu, Dec 27, 2012 at 6:37
.. :) Hope I make it
> clear for you.
>
> -Anoop-
>
> From: Shengjie Min [kelvin@gmail.com]
> Sent: Thursday, December 27, 2012 4:53 PM
> To: user@hbase.apache.org
> Subject: Re: HBase - Secondary Index
>
> Hi Anoop,
>
>
mat.. :) Hope I make it clear for you.
-Anoop-
From: Shengjie Min [kelvin@gmail.com]
Sent: Thursday, December 27, 2012 4:53 PM
To: user@hbase.apache.org
Subject: Re: HBase - Secondary Index
Hi Anoop,
>First all there will be same number of regi
src the solution yet.
> Any particular query you come acorss pls feel free to aske me :)
> You can see the HalfStoreFileReader class 1st..
>
> -Anoop-
>
> From: anil gupta [anilgupt...@gmail.com]
> Sent: Friday, December 14, 201
le due to Christmas
> & New year holidays :)
>
> -Anoop-
>
> From: Andrew Purtell [apurt...@apache.org]
> Sent: Wednesday, December 19, 2012 6:21 AM
> To: user@hbase.apache.org
> Cc: d...@hbase.apache.org
> Subject: Re: HBase - Sec
David
Not using any existing library like Lucene. The index data of a table will
be written in another HBase table.
-Anoop-
From: David Arthur [mum...@gmail.com]
Sent: Thursday, December 20, 2012 8:17 AM
To: user@hbase.apache.org
Subject: Re: HBase
Sent: Wednesday, December 19, 2012 6:21 AM
To: user@hbase.apache.org
Cc: d...@hbase.apache.org
Subject: Re: HBase - Secondary Index
Hi Anoop,
What Nick asked. I've also heard people wonder this out loud in a few
places.
On Tue, Dec 18, 2012 at 9:48 AM, Nick Dimiduk wrote:
> Hi Anoop,
>
>
Very cool design. Just curious, for the index did you write something
custom or using an existing library like Lucene?
-David
On 12/4/12 3:10 AM, Anoop Sam John wrote:
Hi All
Last week I got a chance to present the secondary indexing
solution what we have done in Huawei at the C
nt be having customer id for all the queries. Hence i
> >> cannot use customer_id as a prefix in rowkey of my Secondary Table.
> >>
> >>>
> >>> I feel your use case perfectly fit with our model
> >> Anil: Somehow i am unable to fit your implementation into my use case
> due
> >> to the
I have shared, there also
> it is saying the same thing. It is showing what is happening at the server
> side.
>
> -Anoop-
>
> ____________
> From: anil gupta [anilgupt...@gmail.com]
> Sent: Tuesday, December 18, 2012 1:58 PM
> To: user@hbase.apache.org
> Subject: Re: HBase
Hi Anoop,
What Nick asked. I've also heard people wonder this out loud in a few
places.
On Tue, Dec 18, 2012 at 9:48 AM, Nick Dimiduk wrote:
> Hi Anoop,
>
> Your presentation has garnered quite a bit of community interest. Have you
> considered providing your implementation to the community, p
Hi Anoop,
Your presentation has garnered quite a bit of community interest. Have you
considered providing your implementation to the community, perhaps in an
HBase-contrib module?
Thanks,
Nick
On Tue, Dec 4, 2012 at 12:10 AM, Anoop Sam John wrote:
> Hi All
>
> Last week I got a cha
gets to get the main table data. Correct Anil?
Just making it clear... :)
-Anoop-
From: Michel Segel [michael_se...@hotmail.com]
Sent: Tuesday, December 18, 2012 2:32 PM
To: user@hbase.apache.org
Cc: user@hbase.apache.org
Subject: Re: HBase - Secondary In
u got it now :)
>
Anil: I hope now we are on same page. Thanks a lot for your valuable time
to discuss this stuff.
>
> -Anoop-
> ________
> From: anil gupta [anilgupt...@gmail.com]
> Sent: Friday, December 14, 2012 11:31 PM
> To: user@hbase.
e scan
>> object and set startRow on that it need not be the full rowkey. It can be
>> part of the rowkey also. Yes like prefix.
>>
>> Hope u got it now :)
> Anil: I hope now we are on same page. Thanks a lot for your valuable time
> to discuss this stuff.
>
>>
>> -An
u got it now :)
>
Anil: I hope now we are on same page. Thanks a lot for your valuable time
to discuss this stuff.
>
> -Anoop-
> ________
> From: anil gupta [anilgupt...@gmail.com]
> Sent: Friday, December 14, 2012 11:31 PM
> To: user@hbase.apache.org
> Subject: Re
Hope u got it now :)
-Anoop-
From: anil gupta [anilgupt...@gmail.com]
Sent: Friday, December 14, 2012 11:31 PM
To: user@hbase.apache.org
Subject: Re: HBase - Secondary Index
On Fri, Dec 14, 2012 at 12:54 AM, Anoop Sam John wrote:
> Hi Anil,
>
> >
feel free to aske me :)
> You can see the HalfStoreFileReader class 1st..
>
> -Anoop-
>
> From: anil gupta [anilgupt...@gmail.com]
> Sent: Friday, December 14, 2012 2:11 PM
> To: user@hbase.apache.org
> Subject: Re: HBase
IndexHalfStoreFileReader?
> Sorry I am not able to publish any design doc or code as the company has
> not decided to open src the solution yet.
> Any particular query you come acorss pls feel free to aske me :)
> You can see the HalfStoreFileReader class 1st..
>
> -Anoop-
> __
particular query you come acorss pls feel free to aske me :)
You can see the HalfStoreFileReader class 1st..
-Anoop-
From: anil gupta [anilgupt...@gmail.com]
Sent: Friday, December 14, 2012 2:11 PM
To: user@hbase.apache.org
Subject: Re: HBase - Secondary Ind
Hi Anoop,
Nice presentation and seems like a smart implementation. Since the
presentation only covered bullet points so i have couple of questions on
your implementation. :)
Here is a recap to my implementation and our previous discussion on
Secondary index:
Here is the link to previous email th
Hi there-
There is a mention of this in the Hbase book.
http://hbase.apache.org/book.html#secondary.indices
On 7/7/11 5:18 AM, "Florin P" wrote:
>Hello!
> Perhaps this question occurs all the time. Maybe it should be in the
>FAQ.
>So, what are the options to achieve secondary index on H
Thanks Steve, Mike! I will try using two tables and manage my own indexing on
the second table.
Jack.
-Original Message-
From: Steven Noels [mailto:stev...@outerthought.org]
Sent: Tuesday, November 02, 2010 9:09 AM
To: user
Subject: Re: HBase Secondary Index
On Tue, Nov 2, 2010 at 12
ing an image or 'blob' in HBase.
> From: j...@local.com
> To: user@hbase.apache.org
> Date: Mon, 1 Nov 2010 17:04:40 -0700
> Subject: RE: HBase Secondary Index
>
> Thanks again Mike!
>
> Two tables to store the same data and the only difference is just the row
>
On Tue, Nov 2, 2010 at 12:36 AM, Michael Segel wrote:
>
> Jack...
>
> When we looked at the secondary index, it started off ok, but we started
> having issues.
> There were some serious problems that needed to be addressed and the
> contributor wasn't keeping things up to date. (Such is life)
>
>
...@local.com
> To: user@hbase.apache.org
> Date: Mon, 1 Nov 2010 15:43:22 -0700
> Subject: RE: HBase Secondary Index
>
> Hi Michael, thanks a lot!
>
> Is adding a secondary index to HTable a bad idea given that my data will
> never be updated once added to HTable?
&g
cal.com
> To: user@hbase.apache.org
> Date: Mon, 1 Nov 2010 15:43:22 -0700
> Subject: RE: HBase Secondary Index
>
> Hi Michael, thanks a lot!
>
> Is adding a secondary index to HTable a bad idea given that my data will
> never be updated once added to HTable?
>
> I am
different
time stamps.
Thanks for the advices!
Jack.
-Original Message-
From: Michael Segel [mailto:michael_se...@hotmail.com]
Sent: Monday, November 01, 2010 3:36 PM
To: user@hbase.apache.org
Subject: RE: HBase Secondary Index
Jack,
Its not.
Long story short, the transactional jar
Jack,
Its not.
Long story short, the transactional jar is out in git hub.
As to 0.89 support, wasn't there the last time I checked.
> From: j...@local.com
> To: user@hbase.apache.org
> Date: Mon, 1 Nov 2010 15:25:02 -0700
> Subject: HBase Secondary Index
>
> Hi All,
>
> I'm learning how to a
2010/9/6 Murali Krishna. P :
> Hi,
> My row size is around 300 bytes with total 20 columns. I tried the custom
> indexing without the write to WAL. Currently having only 2 tables, one for the
> main table and another for all 20 indexes. My key to the index table is
> columnValue+columnName+rowKey
se the key
generation-> cName+ cValue + rowKey. I went for this schema to reduce the
number of tables involved.
Thanks,
Murali Krishna
From: Ted Yu
To: user@hbase.apache.org
Sent: Mon, 6 September, 2010 7:23:22 PM
Subject: Re: HBase secondary index performa
Thanks,
> Murali Krishna
>
>
>
>
>
> From: Andrey Stepachev
> To: user@hbase.apache.org
> Sent: Sun, 5 September, 2010 11:54:45 PM
> Subject: Re: HBase secondary index performance
>
> 2010/9/5 Murali Krishna. P :
> > Hi,
> &
it be better during the initial write
itself (since each table has different region) ??
Thanks,
Murali Krishna
From: Andrey Stepachev
To: user@hbase.apache.org
Sent: Sun, 5 September, 2010 11:54:45 PM
Subject: Re: HBase secondary index performance
2010/9/5 M
non indexed region (disable
indexedtables extension)?
What numbers did you got?
>
> Thanks,
> Murali Krishna
>
>
>
>
>
> From: Andrey Stepachev
> To: user@hbase.apache.org
> Sent: Sun, 5 September, 2010 3:53:26 AM
> Subject: Re
2010/9/5 Samuru Jackson :
> Hi,
>
>> where key will be [value:key] and insert rows every time, when we insert
>> our values. We will got 30k rows/s/node.
>
> Could you specify on what kind of hardware you did this?
3 node "cluster", 16Gb core2duo. sas raid10.
> How did you design your indexer? Is
: Andrey Stepachev
To: user@hbase.apache.org
Sent: Sun, 5 September, 2010 3:53:26 AM
Subject: Re: HBase secondary index performance
2010/9/3 Murali Krishna. P :
>* custom indexing is good, but our data keeps changing every day. So,
>probably
> indextable is the best option for us
Hi,
> where key will be [value:key] and insert rows every time, when we insert
> our values. We will got 30k rows/s/node.
Could you specify on what kind of hardware you did this? How did you
design your indexer? Is it multithreaded?
/SJ
---
http://uncinuscloud.blogspot.com/
2010/9/3 Murali Krishna. P :
> * custom indexing is good, but our data keeps changing every day. So,
> probably
> indextable is the best option for us
In case of custom indexing you can use timestamps to check, that index
record still valid.
(or ever simply recheck existance of the value)
On Fri, Sep 3, 2010 at 7:57 AM, Michael Segel wrote:
>
>
>
> > Date: Fri, 3 Sep 2010 18:00:42 +0530
> > From: muralikpb...@yahoo.com
> > Subject: Re: HBase secondary index performance
> > To: user@hbase.apache.org
> >
> > Thanks Andrey,
>
uch that daily updates are around 10 million records
> where
> most of it are just updates and we want it to be real time (or NRT). Any
> suggestions are appreciated.
>
> Thanks,
> Murali Krishna
>
>
>
>
>
> From: Samuru Jackson
>
From: Samuru Jackson
To: user@hbase.apache.org
Sent: Fri, 3 September, 2010 6:24:16 PM
Subject: Re: HBase secondary index performance
Hi,
I wrote my own Indexer and actually I have a pretty good performance.
However, there are still known places where I could gain even more
performance (just not h
> Date: Fri, 3 Sep 2010 18:00:42 +0530
> From: muralikpb...@yahoo.com
> Subject: Re: HBase secondary index performance
> To: user@hbase.apache.org
>
> Thanks Andrey,
>
> * Setting the autoflush to false and increasing the writeBuffer size to
> 12MB
>
___
> From: Andrey Stepachev
> To: user@hbase.apache.org
> Sent: Fri, 3 September, 2010 12:14:29 AM
> Subject: Re: HBase secondary index performance
>
> First, check that you connection not in autoflash mode.
> Second, you can think about custom indexing instead
>
From: Andrey Stepachev
To: user@hbase.apache.org
Sent: Fri, 3 September, 2010 12:14:29 AM
Subject: Re: HBase secondary index performance
First, check that you connection not in autoflash mode.
Second, you can think about custom indexing instead
of using
First, check that you connection not in autoflash mode.
Second, you can think about custom indexing instead
of using indexedtable. In my experience custom idexing
(especially if data doesn't modified), is much more performant.
Problem with indexedtable is in fact, that on every insert
hbase perform
70 matches
Mail list logo