[jira] [Created] (HBASE-13260) Bootstrap Tables for fun and profit

Enis Soztutar created HBASE-13260: - Summary: Bootstrap Tables for fun and profit Key: HBASE-13260 URL: https://issues.apache.org/jira/browse/HBASE-13260 Project: HBase Issue Type: Bug

[jira] [Created] (HBASE-13259) mmap() based BucketCache IOEngine

Zee Chen created HBASE-13259: Summary: mmap() based BucketCache IOEngine Key: HBASE-13259 URL: https://issues.apache.org/jira/browse/HBASE-13259 Project: HBase Issue Type: New Feature C

[jira] [Created] (HBASE-13258) Promote TestHRegion to LargeTests

zhangduo created HBASE-13258: Summary: Promote TestHRegion to LargeTests Key: HBASE-13258 URL: https://issues.apache.org/jira/browse/HBASE-13258 Project: HBase Issue Type: Sub-task Comp

Re: Rough goal timelines for 1.1 and 2.0

+1 on Nick being the RM for release 1.1 On Mon, Mar 16, 2015 at 11:50 AM, Enis Söztutar wrote: > I would love to see 1.1 in or before May. We already have good stuff in > branch-1, enough to justify a minor release. Some of the "features" are > still in the pipeline waiting to be finished (MOB,

[jira] [Created] (HBASE-13257) Show coverage report on jenkins

zhangduo created HBASE-13257: Summary: Show coverage report on jenkins Key: HBASE-13257 URL: https://issues.apache.org/jira/browse/HBASE-13257 Project: HBase Issue Type: Task Reporter

[jira] [Resolved] (HBASE-7781) Update security unit tests to use a KDC if available

[ https://issues.apache.org/jira/browse/HBASE-7781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangduo resolved HBASE-7781. - Resolution: Fixed Fix Version/s: 1.1.0 2.0.0 Assignee: zhangduo Update

Re: Rough goal timelines for 1.1 and 2.0

I'd be a little nervous about procedureV2 since it's a larger change, and it would be landing later in the cycle. What do others think of a 1.1 without that ? On Mon, Mar 16, 2015 at 12:35 PM, Andrew Purtell wrote: > Agreed! Since Nick has volunteered to RM 1.1 please let me withdraw my > earlie

[jira] [Created] (HBASE-13256) HBaseConfiguration#checkDefaultsVersion(Configuration) has spelling error

Josh Elser created HBASE-13256: -- Summary: HBaseConfiguration#checkDefaultsVersion(Configuration) has spelling error Key: HBASE-13256 URL: https://issues.apache.org/jira/browse/HBASE-13256 Project: HBase

Re: Question on EnableTableHandler code

Now (1) is under control (HBASE-13254). Let us talk about (2). Looks like we are doing best effort to online all regions of a table during 'enable table' operation. My argument is that we should be consistent with all conditions. Currently, we fail if bulk assignment failed with some reason; bu

[jira] [Created] (HBASE-13255) Bad grammar in RegionServer status page

Josh Elser created HBASE-13255: -- Summary: Bad grammar in RegionServer status page Key: HBASE-13255 URL: https://issues.apache.org/jira/browse/HBASE-13255 Project: HBase Issue Type: Improvement

[jira] [Created] (HBASE-13254) EnableTableHandler#prepare would not throw TableNotFoundException during recovery

Stephen Yuan Jiang created HBASE-13254: -- Summary: EnableTableHandler#prepare would not throw TableNotFoundException during recovery Key: HBASE-13254 URL: https://issues.apache.org/jira/browse/HBASE-13254

Re: Question on EnableTableHandler code

Stephen , you are right , that is my code and that thing was overlooked :) I think we need completely remove state cleanup code. Actually how it done tablestate manager could not return table which later will be rendered as nonexistent. Basically that means, that if we got nonexistent table in Ena

Re: mmap() based BucketCache IOEngine

Yes please! On Mon, Mar 16, 2015 at 12:43 PM, Zee Chen wrote: > Of the existing BucketCache IOEngines, the FileIOEngine uses pread() to > copy data from kernel space to user space. This is a good choice when the > total working set size is much bigger than the available RAM and the > latency is

Re: Status of Huawei's 2' Indexing?

Hi Rose, Sorry for late reply. bq. Is there work on this that I don’t see? You can try this [1] for checking something with 0.98.3 version(sorry not that much latest). We thought of making it independent from HBase. Trying to do when ever find time(only few kernel changes left in bulkload to prep

Re: Question on EnableTableHandler code

thanks, Rajeshbabu, HBASE-10215 is not the last change, The HBASE-7767 (hello, Andrey [?]) removed the exception throw code after setting up the table state, what we really want is as follows (if Andrey agrees with the change, I will create a JIRA and send out the patch today): // Check if t

mmap() based BucketCache IOEngine

Of the existing BucketCache IOEngines, the FileIOEngine uses pread() to copy data from kernel space to user space. This is a good choice when the total working set size is much bigger than the available RAM and the latency is dominated by IO access. However, when the entire working set is small eno

Re: Rough goal timelines for 1.1 and 2.0

Agreed! Since Nick has volunteered to RM 1.1 please let me withdraw my earlier volunteerism for that task, unless Nick declines. On Mon, Mar 16, 2015 at 12:15 PM, Stack wrote: > Thanks for raising this topic Mr Busbey. > > A 1.1 before hbasecon would be sweet. As has been said already, 1.1 has a

Re: Status of Huawei's 2' Indexing?

You miss the point. Your index is going to be orthogonal to your base table. Again, how do you handle joins? In terms of indexing… you have to ways of building your index. 1) In a separate M/R job. 2) As each row is inserted, the coprocessor inserts the data in to the secondary indexes. Mor

Re: Rough goal timelines for 1.1 and 2.0

Thanks for raising this topic Mr Busbey. A 1.1 before hbasecon would be sweet. As has been said already, 1.1 has a bunch of good stuff in it already -- e.g. flush by column family -- so worthwhile pushing it out soon. +1 on Nick for RM because it is good to spread the RM'ing load. St.Ack On Mon

Re: Jira role cleanup

On Mon, Mar 16, 2015 at 1:28 PM, Andrew Purtell wrote: > On Mon, Mar 16, 2015 at 11:02 AM, Nick Dimiduk wrote: > > > bq. Our commit log conventions aren't universally followed, due to human > > error > > > > Going forward, I think we can alleviate this issue with a git hook and a > > regexp. > >

Re: Question on EnableTableHandler code

Hi Stephen and Andrey, The first step added to remove stale znodes if table creation fails after znode creation. See HBASE-10215 . Not sure still we need it or not. Thanks, Rajeshbabu. On Tue, Mar 17, 2015 at 12:18 AM, Andrey Stepachev wrot

[jira] [Created] (HBASE-13253) LoadIncrementalHFiles unify hfiles discovery

Matteo Bertozzi created HBASE-13253: --- Summary: LoadIncrementalHFiles unify hfiles discovery Key: HBASE-13253 URL: https://issues.apache.org/jira/browse/HBASE-13253 Project: HBase Issue Type

Re: Status of Huawei's 2' Indexing?

Joseph, First, I would strongly recommend against using HBase… but since you insist. Lets start with your row key. 1) REMEMBER HIPPA 2) How are you going to access the base table? So if for example, you’re never going to do a “Get me Mary Smith’s record” but more “Show me all of the patie

[jira] [Created] (HBASE-13252) Git rid of managed connections and connection caching

Mikhail Antonov created HBASE-13252: --- Summary: Git rid of managed connections and connection caching Key: HBASE-13252 URL: https://issues.apache.org/jira/browse/HBASE-13252 Project: HBase I

Re: Rough goal timelines for 1.1 and 2.0

I would love to see 1.1 in or before May. We already have good stuff in branch-1, enough to justify a minor release. Some of the "features" are still in the pipeline waiting to be finished (MOB, procV2, etc). Personally, I think we should get HBASE-12972, and ProcV2, RPC quotas (and other multi-ten

Re: Status of Huawei's 2' Indexing?

Thanks, Wilm. I’ll look for the thread there. Obviously I didn’t realize there was so much back story: I was asking about this specific implementation because it seems to be fairly well thought out and have good commentary in the Jira ticket (HBASE-9203). At the time I thought it was mostly a dev

Re: Question on EnableTableHandler code

Thanks Stephen. on (2): I think that much better to guarantee that table was enabled (i.e. all internal structures reflect that fact and balancer knows about new table). But result of that could be checked asyncronically from Admin. Does it make sense? On Mon, Mar 16, 2015 at 6:10 PM, Stephen Jia

[jira] [Created] (HBASE-13251) Correct 'HBase, MapReduce, and the CLASSPATH' section in HBase Ref Guide

Jerry He created HBASE-13251: Summary: Correct 'HBase, MapReduce, and the CLASSPATH' section in HBase Ref Guide Key: HBASE-13251 URL: https://issues.apache.org/jira/browse/HBASE-13251 Project: HBase

Re: Jira role cleanup

On Mon, Mar 16, 2015 at 11:02 AM, Nick Dimiduk wrote: > bq. Our commit log conventions aren't universally followed, due to human > error > > Going forward, I think we can alleviate this issue with a git hook and a > regexp. > That's a good idea. > On Mon, Mar 16, 2015 at 10:38 AM, Andrew Purt

Re: Status of Huawei's 2' Indexing?

​That's patently untrue and pure paranoia. ​The comment about having a civilized discussion had nothing to do with you Michael. Joseph said: "HBASE-9203 (the ticket that deals with 2’ indexes) is pretty civilized (as are most of them, it seems)" and so I responded as you saw. I was not thinking

Re: Question on EnableTableHandler code

Andrey, I will take care of (1). And (2) :-) if your guys agree. Because it is not consistent, if the bulk assigned failed, we would fail the enabling table; however, if the bulk assign not starts, we would enable table with offline regions - really inconsistent - we either all fail in those scen

Re: Status of Huawei's 2' Indexing?

Hi Joseph, I think that you kicked off this discussion, because to implement an indexing mechanism for hbase in general is much more complicate than your specific problem. The people on this list want to bear every possible (or at least A LOT) of applications in mind. A too easy mechanism wouldn't

Re: Rough goal timelines for 1.1 and 2.0

​FWIW, the Region proposal (HBASE-12972) is ready for review. The companion issue for SplitTransaction and RegionMergeTransaction (HBASE-12975) needs more discussion but could be ready to go in a <= one month timeframe. On Mon, Mar 16, 2015 at 10:30 AM, Nick Dimiduk wrote: > I think we can learn

Re: Question on EnableTableHandler code

Thanks Stephen. Looks like you are right. For (1) case we really don't need there state cleanup. That is a bug. Should throw TableNotFoundException. As for (2) in case of no online region servers available we could leave table enabled, but no regions would be assigned. Actually that rises good q

Re: Jira role cleanup

bq. Our commit log conventions aren't universally followed, due to human error Going forward, I think we can alleviate this issue with a git hook and a regexp. On Mon, Mar 16, 2015 at 10:38 AM, Andrew Purtell wrote: > > I think Jira management should be left to the committers. One can pretty >

Re: Jira role cleanup

Okay, it sounds like there's decent consensus. How much of this cleanup can I take care of before the PMC meets? Everyone fine if I do the earlier pruning we talked about and look into the "anyone is assignable" bit? On Mon, Mar 16, 2015 at 12:38 PM, Andrew Purtell wrote: > > I think Jira man

Re: Question on EnableTableHandler code

Stephen, would you like to create jira for case (1)? Thank you. On Mon, Mar 16, 2015 at 5:58 PM, Andrey Stepachev wrote: > Thanks Stephen. > > Looks like you are right. For (1) case we really don't need there > state cleanup. That is a bug. Should throw TableNotFoundException. > > As for (2) in

Re: Status of Huawei's 2' Indexing?

Dude... Relax... Let's keep it cordial, please. To the topic: Any CS 101 student can implement an eventually consistent index on top of HBase. The part that is always missed is: How do you keep it consistent?There you have essentially two choices: (1) every update to an indexed table becomes a d

Re: Status of Huawei's 2' Indexing?

Alright, let’s see if I can get this discussion back on track. I have a sensibly defined table for patient data; its rowkey is simply lastname:firstname, since it’s convenient for the bulk of my lookups. Unfortunately I also need to efficiently find patients using an ID string, whose literal value

Re: Rough goal timelines for 1.1 and 2.0

And, now we have *two* volunteers for RM for 1.1. (Nick and myself). Let's take that as interest in getting it done and do it. As far as I'm concerned, it's all yours Nick, have at it! On Mon, Mar 16, 2015 at 10:30 AM, Nick Dimiduk wrote: > I think we can learn a lesson or two from the vendor

Re: Jira role cleanup

> I think Jira management should be left to the committers. One can pretty much mess up a release, and make it hard to account for what's in and what's not when jiras are changed the around (the ultimate truth can be reconstructed from the git commit records, but that's tedious). I agree we should

Question on EnableTableHandler code

I want to make sure that the following logic in EnableTableHandler is correct: (1). In EnableTableHandler#prepare - if the table is not existed, it marked the table as deleted and not throw exception. The result is the table lock is released and the caller has no knowledge that the table not exis

Re: Rough goal timelines for 1.1 and 2.0

I think we can learn a lesson or two from the vendor marketing machines -- a release timed with HBaseCon would be ideal in this regard. My obligations to the event are minimal, so I'm willing to volunteer as RM for 1.1. Do we think we can make some of these decisions in time for spinning RC's in mi

Re: Status of Huawei's 2' Indexing?

Andrew, because 2+ years ago, Phoenix wasn’t an Apache project. At the time, Huawei was releasing their research on it and Salesforce was implementing it. I mention the company names because those were the parties involved in the work as well as the discussion. Also those companies are mentio

Re: Status of Huawei's 2' Indexing?

I don't understand the repeated mention of "Salesforce" in that invective. As point of fact the work of adding local mutable indexes to Phoenix was done by a contributor from Huawei, who has since moved over to Hortonworks, if I'm not mistaken - but not like affiliation matters, it really doesn't.

Re: Status of Huawei's 2' Indexing?

Sigh. Here we go again… 1) Complexity? 2) Speed when looking at the indexes in a more general case. 3) Resources required to do the search become excessive... ... Again, your indexes will be orthogonal to the base table. If you can’t understand that… then you need to sit back, drink a few

Re: Status of Huawei's 2' Indexing?

On Mon, Mar 16, 2015 at 8:14 AM, Michael Segel wrote: > You’ll have to excuse Andy. > > He’s a bit slow. ... I gave up trying to have an intelligent/civilized conversation with Andrew > because he just couldn’t grasp the basics. ;-) > > > Michael: Quit insult and ad hominem. Stick to the te

Re: Status of Huawei's 2' Indexing?

There is nothing wrong with co-locating index and data on a same RS. This will greatly improve single table search. Joins are evil anyway. Leave them to RDBMS Zoo. -Vlad On Mon, Mar 16, 2015 at 8:14 AM, Michael Segel wrote: > You’ll have to excuse Andy. > > He’s a bit slow. HBASE-13044 should

Re: Status of Huawei's 2' Indexing?

Michael, I don’t understand the invective. I’m sure you have something to contribute but when bring on this tone the only thing I hear are the snide comments. -j P.s., I’ll refer you to this: https://hbase.apache.org/book.html#_joins On 3/16/15, 11:15 AM, "Michael Segel" wrote: >You’ll hav

Re: Status of Huawei's 2' Indexing?

You’ll have to excuse Andy. He’s a bit slow. HBASE-13044 should have been done 2 years ago. And it was trivial. Just got done last month…. But I digress… The long story short… HBASE-9203 was brain dead from inception. Huawei’s idea was to index on the region which had two problems. 1) Co

Re: Jira role cleanup

bq. Beyond that I agree that we should limit this to a known set of people (the contributors). +1 bq. Maybe discuss this briefly at the next PMC meeting +1 too. On Sun, Mar 15, 2015 at 11:12 PM, lars hofhansl wrote: > Hmm... This is interesting. I think Jira management should be left to the > c

[jira] [Created] (HBASE-13250) chown of ExportSnapshot does not cover all path and files

He Liangliang created HBASE-13250: - Summary: chown of ExportSnapshot does not cover all path and files Key: HBASE-13250 URL: https://issues.apache.org/jira/browse/HBASE-13250 Project: HBase I

[jira] [Created] (HBASE-13249) Concurrency issue in SnapshotFileCache

He Liangliang created HBASE-13249: - Summary: Concurrency issue in SnapshotFileCache Key: HBASE-13249 URL: https://issues.apache.org/jira/browse/HBASE-13249 Project: HBase Issue Type: Bug

[jira] [Resolved] (HBASE-13217) Flush procedure fails in trunk due to ZK issue

[ https://issues.apache.org/jira/browse/HBASE-13217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-13217. Resolution: Invalid I am closing this issue for now. It still occurs for m