Build failed in Jenkins: HBase-0.92 #540

2012-08-28 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-0.92/540/ -- [...truncated 1797 lines...] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.648 sec Running org.apache.hadoop.hbase.regionserver.wal.TestWALActionsListener Tests run: 1, Failures: 0,

RE: Improving Coprocessor postSplit/postOpen synchronization

2012-08-28 Thread Ramkrishna.S.Vasudevan
Hi Kevin I am very much interested to see this. We have done something similar internally but along with the new coprocessor hooks that we added, we also tweaked a bit on the kernel side. It is something like, Divide your splits steps into two parts Steps before PONR and steps after PONR.

A general question on maxVersion handling when we have Secondary index tables

2012-08-28 Thread Ramkrishna.S.Vasudevan
Hi All When we try to build any type of secondary indices for a given table how can one handle maxVersions in the secondary index tables. For eg, I have inserted Row1 - Val1 = t Row1 - Val2 = t+1 Row1 - Val3. = t+2 Ideally if my max versions is only one then Val3 should be my

Re: A general question on maxVersion handling when we have Secondary index tables

2012-08-28 Thread Jesse Yates
Ram, If I understand correctly, I think you can design your index such that you don't actually use the timestamp (e.g. everything gets put with a TS = 10 - or some other non-special, relatively small number that's not 0 as I'd worry about that in HBase ;) Then when you set maxVersions to 1,

RE: A general question on maxVersion handling when we have Secondary index tables

2012-08-28 Thread Ramkrishna.S.Vasudevan
Hi Jesse Thanks lot for your reply. - Not maintaining timestamps in the sec index may cause problems when I issue an delete on the main table and the corresponding things needs to be deleted in the sec index. - As in the case I mentioned below Index table will have Val1_row1 (t) Val2_row1 (t+2)

Jenkins build is back to normal : HBase-0.92 #541

2012-08-28 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-0.92/541/

Re: Improving Coprocessor postSplit/postOpen synchronization

2012-08-28 Thread Andrew Purtell
What about writing a marker (a file) into the region at split (from preSplit) which is then existence checked and read at open (postOpen)? This file would contain whatever indexing metadata is required. Also, splits are nearly instant because the daughters are created with reference files to the

Re: Improving Coprocessor postSplit/postOpen synchronization

2012-08-28 Thread Andrew Purtell
Never mind, I went to look at the code. Should have done that first. Looking at 0.94 sources, in SplitTransaction, first we notify the master that the split has happened, and wait for the master to process it (which opens daughters), and then call up to the CP with the daughter regions as

Build failed in Jenkins: HBase-0.92 #542

2012-08-28 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-0.92/542/ -- [...truncated 1803 lines...] Running org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.954 sec Running

Build failed in Jenkins: HBase-0.92 #543

2012-08-28 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-0.92/543/ -- [...truncated 1803 lines...] Running org.apache.hadoop.hbase.regionserver.wal.TestLogRolling Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 393.32 sec Running

Re: A general question on maxVersion handling when we have Secondary index tables

2012-08-28 Thread Wei Tan
Thanks for sharing a pointer to your implementation. My two cents: timestamp is a way to do MVCC and setting every KV with the same TS will get concurrency control very tricky and error prone, if not impossible I think Ram is talking about the dead entry in the index table rather than data

Build failed in Jenkins: HBase-TRUNK #3289

2012-08-28 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-TRUNK/3289/changes Changes: [Tedyu] HBASE-6671 Kerberos authenticated super user should be able to retrieve proxied delegation tokens (Francis) -- [...truncated 2646 lines...] Tests run: 1, Failures: 0, Errors: 0,

Re: A general question on maxVersion handling when we have Secondary index tables

2012-08-28 Thread Ted Yu
I think this discussion should be on HBASE JIRA. Another dimension to secondary indexing is the co-location (or pairing) of data table region and index table region. Related regions from the two tables should be placed on the same region server. Cheers On Tue, Aug 28, 2012 at 8:52 AM, Wei Tan

Jenkins build is back to normal : HBase-0.92 #544

2012-08-28 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-0.92/544/changes

Re: A general question on maxVersion handling when we have Secondary index tables

2012-08-28 Thread Jesse Yates
@Ted: Are you proposing re-opening the should we have secondary indexes in HBase discussion? If so, I'm +1 on adding them. Wanna file a jira? @Wei Tan: Yeah, I generally agree. However, I think you can get away with ignoring MVCC and just keep an index on the latest key (where key _includes_ the

ANN: The first hbase 0.92.2 release candidate is available for download

2012-08-28 Thread Ted Yu
Hi, The first 0.92.2 RC is available for download here: http://people.apache.org/~tedyu/hbase-0.92.2-rc0/ I signed the tarball. My gpg key (signed by stack) is available from pgp.mit.edu. Key id: F48B08A4http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0xCBC29E86F48B08A4 HBase

Re: A general question on maxVersion handling when we have Secondary index tables

2012-08-28 Thread Ted Yu
I think we should revive secondary indexes discussion (actually it has been revived) Since Ramkrishna has design in mind, he would be the best person to log a new JIRA. Cheers On Tue, Aug 28, 2012 at 10:03 AM, Jesse Yates jesse.k.ya...@gmail.comwrote: @Ted: Are you proposing re-opening the

Re: ANN: The first hbase 0.92.2 release candidate is available for download

2012-08-28 Thread Stack
Something is wrong w/ the email below Ted. Its signed by Lars rather than you and its all indented. Did you mean to send it? Over here in this thread, it looks like a few of us were trying to fix the flappying 0.92.2 unit tests before putting up an RC:

Re: Improving Coprocessor postSplit/postOpen synchronization

2012-08-28 Thread lars hofhansl
That approach sounds good to me. - Original Message - From: Andrew Purtell apurt...@apache.org To: dev@hbase.apache.org Cc: Sent: Tuesday, August 28, 2012 3:05 AM Subject: Re: Improving Coprocessor postSplit/postOpen synchronization Never mind, I went to look at the code. Should have

Re: ANN: The first hbase 0.92.2 release candidate is available for download

2012-08-28 Thread Ted Yu
Stack: Just saw your comment in HBASE-6660. I will look through that issue. Sorry for the malformed email. Expect another RC. Cheers On Tue, Aug 28, 2012 at 11:10 AM, Stack st...@duboce.net wrote: Something is wrong w/ the email below Ted. Its signed by Lars rather than you and its all

Build failed in Jenkins: HBase-0.92-security #118

2012-08-28 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-0.92-security/118/changes Changes: [jmhsieh] HBASE-6586 Quarantine Corrupted HFiles with hbck -- [...truncated 7211 lines...] Forking command line: /bin/sh -c cd

Re: ANN: The first hbase 0.92.2 release candidate is available for download

2012-08-28 Thread Stack
On Tue, Aug 28, 2012 at 11:17 AM, Ted Yu yuzhih...@gmail.com wrote: Expect another RC. Would suggest you wait till closure over on the thread [Discuss] Release 0.92.2? [1] before you cut another RC. Good on you Ted, St.Ack 1.

Re: [Discuss] Release 0.92.2?

2012-08-28 Thread Ted Yu
About test failures, I saw the following in https://builds.apache.org/job/HBase-0.92-securityhttps://builds.apache.org/job/HBase-0.92-security/ws/trunk/target/surefire-reportsbuild 118: Failed tests: queueFailover(org.apache. hadoop.hbase.replication.TestReplication): Waited too much time for

Build failed in Jenkins: HBase-TRUNK #3290

2012-08-28 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-TRUNK/3290/changes Changes: [stack] HBASE-6052 Convert .META. and -ROOT- content to pb; ADDENDUM REMOVE UNUSED hbase-4388-root.dir.tgz -- [...truncated 2653 lines...] Tests run: 1, Failures: 0, Errors: 0, Skipped:

Build failed in Jenkins: HBase-0.94 #442

2012-08-28 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-0.94/442/changes Changes: [Tedyu] HBASE-6671 Kerberos authenticated super user should be able to retrieve proxied delegation tokens (Francis) -- [...truncated 614 lines...] Running

Build failed in Jenkins: HBase-0.92-security #119

2012-08-28 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-0.92-security/119/ -- [...truncated 7185 lines...] Forking command line: /bin/sh -c cd https://builds.apache.org/job/HBase-0.92-security/ws/trunk /home/jenkins/tools/java/jdk1.6.0_27-32/jre/bin/java

Build failed in Jenkins: HBase-TRUNK-on-Hadoop-2.0.0 #153

2012-08-28 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/153/changes Changes: [stack] HBASE-6052 Convert .META. and -ROOT- content to pb; ADDENDUM REMOVE UNUSED hbase-4388-root.dir.tgz [Tedyu] HBASE-6671 Kerberos authenticated super user should be able to retrieve proxied delegation

Build failed in Jenkins: HBase-0.92-security #120

2012-08-28 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-0.92-security/120/ -- [...truncated 7181 lines...] Running org.apache.hadoop.hbase.io.hfile.TestHFileBlockIndex Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.998 sec Forking command line: /bin/sh

Build failed in Jenkins: HBase-0.92-security #121

2012-08-28 Thread Apache Jenkins Server
See https://builds.apache.org/job/HBase-0.92-security/121/ -- [...truncated 7212 lines...] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.068 sec Forking command line: /bin/sh -c cd

RE: A general question on maxVersion handling when we have Secondary index tables

2012-08-28 Thread Ramkrishna.S.Vasudevan
Hi Yes I was talking about the dead entry in the index table rather than the actual data table. Regards Ram -Original Message- From: Wei Tan [mailto:w...@us.ibm.com] Sent: Tuesday, August 28, 2012 9:22 PM To: dev@hbase.apache.org Cc: Sandeep Tata Subject: Re: A general question

RE: Improving Coprocessor postSplit/postOpen synchronization

2012-08-28 Thread Ramkrishna.S.Vasudevan
Hi As per Andrew the postSplit needs to move up before even the transitioning of the ZK nodes are done. But in case of splits it is like if the META is updated with the updated daughter info, the clients will tend to send in the data to those updated regions. So it is better we can add a new