[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly
[ https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422054#comment-13422054 ] Ferdy Galema commented on HBASE-2214: - How can this break compatibily? (I would say that the Scan version number actually maintains compatibility). Anyway, I think it would be best if someone else picked this up. That will be much faster to implement it correctly. (As I'm not involved enough). Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly - Key: HBASE-2214 URL: https://issues.apache.org/jira/browse/HBASE-2214 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Ferdy Galema Fix For: 0.96.0, 0.94.2 Attachments: HBASE-2214-0.94-v2.txt, HBASE-2214-0.94-v3.txt, HBASE-2214-0.94.txt, HBASE-2214-v4.txt, HBASE-2214-v5.txt, HBASE-2214-v6.txt, HBASE-2214-v7.txt, HBASE-2214_with_broken_TestShell.txt The notion that you set size rather than row count specifying how many rows a scanner should return in each cycle was raised over in hbase-1966. Its a good one making hbase regular though the data under it may vary. HBase-1966 was committed but the patch was constrained by the fact that it needed to not change RPC interface. This issue is about doing hbase-1966 for 0.21 in a clean, unconstrained way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly
[ https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferdy Galema updated HBASE-2214: Assignee: (was: Ferdy Galema) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly - Key: HBASE-2214 URL: https://issues.apache.org/jira/browse/HBASE-2214 Project: HBase Issue Type: New Feature Reporter: stack Fix For: 0.96.0, 0.94.2 Attachments: HBASE-2214-0.94-v2.txt, HBASE-2214-0.94-v3.txt, HBASE-2214-0.94.txt, HBASE-2214-v4.txt, HBASE-2214-v5.txt, HBASE-2214-v6.txt, HBASE-2214-v7.txt, HBASE-2214_with_broken_TestShell.txt The notion that you set size rather than row count specifying how many rows a scanner should return in each cycle was raised over in hbase-1966. Its a good one making hbase regular though the data under it may vary. HBase-1966 was committed but the patch was constrained by the fact that it needed to not change RPC interface. This issue is about doing hbase-1966 for 0.21 in a clean, unconstrained way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6447) Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent
[ https://issues.apache.org/jira/browse/HBASE-6447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-6447. -- Resolution: Fixed Fix Version/s: 0.94.1 0.92.2 Hadoop Flags: Reviewed Committed to trunk and to 0.94 and 0.92 branches. Thanks for review Ted. Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent - Key: HBASE-6447 URL: https://issues.apache.org/jira/browse/HBASE-6447 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0, 0.96.0 Reporter: stack Assignee: stack Fix For: 0.92.2, 0.94.1 Attachments: testzk.txt, testzkTRUNK.txt Studying 0.94 failures of late, in the last 15 or so builds, TestZooKeeper has failed at least three times. Running it local, it fails 50% of the time: once with testCreateSilentIsReallySilent and then less frequently with testMasterExpiration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6447) Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent
[ https://issues.apache.org/jira/browse/HBASE-6447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422137#comment-13422137 ] Hudson commented on HBASE-6447: --- Integrated in HBase-0.94 #368 (See [https://builds.apache.org/job/HBase-0.94/368/]) HBASE-6447 Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent (Revision 1365503) Result = FAILURE stack : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent - Key: HBASE-6447 URL: https://issues.apache.org/jira/browse/HBASE-6447 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0, 0.96.0 Reporter: stack Assignee: stack Fix For: 0.92.2, 0.94.1 Attachments: testzk.txt, testzkTRUNK.txt Studying 0.94 failures of late, in the last 15 or so builds, TestZooKeeper has failed at least three times. Running it local, it fails 50% of the time: once with testCreateSilentIsReallySilent and then less frequently with testMasterExpiration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6453) Hbase Replication point in time feature
terry zhang created HBASE-6453: -- Summary: Hbase Replication point in time feature Key: HBASE-6453 URL: https://issues.apache.org/jira/browse/HBASE-6453 Project: HBase Issue Type: New Feature Components: replication Affects Versions: 0.94.0 Reporter: terry zhang Assignee: terry zhang Now we can not control when hbase replication start to work. this patch support we set a time stamp filter . All the row which is below this time stamp will not be replicated. We also can delete and show this time stamp in hbase shell if we want to change it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6453) Hbase Replication point in time feature
[ https://issues.apache.org/jira/browse/HBASE-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] terry zhang updated HBASE-6453: --- Attachment: hbase-6453-v1.patch Hbase Replication point in time feature --- Key: HBASE-6453 URL: https://issues.apache.org/jira/browse/HBASE-6453 Project: HBase Issue Type: New Feature Components: replication Affects Versions: 0.94.0 Reporter: terry zhang Assignee: terry zhang Attachments: hbase-6453-v1.patch Now we can not control when hbase replication start to work. this patch support we set a time stamp filter . All the row which is below this time stamp will not be replicated. We also can delete and show this time stamp in hbase shell if we want to change it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6453) Hbase Replication point in time feature
[
https://issues.apache.org/jira/browse/HBASE-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422147#comment-13422147
]
terry zhang commented on HBASE-6453:
{code}
hbase(main):001:0 set_timefilter
ERROR: wrong number of arguments (0 for 2)
Here is some help for this command:
Set a peer cluster time filter to replicate to, the row which time stamp is
before
the timestamp will be filtered.
Examples:
hbase set_timefilter '1', 1329896850047
hbase set_timefilter '2', 1329896850047
hbase(main):002:0 set_timefilter '1',1329896850047
0 row(s) in 0.3000 seconds
{code}
set time stamp to 1329896850047. Them all the kvs which is early than
1329896850047 will be filterd
Hbase Replication point in time feature
---
Key: HBASE-6453
URL: https://issues.apache.org/jira/browse/HBASE-6453
Project: HBase
Issue Type: New Feature
Components: replication
Affects Versions: 0.94.0
Reporter: terry zhang
Assignee: terry zhang
Attachments: hbase-6453-v1.patch
Now we can not control when hbase replication start to work. this patch
support we set a time stamp filter . All the row which is below this time
stamp will not be replicated. We also can delete and show this time stamp in
hbase shell if we want to change it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6453) Hbase Replication point in time feature
[
https://issues.apache.org/jira/browse/HBASE-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422148#comment-13422148
]
terry zhang commented on HBASE-6453:
{code}
hbase(main):003:0 get_timefilter '1'
PEER_IDTIME_FILTER
1 1329896850047
{code}
we can show the time stamp by clusterId and check if it is set correctly
Hbase Replication point in time feature
---
Key: HBASE-6453
URL: https://issues.apache.org/jira/browse/HBASE-6453
Project: HBase
Issue Type: New Feature
Components: replication
Affects Versions: 0.94.0
Reporter: terry zhang
Assignee: terry zhang
Attachments: hbase-6453-v1.patch
Now we can not control when hbase replication start to work. this patch
support we set a time stamp filter . All the row which is below this time
stamp will not be replicated. We also can delete and show this time stamp in
hbase shell if we want to change it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6453) Hbase Replication point in time feature
[
https://issues.apache.org/jira/browse/HBASE-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422149#comment-13422149
]
terry zhang commented on HBASE-6453:
we also can drop the time filter. after we drop it , time stamp change to zero
. and all the kvs will be replicated.
{code}
hbase(main):004:0 drop_timefilter '1'
0 row(s) in 0.0030 seconds
hbase(main):005:0 get_timefilter '1'
PEER_IDTIME_FILTER
1 0
{code}
Hbase Replication point in time feature
---
Key: HBASE-6453
URL: https://issues.apache.org/jira/browse/HBASE-6453
Project: HBase
Issue Type: New Feature
Components: replication
Affects Versions: 0.94.0
Reporter: terry zhang
Assignee: terry zhang
Attachments: hbase-6453-v1.patch
Now we can not control when hbase replication start to work. this patch
support we set a time stamp filter . All the row which is below this time
stamp will not be replicated. We also can delete and show this time stamp in
hbase shell if we want to change it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6447) Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent
[ https://issues.apache.org/jira/browse/HBASE-6447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422155#comment-13422155 ] Hudson commented on HBASE-6447: --- Integrated in HBase-TRUNK #3171 (See [https://builds.apache.org/job/HBase-TRUNK/3171/]) HBASE-6447 Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent (Revision 1365502) Result = SUCCESS stack : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent - Key: HBASE-6447 URL: https://issues.apache.org/jira/browse/HBASE-6447 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0, 0.96.0 Reporter: stack Assignee: stack Fix For: 0.92.2, 0.94.1 Attachments: testzk.txt, testzkTRUNK.txt Studying 0.94 failures of late, in the last 15 or so builds, TestZooKeeper has failed at least three times. Running it local, it fails 50% of the time: once with testCreateSilentIsReallySilent and then less frequently with testMasterExpiration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6447) Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent
[ https://issues.apache.org/jira/browse/HBASE-6447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422170#comment-13422170 ] Hudson commented on HBASE-6447: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #109 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/109/]) HBASE-6447 Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent (Revision 1365502) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent - Key: HBASE-6447 URL: https://issues.apache.org/jira/browse/HBASE-6447 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0, 0.96.0 Reporter: stack Assignee: stack Fix For: 0.92.2, 0.94.1 Attachments: testzk.txt, testzkTRUNK.txt Studying 0.94 failures of late, in the last 15 or so builds, TestZooKeeper has failed at least three times. Running it local, it fails 50% of the time: once with testCreateSilentIsReallySilent and then less frequently with testMasterExpiration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6407) Investigate moving to DI (guice) framework for plugin arch.
[ https://issues.apache.org/jira/browse/HBASE-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1347#comment-1347 ] stack commented on HBASE-6407: -- I agree w/ Andrew. Investigate moving to DI (guice) framework for plugin arch. --- Key: HBASE-6407 URL: https://issues.apache.org/jira/browse/HBASE-6407 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-6407-1.patch Investigate using Guice to inject the correct compat object provided by compat plugins -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6407) Investigate moving to DI (guice) framework for plugin arch.
[ https://issues.apache.org/jira/browse/HBASE-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422234#comment-13422234 ] stack commented on HBASE-6407: -- Looked at patch. RB lost my comments. In general, do we have to have the guice package? Can Interfaces/factories go into respective packages? Put fundamental stuff at top-level (ok if guice becomes a fundamental). Seems way more scaffolding than I'd expect though I suppose its more a DI retrofit than it is scaffolding, is that right E? Investigate moving to DI (guice) framework for plugin arch. --- Key: HBASE-6407 URL: https://issues.apache.org/jira/browse/HBASE-6407 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-6407-1.patch Investigate using Guice to inject the correct compat object provided by compat plugins -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6450) HBase startup should be with MALLOC_MAX_ARENA set
[ https://issues.apache.org/jira/browse/HBASE-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6450: - Resolution: Fixed Fix Version/s: 0.94.2 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 0.92, 0.94 and trunk. Thanks for patch DD. HBase startup should be with MALLOC_MAX_ARENA set - Key: HBASE-6450 URL: https://issues.apache.org/jira/browse/HBASE-6450 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.92.2, 0.94.2 Attachments: 6450.patch I think we should do the same as what HADOOP-7154 has done in terms of starting up daemons with MALLOC_MAX_ARENA set. I recently noticed that there were RS crashes on RHEL6 due to memory (could be avoided by setting MALLOC_MAX_ARENA). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6449) Dapper like tracing
[ https://issues.apache.org/jira/browse/HBASE-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422270#comment-13422270 ] stack commented on HBASE-6449: -- Excellent Dapper like tracing --- Key: HBASE-6449 URL: https://issues.apache.org/jira/browse/HBASE-6449 Project: HBase Issue Type: New Feature Components: client, ipc Affects Versions: 0.96.0 Reporter: Jonathan Leavitt Labels: tracing Attachments: htrace1.diff, trace.png Add [Dapper|http://research.google.com/pubs/pub36356.html] like tracing to HBase. [Accumulo|http://accumulo.apache.org] added something similar with their cloudtrace package. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6415) Add Rat checking to the pre-checkin script.
[ https://issues.apache.org/jira/browse/HBASE-6415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422274#comment-13422274 ] stack commented on HBASE-6415: -- So what should we do here? Check in this patch Elliott? Add Rat checking to the pre-checkin script. --- Key: HBASE-6415 URL: https://issues.apache.org/jira/browse/HBASE-6415 Project: HBase Issue Type: Task Reporter: Elliott Clark Assignee: Elliott Clark -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6396) Fix NoSuchMethodError running against hadoop 2.0
[ https://issues.apache.org/jira/browse/HBASE-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422300#comment-13422300 ] stack commented on HBASE-6396: -- We should close this as Won't fix then? Fix NoSuchMethodError running against hadoop 2.0 Key: HBASE-6396 URL: https://issues.apache.org/jira/browse/HBASE-6396 Project: HBase Issue Type: Bug Affects Versions: 0.94.0 Reporter: Zhihong Ted Yu Assignee: Zhihong Ted Yu Labels: hadoop-2.0 Fix For: 0.96.0 Attachments: 6396-v2.txt HADOOP-8350 changed the signature of NetUtils.getInputStream() This leads to NoSuchMethodError in HBaseClient$Connection.setupIOstreams(). See https://issues.apache.org/jira/browse/HADOOP-8350?focusedCommentId=13414276page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13414276 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5447) Support for custom filters with PB-based RPC
[ https://issues.apache.org/jira/browse/HBASE-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422305#comment-13422305 ] stack commented on HBASE-5447: -- @Gregory I don't think there many user-made filters that are not part of core code base. Would go along w/ your option A above. We can help those w/ user-made filters do convertion to 0.96 when/if they show up trying to figure how to convert. Thanks. Support for custom filters with PB-based RPC Key: HBASE-5447 URL: https://issues.apache.org/jira/browse/HBASE-5447 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Todd Lipcon Assignee: Todd Lipcon -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5705) Introduce Protocol Buffer RPC engine
[ https://issues.apache.org/jira/browse/HBASE-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422313#comment-13422313 ] stack commented on HBASE-5705: -- I'd vote for removing Writables engine once all pb converted. Introduce Protocol Buffer RPC engine Key: HBASE-5705 URL: https://issues.apache.org/jira/browse/HBASE-5705 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.96.0 Attachments: 5705-1.patch, 5705-2.1.patch Introduce Protocol Buffer RPC engine in the RPC core. Protocols that are PB aware can be made to go through this RPC engine. The approach, in my current thinking, would be similar to HADOOP-7773. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly
[ https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422334#comment-13422334 ] Lars Hofhansl commented on HBASE-2214: -- Hi Ferdy, we definitely appreciate all the work you put in for this patch! It is still in 0.96. :) We'll try to find a way to use your patch without changing the version number. The problem with changing the version number (in 0.94 and prior) is that then the newer version of the object cannot be deserialized by an older version. You can see this Scan.readFields(), which throws an exception if it reads a version newer than the one it expects. In practice this means that now a new client cannot issue a scan request to an older server, because the server cannot deserialize the scan object. In 0.96 and later these kind of problems are solved by protobufs. Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly - Key: HBASE-2214 URL: https://issues.apache.org/jira/browse/HBASE-2214 Project: HBase Issue Type: New Feature Reporter: stack Fix For: 0.96.0, 0.94.2 Attachments: HBASE-2214-0.94-v2.txt, HBASE-2214-0.94-v3.txt, HBASE-2214-0.94.txt, HBASE-2214-v4.txt, HBASE-2214-v5.txt, HBASE-2214-v6.txt, HBASE-2214-v7.txt, HBASE-2214_with_broken_TestShell.txt The notion that you set size rather than row count specifying how many rows a scanner should return in each cycle was raised over in hbase-1966. Its a good one making hbase regular though the data under it may vary. HBase-1966 was committed but the patch was constrained by the fact that it needed to not change RPC interface. This issue is about doing hbase-1966 for 0.21 in a clean, unconstrained way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5705) Introduce Protocol Buffer RPC engine
[ https://issues.apache.org/jira/browse/HBASE-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422338#comment-13422338 ] stack commented on HBASE-5705: -- RB is not working for me at mo so reviews here: I asked about comment in ClientCache in previous reviews but it went unanswered. I'm talking about the old Doug comment on a. vs. b. choice. I didn't understand it then and less so now. Is it a copy/paste error? The writing to a buffer before we write on the line goes away if we move to a pure pb engine? Is the size used here right for case where we have an exception? (Or its just a hint?) ByteBufferOutputStream buf = new ByteBufferOutputStream(size); And RPCRequestWritable and RPCResponseWritable will go away too when we do pb? (You've answered yes to this multiple times I believe). Why the need for a setRPC w/ no params? This stuff passed to Server, whats it about? Is this some Hadoop thing about being able to do multiple serializations? Class? extends Writable paramClass, When we do logResponse, we no longer take a Call but we pass in its name and methodnames and params is that a regression? Would it be better to take the Invocation? The below doesn't look too bad: hrpc-bytearray'5'-bytelength-of-serialized-ConnectionHeader-obj[int32]ConnectionHeader-object serialized In header can we say if its pbs that follow? Or will we have to always calc message size before sending? Why we need this: clientProtocolVersion in request? Is this Writables thing? It goes away when we go pb? My general thought on this is we commit. This patch has same shape as our current rpc'ing; same classes, etc. just moved over some to support pb. We need to go pure pb and get rid of Writables in this rpc call path. I see this as a stepping stone so we should get it in so we can undo it later w/ pb engine. Introduce Protocol Buffer RPC engine Key: HBASE-5705 URL: https://issues.apache.org/jira/browse/HBASE-5705 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.96.0 Attachments: 5705-1.patch, 5705-2.1.patch Introduce Protocol Buffer RPC engine in the RPC core. Protocols that are PB aware can be made to go through this RPC engine. The approach, in my current thinking, would be similar to HADOOP-7773. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly
[ https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422347#comment-13422347 ] Ferdy Galema commented on HBASE-2214: - Thanks Lars. Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly - Key: HBASE-2214 URL: https://issues.apache.org/jira/browse/HBASE-2214 Project: HBase Issue Type: New Feature Reporter: stack Fix For: 0.96.0, 0.94.2 Attachments: HBASE-2214-0.94-v2.txt, HBASE-2214-0.94-v3.txt, HBASE-2214-0.94.txt, HBASE-2214-v4.txt, HBASE-2214-v5.txt, HBASE-2214-v6.txt, HBASE-2214-v7.txt, HBASE-2214_with_broken_TestShell.txt The notion that you set size rather than row count specifying how many rows a scanner should return in each cycle was raised over in hbase-1966. Its a good one making hbase regular though the data under it may vary. HBase-1966 was committed but the patch was constrained by the fact that it needed to not change RPC interface. This issue is about doing hbase-1966 for 0.21 in a clean, unconstrained way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5705) Introduce Protocol Buffer RPC engine
[ https://issues.apache.org/jira/browse/HBASE-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422356#comment-13422356 ] Devaraj Das commented on HBASE-5705: Sorry, seems like I forgot to publish my responses to the comments on RB on some occasions. I published all of them now. Apologize for the out-of-order responses. I'll respond to your comments shortly, Stack (thanks for looking). Introduce Protocol Buffer RPC engine Key: HBASE-5705 URL: https://issues.apache.org/jira/browse/HBASE-5705 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.96.0 Attachments: 5705-1.patch, 5705-2.1.patch Introduce Protocol Buffer RPC engine in the RPC core. Protocols that are PB aware can be made to go through this RPC engine. The approach, in my current thinking, would be similar to HADOOP-7773. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly
[ https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422355#comment-13422355 ] Zhihong Ted Yu commented on HBASE-2214: --- Scan class should be flexible in serializing scan version. When the older server throws exception, Scan object can downgrade to earlier scan version and retry. Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly - Key: HBASE-2214 URL: https://issues.apache.org/jira/browse/HBASE-2214 Project: HBase Issue Type: New Feature Reporter: stack Fix For: 0.96.0, 0.94.2 Attachments: HBASE-2214-0.94-v2.txt, HBASE-2214-0.94-v3.txt, HBASE-2214-0.94.txt, HBASE-2214-v4.txt, HBASE-2214-v5.txt, HBASE-2214-v6.txt, HBASE-2214-v7.txt, HBASE-2214_with_broken_TestShell.txt The notion that you set size rather than row count specifying how many rows a scanner should return in each cycle was raised over in hbase-1966. Its a good one making hbase regular though the data under it may vary. HBase-1966 was committed but the patch was constrained by the fact that it needed to not change RPC interface. This issue is about doing hbase-1966 for 0.21 in a clean, unconstrained way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6450) HBase startup should be with MALLOC_MAX_ARENA set
[ https://issues.apache.org/jira/browse/HBASE-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422380#comment-13422380 ] Hudson commented on HBASE-6450: --- Integrated in HBase-0.92 #481 (See [https://builds.apache.org/job/HBase-0.92/481/]) HBASE-6450 HBase startup should be with MALLOC_MAX_ARENA set (Revision 1365582) Result = SUCCESS stack : Files : * /hbase/branches/0.92/bin/hbase-config.sh HBase startup should be with MALLOC_MAX_ARENA set - Key: HBASE-6450 URL: https://issues.apache.org/jira/browse/HBASE-6450 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.92.2, 0.94.2 Attachments: 6450.patch I think we should do the same as what HADOOP-7154 has done in terms of starting up daemons with MALLOC_MAX_ARENA set. I recently noticed that there were RS crashes on RHEL6 due to memory (could be avoided by setting MALLOC_MAX_ARENA). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6447) Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent
[ https://issues.apache.org/jira/browse/HBASE-6447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422379#comment-13422379 ] Hudson commented on HBASE-6447: --- Integrated in HBase-0.92 #481 (See [https://builds.apache.org/job/HBase-0.92/481/]) HBASE-6447 Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent (Revision 1365581) Result = SUCCESS stack : Files : * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java Common TestZooKeeper failures on jenkins: testMasterSessionExpired and testCreateSilentIsReallySilent - Key: HBASE-6447 URL: https://issues.apache.org/jira/browse/HBASE-6447 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0, 0.96.0 Reporter: stack Assignee: stack Fix For: 0.92.2, 0.94.1 Attachments: testzk.txt, testzkTRUNK.txt Studying 0.94 failures of late, in the last 15 or so builds, TestZooKeeper has failed at least three times. Running it local, it fails 50% of the time: once with testCreateSilentIsReallySilent and then less frequently with testMasterExpiration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly
[ https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422386#comment-13422386 ] Lars Hofhansl commented on HBASE-2214: -- Yep. Scan can check the version in readFields and then act with a bit more smarts than just throwing up its hands (an exception). Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly - Key: HBASE-2214 URL: https://issues.apache.org/jira/browse/HBASE-2214 Project: HBase Issue Type: New Feature Reporter: stack Fix For: 0.96.0, 0.94.2 Attachments: HBASE-2214-0.94-v2.txt, HBASE-2214-0.94-v3.txt, HBASE-2214-0.94.txt, HBASE-2214-v4.txt, HBASE-2214-v5.txt, HBASE-2214-v6.txt, HBASE-2214-v7.txt, HBASE-2214_with_broken_TestShell.txt The notion that you set size rather than row count specifying how many rows a scanner should return in each cycle was raised over in hbase-1966. Its a good one making hbase regular though the data under it may vary. HBase-1966 was committed but the patch was constrained by the fact that it needed to not change RPC interface. This issue is about doing hbase-1966 for 0.21 in a clean, unconstrained way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6427) Pluggable policy for smallestReadPoint in HRegion
[ https://issues.apache.org/jira/browse/HBASE-6427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422394#comment-13422394 ] Lars Hofhansl commented on HBASE-6427: -- The other scenario where this is useful is for M/R based incremental backups (as described here: http://hadoop-hbase.blogspot.com/2012/04/timestamp-consistent-backups-in-hbase.html). The backup tools can then control exactly what data to keep, while the backups are running. The actual plugged policy would probably coordinate via ZK. Pluggable policy for smallestReadPoint in HRegion - Key: HBASE-6427 URL: https://issues.apache.org/jira/browse/HBASE-6427 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Priority: Minor When implementing higher level stores on top of HBase it is necessary to allow dynamic control over how long KVs must be kept around. Semi-static config options for ColumnFamilies (# of version or TTL) is not sufficient. The simplest way to achieve this is to have a pluggable class to determine the smallestReadpoint for Region. That way outside code can control what KVs to retain. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6444) Expose the ability to set custom HTTP Request Headers for the REST client used by RemoteHTable
[ https://issues.apache.org/jira/browse/HBASE-6444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422409#comment-13422409 ] Andrew Purtell commented on HBASE-6444: --- +1 Expose the ability to set custom HTTP Request Headers for the REST client used by RemoteHTable -- Key: HBASE-6444 URL: https://issues.apache.org/jira/browse/HBASE-6444 Project: HBase Issue Type: Improvement Components: rest Reporter: Erich Hochmuth Attachments: HBASE-6444-0.94.patch, HBASE-6444.patch Original Estimate: 48h Remaining Estimate: 48h My corporate security office (ISO) requires that all http traffic get routed through a Web Access Management layer (http://en.wikipedia.org/wiki/Web_access_management) Our Hadoop cluster has been segmented by a virtual network with all access to HBase from outside clients being managed through HBase Stargate rest server. The corporate WAM system requires that all http clients authenticate with it first before making any http request to any http service in the corporate network. After the http client authenticates with the WAM system the WAM system returns the client a set of values that must be inserted into a http cookie and request header of all future http requests to other http clients. This would mean that all requests through the RemoteHTable interface would require that this cookie and request header be set as part of the http request. org.apache.hadoop.hbase.rest.client.Client looks like the appropriate place that this functionality would need to be plugged into. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6444) Expose the ability to set custom HTTP Request Headers for the REST client used by RemoteHTable
[ https://issues.apache.org/jira/browse/HBASE-6444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422423#comment-13422423 ] Jimmy Xiang commented on HBASE-6444: Another choice is to extend the REST client and override some function, for example, executeURI(method, headers, path)? Will this solve your problem, Erich? Expose the ability to set custom HTTP Request Headers for the REST client used by RemoteHTable -- Key: HBASE-6444 URL: https://issues.apache.org/jira/browse/HBASE-6444 Project: HBase Issue Type: Improvement Components: rest Reporter: Erich Hochmuth Attachments: HBASE-6444-0.94.patch, HBASE-6444.patch Original Estimate: 48h Remaining Estimate: 48h My corporate security office (ISO) requires that all http traffic get routed through a Web Access Management layer (http://en.wikipedia.org/wiki/Web_access_management) Our Hadoop cluster has been segmented by a virtual network with all access to HBase from outside clients being managed through HBase Stargate rest server. The corporate WAM system requires that all http clients authenticate with it first before making any http request to any http service in the corporate network. After the http client authenticates with the WAM system the WAM system returns the client a set of values that must be inserted into a http cookie and request header of all future http requests to other http clients. This would mean that all requests through the RemoteHTable interface would require that this cookie and request header be set as part of the http request. org.apache.hadoop.hbase.rest.client.Client looks like the appropriate place that this functionality would need to be plugged into. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6444) Expose the ability to set custom HTTP Request Headers for the REST client used by RemoteHTable
[ https://issues.apache.org/jira/browse/HBASE-6444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422435#comment-13422435 ] Erich Hochmuth commented on HBASE-6444: --- That would require the following changes: * Make the httpClient protected instead of private or provide a getter method for the member variable * Create an interface for the rest.Client that RemoteTable uses I like having RemoteHTable use a rest.Client interface, i think that's a cleaner implementation then whats there today. I'd wan't the new Client impl contributed to the HBase project. I don't like having my application extend an HBase class to expose this functionality. I believe this is a capability that should be part of the REST client. Expose the ability to set custom HTTP Request Headers for the REST client used by RemoteHTable -- Key: HBASE-6444 URL: https://issues.apache.org/jira/browse/HBASE-6444 Project: HBase Issue Type: Improvement Components: rest Reporter: Erich Hochmuth Attachments: HBASE-6444-0.94.patch, HBASE-6444.patch Original Estimate: 48h Remaining Estimate: 48h My corporate security office (ISO) requires that all http traffic get routed through a Web Access Management layer (http://en.wikipedia.org/wiki/Web_access_management) Our Hadoop cluster has been segmented by a virtual network with all access to HBase from outside clients being managed through HBase Stargate rest server. The corporate WAM system requires that all http clients authenticate with it first before making any http request to any http service in the corporate network. After the http client authenticates with the WAM system the WAM system returns the client a set of values that must be inserted into a http cookie and request header of all future http requests to other http clients. This would mean that all requests through the RemoteHTable interface would require that this cookie and request header be set as part of the http request. org.apache.hadoop.hbase.rest.client.Client looks like the appropriate place that this functionality would need to be plugged into. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based read/write locks
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422437#comment-13422437 ] Alex Feinberg commented on HBASE-5991: -- Working on this right now. Introduce sequential ZNode based read/write locks -- Key: HBASE-5991 URL: https://issues.apache.org/jira/browse/HBASE-5991 Project: HBase Issue Type: Improvement Reporter: Alex Feinberg Assignee: Alex Feinberg This is a continuation of HBASE-5494: Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6427) Pluggable policy for smallestReadPoint in HRegion
[ https://issues.apache.org/jira/browse/HBASE-6427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422438#comment-13422438 ] Lars Hofhansl commented on HBASE-6427: -- Upon further scrutiny, that would actually not work, because any external code would have no knowledge about the internal memstoreTSs. Perhaps a better option would be to make the expiration policy of a column family pluggable. That way TTL and # versions could be controlled from the outside. Pluggable policy for smallestReadPoint in HRegion - Key: HBASE-6427 URL: https://issues.apache.org/jira/browse/HBASE-6427 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Priority: Minor When implementing higher level stores on top of HBase it is necessary to allow dynamic control over how long KVs must be kept around. Semi-static config options for ColumnFamilies (# of version or TTL) is not sufficient. The simplest way to achieve this is to have a pluggable class to determine the smallestReadpoint for Region. That way outside code can control what KVs to retain. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6446) Replication source will throw EOF exception when hlog size is 0
[ https://issues.apache.org/jira/browse/HBASE-6446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422446#comment-13422446 ] Jean-Daniel Cryans commented on HBASE-6446: --- bq. I think we should not keep retrying. We can retry for some configurable times, then log an error and skip this file. A bad file should not break the system In which case? Let's say the RS got a 0 length file, then creates a new HLog. Here the replication code will do the right thing and move on. bq. OK，Daniel. Let's change it from warning level to debug level to prevent warning in online cluster. I was also thinking that we should not print the stack trace. Replication source will throw EOF exception when hlog size is 0 --- Key: HBASE-6446 URL: https://issues.apache.org/jira/browse/HBASE-6446 Project: HBase Issue Type: Bug Components: replication Affects Versions: 0.94.0 Reporter: terry zhang Attachments: hbase-6446-v2.patch, hbase-6446.patch when master cluster startup new hlog which size is 0 will be created. if we start replication, replication source will print many EOF exception when openreader. I think we need to ignore this case and do not print so many exception warning log . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5991) Introduce sequential ZNode based read/write locks
[ https://issues.apache.org/jira/browse/HBASE-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422458#comment-13422458 ] Jesse Yates commented on HBASE-5991: @Alex Sweet, looking forward to it. Introduce sequential ZNode based read/write locks -- Key: HBASE-5991 URL: https://issues.apache.org/jira/browse/HBASE-5991 Project: HBase Issue Type: Improvement Reporter: Alex Feinberg Assignee: Alex Feinberg This is a continuation of HBASE-5494: Currently table-level write locks have been implemented using non-sequential ZNodes as part of HBASE-5494 and committed to 89-fb branch. This issue is to track converting the table-level locks to sequential ZNodes and supporting read-write locks, as to solve the issue of preventing schema changes during region splits or merges. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6454) Write PB definitions for filters
Gregory Chanan created HBASE-6454: - Summary: Write PB definitions for filters Key: HBASE-6454 URL: https://issues.apache.org/jira/browse/HBASE-6454 Project: HBase Issue Type: Task Components: ipc, migration Reporter: Gregory Chanan Assignee: Gregory Chanan Fix For: 0.96.0 See HBASE-5447. Conversion to protobuf requires writing protobuf definitions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6415) Add Rat checking to the pre-checkin script.
[ https://issues.apache.org/jira/browse/HBASE-6415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422467#comment-13422467 ] Elliott Clark commented on HBASE-6415: -- I don't have a patch yet. My current thought is that there's some different configs for the rat being run on a normal build and the one run for the hadoopqa script. But I've been heads down on 6407 so I haven't gotten too far on this yet. Add Rat checking to the pre-checkin script. --- Key: HBASE-6415 URL: https://issues.apache.org/jira/browse/HBASE-6415 Project: HBase Issue Type: Task Reporter: Elliott Clark Assignee: Elliott Clark -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6386) Audit log messages do not include column family / qualifier information consistently
[ https://issues.apache.org/jira/browse/HBASE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HBASE-6386: -- Attachment: hbase-6386-v1.patch Consistently include column family / qualifier info in audit logs. This patch makes the AccessController consistently add information about column families and qualifiers into audit logs. Since there may be more than one cf / qualifier involved, the format of the audit log message changed slightly (coupled with some pretty ugly code to create the message). TestAccessController passes; also tested on live HBase 0.92 (with a backported version of the patch). Audit log messages do not include column family / qualifier information consistently Key: HBASE-6386 URL: https://issues.apache.org/jira/browse/HBASE-6386 Project: HBase Issue Type: Improvement Components: security Reporter: Marcelo Vanzin Attachments: hbase-6386-v1.patch The code related to this issue is in AccessController.java:permissionGranted(). When creating audit logs, that method will do one of the following: * grant access, create audit log with table name only * deny access because of table permission, create audit log with table name only * deny access because of column family / qualifier permission, create audit log with specific family / qualifier So, in the case where more than one column family and/or qualifier are in the same request, there will be a loss of information. Even in the case where only one column family and/or qualifier is involved, information may be lost. It would be better if this behavior consistently included all the information in the request; regardless of access being granted or denied, and regardless which permission caused the denial, the column family and qualifier info should be part of the audit log message. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6386) Audit log messages do not include column family / qualifier information consistently
[ https://issues.apache.org/jira/browse/HBASE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HBASE-6386: -- Affects Version/s: 0.96.0 Status: Patch Available (was: Open) Audit log messages do not include column family / qualifier information consistently Key: HBASE-6386 URL: https://issues.apache.org/jira/browse/HBASE-6386 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.96.0 Reporter: Marcelo Vanzin Attachments: hbase-6386-v1.patch The code related to this issue is in AccessController.java:permissionGranted(). When creating audit logs, that method will do one of the following: * grant access, create audit log with table name only * deny access because of table permission, create audit log with table name only * deny access because of column family / qualifier permission, create audit log with specific family / qualifier So, in the case where more than one column family and/or qualifier are in the same request, there will be a loss of information. Even in the case where only one column family and/or qualifier is involved, information may be lost. It would be better if this behavior consistently included all the information in the request; regardless of access being granted or denied, and regardless which permission caused the denial, the column family and qualifier info should be part of the audit log message. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6407) Investigate moving to DI (guice) framework for plugin arch.
[ https://issues.apache.org/jira/browse/HBASE-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6407: - Attachment: HBASE-6407-2.patch Investigate moving to DI (guice) framework for plugin arch. --- Key: HBASE-6407 URL: https://issues.apache.org/jira/browse/HBASE-6407 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-6407-1.patch, HBASE-6407-2.patch Investigate using Guice to inject the correct compat object provided by compat plugins -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6407) Investigate moving to DI (guice) framework for plugin arch.
[ https://issues.apache.org/jira/browse/HBASE-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422519#comment-13422519 ] Elliott Clark commented on HBASE-6407: -- bq.It's fine if you just want to contribute a big patch, but for commit it might make sense to put in the core of the work (POM update, HBaseGuice, ExplicitConfModule). Then Sleeper. Then the modularization of LocalHBaseCluster and therefore factories for HMaster, HRegionServer, etc. Then the modularization of HadoopCompat. Then the replication modules. It looks not to difficult to tease this apart that way (so far). What do other committers think? I'm up for that once I've got things in a pretty good state and there are some tests showing how to use the new injector stuff I'll think about splitting. bq.In general, do we have to have the guice package? Nope. That was one of the things I was unsure of about my design. And as I'm adding more and more modules that package is getting a little crowded. I'll work on moving those things around in a little bit. bq.Put fundamental stuff at top-level Hopefully there won't be much. Should just be HBaseGuice which creates the different injector types. bq.Seems way more scaffolding than I'd expect though I suppose its more a DI retrofit than it is scaffolding, is that right E? There is a lot of scaffolding. :-/ Most of that is due to the fact that we pass around Abortable/Stopable/HRegionServer/Other things that are created async everywhere so Guice isn't able to provide those args. Since guice can't provide those directly a factory interface is needed for all those classes. In my latest commit I've gotten rid of a couple factories. I have a little bit more to do to fully guice HMaster and HRegionServer. After those two classes are complete I have some tests and then I'll start on splitting it as Andrew and Stack suggested. There will still be a lot more to do to make everything fully injectable but I don't want this to get too hard to review for comitters. Investigate moving to DI (guice) framework for plugin arch. --- Key: HBASE-6407 URL: https://issues.apache.org/jira/browse/HBASE-6407 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-6407-1.patch, HBASE-6407-2.patch Investigate using Guice to inject the correct compat object provided by compat plugins -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6386) Audit log messages do not include column family / qualifier information consistently
[ https://issues.apache.org/jira/browse/HBASE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422543#comment-13422543 ] Hadoop QA commented on HBASE-6386: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12537874/hbase-6386-v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestAssignmentManager org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2436//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2436//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2436//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2436//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2436//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2436//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2436//console This message is automatically generated. Audit log messages do not include column family / qualifier information consistently Key: HBASE-6386 URL: https://issues.apache.org/jira/browse/HBASE-6386 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.96.0 Reporter: Marcelo Vanzin Attachments: hbase-6386-v1.patch The code related to this issue is in AccessController.java:permissionGranted(). When creating audit logs, that method will do one of the following: * grant access, create audit log with table name only * deny access because of table permission, create audit log with table name only * deny access because of column family / qualifier permission, create audit log with specific family / qualifier So, in the case where more than one column family and/or qualifier are in the same request, there will be a loss of information. Even in the case where only one column family and/or qualifier is involved, information may be lost. It would be better if this behavior consistently included all the information in the request; regardless of access being granted or denied, and regardless which permission caused the denial, the column family and qualifier info should be part of the audit log message. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6386) Audit log messages do not include column family / qualifier information consistently
[ https://issues.apache.org/jira/browse/HBASE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422552#comment-13422552 ] Marcelo Vanzin commented on HBASE-6386: --- Tests above are passing for me locally. Audit log messages do not include column family / qualifier information consistently Key: HBASE-6386 URL: https://issues.apache.org/jira/browse/HBASE-6386 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.96.0 Reporter: Marcelo Vanzin Attachments: hbase-6386-v1.patch The code related to this issue is in AccessController.java:permissionGranted(). When creating audit logs, that method will do one of the following: * grant access, create audit log with table name only * deny access because of table permission, create audit log with table name only * deny access because of column family / qualifier permission, create audit log with specific family / qualifier So, in the case where more than one column family and/or qualifier are in the same request, there will be a loss of information. Even in the case where only one column family and/or qualifier is involved, information may be lost. It would be better if this behavior consistently included all the information in the request; regardless of access being granted or denied, and regardless which permission caused the denial, the column family and qualifier info should be part of the audit log message. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6282) The introspection, etc. of objects in the RPC has to be handled for PB objects
[ https://issues.apache.org/jira/browse/HBASE-6282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422564#comment-13422564 ] Devaraj Das commented on HBASE-6282: MonitoredRPCHandlerImpl.toMap() also needs to be updated. The introspection, etc. of objects in the RPC has to be handled for PB objects -- Key: HBASE-6282 URL: https://issues.apache.org/jira/browse/HBASE-6282 Project: HBase Issue Type: Bug Components: ipc Reporter: Devaraj Das Fix For: 0.96.0 The places where the type of objects are inspected need to be updated to take into consideration PB types. I have noticed Objects.describeQuantity being used, and the private WritableRpcEngine.Server.logResponse method also needs updating (in the PB world, all information about operations/tablenames is contained in one PB argument). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6282) The introspection, etc. of objects in the RPC has to be handled for PB objects
[ https://issues.apache.org/jira/browse/HBASE-6282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-6282: --- Priority: Blocker (was: Major) Upgrading the Priority so that it gets due attention (maybe after a couple of more PB jira fixes). The introspection, etc. of objects in the RPC has to be handled for PB objects -- Key: HBASE-6282 URL: https://issues.apache.org/jira/browse/HBASE-6282 Project: HBase Issue Type: Bug Components: ipc Reporter: Devaraj Das Priority: Blocker Fix For: 0.96.0 The places where the type of objects are inspected need to be updated to take into consideration PB types. I have noticed Objects.describeQuantity being used, and the private WritableRpcEngine.Server.logResponse method also needs updating (in the PB world, all information about operations/tablenames is contained in one PB argument). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6386) Audit log messages do not include column family / qualifier information consistently
[ https://issues.apache.org/jira/browse/HBASE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422601#comment-13422601 ] stack commented on HBASE-6386: -- The AccessController class is missing access, the InterfaceAudience.* annotations. Does this mean it public? Do we need to bring people along w/ the API changes? Otherwise +1 on patch. Should we surface another JIRA Marcelo where we fix the 'suck's issue of not having standard way of doing family maps (the point you raise above)? Audit log messages do not include column family / qualifier information consistently Key: HBASE-6386 URL: https://issues.apache.org/jira/browse/HBASE-6386 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.96.0 Reporter: Marcelo Vanzin Attachments: hbase-6386-v1.patch The code related to this issue is in AccessController.java:permissionGranted(). When creating audit logs, that method will do one of the following: * grant access, create audit log with table name only * deny access because of table permission, create audit log with table name only * deny access because of column family / qualifier permission, create audit log with specific family / qualifier So, in the case where more than one column family and/or qualifier are in the same request, there will be a loss of information. Even in the case where only one column family and/or qualifier is involved, information may be lost. It would be better if this behavior consistently included all the information in the request; regardless of access being granted or denied, and regardless which permission caused the denial, the column family and qualifier info should be part of the audit log message. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6427) Pluggable policy for smallestReadPoint in HRegion
[ https://issues.apache.org/jira/browse/HBASE-6427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422609#comment-13422609 ] Lars Hofhansl commented on HBASE-6427: -- Yet another way of looking at is new coprocessor hook. That would be a hook that sits before the StoreScanner is created (in Store.internalFlushCache and Store.compact) and be passed the set of scanners to use, the store and whether this is a major compaction or not (in the compaction case). Then this hook could optionally return a scanner, and if non-null scanner is return that will be used for the flush/compaction. Now, there already are preFlush and preCompact hooks (interestingly the preFlush is at the region level, whereas the preCompact is at the store level, which is not quite right I think, I wonder whether we can change that), so I'm having a hard time naming these hooks accordingly. preScannerFlush, preScannerCompact doesn't quite sound right. [~apurtell] Do you have an opinion? Pluggable policy for smallestReadPoint in HRegion - Key: HBASE-6427 URL: https://issues.apache.org/jira/browse/HBASE-6427 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Priority: Minor When implementing higher level stores on top of HBase it is necessary to allow dynamic control over how long KVs must be kept around. Semi-static config options for ColumnFamilies (# of version or TTL) is not sufficient. The simplest way to achieve this is to have a pluggable class to determine the smallestReadpoint for Region. That way outside code can control what KVs to retain. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6386) Audit log messages do not include column family / qualifier information consistently
[ https://issues.apache.org/jira/browse/HBASE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422610#comment-13422610 ] Marcelo Vanzin commented on HBASE-6386: --- stack: I'm not changing any public API in AccessController, as far as I remember. So my changes by themselves have no effect on its InterfaceAudience status. That being said, since it's a coprocessor, I don't think it needs to be InterfaceAudience.Public. It would be nice to have a standard way to refer to the family maps, but I'm not familiar with the other parts of the code to be able to say whether it's feasible to do that. Audit log messages do not include column family / qualifier information consistently Key: HBASE-6386 URL: https://issues.apache.org/jira/browse/HBASE-6386 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.96.0 Reporter: Marcelo Vanzin Attachments: hbase-6386-v1.patch The code related to this issue is in AccessController.java:permissionGranted(). When creating audit logs, that method will do one of the following: * grant access, create audit log with table name only * deny access because of table permission, create audit log with table name only * deny access because of column family / qualifier permission, create audit log with specific family / qualifier So, in the case where more than one column family and/or qualifier are in the same request, there will be a loss of information. Even in the case where only one column family and/or qualifier is involved, information may be lost. It would be better if this behavior consistently included all the information in the request; regardless of access being granted or denied, and regardless which permission caused the denial, the column family and qualifier info should be part of the audit log message. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6386) Audit log messages do not include column family / qualifier information consistently
[
https://issues.apache.org/jira/browse/HBASE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422619#comment-13422619
]
stack commented on HBASE-6386:
--
@Marcelo I was thinking these changes:
{code}
-public static AuthResult allow(String reason, User user, Permission.Action
action, byte[] table) {
- return new AuthResult(true, reason, user, action, table, null, null);
+public static AuthResult allow(String reason, User user,
+Permission.Action action, byte[] table,
+Mapbyte[], ? extends Collection? families) {
{code}
@Andrew You think the above changes an issue? If not I'll commit.
Audit log messages do not include column family / qualifier information
consistently
Key: HBASE-6386
URL: https://issues.apache.org/jira/browse/HBASE-6386
Project: HBase
Issue Type: Improvement
Components: security
Affects Versions: 0.96.0
Reporter: Marcelo Vanzin
Attachments: hbase-6386-v1.patch
The code related to this issue is in
AccessController.java:permissionGranted().
When creating audit logs, that method will do one of the following:
* grant access, create audit log with table name only
* deny access because of table permission, create audit log with table name
only
* deny access because of column family / qualifier permission, create audit
log with specific family / qualifier
So, in the case where more than one column family and/or qualifier are in the
same request, there will be a loss of information. Even in the case where
only one column family and/or qualifier is involved, information may be lost.
It would be better if this behavior consistently included all the information
in the request; regardless of access being granted or denied, and regardless
which permission caused the denial, the column family and qualifier info
should be part of the audit log message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5705) Introduce Protocol Buffer RPC engine
[ https://issues.apache.org/jira/browse/HBASE-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422623#comment-13422623 ] stack commented on HBASE-5705: -- np I responded up on rb. See my other comments above (ignore stuff that is duplicated and already answered). Any useful stuff in above feedback? If so, address and lets get this in. Introduce Protocol Buffer RPC engine Key: HBASE-5705 URL: https://issues.apache.org/jira/browse/HBASE-5705 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.96.0 Attachments: 5705-1.patch, 5705-2.1.patch Introduce Protocol Buffer RPC engine in the RPC core. Protocols that are PB aware can be made to go through this RPC engine. The approach, in my current thinking, would be similar to HADOOP-7773. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6453) Hbase Replication point in time feature
[ https://issues.apache.org/jira/browse/HBASE-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422631#comment-13422631 ] stack commented on HBASE-6453: -- So idea is you'd set this timestamp and we'd replicate from that time on? What is the use case? Why not have replication happen when enabled? You write the timestamp to zk under clusterid znode? Into a znode named timefilter? The name should be timestamp? Or start_timestamp? Thats great you added shell commands for setting/getting. Should they have replication mentioned in command name so they are known to be replication facility? Maybe its not necessary as long as these new commands are grouped w/ other replication commands (that is so?) Thanks for the new feature Terry. Hbase Replication point in time feature --- Key: HBASE-6453 URL: https://issues.apache.org/jira/browse/HBASE-6453 Project: HBase Issue Type: New Feature Components: replication Affects Versions: 0.94.0 Reporter: terry zhang Assignee: terry zhang Attachments: hbase-6453-v1.patch Now we can not control when hbase replication start to work. this patch support we set a time stamp filter . All the row which is below this time stamp will not be replicated. We also can delete and show this time stamp in hbase shell if we want to change it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6386) Audit log messages do not include column family / qualifier information consistently
[ https://issues.apache.org/jira/browse/HBASE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422704#comment-13422704 ] Marcelo Vanzin commented on HBASE-6386: --- Ah, yes, right. I'll wait for Andrew's comments; in any case, it's trivial to add the old constructor and method back, they just need to call the new makeFamilyMap() helper method. Audit log messages do not include column family / qualifier information consistently Key: HBASE-6386 URL: https://issues.apache.org/jira/browse/HBASE-6386 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.96.0 Reporter: Marcelo Vanzin Attachments: hbase-6386-v1.patch The code related to this issue is in AccessController.java:permissionGranted(). When creating audit logs, that method will do one of the following: * grant access, create audit log with table name only * deny access because of table permission, create audit log with table name only * deny access because of column family / qualifier permission, create audit log with specific family / qualifier So, in the case where more than one column family and/or qualifier are in the same request, there will be a loss of information. Even in the case where only one column family and/or qualifier is involved, information may be lost. It would be better if this behavior consistently included all the information in the request; regardless of access being granted or denied, and regardless which permission caused the denial, the column family and qualifier info should be part of the audit log message. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6431) Some FilterList Constructors break addFilter
[ https://issues.apache.org/jira/browse/HBASE-6431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422746#comment-13422746 ] Alex Newman commented on HBASE-6431: Is there anything I need to do here? Some FilterList Constructors break addFilter Key: HBASE-6431 URL: https://issues.apache.org/jira/browse/HBASE-6431 Project: HBase Issue Type: Bug Components: filters Affects Versions: 0.92.1, 0.94.0 Reporter: Alex Newman Assignee: Alex Newman Priority: Minor Attachments: 0001-HBASE-6431.-Some-FilterList-Constructors-break-addFi.patch Some of the constructors for FilterList set the internal list of filters to list types which don't support the add operation. As a result FilterList(final ListFilter rowFilters) FilterList(final Filter... rowFilters) FilterList(final Operator operator, final ListFilter rowFilters) FilterList(final Operator operator, final Filter... rowFilters) may init private ListFilter filters = new ArrayListFilter(); incorrectly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6450) HBase startup should be with MALLOC_MAX_ARENA set
[ https://issues.apache.org/jira/browse/HBASE-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422761#comment-13422761 ] Hudson commented on HBASE-6450: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #110 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/110/]) HBASE-6450 HBase startup should be with MALLOC_MAX_ARENA set (Revision 1365584) Result = FAILURE stack : Files : * /hbase/trunk/bin/hbase-config.sh HBase startup should be with MALLOC_MAX_ARENA set - Key: HBASE-6450 URL: https://issues.apache.org/jira/browse/HBASE-6450 Project: HBase Issue Type: Bug Affects Versions: 0.92.1 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.92.2, 0.94.2 Attachments: 6450.patch I think we should do the same as what HADOOP-7154 has done in terms of starting up daemons with MALLOC_MAX_ARENA set. I recently noticed that there were RS crashes on RHEL6 due to memory (could be avoided by setting MALLOC_MAX_ARENA). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6455) org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
Aditya Kishore created HBASE-6455: - Summary: org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path Key: HBASE-6455 URL: https://issues.apache.org/jira/browse/HBASE-6455 Project: HBase Issue Type: Bug Components: performance Affects Versions: 0.94.0, 0.96.0 Reporter: Aditya Kishore Assignee: Aditya Kishore Priority: Minor Attachments: HBASE-6455_trunk.patch I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path. The attached patch addresses both the issues. # Creates separate input and output path rooted under a single folder e.g. user_home/performance_evaluation/MMddHHmmss/inputs and user_home/performance_evaluation/MMddHHmmss/outputs, and # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6455) org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Kishore updated HBASE-6455: -- Attachment: HBASE-6455_trunk.patch Patch attached for trunk org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path -- Key: HBASE-6455 URL: https://issues.apache.org/jira/browse/HBASE-6455 Project: HBase Issue Type: Bug Components: performance Affects Versions: 0.94.0, 0.96.0 Reporter: Aditya Kishore Assignee: Aditya Kishore Priority: Minor Labels: benchmark Attachments: HBASE-6455_trunk.patch I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path. The attached patch addresses both the issues. # Creates separate input and output path rooted under a single folder e.g. user_home/performance_evaluation/MMddHHmmss/inputs and user_home/performance_evaluation/MMddHHmmss/outputs, and # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6455) org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Kishore updated HBASE-6455: -- Fix Version/s: 0.96.0 Status: Patch Available (was: Open) org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path -- Key: HBASE-6455 URL: https://issues.apache.org/jira/browse/HBASE-6455 Project: HBase Issue Type: Bug Components: performance Affects Versions: 0.94.0, 0.96.0 Reporter: Aditya Kishore Assignee: Aditya Kishore Priority: Minor Labels: benchmark Fix For: 0.96.0 Attachments: HBASE-6455_trunk.patch I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path. The attached patch addresses both the issues. # Creates separate input and output path rooted under a single folder e.g. user_home/performance_evaluation/MMddHHmmss/inputs and user_home/performance_evaluation/MMddHHmmss/outputs, and # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6431) Some FilterList Constructors break addFilter
[
https://issues.apache.org/jira/browse/HBASE-6431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422780#comment-13422780
]
Zhihong Ted Yu commented on HBASE-6431:
---
{code}
- private ListFilter filters = new ArrayListFilter();
+ private ArrayListFilter filters = new ArrayListFilter();
{code}
Can we preserve List type for filters field ? That is generic.
{code}
public FilterList(final ListFilter rowFilters) {
-this.filters = rowFilters;
+this.filters = new ArrayListFilter(rowFilters);
{code}
Please check whether rowFilters is already ArrayList.
{code}
-this.filters = Arrays.asList(rowFilters);
+this.filters = new ArrayListFilter(Arrays.asList(rowFilters));
{code}
Why is the above needed ?
See http://www.docjar.com/html/api/java/util/Arrays.java.html:
{code}
2827 public static T ListT asList(T... a) {
2828 return new ArrayList(a);
{code}
Some FilterList Constructors break addFilter
Key: HBASE-6431
URL: https://issues.apache.org/jira/browse/HBASE-6431
Project: HBase
Issue Type: Bug
Components: filters
Affects Versions: 0.92.1, 0.94.0
Reporter: Alex Newman
Assignee: Alex Newman
Priority: Minor
Attachments:
0001-HBASE-6431.-Some-FilterList-Constructors-break-addFi.patch
Some of the constructors for FilterList set the internal list of filters to
list types which don't support the add operation. As a result
FilterList(final ListFilter rowFilters)
FilterList(final Filter... rowFilters)
FilterList(final Operator operator, final ListFilter rowFilters)
FilterList(final Operator operator, final Filter... rowFilters)
may init private ListFilter filters = new ArrayListFilter(); incorrectly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5705) Introduce Protocol Buffer RPC engine
[ https://issues.apache.org/jira/browse/HBASE-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-5705: --- Attachment: 5705-2.2.patch bq. I asked about comment in ClientCache in previous reviews but it went unanswered. I'm talking about the old Doug comment on a. vs. b. choice. I didn't understand it then and less so now. Is it a copy/paste error? Answered on RB but a little bit more here - I think it makes sense to remove this confusing comment (since now the ClientCache class has some javadoc). I'll remove it in the next patch. bq. The writing to a buffer before we write on the line goes away if we move to a pure pb engine? Right. bq. Is the size used here right for case where we have an exception? (Or its just a hint?) Hint. bq. And RPCRequestWritable and RPCResponseWritable will go away too when we do pb? (You've answered yes to this multiple times I believe). Yes (AFAICT). bq. Why the need for a setRPC w/ no params? Could you please point me to the block of code in the patch where this is.. If you are referring to the setRPC with two params that I added here, that I reverted back in my current update.. (and added a comment up on HBASE-6282). bq. This stuff passed to Server, whats it about? Is this some Hadoop thing about being able to do multiple serializations? This is to handle WritableRpcEngine and ProtobufRpcEngine equally well. In the case of WritableRpcEngine, the RPC request is serialized using Invocation objects and in the case of ProtobufRpcEngine, they are done using RpcRequestWritable (a thin wrapper over PB objects). bq. When we do logResponse, we no longer take a Call but we pass in its name and methodnames and params is that a regression? Would it be better to take the Invocation? This is handled in the way that the method now takes methodname and params explicitly (as opposed to how it is in the current trunk where an Invocation object is passed and the method body extracts the params and methodname). This was needed to support ProtobufRpc. Also, HBASE-6282 is applicable here. bq. In header can we say if its pbs that follow? Or will we have to always calc message size before sending? The RPC requires that any RPC message is preceded by the length of the same. In that sense, we need the message size. bq. Why we need this: clientProtocolVersion in request? Is this Writables thing? It goes away when we go pb? The clientProtocolVersion is currently unused on the server side (and it's optional in the .proto definition), but it could potentially be used in the server to decide how to best service the client's RPC (if there were multiple implementations for a method, and the server could pick an implementation based on client version). If this use case seems like we probably won't need to support, we can drop the clientProtocolVersion (btw it being optional in the .proto definition makes it amenable to easy removal even later). By keeping it, it doesn't hurt either.. Introduce Protocol Buffer RPC engine Key: HBASE-5705 URL: https://issues.apache.org/jira/browse/HBASE-5705 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.96.0 Attachments: 5705-1.patch, 5705-2.1.patch, 5705-2.2.patch Introduce Protocol Buffer RPC engine in the RPC core. Protocols that are PB aware can be made to go through this RPC engine. The approach, in my current thinking, would be similar to HADOOP-7773. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6455) org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422815#comment-13422815 ] Hadoop QA commented on HBASE-6455: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12537934/HBASE-6455_trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 14 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2437//console This message is automatically generated. org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path -- Key: HBASE-6455 URL: https://issues.apache.org/jira/browse/HBASE-6455 Project: HBase Issue Type: Bug Components: performance Affects Versions: 0.94.0, 0.96.0 Reporter: Aditya Kishore Assignee: Aditya Kishore Priority: Minor Labels: benchmark Fix For: 0.96.0 Attachments: HBASE-6455_trunk.patch I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path. The attached patch addresses both the issues. # Creates separate input and output path rooted under a single folder e.g. user_home/performance_evaluation/MMddHHmmss/inputs and user_home/performance_evaluation/MMddHHmmss/outputs, and # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5705) Introduce Protocol Buffer RPC engine
[ https://issues.apache.org/jira/browse/HBASE-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422818#comment-13422818 ] Devaraj Das commented on HBASE-5705: bq. My general thought on this is we commit. This patch has same shape as our current rpc'ing; same classes, etc. just moved over some to support pb. We need to go pure pb and get rid of Writables in this rpc call path. I see this as a stepping stone so we should get it in so we can undo it later w/ pb engine. +1 Introduce Protocol Buffer RPC engine Key: HBASE-5705 URL: https://issues.apache.org/jira/browse/HBASE-5705 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.96.0 Attachments: 5705-1.patch, 5705-2.1.patch, 5705-2.2.patch Introduce Protocol Buffer RPC engine in the RPC core. Protocols that are PB aware can be made to go through this RPC engine. The approach, in my current thinking, would be similar to HADOOP-7773. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6449) Dapper like tracing
[
https://issues.apache.org/jira/browse/HBASE-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422833#comment-13422833
]
Zhihong Ted Yu commented on HBASE-6449:
---
You're right about test categorization.
Please add htrace module to the root pom.xml (this was the reason why htrace
tests were not run):
{code}
Index: pom.xml
===
--- pom.xml (revision 1365854)
+++ pom.xml (working copy)
@@ -46,6 +46,7 @@
modulehbase-hadoop-compat/module
modulehbase-common/module
modulehbase-it/module
+modulehtrace/module
/modules
scm
connectionscm:svn:http://svn.apache.org/repos/asf/hbase/trunk/connection
{code}
Then you should be able to run test individually at workspace root:
{code}
Running org.apache.hadoop.hbase.htrace.CountSamplerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.039 sec
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] HBase . SUCCESS [1.479s]
[INFO] HBase - Common SUCCESS [0.792s]
[INFO] HBase - Hadoop Compatibility .. SUCCESS [0.089s]
[INFO] HBase - Hadoop One Compatibility .. SUCCESS [0.235s]
[INFO] HBase - Server SUCCESS [3.212s]
[INFO] HBase - Hadoop Two Compatibility .. SUCCESS [1.581s]
[INFO] HBase - Integration Tests . SUCCESS [0.376s]
[INFO] htrace SUCCESS [0.553s]
{code}
Please change the name of htrace to 'HBase - Trace' so that it aligns with
other module names.
Looks like you forgot to include a file in the patch:
{code}
Running org.apache.hadoop.hbase.htrace.TestHTrace
Error constructing LocalFileSpanReceiver: test/htrace-test-output-spans.txt (No
such file or directory)
{code}
Dapper like tracing
---
Key: HBASE-6449
URL: https://issues.apache.org/jira/browse/HBASE-6449
Project: HBase
Issue Type: New Feature
Components: client, ipc
Affects Versions: 0.96.0
Reporter: Jonathan Leavitt
Labels: tracing
Attachments: htrace1.diff, trace.png
Add [Dapper|http://research.google.com/pubs/pub36356.html] like tracing to
HBase. [Accumulo|http://accumulo.apache.org] added something similar with
their cloudtrace package.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6431) Some FilterList Constructors break addFilter
[ https://issues.apache.org/jira/browse/HBASE-6431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422834#comment-13422834 ] Alex Newman commented on HBASE-6431: Question: Can we preserve List type for filters field ? Answer: We can only support implementations of lists which support add. There is nothing in between list and for instance arraylist. If we don't convert it, we will crash when they try to add. question: Please check whether rowFilters is already ArrayList. Answer: I'm confused how could it be, it's an array question: Why is the above needed. answer: Sorry what's this question asking? Some FilterList Constructors break addFilter Key: HBASE-6431 URL: https://issues.apache.org/jira/browse/HBASE-6431 Project: HBase Issue Type: Bug Components: filters Affects Versions: 0.92.1, 0.94.0 Reporter: Alex Newman Assignee: Alex Newman Priority: Minor Attachments: 0001-HBASE-6431.-Some-FilterList-Constructors-break-addFi.patch Some of the constructors for FilterList set the internal list of filters to list types which don't support the add operation. As a result FilterList(final ListFilter rowFilters) FilterList(final Filter... rowFilters) FilterList(final Operator operator, final ListFilter rowFilters) FilterList(final Operator operator, final Filter... rowFilters) may init private ListFilter filters = new ArrayListFilter(); incorrectly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5705) Introduce Protocol Buffer RPC engine
[ https://issues.apache.org/jira/browse/HBASE-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422838#comment-13422838 ] Hadoop QA commented on HBASE-5705: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12537940/5705-2.2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 17 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 15 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2438//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2438//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2438//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2438//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2438//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2438//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2438//console This message is automatically generated. Introduce Protocol Buffer RPC engine Key: HBASE-5705 URL: https://issues.apache.org/jira/browse/HBASE-5705 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.96.0 Attachments: 5705-1.patch, 5705-2.1.patch, 5705-2.2.patch Introduce Protocol Buffer RPC engine in the RPC core. Protocols that are PB aware can be made to go through this RPC engine. The approach, in my current thinking, would be similar to HADOOP-7773. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6431) Some FilterList Constructors break addFilter
[
https://issues.apache.org/jira/browse/HBASE-6431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422849#comment-13422849
]
Zhihong Ted Yu commented on HBASE-6431:
---
The second question above was about this:
{code}
public FilterList(final ListFilter rowFilters) {
{code}
Not this:
{code}
public FilterList(final Filter... rowFilters) {
{code}
The third question pertains to the fact that Arrays.asList() creates ArrayList
internally. I don't see the need to wrap it with new ArrayListFilter.
Some FilterList Constructors break addFilter
Key: HBASE-6431
URL: https://issues.apache.org/jira/browse/HBASE-6431
Project: HBase
Issue Type: Bug
Components: filters
Affects Versions: 0.92.1, 0.94.0
Reporter: Alex Newman
Assignee: Alex Newman
Priority: Minor
Attachments:
0001-HBASE-6431.-Some-FilterList-Constructors-break-addFi.patch
Some of the constructors for FilterList set the internal list of filters to
list types which don't support the add operation. As a result
FilterList(final ListFilter rowFilters)
FilterList(final Filter... rowFilters)
FilterList(final Operator operator, final ListFilter rowFilters)
FilterList(final Operator operator, final Filter... rowFilters)
may init private ListFilter filters = new ArrayListFilter(); incorrectly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6431) Some FilterList Constructors break addFilter
[ https://issues.apache.org/jira/browse/HBASE-6431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422853#comment-13422853 ] Alex Newman commented on HBASE-6431: Internally arrays.aslist isn't an arraylist on my vm.could you double check? Some FilterList Constructors break addFilter Key: HBASE-6431 URL: https://issues.apache.org/jira/browse/HBASE-6431 Project: HBase Issue Type: Bug Components: filters Affects Versions: 0.92.1, 0.94.0 Reporter: Alex Newman Assignee: Alex Newman Priority: Minor Attachments: 0001-HBASE-6431.-Some-FilterList-Constructors-break-addFi.patch Some of the constructors for FilterList set the internal list of filters to list types which don't support the add operation. As a result FilterList(final ListFilter rowFilters) FilterList(final Filter... rowFilters) FilterList(final Operator operator, final ListFilter rowFilters) FilterList(final Operator operator, final Filter... rowFilters) may init private ListFilter filters = new ArrayListFilter(); incorrectly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6431) Some FilterList Constructors break addFilter
[ https://issues.apache.org/jira/browse/HBASE-6431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-6431: -- Attachment: 6431-v2.txt I was wrong about the implementation of Arrays.asList(). Here is a patch illustrating my other two points. TestFilterList#testAddFilter passes based on patch v2. Some FilterList Constructors break addFilter Key: HBASE-6431 URL: https://issues.apache.org/jira/browse/HBASE-6431 Project: HBase Issue Type: Bug Components: filters Affects Versions: 0.92.1, 0.94.0 Reporter: Alex Newman Assignee: Alex Newman Priority: Minor Attachments: 0001-HBASE-6431.-Some-FilterList-Constructors-break-addFi.patch, 6431-v2.txt Some of the constructors for FilterList set the internal list of filters to list types which don't support the add operation. As a result FilterList(final ListFilter rowFilters) FilterList(final Filter... rowFilters) FilterList(final Operator operator, final ListFilter rowFilters) FilterList(final Operator operator, final Filter... rowFilters) may init private ListFilter filters = new ArrayListFilter(); incorrectly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6455) org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path
[ https://issues.apache.org/jira/browse/HBASE-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422855#comment-13422855 ] Zhihong Ted Yu commented on HBASE-6455: --- Patch looks good to me. org.apache.hadoop.hbase.PerformanceEvaluation sets the map reduce output path as a child of input path -- Key: HBASE-6455 URL: https://issues.apache.org/jira/browse/HBASE-6455 Project: HBase Issue Type: Bug Components: performance Affects Versions: 0.94.0, 0.96.0 Reporter: Aditya Kishore Assignee: Aditya Kishore Priority: Minor Labels: benchmark Fix For: 0.96.0 Attachments: HBASE-6455_trunk.patch I was the running PerformanceEvaluation test with a modified job configuration where the job output path is created before the splits and that unmasked the issue because the the PeInputFormat.getSplits() function expects to find only files under the input path. The attached patch addresses both the issues. # Creates separate input and output path rooted under a single folder e.g. user_home/performance_evaluation/MMddHHmmss/inputs and user_home/performance_evaluation/MMddHHmmss/outputs, and # The PeInputFormat.getSplits(), now skips any folder found under the input path and process only files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
