[jira] [Updated] (HBASE-9533) List of dependency jars for MR jobs is hard-coded and does not include netty, breaking MRv1 jobs
[
https://issues.apache.org/jira/browse/HBASE-9533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Shulman updated HBASE-9533:
-
Summary: List of dependency jars for MR jobs is hard-coded and does not
include netty, breaking MRv1 jobs (was: List of dependancy jars for MR jobs is
hard-coded and does not include netty, breaking MRv1 jobs)
> List of dependency jars for MR jobs is hard-coded and does not include netty,
> breaking MRv1 jobs
>
>
> Key: HBASE-9533
> URL: https://issues.apache.org/jira/browse/HBASE-9533
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 0.95.2, 0.96.1
>Reporter: Aleksandr Shulman
>Assignee: Matteo Bertozzi
> Fix For: 0.95.2, 0.96.1
>
> Attachments: failed_mrv1_rowcounter_tt_taskoutput.out
>
>
> Observed behavior:
> Against trunk, using MRv1 with hadoop 1.0.4, r1393290, I am able to run MRv1
> jobs (e.g. pi 2 4).
> However, when I use it to run MR over HBase jobs, they fail with the stack
> trace below.
> From the trace, the issue seems to be that it cannot find a class that the
> netty jar contains. This would make sense, given that the dependency jars
> that we use for the MapReduce job are hard-coded, and that the netty jar is
> not one of them.
> https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java#L519
> Strangely, this is only an issue in trunk, not 0.95, even though the code
> hasn't changed.
> Command:
> {code}/bin/hbase org.apache.hadoop.hbase.mapreduce.RowCounter
> sampletable{code}
> TT logs (attached)
> Output from console running job:
> {code}13/09/13 16:02:58 INFO mapred.JobClient: Task Id :
> attempt_201309131601_0002_m_00_2, Status : FAILED
> java.io.IOException: Cannot create a record reader because of a previous
> error. Please look at the previous logs lines from the task's full log for
> more details.
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.createRecordReader(TableInputFormatBase.java:119)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.(MapTask.java:489)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:731)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> 13/09/13 16:03:09 INFO mapred.JobClient: Job complete: job_201309131601_0002
> 13/09/13 16:03:09 INFO mapred.JobClient: Counters: 7
> 13/09/13 16:03:09 INFO mapred.JobClient: Job Counters
> 13/09/13 16:03:09 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=29913
> 13/09/13 16:03:09 INFO mapred.JobClient: Total time spent by all reduces
> waiting after reserving slots (ms)=0
> 13/09/13 16:03:09 INFO mapred.JobClient: Total time spent by all maps
> waiting after reserving slots (ms)=0
> 13/09/13 16:03:09 INFO mapred.JobClient: Launched map tasks=4
> 13/09/13 16:03:09 INFO mapred.JobClient: Data-local map tasks=4
> 13/09/13 16:03:09 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
> 13/09/13 16:03:09 INFO mapred.JobClient: Failed map tasks=1{code}
> Expected behavior:
> As a stopgap, the netty jar should be included in that list. More generally,
> there should be a more elegant way to include the jars that are needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9533) List of dependancy jars for MR jobs is hard-coded and does not include netty, breaking MRv1 jobs
[
https://issues.apache.org/jira/browse/HBASE-9533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Shulman updated HBASE-9533:
-
Summary: List of dependancy jars for MR jobs is hard-coded and does not
include netty, breaking MRv1 jobs (was: List of imported jars is hard-coded
and does not include netty, breaking MRv1 jobs)
> List of dependancy jars for MR jobs is hard-coded and does not include netty,
> breaking MRv1 jobs
>
>
> Key: HBASE-9533
> URL: https://issues.apache.org/jira/browse/HBASE-9533
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 0.95.2, 0.96.1
>Reporter: Aleksandr Shulman
>Assignee: Matteo Bertozzi
> Fix For: 0.95.2, 0.96.1
>
> Attachments: failed_mrv1_rowcounter_tt_taskoutput.out
>
>
> Observed behavior:
> Against trunk, using MRv1 with hadoop 1.0.4, r1393290, I am able to run MRv1
> jobs (e.g. pi 2 4).
> However, when I use it to run MR over HBase jobs, they fail with the stack
> trace below.
> From the trace, the issue seems to be that it cannot find a class that the
> netty jar contains. This would make sense, given that the dependency jars
> that we use for the MapReduce job are hard-coded, and that the netty jar is
> not one of them.
> https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java#L519
> Strangely, this is only an issue in trunk, not 0.95, even though the code
> hasn't changed.
> Command:
> {code}/bin/hbase org.apache.hadoop.hbase.mapreduce.RowCounter
> sampletable{code}
> TT logs (attached)
> Output from console running job:
> {code}13/09/13 16:02:58 INFO mapred.JobClient: Task Id :
> attempt_201309131601_0002_m_00_2, Status : FAILED
> java.io.IOException: Cannot create a record reader because of a previous
> error. Please look at the previous logs lines from the task's full log for
> more details.
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.createRecordReader(TableInputFormatBase.java:119)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.(MapTask.java:489)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:731)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> 13/09/13 16:03:09 INFO mapred.JobClient: Job complete: job_201309131601_0002
> 13/09/13 16:03:09 INFO mapred.JobClient: Counters: 7
> 13/09/13 16:03:09 INFO mapred.JobClient: Job Counters
> 13/09/13 16:03:09 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=29913
> 13/09/13 16:03:09 INFO mapred.JobClient: Total time spent by all reduces
> waiting after reserving slots (ms)=0
> 13/09/13 16:03:09 INFO mapred.JobClient: Total time spent by all maps
> waiting after reserving slots (ms)=0
> 13/09/13 16:03:09 INFO mapred.JobClient: Launched map tasks=4
> 13/09/13 16:03:09 INFO mapred.JobClient: Data-local map tasks=4
> 13/09/13 16:03:09 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
> 13/09/13 16:03:09 INFO mapred.JobClient: Failed map tasks=1{code}
> Expected behavior:
> As a stopgap, the netty jar should be included in that list. More generally,
> there should be a more elegant way to include the jars that are needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767344#comment-13767344 ] Aleksandr Shulman commented on HBASE-9153: -- Hi [~ndimiduk], if there are no other objections, would it be possible to get this committed? > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Task >Reporter: Jonathan Hsieh > Attachments: HBASE-9153-v1.patch, HBASE-9153-v3.patch, > HBASE-9153-v4-0.94.patch, HBASE-9153-v4-0.95.patch, HBASE-9153-v4-trunk.patch > > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9533) List of imported jars is hard-coded and does not include netty, breaking MRv1 jobs
[
https://issues.apache.org/jira/browse/HBASE-9533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Shulman updated HBASE-9533:
-
Attachment: failed_mrv1_rowcounter_tt_taskoutput.out
Attached are the tasktracker logs, which are more valuable. The salient portion
is this:
{code}Caused by: java.lang.NoClassDefFoundError:
org/jboss/netty/channel/ChannelFactory
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:802)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:847)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:873)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.(HConnectionManager.java:675)
... 19 more
Caused by: java.lang.ClassNotFoundException:
org.jboss.netty.channel.ChannelFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 25 more{code}
> List of imported jars is hard-coded and does not include netty, breaking MRv1
> jobs
> --
>
> Key: HBASE-9533
> URL: https://issues.apache.org/jira/browse/HBASE-9533
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 0.95.2, 0.96.1
>Reporter: Aleksandr Shulman
>Assignee: Matteo Bertozzi
> Fix For: 0.95.2, 0.96.1
>
> Attachments: failed_mrv1_rowcounter_tt_taskoutput.out
>
>
> Observed behavior:
> Against trunk, using MRv1 with hadoop 1.0.4, r1393290, I am able to run MRv1
> jobs (e.g. pi 2 4).
> However, when I use it to run MR over HBase jobs, they fail with the stack
> trace below.
> From the trace, the issue seems to be that it cannot find a class that the
> netty jar contains. This would make sense, given that the dependency jars
> that we use for the MapReduce job are hard-coded, and that the netty jar is
> not one of them.
> https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java#L519
> Strangely, this is only an issue in trunk, not 0.95, even though the code
> hasn't changed.
> Command:
> {code}/bin/hbase org.apache.hadoop.hbase.mapreduce.RowCounter
> sampletable{code}
> TT logs (attached)
> Output from console running job:
> {code}13/09/13 16:02:58 INFO mapred.JobClient: Task Id :
> attempt_201309131601_0002_m_00_2, Status : FAILED
> java.io.IOException: Cannot create a record reader because of a previous
> error. Please look at the previous logs lines from the task's full log for
> more details.
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.createRecordReader(TableInputFormatBase.java:119)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.(MapTask.java:489)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:731)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> 13/09/13 16:03:09 INFO mapred.JobClient: Job complete: job_201309131601_0002
> 13/09/13 16:03:09 INFO mapred.JobClient: Counters: 7
> 13/09/13 16:03:09 INFO mapred.JobClient: Job Counters
> 13/09/13 16:03:09 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=29913
> 13/09/13 16:03:09 INFO mapred.JobClient: Total time spent by all reduces
> waiting after reserving slots (ms)=0
> 13/09/13 16:03:09 INFO mapred.JobClient: Total time spent by all maps
> waiting after reserving slots (ms)=0
> 13/09/13 16:03:09 INFO mapred.JobClient: Launched map tasks=4
> 13/09/13 16:03:09 INFO mapred.JobClient: Data-local map tasks=4
> 13/09/13 16:03:09 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
> 13/09/13 16:03:09 INFO mapred.JobClient: Failed map tasks=1{code}
> Expected behavior:
> As a stopgap, the netty jar should be included in that list. More generally,
> there should be a more elegant way to include the jars that are needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For
[jira] [Created] (HBASE-9533) List of imported jars is hard-coded and does not include netty, breaking MRv1 jobs
Aleksandr Shulman created HBASE-9533:
Summary: List of imported jars is hard-coded and does not include
netty, breaking MRv1 jobs
Key: HBASE-9533
URL: https://issues.apache.org/jira/browse/HBASE-9533
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.95.2, 0.96.1
Reporter: Aleksandr Shulman
Assignee: Matteo Bertozzi
Fix For: 0.96.1, 0.95.2
Observed behavior:
Against trunk, using MRv1 with hadoop 1.0.4, r1393290, I am able to run MRv1
jobs (e.g. pi 2 4).
However, when I use it to run MR over HBase jobs, they fail with the stack
trace below.
>From the trace, the issue seems to be that it cannot find a class that the
>netty jar contains. This would make sense, given that the dependency jars that
>we use for the MapReduce job are hard-coded, and that the netty jar is not one
>of them.
https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java#L519
Strangely, this is only an issue in trunk, not 0.95, even though the code
hasn't changed.
Command:
{code}/bin/hbase org.apache.hadoop.hbase.mapreduce.RowCounter sampletable{code}
TT logs (attached)
Output from console running job:
{code}13/09/13 16:02:58 INFO mapred.JobClient: Task Id :
attempt_201309131601_0002_m_00_2, Status : FAILED
java.io.IOException: Cannot create a record reader because of a previous error.
Please look at the previous logs lines from the task's full log for more
details.
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.createRecordReader(TableInputFormatBase.java:119)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.(MapTask.java:489)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:731)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
13/09/13 16:03:09 INFO mapred.JobClient: Job complete: job_201309131601_0002
13/09/13 16:03:09 INFO mapred.JobClient: Counters: 7
13/09/13 16:03:09 INFO mapred.JobClient: Job Counters
13/09/13 16:03:09 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=29913
13/09/13 16:03:09 INFO mapred.JobClient: Total time spent by all reduces
waiting after reserving slots (ms)=0
13/09/13 16:03:09 INFO mapred.JobClient: Total time spent by all maps
waiting after reserving slots (ms)=0
13/09/13 16:03:09 INFO mapred.JobClient: Launched map tasks=4
13/09/13 16:03:09 INFO mapred.JobClient: Data-local map tasks=4
13/09/13 16:03:09 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
13/09/13 16:03:09 INFO mapred.JobClient: Failed map tasks=1{code}
Expected behavior:
As a stopgap, the netty jar should be included in that list. More generally,
there should be a more elegant way to include the jars that are needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9497) Old .META. .tableinfo file kills HMaster
[
https://issues.apache.org/jira/browse/HBASE-9497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13764997#comment-13764997
]
Aleksandr Shulman commented on HBASE-9497:
--
Thanks [~saint@gmail.com] :)
Yep, we should test both clean shutdown and messy shutdown as part of general
upgrade testing.
> Old .META. .tableinfo file kills HMaster
>
>
> Key: HBASE-9497
> URL: https://issues.apache.org/jira/browse/HBASE-9497
> Project: HBase
> Issue Type: Bug
> Components: master, migration
>Affects Versions: 0.95.2
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.98.0, 0.96.0
>
> Attachments: HBase-9497.patch
>
>
> In pre-0.96, .META. has .tableinfo files which refer to .META. On startup,
> master tries to read it and aborts since the table name has changed. The
> .META. .tableinfo files are not being created in 0.94.x (fixed for 96 in
> HBASE-6971; but this can be reproduced when migrating from 0.92 -> 0.94 ->
> 0.96. Our old users would be affected by this.
> {code}
> java.lang.IllegalArgumentException: .META. no longer exists. The table has
> been renamed to hbase:meta
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:291)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:283)
> at
> org.apache.hadoop.hbase.HTableDescriptor.readFields(HTableDescriptor.java:960)
> at
> org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:131)
> at
> org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:101)
> at
> org.apache.hadoop.hbase.HTableDescriptor.parseFrom(HTableDescriptor.java:1407)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.readTableDescriptor(FSTableDescriptors.java:521)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.createTableDescriptorForTableDirectory(FSTableDescriptors.java:707)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.createTableDescriptor(FSTableDescriptors.java:683)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.createTableDescriptor(FSTableDescriptors.java:670)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:485)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:145)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:129)
> at
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:761)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9153) Create a deprecation policy enforcement check
[
https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13759717#comment-13759717
]
Aleksandr Shulman commented on HBASE-9153:
--
The usage is a little confusing. The arguments come in pairs. So if you specify
a local source dir for $1, then $2 is the name of that code. It's arbitrary.
The same is true for 3 and 4, respectively. If you specify a git repo as
argument $1 or argument $3, then arguments $2 or $4, respectively, is the name
of the branch. This is not arbitrary.
In either case, arguments $2 and $4 are used as labels in the report.
You can have two references from the same git repo, but you'd have to specify
it twice, once as argument $1 and once as argument $3.
{quote}how do I specify a local repo{quote}
You can use the absolute location of the /hbase folder. So if my code path
looked like /home/aleks/code/hbase/0.95/hbase/hbase-server/... then I would
give /home/aleks/code/hbase/0.95/hbase as the argument.
{quote}I'm seeing lots of errors{quote}
I have seen it as well. I don't know the cause but the resulting reports look
correct and it doesn't cause the operation to fail.
We should look into it is being generated, however.
Also, the script will appear to hang at times, but it's not actually hanging.
Note that it takes ~30 minutes to do a run.
> Create a deprecation policy enforcement check
> -
>
> Key: HBASE-9153
> URL: https://issues.apache.org/jira/browse/HBASE-9153
> Project: HBase
> Issue Type: Task
>Reporter: Jonathan Hsieh
> Attachments: HBASE-9153-v1.patch, HBASE-9153-v3.patch,
> HBASE-9153-v4-0.94.patch, HBASE-9153-v4-0.95.patch, HBASE-9153-v4-trunk.patch
>
>
> We've had a few issues now where we've removed API's without deprecating or
> deprecating in the late release. (HBASE-9142, HBASE-9093) We should just
> have a tool that enforces our api deprecation policy as a release time check
> or as a precommit check.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13759656#comment-13759656 ] Aleksandr Shulman commented on HBASE-9153: -- Oops. Yes, I do. Would you like me to submit a new patch with the fix? > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Task >Reporter: Jonathan Hsieh > Attachments: HBASE-9153-v1.patch, HBASE-9153-v3.patch, > HBASE-9153-v4-0.94.patch, HBASE-9153-v4-0.95.patch, HBASE-9153-v4-trunk.patch > > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9153) Create a deprecation policy enforcement check
[
https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13759392#comment-13759392
]
Aleksandr Shulman commented on HBASE-9153:
--
{quote}These are the steps I was expecting. Do you plan to include those as
further patch revisions on this ticket or open followup tickets?{quote}
That's a good question. I did a first-pass at steps 1 and 2 and I could not
find a quick one-off to the tool to allow that. Therefore, it'll probably
involve a non-trivial amount of work. That said, they are on my back-burner.
Meanwhile, it would be great if other folks wanted to maybe take a look.
{quote}Indeed. And here I thought you were doing to reveal some of the secret
sauce behind your recent blog post {quote}
:)
> Create a deprecation policy enforcement check
> -
>
> Key: HBASE-9153
> URL: https://issues.apache.org/jira/browse/HBASE-9153
> Project: HBase
> Issue Type: Task
>Reporter: Jonathan Hsieh
> Attachments: HBASE-9153-v1.patch, HBASE-9153-v3.patch,
> HBASE-9153-v4-0.94.patch, HBASE-9153-v4-0.95.patch, HBASE-9153-v4-trunk.patch
>
>
> We've had a few issues now where we've removed API's without deprecating or
> deprecating in the late release. (HBASE-9142, HBASE-9093) We should just
> have a tool that enforces our api deprecation policy as a release time check
> or as a precommit check.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-9153: - Attachment: HBASE-9153-v4-trunk.patch HBASE-9153-v4-0.94.patch HBASE-9153-v4-0.95.patch Latest diffs from code review with a minor comment change. For 0.94 and 0.95, the diff is different because the files are a pure addition. For trunk the jdiff tool already existed, and the patch will modify that tool. When all 3 applied to their respective branches, the jdiff tool will be at parity across all 3. > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Task >Reporter: Jonathan Hsieh > Attachments: HBASE-9153-v1.patch, HBASE-9153-v3.patch, > HBASE-9153-v4-0.94.patch, HBASE-9153-v4-0.95.patch, HBASE-9153-v4-trunk.patch > > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13759332#comment-13759332 ] Aleksandr Shulman commented on HBASE-9153: -- Hi [~ndimiduk], I haven't attached the patch yet. I will do so shortly. You're correct in that the checkin does not do any enforcement, nor does it have an automatic way to detect incompatible changes. These are potential next steps. Other next steps: 1. Have JDiff only examine a specific set of classes/packagages that are client-facing. 2. Have it raise an alert anytime something incompatible is introduced. 2a. Have it mute known incompatibilities 3. Add it as a pre-checkin hook Finally, I recall you had a question about text format. So far I haven't found a way to generate a text-only format. It would definitely be useful for parsing and so forth! > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Task >Reporter: Jonathan Hsieh > Attachments: HBASE-9153-v1.patch, HBASE-9153-v3.patch > > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-9153: - Attachment: HBASE-9153-v3.patch > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Task >Reporter: Jonathan Hsieh > Attachments: HBASE-9153-v1.patch, HBASE-9153-v3.patch > > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13758631#comment-13758631 ] Aleksandr Shulman commented on HBASE-9153: -- >From the feedback of the previous review, I made some pretty sweeping changes. >I'll go ahead and put it up on ReviewBoard shortly. At a high level: - Report can be run anywhere - User does not need to specify whether the versions are before, across, or after singularity. This is autodetected. - Bits can be sourced from either git or from local branch https://reviews.apache.org/r/13788/ > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Task >Reporter: Jonathan Hsieh > Attachments: HBASE-9153-v1.patch, HBASE-9153-v3.patch > > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13753923#comment-13753923 ] Aleksandr Shulman commented on HBASE-9153: -- [~apurtell], sorry for the delay in responding. Right now it's git only. If you would like to add support for SVN, I can help you with that. Singularity - I'm not fond of the name either. IThe tool could detect the directory structure and just decide which template to apply. In fact, that sounds like an easy fix. I'll add it to the next revision. It'll also make it more elegant with fewer files. > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Task >Reporter: Jonathan Hsieh > Attachments: HBASE-9153-v1.patch > > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-9153: - Status: Patch Available (was: Open) > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Task >Reporter: Jonathan Hsieh > Attachments: HBASE-9153-v1.patch > > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-9153: - Attachment: HBASE-9153-v1.patch First revision of the patch. Also on reviewboard. > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Task >Reporter: Jonathan Hsieh > Attachments: HBASE-9153-v1.patch > > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749133#comment-13749133 ] Aleksandr Shulman commented on HBASE-9153: -- Hey folks, I have a review up. https://reviews.apache.org/r/13763/ Feedback appreciated! > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Task >Reporter: Jonathan Hsieh > Attachments: HBASE-9153-v1.patch > > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13744528#comment-13744528 ] Aleksandr Shulman commented on HBASE-9153: -- I am working on a modification of my JDiff tool that'll work with the directory formats before and after the singularity between 0.94 and 0.95. I'll put it under formal review soon, but if folks are interested at a sneak peek: https://github.com/AleksandrShulman/hbase_jdiff 1. The initial report between 0.94 and 0.95 is up in https://github.com/AleksandrShulman/hbase_jdiff/tree/master/reports. You'll need to clone the repo since it's too large to download raw. 2. The scripts to do this yourself are also available in that project. For now it is up to the user to manually examine the report. I haven't yet built a way for it to automatically detect an incompatible change. That part may come next. > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-9153: - Issue Type: Task (was: Bug) > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Task >Reporter: Jonathan Hsieh > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman reassigned HBASE-9153: Assignee: Aleksandr Shulman > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh >Assignee: Aleksandr Shulman > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-9153) Create a deprecation policy enforcement check
[ https://issues.apache.org/jira/browse/HBASE-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman reassigned HBASE-9153: Assignee: (was: Aleksandr Shulman) > Create a deprecation policy enforcement check > - > > Key: HBASE-9153 > URL: https://issues.apache.org/jira/browse/HBASE-9153 > Project: HBase > Issue Type: Bug >Reporter: Jonathan Hsieh > > We've had a few issues now where we've removed API's without deprecating or > deprecating in the late release. (HBASE-9142, HBASE-9093) We should just > have a tool that enforces our api deprecation policy as a release time check > or as a precommit check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-9223) HBase ZKCli gives error because it is looking for class removed in 0.95: org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
[
https://issues.apache.org/jira/browse/HBASE-9223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Shulman resolved HBASE-9223.
--
Resolution: Cannot Reproduce
> HBase ZKCli gives error because it is looking for class removed in 0.95:
> org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
> -
>
> Key: HBASE-9223
> URL: https://issues.apache.org/jira/browse/HBASE-9223
> Project: HBase
> Issue Type: Bug
> Components: util, Zookeeper
>Affects Versions: 0.95.2
>Reporter: Aleksandr Shulman
>
> Observed behavior:
> Command: hbase zkcli
> There is an error message about a missing class:
> {code}[root@data-compat-3 hbase]# hbase zkcli
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/zookeeper/ZooKeeperMainServerArg
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> {code}
> It looks like this class is no longer in 0.95.
> ZKCli still continues in its startup and continues working (at least reads
> continue to work)
> Expected behavior:
> HBase Zkcli should not be looking for classes that are no longer part of
> HBase. This indicates an underlying issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9223) HBase ZKCli gives error because it is looking for class removed in 0.95: org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
[
https://issues.apache.org/jira/browse/HBASE-9223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13740763#comment-13740763
]
Aleksandr Shulman commented on HBASE-9223:
--
It must be something more subtle then. I've eliminated JDK6 vs. JDK7. For now,
since this works cleanly on stock 0.95.2, let's assume it's an environmental
issue somewhere on my end.
> HBase ZKCli gives error because it is looking for class removed in 0.95:
> org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
> -
>
> Key: HBASE-9223
> URL: https://issues.apache.org/jira/browse/HBASE-9223
> Project: HBase
> Issue Type: Bug
> Components: util, Zookeeper
>Affects Versions: 0.95.2
>Reporter: Aleksandr Shulman
>
> Observed behavior:
> Command: hbase zkcli
> There is an error message about a missing class:
> {code}[root@data-compat-3 hbase]# hbase zkcli
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/zookeeper/ZooKeeperMainServerArg
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> {code}
> It looks like this class is no longer in 0.95.
> ZKCli still continues in its startup and continues working (at least reads
> continue to work)
> Expected behavior:
> HBase Zkcli should not be looking for classes that are no longer part of
> HBase. This indicates an underlying issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9223) HBase ZKCli gives error because it is looking for class removed in 0.95: org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
[
https://issues.apache.org/jira/browse/HBASE-9223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13740751#comment-13740751
]
Aleksandr Shulman commented on HBASE-9223:
--
The tool still works. At face value, it's just a warning that comes up.
However, that warning might be indicative of other problems.
> HBase ZKCli gives error because it is looking for class removed in 0.95:
> org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
> -
>
> Key: HBASE-9223
> URL: https://issues.apache.org/jira/browse/HBASE-9223
> Project: HBase
> Issue Type: Bug
> Components: util, Zookeeper
>Affects Versions: 0.95.2
>Reporter: Aleksandr Shulman
>
> Observed behavior:
> Command: hbase zkcli
> There is an error message about a missing class:
> {code}[root@data-compat-3 hbase]# hbase zkcli
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/zookeeper/ZooKeeperMainServerArg
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> {code}
> It looks like this class is no longer in 0.95.
> ZKCli still continues in its startup and continues working (at least reads
> continue to work)
> Expected behavior:
> HBase Zkcli should not be looking for classes that are no longer part of
> HBase. This indicates an underlying issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9223) HBase ZKCli gives error because it is looking for class removed in 0.95: org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
[
https://issues.apache.org/jira/browse/HBASE-9223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13740744#comment-13740744
]
Aleksandr Shulman commented on HBASE-9223:
--
Commit 90d74327e21d74386e90eb35b8b7276d5576efa0 on the 0.95 branch on June
19th, 2013 renamed oahh.zookeeper.ZooKeeperMainServerArg.java to
oahh.zookeeper.ZooKeeperMainServer.java
https://github.com/apache/hbase/commit/90d74327e21d74386e90eb35b8b7276d5576efa0
> HBase ZKCli gives error because it is looking for class removed in 0.95:
> org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
> -
>
> Key: HBASE-9223
> URL: https://issues.apache.org/jira/browse/HBASE-9223
> Project: HBase
> Issue Type: Bug
> Components: util, Zookeeper
>Affects Versions: 0.95.2
>Reporter: Aleksandr Shulman
> Fix For: 0.95.2
>
>
> Observed behavior:
> Command: hbase zkcli
> There is an error message about a missing class:
> {code}[root@data-compat-3 hbase]# hbase zkcli
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/zookeeper/ZooKeeperMainServerArg
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> {code}
> It looks like this class is no longer in 0.95.
> ZKCli still continues in its startup and continues working (at least reads
> continue to work)
> Expected behavior:
> HBase Zkcli should not be looking for classes that are no longer part of
> HBase. This indicates an underlying issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9223) HBase ZKCli gives error because it is looking for class removed in 0.95: org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
[
https://issues.apache.org/jira/browse/HBASE-9223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13740387#comment-13740387
]
Aleksandr Shulman commented on HBASE-9223:
--
For folks attempting to repro, this is built from commit:
4b9e81d93e563eb15d7ee86e80c6029d8e3b79aa on the 0.95 branch.
> HBase ZKCli gives error because it is looking for class removed in 0.95:
> org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
> -
>
> Key: HBASE-9223
> URL: https://issues.apache.org/jira/browse/HBASE-9223
> Project: HBase
> Issue Type: Bug
> Components: util, Zookeeper
>Affects Versions: 0.95.2
>Reporter: Aleksandr Shulman
> Fix For: 0.95.2
>
>
> Observed behavior:
> Command: hbase zkcli
> There is an error message about a missing class:
> {code}[root@data-compat-3 hbase]# hbase zkcli
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/zookeeper/ZooKeeperMainServerArg
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> {code}
> It looks like this class is no longer in 0.95.
> ZKCli still continues in its startup and continues working (at least reads
> continue to work)
> Expected behavior:
> HBase Zkcli should not be looking for classes that are no longer part of
> HBase. This indicates an underlying issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9223) HBase ZKCli gives error because it is looking for class removed in 0.95: org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
Aleksandr Shulman created HBASE-9223:
Summary: HBase ZKCli gives error because it is looking for class
removed in 0.95: org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
Key: HBASE-9223
URL: https://issues.apache.org/jira/browse/HBASE-9223
Project: HBase
Issue Type: Bug
Components: util, Zookeeper
Affects Versions: 0.95.2
Reporter: Aleksandr Shulman
Fix For: 0.95.2
Observed behavior:
Command: hbase zkcli
There is an error message about a missing class:
{code}[root@data-compat-3 hbase]# hbase zkcli
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/hbase/zookeeper/ZooKeeperMainServerArg
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
{code}
It looks like this class is no longer in 0.95.
ZKCli still continues in its startup and continues working (at least reads
continue to work)
Expected behavior:
HBase Zkcli should not be looking for classes that are no longer part of HBase.
This indicates an underlying issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8619) [HBase Client]: Improve client to retry if master is down when requesting getHTableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13736179#comment-13736179
]
Aleksandr Shulman commented on HBASE-8619:
--
I'm seeing this occur in my testing. It's a race condition so it's hard to
repro, unfortunately.
The latest stack trace where I saw it looked like this:
{code} Exception in thread "main"
java.lang.reflect.UndeclaredThrowableException
20:30:00at $Proxy7.getHTableDescriptors(Unknown Source)
20:30:00at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHTableDescriptor(HConnectionManager.java:1863)
20:30:00at
org.apache.hadoop.hbase.client.HTable.getTableDescriptor(HTable.java:440)
20:30:00at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureCompression(HFileOutputFormat.java:458)
20:30:00at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:375)
20:30:00at
org.apache.hadoop.hbase.mapreduce.ImportTsv.createSubmittableJob(ImportTsv.java:280)
20:30:00at
org.apache.hadoop.hbase.mapreduce.ImportTsv.main(ImportTsv.java:424)
20:30:00 Caused by: java.io.IOException: Call to
hbase-rolling-6.ent.cloudera.com/10.20.186.99:22001 failed on local exception:
java.io.IOException: Connection reset by peer
20:30:00at
org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:1033)
20:30:00at
org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1002)
20:30:00at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:86)
20:30:00... 7 more
20:30:00 Caused by: java.io.IOException: Connection reset by peer
20:30:00at sun.nio.ch.FileDispatcher.read0(Native Method)
20:30:00at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
20:30:00at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198)
20:30:00at sun.nio.ch.IOUtil.read(IOUtil.java:171)
20:30:00at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:243)
20:30:00at
org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:56)
20:30:00at
org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:143)
20:30:00at
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:156)
20:30:00at
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:129)
20:30:00at java.io.FilterInputStream.read(FilterInputStream.java:116)
20:30:00at java.io.FilterInputStream.read(FilterInputStream.java:116)
20:30:00at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection$PingInputStream.read(HBaseClient.java:373)
20:30:00at
java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
20:30:00at
java.io.BufferedInputStream.read(BufferedInputStream.java:237)
20:30:00at java.io.DataInputStream.readInt(DataInputStream.java:370)
20:30:00at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:646)
20:30:00at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:580)
{code}
> [HBase Client]: Improve client to retry if master is down when requesting
> getHTableDescriptor
> -
>
> Key: HBASE-8619
> URL: https://issues.apache.org/jira/browse/HBASE-8619
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.94.8
>Reporter: Aleksandr Shulman
>Assignee: Elliott Clark
>Priority: Minor
> Fix For: 0.94.12
>
>
> In our rolling upgrade testing, running ImportTsv failed in the job
> submission phase when the master was down. This was when the master was
> failing over to the backup master. In this case, a retry would have been
> helpful and made sure that the job would get submitted.
> A good solution would be to refresh the master information before placing the
> call to getHTableDescriptor.
> Command:
> {code} sudo -u hbase hbase org.apache.hadoop.hbase.mapreduce.ImportTsv
> -Dimporttsv.columns=HBASE_ROW_KEY,c1,c2,c3
> -Dimporttsv.bulk.output=/user/hbase/storeFiles2_2/import2_table1369439156
> import2_table1369439156 /user/hbase/tsv2{code}
> Here is the stack trace:
> {code} 13/05/24 16:55:49 INFO compress.CodecPool: Got brand-new compressor
> [.deflate]
> 16:45:44 Exception in thread "main"
> java.lang.reflect.UndeclaredThrowableException
> 16:45:44 at $Proxy7.getHTableDescriptors(Unknown Source)
> 16:45:44 at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHTableDescriptor(HConnectionManager.java:1861)
> 16:45:44 at
> org.apache.hadoop.hbase.client.HTable.getTableDescri
[jira] [Commented] (HBASE-8200) Integration Test classes are not part of the default HBase classpath. We should fix this or note workaround in documentation
[ https://issues.apache.org/jira/browse/HBASE-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13703811#comment-13703811 ] Aleksandr Shulman commented on HBASE-8200: -- Patch looks good. Thanks Enis! > Integration Test classes are not part of the default HBase classpath. We > should fix this or note workaround in documentation > > > Key: HBASE-8200 > URL: https://issues.apache.org/jira/browse/HBASE-8200 > Project: HBase > Issue Type: Bug > Components: test > Environment: ubuntu >Reporter: Aleksandr Shulman >Assignee: Enis Soztutar > Fix For: 0.98.0, 0.95.2 > > Attachments: hbase-8200_v1.patch > > > Expected behavior: > Following the instructions here: > http://hbase.apache.org/book/hbase.tests.html#integration.tests > After pulling down the latest bits from trunk and running, > > mvn clean compile test-compile package install assembly:assembly -DskipTests > I should be able to run integration tests with the following command: > > bin/hbase --config /path/to/configs > > org.apache.hadoop.hbase.IntegrationTestsDriver > According to the documentation, this should kick off the tests. > Observed behavior: > Upon executing > > bin/hbase --config /path/to/configs > > org.apache.hadoop.hbase.IntegrationTestsDriver > I see the following: > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/hadoop/hbase/IntegrationTestsDriver > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.hbase.IntegrationTestsDriver > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > I can fix it by adding the following line to the hbase-env.sh file > corresponding to the configs: > export HBASE_CLASSPATH=/path/to/project/hbase/hbase-it/target/test-classes/ > This produces the correct output. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8756) Add open regions for writes without waiting on log replay in current distributed log splitting scheme
[ https://issues.apache.org/jira/browse/HBASE-8756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698462#comment-13698462 ] Aleksandr Shulman commented on HBASE-8756: -- It looks like this item is covered as part 2 of HBASE-7006. Can I dupe this one out, or is there more to this than what is in HBASE-7006? > Add open regions for writes without waiting on log replay in current > distributed log splitting scheme > - > > Key: HBASE-8756 > URL: https://issues.apache.org/jira/browse/HBASE-8756 > Project: HBase > Issue Type: Improvement > Components: Performance >Reporter: stack > Fix For: 0.95.2 > > > distributed log replay would add opening regions to take writes before > waiting on distributed log replay to finish. > We can add this facility to our current distributed log splitting mechanism > too? > Depends on HBASE-8741 or something like it so our sequenceids are good when > the region opens. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8672) Create an Integration test for Bulk Loads
[
https://issues.apache.org/jira/browse/HBASE-8672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13675325#comment-13675325
]
Aleksandr Shulman commented on HBASE-8672:
--
Minor issue -- in {code}runLinkedListMRJob(int iteration){code}, your delete
comes at the end of the function. Should there be an issue and the test fails,
it will not reach that point and there will be leftover cruft that could affect
other tests. One way to get around this is to use a try-catch-finally (or just
try-finally) pattern, with the deletion being in the 'finally' portion.
Granted, you only have one test, but if you write more, or if you run this on
an actual cluster, cleanup becomes important.
> Create an Integration test for Bulk Loads
> -
>
> Key: HBASE-8672
> URL: https://issues.apache.org/jira/browse/HBASE-8672
> Project: HBase
> Issue Type: Bug
> Components: regionserver, test
>Affects Versions: 0.98.0, 0.95.1
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-8672-0.patch, HBASE-8672-1.patch,
> HBASE-8672-2.patch, HBASE-8672-3.patch
>
>
> Bulk loads and MR are not well tested using our IT tests. We should add a
> test that bulk loads hfiles and then scans over the resulting table to make
> sure that all the data is there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8672) Create an Integration test for Bulk Loads
[ https://issues.apache.org/jira/browse/HBASE-8672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13675164#comment-13675164 ] Aleksandr Shulman commented on HBASE-8672: -- First read of the patch looks good. Will take a second look shortly. > Create an Integration test for Bulk Loads > - > > Key: HBASE-8672 > URL: https://issues.apache.org/jira/browse/HBASE-8672 > Project: HBase > Issue Type: Bug > Components: regionserver, test >Affects Versions: 0.98.0, 0.95.1 >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-8672-0.patch, HBASE-8672-1.patch > > > Bulk loads and MR are not well tested using our IT tests. We should add a > test that bulk loads hfiles and then scans over the resulting table to make > sure that all the data is there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8619) [HBase Client]: Improve client to retry if master is down when requesting getHTableDescriptor
Aleksandr Shulman created HBASE-8619:
Summary: [HBase Client]: Improve client to retry if master is down
when requesting getHTableDescriptor
Key: HBASE-8619
URL: https://issues.apache.org/jira/browse/HBASE-8619
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.94.8
Reporter: Aleksandr Shulman
Assignee: Elliott Clark
Priority: Minor
Fix For: 0.94.8
In our rolling upgrade testing, running ImportTsv failed in the job submission
phase when the master was down. This was when the master was failing over to
the backup master. In this case, a retry would have been helpful and made sure
that the job would get submitted.
A good solution would be to refresh the master information before placing the
call to getHTableDescriptor.
Command:
{code} sudo -u hbase hbase org.apache.hadoop.hbase.mapreduce.ImportTsv
-Dimporttsv.columns=HBASE_ROW_KEY,c1,c2,c3
-Dimporttsv.bulk.output=/user/hbase/storeFiles2_2/import2_table1369439156
import2_table1369439156 /user/hbase/tsv2{code}
Here is the stack trace:
{code} 13/05/24 16:55:49 INFO compress.CodecPool: Got brand-new compressor
[.deflate]
16:45:44 Exception in thread "main"
java.lang.reflect.UndeclaredThrowableException
16:45:44at $Proxy7.getHTableDescriptors(Unknown Source)
16:45:44at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHTableDescriptor(HConnectionManager.java:1861)
16:45:44at
org.apache.hadoop.hbase.client.HTable.getTableDescriptor(HTable.java:440)
16:45:44at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureCompression(HFileOutputFormat.java:458)
16:45:44at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:375)
16:45:44at
org.apache.hadoop.hbase.mapreduce.ImportTsv.createSubmittableJob(ImportTsv.java:280)
16:45:44at
org.apache.hadoop.hbase.mapreduce.ImportTsv.main(ImportTsv.java:424)
16:45:44 Caused by: java.io.IOException: Call to
hbase-rolling-6.ent.cloudera.com/10.20.186.99:22001 failed on local exception:
java.io.EOFException
16:45:44at
org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:1030)
16:45:44at
org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:999)
16:45:44at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:86)
16:45:44... 7 more
16:45:44 Caused by: java.io.EOFException
16:45:44at java.io.DataInputStream.readInt(DataInputStream.java:375)
16:45:44at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:646)
16:45:44at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:580){code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8559) increase unit-test coverage of package org.apache.hadoop.hbase.coprocessor
[ https://issues.apache.org/jira/browse/HBASE-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1384#comment-1384 ] Aleksandr Shulman commented on HBASE-8559: -- Is there a reviewBoard review associated with this? > increase unit-test coverage of package org.apache.hadoop.hbase.coprocessor > -- > > Key: HBASE-8559 > URL: https://issues.apache.org/jira/browse/HBASE-8559 > Project: HBase > Issue Type: Test >Affects Versions: 0.98.0, 0.94.8, 0.95.2 >Reporter: Ivan A. Veselovsky > Attachments: HBASE-8559-0.94--N2.patch, HBASE-8559-0.94--N3.patch, > HBASE-8559-trunk--N2.patch, HBASE-8559-trunk--N3.patch > > > increase unit-test coverage of package org.apache.hadoop.hbase.coprocessor up > to 80%. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8595) Add rename operation in hbase shell
[ https://issues.apache.org/jira/browse/HBASE-8595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8595: - Affects Version/s: 0.95.1 0.94.8 > Add rename operation in hbase shell > --- > > Key: HBASE-8595 > URL: https://issues.apache.org/jira/browse/HBASE-8595 > Project: HBase > Issue Type: New Feature > Components: shell >Affects Versions: 0.94.8, 0.95.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.94.8, 0.95.1 > > > We can use a set of snapshot commands to elegantly rename a table. It would > be nice to wrap all those commands in a single call. > http://hbase.apache.org/book.html#table.rename > Also -- the documentation is missing the last step where the original table > needs to be deleted. I can add that to the docbook. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8595) Add rename operation in hbase shell
Aleksandr Shulman created HBASE-8595: Summary: Add rename operation in hbase shell Key: HBASE-8595 URL: https://issues.apache.org/jira/browse/HBASE-8595 Project: HBase Issue Type: New Feature Reporter: Aleksandr Shulman Assignee: Aleksandr Shulman Priority: Minor Fix For: 0.94.8, 0.95.1 We can use a set of snapshot commands to elegantly rename a table. It would be nice to wrap all those commands in a single call. http://hbase.apache.org/book.html#table.rename Also -- the documentation is missing the last step where the original table needs to be deleted. I can add that to the docbook. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8595) Add rename operation in hbase shell
[ https://issues.apache.org/jira/browse/HBASE-8595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8595: - Component/s: shell > Add rename operation in hbase shell > --- > > Key: HBASE-8595 > URL: https://issues.apache.org/jira/browse/HBASE-8595 > Project: HBase > Issue Type: New Feature > Components: shell >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.94.8, 0.95.1 > > > We can use a set of snapshot commands to elegantly rename a table. It would > be nice to wrap all those commands in a single call. > http://hbase.apache.org/book.html#table.rename > Also -- the documentation is missing the last step where the original table > needs to be deleted. I can add that to the docbook. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8584) increase unit-test coverage of class org.apache.hadoop.hbase.coprocessor.BaseMasterObserver
[ https://issues.apache.org/jira/browse/HBASE-8584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13663407#comment-13663407 ] Aleksandr Shulman commented on HBASE-8584: -- Although code coverage is a useful metric to tell us if we've skipped testing areas of the code, I think adding methods that merely touch the code and nothing else might actually do more harm that good. In situations like these, a good question to ask might be what kind of bugs would this test actually catch (that other tests would not)? In my mind, the pros of adding these tests must outweigh the cons. The cons of adding more tests just to improve coverage is: 1. More tests means longer running time and more log/report noise 2. If we don't have testing this in this area, let's add useful tests that will catch regressions that will affect user workflows. Adding tests that merely touch the code might mask a real need for testing and give us a false sense of security. That said, I'm open to hearing the upside of these tests and what benefits they may offer us and I think it's great that we're paying attention to these metrics. > increase unit-test coverage of class > org.apache.hadoop.hbase.coprocessor.BaseMasterObserver > --- > > Key: HBASE-8584 > URL: https://issues.apache.org/jira/browse/HBASE-8584 > Project: HBase > Issue Type: Test >Affects Versions: 0.98.0, 0.95.1, 0.94.9 >Reporter: Ivan A. Veselovsky > Attachments: HBASE-8584-0.94--N1.patch, HBASE-8584-trunk--N1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8588) [Documentation]: Add information about adding REST and Thrift API kerberos principals to HBase ACL table
[ https://issues.apache.org/jira/browse/HBASE-8588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8588: - Attachment: HBASE-8588-v1.patch > [Documentation]: Add information about adding REST and Thrift API kerberos > principals to HBase ACL table > > > Key: HBASE-8588 > URL: https://issues.apache.org/jira/browse/HBASE-8588 > Project: HBase > Issue Type: Bug > Components: documentation, REST, security, Thrift >Affects Versions: 0.94.8, 0.95.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.94.8, 0.95.1 > > Attachments: HBASE-8588-v1.patch > > > When users set up secure REST and Thrift API gateways, those principals will > need entries in the HBase ACL table in order to interact with HBase. > We should add that to the documentation so that they can have success with it > right away. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8588) [Documentation]: Add information about adding REST and Thrift API kerberos principals to HBase ACL table
Aleksandr Shulman created HBASE-8588: Summary: [Documentation]: Add information about adding REST and Thrift API kerberos principals to HBase ACL table Key: HBASE-8588 URL: https://issues.apache.org/jira/browse/HBASE-8588 Project: HBase Issue Type: Bug Components: documentation, REST, security, Thrift Affects Versions: 0.94.8, 0.95.1 Reporter: Aleksandr Shulman Assignee: Aleksandr Shulman Priority: Minor Fix For: 0.94.8, 0.95.1 Attachments: HBASE-8588-v1.patch When users set up secure REST and Thrift API gateways, those principals will need entries in the HBase ACL table in order to interact with HBase. We should add that to the documentation so that they can have success with it right away. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8345) Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin
[ https://issues.apache.org/jira/browse/HBASE-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8345: - Attachment: HBASE-8345-v6-94.patch HBASE-8345-v6-trunk.patch These are patches for commit. The trunk patch should apply cleanly to both 95 and trunk. It has been reviewed and approved. > Add all available resources in o.a.h.h.rest.RootResource and VersionResource > to o.a.h.h.rest.client.RemoteAdmin > --- > > Key: HBASE-8345 > URL: https://issues.apache.org/jira/browse/HBASE-8345 > Project: HBase > Issue Type: Improvement > Components: Client, REST >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Labels: rest_api > Fix For: 0.94.8 > > Attachments: HBASE-8345-v1.patch, HBASE-8345-v6-94.patch, > HBASE-8345-v6-trunk.patch > > > In our built-in REST clients, we should add in more of the available REST > resources. This will allow more thorough testing of the REST API, > particularly with IntegrationTest. > These clients are located in the o.a.h.h.rest.client package. > In this case, I want to add the resources not already included in / and > /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, > /version/rest and /version/cluster, among others. > The RemoteAdmin class is a logical place for these methods because it is not > related to a specific table (those methods should go into RemoteHTable). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7192) Move TestHBase7051.java into TestAtomicOperation.java
[ https://issues.apache.org/jira/browse/HBASE-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-7192: - Attachment: HBASE-7192-v2.patch Here is a revised version that is applicable to trunk. It deletes TestHBase7051 and places the code into TestAtomicOperation. I ran the tests a few times and all tests pass. > Move TestHBase7051.java into TestAtomicOperation.java > - > > Key: HBASE-7192 > URL: https://issues.apache.org/jira/browse/HBASE-7192 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.95.2 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Attachments: HBASE-7192-v1.patch, HBASE-7192-v2.patch > > > This test class only has one test function and would be better served by > being located in TestAtomic.java, since anyone looking for atomicity-related > tests would know to look there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8049) If a RS cannot use a compression codec, it should have a retry limit on checking results of CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13640061#comment-13640061
]
Aleksandr Shulman commented on HBASE-8049:
--
Sorry, I should have disambiguated. I meant to say that we should backport
HBASE-8144 to 0.94. That would resolve this (HBASE-8049). [~jxiang], do you
think this backport of HBASE-8144 is realistic for 0.94?
> If a RS cannot use a compression codec, it should have a retry limit on
> checking results of CompressionTest
> ---
>
> Key: HBASE-8049
> URL: https://issues.apache.org/jira/browse/HBASE-8049
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.6, 0.92.3, 0.94.7, 0.95.0
> Environment: Including, but not limited to, Centos6_64
>Reporter: Aleksandr Shulman
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.98.0
>
>
> Observed Behavior:
> When a user attempts to create a table but there is an issue with the codec,
> the attempt continues repeatedly. The shell command times out but the RS and
> Master are both occupied, leading to HBase being down. Further, HBase creates
> the folders for the table in HDFS.
> The only way to restore the service is by disabling and dropping the table.
> Here are the log lines when a table, t8, is created with this definition:
> create 't8', {NAME=>'f1',COMPRESSION=>'lzo'}
> Error from shell:
> hbase(main):003:0> create 't8', {NAME=>'f1',BLOOMFILTER=>'row',
> COMPRESSION=>'lzo'}
> ERROR: org.apache.hadoop.hbase.client.RegionOfflineException: Only 0 of 1
> regions are online; retries exhausted.
> Log lines on Master (repeats a few times/second):
> 2013-03-07 22:55:31,389 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan for
> region t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.;
> plan=hri=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4., src=,
> dest=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,389 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
> t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4. to
> upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,398 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENING,
> server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_FAILED_OPEN,
> server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED
> event for 311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
> was=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4. state=CLOSED,
> ts=1362725731398, server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,406 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x13d47d21483 Creating (or updating) unassigned node for
> 311edabcc1fe52001cb00e7c3e7f75d4 with OFFLINE state
> 2013-03-07 22:55:31,414 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=M_ZK_REGION_OFFLINE, server=upgrade-vm-1.ent.cloudera.com:6,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> Log lines on RS (repeats a few times/second):
> 2013-03-07 22:58:23,323 ERROR
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
> of region=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.
> java.io.IOException: Compression algorithm 'lzo' previously failed test.
> at
> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:78)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2797)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2786)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2774)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:319)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:105)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:163)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Expected behavior:
> We expect to fail fast (after a few retries). This sh
[jira] [Commented] (HBASE-8049) If a RS cannot use a compression codec, it should have a retry limit on checking results of CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13640047#comment-13640047
]
Aleksandr Shulman commented on HBASE-8049:
--
Looking at the patch in HBASE-8144, the testing added looks good.
Can we get this backported into 0.94?
> If a RS cannot use a compression codec, it should have a retry limit on
> checking results of CompressionTest
> ---
>
> Key: HBASE-8049
> URL: https://issues.apache.org/jira/browse/HBASE-8049
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.6, 0.92.3, 0.94.7, 0.95.0
> Environment: Including, but not limited to, Centos6_64
>Reporter: Aleksandr Shulman
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.98.0
>
>
> Observed Behavior:
> When a user attempts to create a table but there is an issue with the codec,
> the attempt continues repeatedly. The shell command times out but the RS and
> Master are both occupied, leading to HBase being down. Further, HBase creates
> the folders for the table in HDFS.
> The only way to restore the service is by disabling and dropping the table.
> Here are the log lines when a table, t8, is created with this definition:
> create 't8', {NAME=>'f1',COMPRESSION=>'lzo'}
> Error from shell:
> hbase(main):003:0> create 't8', {NAME=>'f1',BLOOMFILTER=>'row',
> COMPRESSION=>'lzo'}
> ERROR: org.apache.hadoop.hbase.client.RegionOfflineException: Only 0 of 1
> regions are online; retries exhausted.
> Log lines on Master (repeats a few times/second):
> 2013-03-07 22:55:31,389 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan for
> region t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.;
> plan=hri=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4., src=,
> dest=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,389 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
> t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4. to
> upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,398 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENING,
> server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_FAILED_OPEN,
> server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED
> event for 311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
> was=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4. state=CLOSED,
> ts=1362725731398, server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,406 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x13d47d21483 Creating (or updating) unassigned node for
> 311edabcc1fe52001cb00e7c3e7f75d4 with OFFLINE state
> 2013-03-07 22:55:31,414 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=M_ZK_REGION_OFFLINE, server=upgrade-vm-1.ent.cloudera.com:6,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> Log lines on RS (repeats a few times/second):
> 2013-03-07 22:58:23,323 ERROR
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
> of region=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.
> java.io.IOException: Compression algorithm 'lzo' previously failed test.
> at
> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:78)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2797)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2786)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2774)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:319)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:105)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:163)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Expected behavior:
> We expect to fail fast (after a few retries). This should take <1 sec.
--
This message is automatically generated by JIRA.
If you think it was sent incorrect
[jira] [Commented] (HBASE-7192) Move TestHBase7051.java into TestAtomicOperation.java
[ https://issues.apache.org/jira/browse/HBASE-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13640025#comment-13640025 ] Aleksandr Shulman commented on HBASE-7192: -- Hmm -- that's odd. The patch is from a while ago. I can re-regenerate that patch. > Move TestHBase7051.java into TestAtomicOperation.java > - > > Key: HBASE-7192 > URL: https://issues.apache.org/jira/browse/HBASE-7192 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.95.2 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Attachments: HBASE-7192-v1.patch > > > This test class only has one test function and would be better served by > being located in TestAtomic.java, since anyone looking for atomicity-related > tests would know to look there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8385) [SNAPSHOTS]: Restore fails to restore snapshot of a deleted table
[
https://issues.apache.org/jira/browse/HBASE-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637008#comment-13637008
]
Aleksandr Shulman commented on HBASE-8385:
--
The functionality to restore a snapshot if the original table is not there is
found in clone, so it's not a data loss issue.
It's not a stretch to assume that someone who does accidentally delete a table
will see the 'restore' command and believe it will bring the table back.
Currently, this operations will fail, and that will be disappointing to that
user. We should be proactive and avoid creating that disappointment. It
wouldn't be too hard to just add a check to bypass the snapshot attempt if the
table doesn't exist.
> [SNAPSHOTS]: Restore fails to restore snapshot of a deleted table
> -
>
> Key: HBASE-8385
> URL: https://issues.apache.org/jira/browse/HBASE-8385
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.98.0, 0.94.7, 0.95.1
>Reporter: Aleksandr Shulman
>Assignee: Aleksandr Shulman
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: HBASE-8385-test-v1.patch
>
>
> Expected behavior:
> A user should be able to:
> 1. Take a snapshot of a table
> 2. Delete that table
> 3. Use the snapshot to restore that deleted table
> Observed behavior:
> During a restore, we attempt to create a snapshot of the table should the
> restore go awry. However, the snapshot fails because the table that we want
> to snapshot is not present.
> {code}
> Stack trace:
> org.apache.hadoop.hbase.exceptions.SnapshotCreationException:
> org.apache.hadoop.hbase.exceptions.SnapshotCreationException: Could not build
> snapshot handler
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
> at
> org.apache.hadoop.hbase.ipc.ProtobufRpcClientEngine$Invoker.invoke(ProtobufRpcClientEngine.java:146)
> at $Proxy24.snapshot(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterProtocolHandler.invoke(HConnectionManager.java:1703)
> at org.apache.hadoop.hbase.client.$Proxy25.snapshot(Unknown Source)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin$18.call(HBaseAdmin.java:2337)
> at org.apache.hadoop.hbase.client.HBaseAdmin$18.call(HBaseAdmin.java:1)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2637)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.execute(HBaseAdmin.java:2612)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.takeSnapshotAsync(HBaseAdmin.java:2334)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.snapshot(HBaseAdmin.java:2279)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.snapshot(HBaseAdmin.java:2252)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.snapshot(HBaseAdmin.java:2204)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.restoreSnapshot(HBaseAdmin.java:2417)
> at
> org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient.testRestoreSnapshotOfDeleted(TestRestoreSnapshotFromClient.java:266)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.stat
[jira] [Updated] (HBASE-8385) [SNAPSHOTS]: Restore fails to restore snapshot of a deleted table
[ https://issues.apache.org/jira/browse/HBASE-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8385: - Description: Expected behavior: A user should be able to: 1. Take a snapshot of a table 2. Delete that table 3. Use the snapshot to restore that deleted table Observed behavior: During a restore, we attempt to create a snapshot of the table should the restore go awry. However, the snapshot fails because the table that we want to snapshot is not present. Stack trace: org.apache.hadoop.hbase.exceptions.SnapshotCreationException: org.apache.hadoop.hbase.exceptions.SnapshotCreationException: Could not build snapshot handler at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79) at org.apache.hadoop.hbase.ipc.ProtobufRpcClientEngine$Invoker.invoke(ProtobufRpcClientEngine.java:146) at $Proxy24.snapshot(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterProtocolHandler.invoke(HConnectionManager.java:1703) at org.apache.hadoop.hbase.client.$Proxy25.snapshot(Unknown Source) at org.apache.hadoop.hbase.client.HBaseAdmin$18.call(HBaseAdmin.java:2337) at org.apache.hadoop.hbase.client.HBaseAdmin$18.call(HBaseAdmin.java:1) at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2637) at org.apache.hadoop.hbase.client.HBaseAdmin.execute(HBaseAdmin.java:2612) at org.apache.hadoop.hbase.client.HBaseAdmin.takeSnapshotAsync(HBaseAdmin.java:2334) at org.apache.hadoop.hbase.client.HBaseAdmin.snapshot(HBaseAdmin.java:2279) at org.apache.hadoop.hbase.client.HBaseAdmin.snapshot(HBaseAdmin.java:2252) at org.apache.hadoop.hbase.client.HBaseAdmin.snapshot(HBaseAdmin.java:2204) at org.apache.hadoop.hbase.client.HBaseAdmin.restoreSnapshot(HBaseAdmin.java:2417) at org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient.testRestoreSnapshotOfDeleted(TestRestoreSnapshotFromClient.java:266) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
[jira] [Updated] (HBASE-8385) [SNAPSHOTS]: Restore fails to restore snapshot of a deleted table
[ https://issues.apache.org/jira/browse/HBASE-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8385: - Attachment: HBASE-8385-test-v1.patch > [SNAPSHOTS]: Restore fails to restore snapshot of a deleted table > - > > Key: HBASE-8385 > URL: https://issues.apache.org/jira/browse/HBASE-8385 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 0.98.0, 0.94.7, 0.95.1, 0.95.2 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Fix For: 0.98.0, 0.94.7, 0.95.1, 0.95.2 > > Attachments: HBASE-8385-test-v1.patch > > > Expected behavior: > A user should be able to: > 1. Take a snapshot of a table > 2. Delete that table > 3. Use the snapshot to restore that deleted table > Observed behavior: > During a restore, we attempt to create a snapshot of the table should the > restore go awry. However, the snapshot fails because the table that we want > to snapshot is not present. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8385) [SNAPSHOTS]: Restore fails to restore snapshot of a deleted table
[ https://issues.apache.org/jira/browse/HBASE-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636904#comment-13636904 ] Aleksandr Shulman commented on HBASE-8385: -- I put together a quick test to verify this. The act of deleting the table causes the restore to fail. Comment out line 263 (the table delete) and the test will pass. > [SNAPSHOTS]: Restore fails to restore snapshot of a deleted table > - > > Key: HBASE-8385 > URL: https://issues.apache.org/jira/browse/HBASE-8385 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 0.98.0, 0.94.7, 0.95.1, 0.95.2 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Fix For: 0.98.0, 0.94.7, 0.95.1, 0.95.2 > > Attachments: HBASE-8385-test-v1.patch > > > Expected behavior: > A user should be able to: > 1. Take a snapshot of a table > 2. Delete that table > 3. Use the snapshot to restore that deleted table > Observed behavior: > During a restore, we attempt to create a snapshot of the table should the > restore go awry. However, the snapshot fails because the table that we want > to snapshot is not present. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8385) [SNAPSHOTS]: Restore fails to restore snapshot of a deleted table
Aleksandr Shulman created HBASE-8385: Summary: [SNAPSHOTS]: Restore fails to restore snapshot of a deleted table Key: HBASE-8385 URL: https://issues.apache.org/jira/browse/HBASE-8385 Project: HBase Issue Type: Bug Components: snapshots Affects Versions: 0.98.0, 0.94.7, 0.95.1, 0.95.2 Reporter: Aleksandr Shulman Assignee: Aleksandr Shulman Fix For: 0.98.0, 0.94.7, 0.95.1, 0.95.2 Expected behavior: A user should be able to: 1. Take a snapshot of a table 2. Delete that table 3. Use the snapshot to restore that deleted table Observed behavior: During a restore, we attempt to create a snapshot of the table should the restore go awry. However, the snapshot fails because the table that we want to snapshot is not present. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8345) Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin
[ https://issues.apache.org/jira/browse/HBASE-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634261#comment-13634261 ] Aleksandr Shulman commented on HBASE-8345: -- Hi [~apurtell], review is up if you want to take a look. I think it's close to its final form, but I wanted your input before we proceed. > Add all available resources in o.a.h.h.rest.RootResource and VersionResource > to o.a.h.h.rest.client.RemoteAdmin > --- > > Key: HBASE-8345 > URL: https://issues.apache.org/jira/browse/HBASE-8345 > Project: HBase > Issue Type: Improvement > Components: Client, REST >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Labels: rest_api > Fix For: 0.94.8 > > Attachments: HBASE-8345-v1.patch > > > In our built-in REST clients, we should add in more of the available REST > resources. This will allow more thorough testing of the REST API, > particularly with IntegrationTest. > These clients are located in the o.a.h.h.rest.client package. > In this case, I want to add the resources not already included in / and > /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, > /version/rest and /version/cluster, among others. > The RemoteAdmin class is a logical place for these methods because it is not > related to a specific table (those methods should go into RemoteHTable). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8345) Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin
[ https://issues.apache.org/jira/browse/HBASE-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13633521#comment-13633521 ] Aleksandr Shulman commented on HBASE-8345: -- That's correct, I'm working on them now. I can ping you before it's committed. > Add all available resources in o.a.h.h.rest.RootResource and VersionResource > to o.a.h.h.rest.client.RemoteAdmin > --- > > Key: HBASE-8345 > URL: https://issues.apache.org/jira/browse/HBASE-8345 > Project: HBase > Issue Type: Improvement > Components: Client, REST >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Labels: rest_api > Fix For: 0.94.8 > > Attachments: HBASE-8345-v1.patch > > > In our built-in REST clients, we should add in more of the available REST > resources. This will allow more thorough testing of the REST API, > particularly with IntegrationTest. > These clients are located in the o.a.h.h.rest.client package. > In this case, I want to add the resources not already included in / and > /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, > /version/rest and /version/cluster, among others. > The RemoteAdmin class is a logical place for these methods because it is not > related to a specific table (those methods should go into RemoteHTable). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8345) Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin
[ https://issues.apache.org/jira/browse/HBASE-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13633328#comment-13633328 ] Aleksandr Shulman commented on HBASE-8345: -- Link to review: https://reviews.apache.org/r/10528/ > Add all available resources in o.a.h.h.rest.RootResource and VersionResource > to o.a.h.h.rest.client.RemoteAdmin > --- > > Key: HBASE-8345 > URL: https://issues.apache.org/jira/browse/HBASE-8345 > Project: HBase > Issue Type: Improvement > Components: Client, REST >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Labels: rest_api > Fix For: 0.94.8 > > Attachments: HBASE-8345-v1.patch > > > In our built-in REST clients, we should add in more of the available REST > resources. This will allow more thorough testing of the REST API, > particularly with IntegrationTest. > These clients are located in the o.a.h.h.rest.client package. > In this case, I want to add the resources not already included in / and > /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, > /version/rest and /version/cluster, among others. > The RemoteAdmin class is a logical place for these methods because it is not > related to a specific table (those methods should go into RemoteHTable). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8345) Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin
[ https://issues.apache.org/jira/browse/HBASE-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13633327#comment-13633327 ] Aleksandr Shulman commented on HBASE-8345: -- Sounds good. I should have put the link to the review in the JIRA. I'll address those issues here, but let's take the conversion to the code review Patch -- Intended for 0.94 only. Parsing -- I'll look into this. Naming -- I felt it was a more useful and descriptive name than 'sb'. > Add all available resources in o.a.h.h.rest.RootResource and VersionResource > to o.a.h.h.rest.client.RemoteAdmin > --- > > Key: HBASE-8345 > URL: https://issues.apache.org/jira/browse/HBASE-8345 > Project: HBase > Issue Type: Improvement > Components: Client, REST >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Labels: rest_api > Fix For: 0.94.8 > > Attachments: HBASE-8345-v1.patch > > > In our built-in REST clients, we should add in more of the available REST > resources. This will allow more thorough testing of the REST API, > particularly with IntegrationTest. > These clients are located in the o.a.h.h.rest.client package. > In this case, I want to add the resources not already included in / and > /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, > /version/rest and /version/cluster, among others. > The RemoteAdmin class is a logical place for these methods because it is not > related to a specific table (those methods should go into RemoteHTable). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8345) Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin
[ https://issues.apache.org/jira/browse/HBASE-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8345: - Description: In our built-in REST clients, we should add in more of the available REST resources. This will allow more thorough testing of the REST API, particularly with IntegrationTest. These clients are located in the o.a.h.h.rest.client package. In this case, I want to add the resources not already included in / and /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, /version/rest and /version/cluster, among others. The RemoteAdmin class is a logical place for these methods because it is not related to a specific table (those methods should go into RemoteHTable). was: In our built-in REST clients, we should add in more of the available REST resources. This will allow users to build more capable RESTful applications against our REST API. These clients are located in the o.a.h.h.rest.client package. In this case, I want to add the resources not already included in / and /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, /version/rest and /version/cluster, among others. The RemoteAdmin class is a logical place for these methods because it is not related to a specific table (those methods should go into RemoteHTable). > Add all available resources in o.a.h.h.rest.RootResource and VersionResource > to o.a.h.h.rest.client.RemoteAdmin > --- > > Key: HBASE-8345 > URL: https://issues.apache.org/jira/browse/HBASE-8345 > Project: HBase > Issue Type: Improvement > Components: Client, REST >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Labels: rest_api > Fix For: 0.94.8 > > Attachments: HBASE-8345-v1.patch > > > In our built-in REST clients, we should add in more of the available REST > resources. This will allow more thorough testing of the REST API, > particularly with IntegrationTest. > These clients are located in the o.a.h.h.rest.client package. > In this case, I want to add the resources not already included in / and > /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, > /version/rest and /version/cluster, among others. > The RemoteAdmin class is a logical place for these methods because it is not > related to a specific table (those methods should go into RemoteHTable). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8345) Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin
[ https://issues.apache.org/jira/browse/HBASE-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8345: - Status: Patch Available (was: Open) Will put this up for review shortly. > Add all available resources in o.a.h.h.rest.RootResource and VersionResource > to o.a.h.h.rest.client.RemoteAdmin > --- > > Key: HBASE-8345 > URL: https://issues.apache.org/jira/browse/HBASE-8345 > Project: HBase > Issue Type: Improvement > Components: Client, REST >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Labels: rest_api > Fix For: 0.94.8 > > Attachments: HBASE-8345-v1.patch > > > In our built-in REST clients, we should add in more of the available REST > resources. This will allow users to build more capable RESTful applications > against our REST API. > These clients are located in the o.a.h.h.rest.client package. > In this case, I want to add the resources not already included in / and > /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, > /version/rest and /version/cluster, among others. > The RemoteAdmin class is a logical place for these methods because it is not > related to a specific table (those methods should go into RemoteHTable). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8345) Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin
[ https://issues.apache.org/jira/browse/HBASE-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8345: - Attachment: HBASE-8345-v1.patch Patch with tests. > Add all available resources in o.a.h.h.rest.RootResource and VersionResource > to o.a.h.h.rest.client.RemoteAdmin > --- > > Key: HBASE-8345 > URL: https://issues.apache.org/jira/browse/HBASE-8345 > Project: HBase > Issue Type: Improvement > Components: Client, REST >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Labels: rest_api > Fix For: 0.94.8 > > Attachments: HBASE-8345-v1.patch > > > In our built-in REST clients, we should add in more of the available REST > resources. This will allow users to build more capable RESTful applications > against our REST API. > These clients are located in the o.a.h.h.rest.client package. > In this case, I want to add the resources not already included in / and > /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, > /version/rest and /version/cluster, among others. > The RemoteAdmin class is a logical place for these methods because it is not > related to a specific table (those methods should go into RemoteHTable). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8345) Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin
[ https://issues.apache.org/jira/browse/HBASE-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632190#comment-13632190 ] Aleksandr Shulman commented on HBASE-8345: -- Thanks for the pointer. Is there someone who typically updates these docs? > Add all available resources in o.a.h.h.rest.RootResource and VersionResource > to o.a.h.h.rest.client.RemoteAdmin > --- > > Key: HBASE-8345 > URL: https://issues.apache.org/jira/browse/HBASE-8345 > Project: HBase > Issue Type: Improvement > Components: Client, REST >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Labels: rest_api > Fix For: 0.94.8 > > > In our built-in REST clients, we should add in more of the available REST > resources. This will allow users to build more capable RESTful applications > against our REST API. > These clients are located in the o.a.h.h.rest.client package. > In this case, I want to add the resources not already included in / and > /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, > /version/rest and /version/cluster, among others. > The RemoteAdmin class is a logical place for these methods because it is not > related to a specific table (those methods should go into RemoteHTable). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8345) Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin
[ https://issues.apache.org/jira/browse/HBASE-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632098#comment-13632098 ] Aleksandr Shulman commented on HBASE-8345: -- [~apurtell], do you know of any documentation that lists the REST resources that we support? If not, we should have that available for developers. > Add all available resources in o.a.h.h.rest.RootResource and VersionResource > to o.a.h.h.rest.client.RemoteAdmin > --- > > Key: HBASE-8345 > URL: https://issues.apache.org/jira/browse/HBASE-8345 > Project: HBase > Issue Type: Improvement > Components: Client, REST >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Labels: rest_api > Fix For: 0.94.7 > > > In our built-in REST clients, we should add in more of the available REST > resources. This will allow users to build more capable RESTful applications > against our REST API. > These clients are located in the o.a.h.h.rest.client package. > In this case, I want to add the resources not already included in / and > /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, > /version/rest and /version/cluster, among others. > The RemoteAdmin class is a logical place for these methods because it is not > related to a specific table (those methods should go into RemoteHTable). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8345) Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin
[ https://issues.apache.org/jira/browse/HBASE-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632093#comment-13632093 ] Aleksandr Shulman commented on HBASE-8345: -- Thanks for the usage clarification. In that case, this JIRA will help in the testing of methods not included in RemoteAdmin. > Add all available resources in o.a.h.h.rest.RootResource and VersionResource > to o.a.h.h.rest.client.RemoteAdmin > --- > > Key: HBASE-8345 > URL: https://issues.apache.org/jira/browse/HBASE-8345 > Project: HBase > Issue Type: Improvement > Components: Client, REST >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Labels: rest_api > Fix For: 0.94.7 > > > In our built-in REST clients, we should add in more of the available REST > resources. This will allow users to build more capable RESTful applications > against our REST API. > These clients are located in the o.a.h.h.rest.client package. > In this case, I want to add the resources not already included in / and > /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, > /version/rest and /version/cluster, among others. > The RemoteAdmin class is a logical place for these methods because it is not > related to a specific table (those methods should go into RemoteHTable). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8345) Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin
Aleksandr Shulman created HBASE-8345: Summary: Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin Key: HBASE-8345 URL: https://issues.apache.org/jira/browse/HBASE-8345 Project: HBase Issue Type: Improvement Components: Client, REST Affects Versions: 0.94.6.1 Reporter: Aleksandr Shulman Assignee: Aleksandr Shulman Priority: Minor Fix For: 0.94.7 In our built-in REST clients, we should add in more of the available REST resources. This will allow users to build more capable RESTful applications against our REST API. These clients are located in the o.a.h.h.rest.client package. In this case, I want to add the resources not already included in / and /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, /version/rest and /version/cluster, among others. The RemoteAdmin class is a logical place for these methods because it is not related to a specific table (those methods should go into RemoteHTable). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8345) Add all available resources in o.a.h.h.rest.RootResource and VersionResource to o.a.h.h.rest.client.RemoteAdmin
[ https://issues.apache.org/jira/browse/HBASE-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8345: - Labels: rest_api (was: ) > Add all available resources in o.a.h.h.rest.RootResource and VersionResource > to o.a.h.h.rest.client.RemoteAdmin > --- > > Key: HBASE-8345 > URL: https://issues.apache.org/jira/browse/HBASE-8345 > Project: HBase > Issue Type: Improvement > Components: Client, REST >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Labels: rest_api > Fix For: 0.94.7 > > > In our built-in REST clients, we should add in more of the available REST > resources. This will allow users to build more capable RESTful applications > against our REST API. > These clients are located in the o.a.h.h.rest.client package. > In this case, I want to add the resources not already included in / and > /version to o.a.h.h.rest.client.RemoteAdmin. This includes, /status/cluster, > /version/rest and /version/cluster, among others. > The RemoteAdmin class is a logical place for these methods because it is not > related to a specific table (those methods should go into RemoteHTable). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8277) [API Compatibility]: Create tool to analyze changes in the HBase Java public API
[ https://issues.apache.org/jira/browse/HBASE-8277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8277: - Status: Patch Available (was: Reopened) > [API Compatibility]: Create tool to analyze changes in the HBase Java public > API > > > Key: HBASE-8277 > URL: https://issues.apache.org/jira/browse/HBASE-8277 > Project: HBase > Issue Type: Bug > Components: Client, test >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Labels: api_compatibility > Attachments: HBase-8277-v3.patch > > > API incompatibilities can be frustrating for customers. Therefore, it is > important to identify incompatibilities and correct them. > There is a tool called JDiff which will compare two public API descriptions. > Having it pull from a git repo it will make it very useful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8277) [API Compatibility]: Create tool to analyze changes in the HBase Java public API
[ https://issues.apache.org/jira/browse/HBASE-8277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8277: - Attachment: HBase-8277-v3.patch Latest patch from reviewboard. > [API Compatibility]: Create tool to analyze changes in the HBase Java public > API > > > Key: HBASE-8277 > URL: https://issues.apache.org/jira/browse/HBASE-8277 > Project: HBase > Issue Type: Bug > Components: Client, test >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Labels: api_compatibility > Attachments: HBase-8277-v3.patch > > > API incompatibilities can be frustrating for customers. Therefore, it is > important to identify incompatibilities and correct them. > There is a tool called JDiff which will compare two public API descriptions. > Having it pull from a git repo it will make it very useful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8277) [API Compatibility]: Create tool to analyze changes in the HBase Java public API
[ https://issues.apache.org/jira/browse/HBASE-8277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8277: - Description: API incompatibilities can be frustrating for customers. Therefore, it is important to identify incompatibilities and correct them. There is a tool called JDiff which will compare two public API descriptions. Having it pull from a git repo it will make it very useful. was: API incompatibilities can be frustrating for customers. Therefore, it is important to identify incompatibilities and correct them. There is a tool called JDiff which will compare two public API descriptions. Having it pull from a git repo it will be very powerful. > [API Compatibility]: Create tool to analyze changes in the HBase Java public > API > > > Key: HBASE-8277 > URL: https://issues.apache.org/jira/browse/HBASE-8277 > Project: HBase > Issue Type: Bug > Components: Client, test >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Labels: api_compatibility > > API incompatibilities can be frustrating for customers. Therefore, it is > important to identify incompatibilities and correct them. > There is a tool called JDiff which will compare two public API descriptions. > Having it pull from a git repo it will make it very useful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8277) [API Compatibility]: Create tool to analyze changes in the HBase Java public API
[ https://issues.apache.org/jira/browse/HBASE-8277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8277: - Description: API incompatibilities can be frustrating for customers. Therefore, it is important to identify incompatibilities and correct them. There is a tool called JDiff which will compare two public API descriptions. Having it pull from a git repo it will be very powerful. was: API incompatibilities can be frustrating for customers. Therefore, it is important to identify incompatibilities and correct them. There is a tool called JDiff which will compare two public API descriptions. If have it pull from a git repo it will be very powerful. > [API Compatibility]: Create tool to analyze changes in the HBase Java public > API > > > Key: HBASE-8277 > URL: https://issues.apache.org/jira/browse/HBASE-8277 > Project: HBase > Issue Type: Bug > Components: Client, test >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Labels: api_compatibility > > API incompatibilities can be frustrating for customers. Therefore, it is > important to identify incompatibilities and correct them. > There is a tool called JDiff which will compare two public API descriptions. > Having it pull from a git repo it will be very powerful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-8277) [API Compatibility]: Create tool to analyze changes in the HBase Java public API
[ https://issues.apache.org/jira/browse/HBASE-8277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman reopened HBASE-8277: -- This is a different tool than the one to which this was dup'd. Specifically, I had in mind a JDiff report, as opposed to what was suggested on that thread. > [API Compatibility]: Create tool to analyze changes in the HBase Java public > API > > > Key: HBASE-8277 > URL: https://issues.apache.org/jira/browse/HBASE-8277 > Project: HBase > Issue Type: Bug > Components: Client, test >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Labels: api_compatibility > > API incompatibilities can be frustrating for customers. Therefore, it is > important to identify incompatibilities and correct them. > There is a tool called JDiff which will compare two public API descriptions. > If have it pull from a git repo it will be very powerful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8277) [API Compatibility]: Create tool to analyze changes in the HBase Java public API
[ https://issues.apache.org/jira/browse/HBASE-8277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8277: - Summary: [API Compatibility]: Create tool to analyze changes in the HBase Java public API (was: [API Compatibility]: Create tool to analyze changes in the HBase public API) > [API Compatibility]: Create tool to analyze changes in the HBase Java public > API > > > Key: HBASE-8277 > URL: https://issues.apache.org/jira/browse/HBASE-8277 > Project: HBase > Issue Type: Bug > Components: Client, test >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Fix For: 0.94.7 > > > API incompatibilities can be frustrating for customers. Therefore, it is > important to identify incompatibilities and correct them. > There is a tool called JDiff which will compare two public API descriptions. > If have it pull from a git repo it will be very powerful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8277) [API Compatibility]: Create tool to analyze changes in the HBase Java public API
[ https://issues.apache.org/jira/browse/HBASE-8277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8277: - Labels: api_compatibility (was: ) > [API Compatibility]: Create tool to analyze changes in the HBase Java public > API > > > Key: HBASE-8277 > URL: https://issues.apache.org/jira/browse/HBASE-8277 > Project: HBase > Issue Type: Bug > Components: Client, test >Affects Versions: 0.94.6.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Labels: api_compatibility > Fix For: 0.94.7 > > > API incompatibilities can be frustrating for customers. Therefore, it is > important to identify incompatibilities and correct them. > There is a tool called JDiff which will compare two public API descriptions. > If have it pull from a git repo it will be very powerful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8277) [API Compatibility]: Create tool to analyze changes in the HBase public API
Aleksandr Shulman created HBASE-8277: Summary: [API Compatibility]: Create tool to analyze changes in the HBase public API Key: HBASE-8277 URL: https://issues.apache.org/jira/browse/HBASE-8277 Project: HBase Issue Type: Bug Components: Client, test Affects Versions: 0.94.6.1 Reporter: Aleksandr Shulman Assignee: Aleksandr Shulman Fix For: 0.94.7 API incompatibilities can be frustrating for customers. Therefore, it is important to identify incompatibilities and correct them. There is a tool called JDiff which will compare two public API descriptions. If have it pull from a git repo it will be very powerful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8275) Tool to test binary compatibility
[ https://issues.apache.org/jira/browse/HBASE-8275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622980#comment-13622980 ] Aleksandr Shulman commented on HBASE-8275: -- That's a great idea. > Tool to test binary compatibility > - > > Key: HBASE-8275 > URL: https://issues.apache.org/jira/browse/HBASE-8275 > Project: HBase > Issue Type: New Feature >Reporter: Jean-Daniel Cryans > Fix For: 0.98.0 > > > Stack and I were discussing of ways to make binary compatibility easier to > test than doing it completely by hand. > One idea would be to have a tool that uses reflection to generate code that > calls all the public methods from a list of classes. You would then compile > this code against the current version you are on, then try it out with > different HBase jars without recompiling. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7401) Remove warning message about running 'hbase migrate'
[
https://issues.apache.org/jira/browse/HBASE-7401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618166#comment-13618166
]
Aleksandr Shulman commented on HBASE-7401:
--
I am running into this issue as well. It looks like there is a patch available,
so maybe we should close the loop on this and get it in?
> Remove warning message about running 'hbase migrate'
>
>
> Key: HBASE-7401
> URL: https://issues.apache.org/jira/browse/HBASE-7401
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.3, 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-7401.patch
>
>
> I was switching version to 0.94 (from 0.95-SNAPSHOT) and got this message
> which refers to a program that does not exist.
> {code}
> org.apache.hadoop.hbase.util.FileSystemVersionException: File system needs to
> be upgraded. You have version null and I want version 7. Run the
> '${HBASE_HOME}/bin/hbase migrate' script.
> at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:324)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:352)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:128)
> at
> org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:113)
> at
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:505)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:362)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.run(HMasterCommandLine.java:226)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7579) HTableDescriptor equals method fails if results are returned in a different order
[ https://issues.apache.org/jira/browse/HBASE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618154#comment-13618154 ] Aleksandr Shulman commented on HBASE-7579: -- Thanks Lars. I think that's a good idea. > HTableDescriptor equals method fails if results are returned in a different > order > - > > Key: HBASE-7579 > URL: https://issues.apache.org/jira/browse/HBASE-7579 > Project: HBase > Issue Type: Bug > Components: Admin >Affects Versions: 0.95.0, 0.94.6 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.95.0, 0.94.7 > > Attachments: HBASE-7579-v1.patch, HBASE-7579-v2.patch, > HBASE-7579-v3.patch, HBASE-7579-v4.patch > > > HTableDescriptor's compareTo function compares a set of HColumnDescriptors > against another set of HColumnDescriptors. It iterates through both, relying > on the fact that they will be in the same order. > In my testing, I may have seen this issue come up, so I decided to fix it. > It's a straightforward fix. I convert the sets into a hashset for O(1) > lookups (at least in theory), then I check that all items in the first set > are found in the second. > Since the sizes are the same, we know that if all elements showed up in the > second set, then they must be equal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7781) Provide the ability to run integration tests with security enabled
[ https://issues.apache.org/jira/browse/HBASE-7781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13616514#comment-13616514 ] Aleksandr Shulman commented on HBASE-7781: -- I was able to run this successfully in 0.94 against a kerberos-enabled secure cluster. Any tried this in 0.96? > Provide the ability to run integration tests with security enabled > -- > > Key: HBASE-7781 > URL: https://issues.apache.org/jira/browse/HBASE-7781 > Project: HBase > Issue Type: Test > Components: security, test >Reporter: Gary Helmling >Priority: Critical > > We currently have large holes in the test coverage of HBase with security > enabled. Two recent examples of bugs which really should have been caught > with testing are HBASE-7771 and HBASE-7772. The long standing problem with > testing with security enabled has been the requirement for supporting > kerberos infrastructure. > We need to close this gap and provide some automated testing with security > enabled, if necessary standing up and provisioning a temporary KDC as an > option for running integration tests, see HADOOP-8078 and HADOOP-9004 where a > similar approach was taken. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8200) Integration Test classes are not part of the default HBase classpath. We should fix this or note workaround in documentation
[ https://issues.apache.org/jira/browse/HBASE-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13615869#comment-13615869 ] Aleksandr Shulman commented on HBASE-8200: -- The documentation seems to say that it's runnable from source: http://hbase.apache.org/book/hbase.tests.html#integration.tests > Integration Test classes are not part of the default HBase classpath. We > should fix this or note workaround in documentation > > > Key: HBASE-8200 > URL: https://issues.apache.org/jira/browse/HBASE-8200 > Project: HBase > Issue Type: Bug > Components: test > Environment: ubuntu >Reporter: Aleksandr Shulman > > Expected behavior: > Following the instructions here: > http://hbase.apache.org/book/hbase.tests.html#integration.tests > After pulling down the latest bits from trunk and running, > > mvn clean compile test-compile package install assembly:assembly -DskipTests > I should be able to run integration tests with the following command: > > bin/hbase --config /path/to/configs > > org.apache.hadoop.hbase.IntegrationTestsDriver > According to the documentation, this should kick off the tests. > Observed behavior: > Upon executing > > bin/hbase --config /path/to/configs > > org.apache.hadoop.hbase.IntegrationTestsDriver > I see the following: > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/hadoop/hbase/IntegrationTestsDriver > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.hbase.IntegrationTestsDriver > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > I can fix it by adding the following line to the hbase-env.sh file > corresponding to the configs: > export HBASE_CLASSPATH=/path/to/project/hbase/hbase-it/target/test-classes/ > This produces the correct output. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8200) Integration Test classes are not part of the default HBase classpath. We should fix this or note workaround in documentation
[ https://issues.apache.org/jira/browse/HBASE-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13614554#comment-13614554 ] Aleksandr Shulman commented on HBASE-8200: -- I'm trying to run it on a remote cluster. > Integration Test classes are not part of the default HBase classpath. We > should fix this or note workaround in documentation > > > Key: HBASE-8200 > URL: https://issues.apache.org/jira/browse/HBASE-8200 > Project: HBase > Issue Type: Bug > Components: test > Environment: ubuntu >Reporter: Aleksandr Shulman > > Expected behavior: > Following the instructions here: > http://hbase.apache.org/book/hbase.tests.html#integration.tests > After pulling down the latest bits from trunk and running, > > mvn clean compile test-compile package install assembly:assembly -DskipTests > I should be able to run integration tests with the following command: > > bin/hbase --config /path/to/configs > > org.apache.hadoop.hbase.IntegrationTestsDriver > According to the documentation, this should kick off the tests. > Observed behavior: > Upon executing > > bin/hbase --config /path/to/configs > > org.apache.hadoop.hbase.IntegrationTestsDriver > I see the following: > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/hadoop/hbase/IntegrationTestsDriver > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.hbase.IntegrationTestsDriver > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > I can fix it by adding the following line to the hbase-env.sh file > corresponding to the configs: > export HBASE_CLASSPATH=/path/to/project/hbase/hbase-it/target/test-classes/ > This produces the correct output. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8200) Integration Test classes are not part of the default HBase classpath. We should fix this or note workaround in documentation
[ https://issues.apache.org/jira/browse/HBASE-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13614505#comment-13614505 ] Aleksandr Shulman commented on HBASE-8200: -- This issue is present in trunk but not in 0.94 because in 0.94, org.apache.hadoop.hbase.IntegrationTestsDriver is placed into ./target/test-classes/org/apache/hadoop/hbase, which is part of the classpath by default. > Integration Test classes are not part of the default HBase classpath. We > should fix this or note workaround in documentation > > > Key: HBASE-8200 > URL: https://issues.apache.org/jira/browse/HBASE-8200 > Project: HBase > Issue Type: Bug > Components: test > Environment: ubuntu >Reporter: Aleksandr Shulman > > Expected behavior: > Following the instructions here: > http://hbase.apache.org/book/hbase.tests.html#integration.tests > After pulling down the latest bits from trunk and running, > > mvn clean compile test-compile package install assembly:assembly -DskipTests > I should be able to run integration tests with the following command: > > bin/hbase --config /path/to/configs > > org.apache.hadoop.hbase.IntegrationTestsDriver > According to the documentation, this should kick off the tests. > Observed behavior: > Upon executing > > bin/hbase --config /path/to/configs > > org.apache.hadoop.hbase.IntegrationTestsDriver > I see the following: > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/hadoop/hbase/IntegrationTestsDriver > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.hbase.IntegrationTestsDriver > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > I can fix it by adding the following line to the hbase-env.sh file > corresponding to the configs: > export HBASE_CLASSPATH=/path/to/project/hbase/hbase-it/target/test-classes/ > This produces the correct output. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8200) Integration Test classes are not part of the default HBase classpath. We should fix this or note workaround in documentation
[
https://issues.apache.org/jira/browse/HBASE-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13614496#comment-13614496
]
Aleksandr Shulman commented on HBASE-8200:
--
The autoformatting mangled the above comment. Here is what is should be:
{code}
#Add integration-test to the classpath
CLASSPATH=${CLASSPATH}:$HBASE_HOME/hbase-it/target/test-classes/
{code}
> Integration Test classes are not part of the default HBase classpath. We
> should fix this or note workaround in documentation
>
>
> Key: HBASE-8200
> URL: https://issues.apache.org/jira/browse/HBASE-8200
> Project: HBase
> Issue Type: Bug
> Components: test
> Environment: ubuntu
>Reporter: Aleksandr Shulman
>
> Expected behavior:
> Following the instructions here:
> http://hbase.apache.org/book/hbase.tests.html#integration.tests
> After pulling down the latest bits from trunk and running,
> > mvn clean compile test-compile package install assembly:assembly -DskipTests
> I should be able to run integration tests with the following command:
> > bin/hbase --config /path/to/configs
> > org.apache.hadoop.hbase.IntegrationTestsDriver
> According to the documentation, this should kick off the tests.
> Observed behavior:
> Upon executing
> > bin/hbase --config /path/to/configs
> > org.apache.hadoop.hbase.IntegrationTestsDriver
> I see the following:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/IntegrationTestsDriver
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.IntegrationTestsDriver
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> I can fix it by adding the following line to the hbase-env.sh file
> corresponding to the configs:
> export HBASE_CLASSPATH=/path/to/project/hbase/hbase-it/target/test-classes/
> This produces the correct output.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8200) Integration Test classes are not part of the default HBase classpath. We should fix this or note workaround in documentation
[
https://issues.apache.org/jira/browse/HBASE-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13614493#comment-13614493
]
Aleksandr Shulman commented on HBASE-8200:
--
A quick fix might be to just add it to /bin/hbase
# Add integration-test to the classpath
CLASSPATH=${CLASSPATH}:$HBASE_HOME/hbase-it/target/test-classes/
> Integration Test classes are not part of the default HBase classpath. We
> should fix this or note workaround in documentation
>
>
> Key: HBASE-8200
> URL: https://issues.apache.org/jira/browse/HBASE-8200
> Project: HBase
> Issue Type: Bug
> Components: test
> Environment: ubuntu
>Reporter: Aleksandr Shulman
>
> Expected behavior:
> Following the instructions here:
> http://hbase.apache.org/book/hbase.tests.html#integration.tests
> After pulling down the latest bits from trunk and running,
> > mvn clean compile test-compile package install assembly:assembly -DskipTests
> I should be able to run integration tests with the following command:
> > bin/hbase --config /path/to/configs
> > org.apache.hadoop.hbase.IntegrationTestsDriver
> According to the documentation, this should kick off the tests.
> Observed behavior:
> Upon executing
> > bin/hbase --config /path/to/configs
> > org.apache.hadoop.hbase.IntegrationTestsDriver
> I see the following:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/IntegrationTestsDriver
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.IntegrationTestsDriver
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> I can fix it by adding the following line to the hbase-env.sh file
> corresponding to the configs:
> export HBASE_CLASSPATH=/path/to/project/hbase/hbase-it/target/test-classes/
> This produces the correct output.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8200) Integration Test classes are not part of the default HBase classpath. We should fix this or note workaround in documentation
[ https://issues.apache.org/jira/browse/HBASE-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-8200: - Summary: Integration Test classes are not part of the default HBase classpath. We should fix this or note workaround in documentation (was: Integration Test is not part of the default HBase classpath. We should fix this or note workaround in documentation) > Integration Test classes are not part of the default HBase classpath. We > should fix this or note workaround in documentation > > > Key: HBASE-8200 > URL: https://issues.apache.org/jira/browse/HBASE-8200 > Project: HBase > Issue Type: Bug > Components: test > Environment: ubuntu >Reporter: Aleksandr Shulman > > Expected behavior: > Following the instructions here: > http://hbase.apache.org/book/hbase.tests.html#integration.tests > After pulling down the latest bits from trunk and running, > > mvn clean compile test-compile package install assembly:assembly -DskipTests > I should be able to run integration tests with the following command: > > bin/hbase --config /path/to/configs > > org.apache.hadoop.hbase.IntegrationTestsDriver > According to the documentation, this should kick off the tests. > Observed behavior: > Upon executing > > bin/hbase --config /path/to/configs > > org.apache.hadoop.hbase.IntegrationTestsDriver > I see the following: > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/hadoop/hbase/IntegrationTestsDriver > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.hbase.IntegrationTestsDriver > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > I can fix it by adding the following line to the hbase-env.sh file > corresponding to the configs: > export HBASE_CLASSPATH=/path/to/project/hbase/hbase-it/target/test-classes/ > This produces the correct output. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8200) Integration Test is not part of the default HBase classpath. We should fix this or note workaround in documentation
Aleksandr Shulman created HBASE-8200: Summary: Integration Test is not part of the default HBase classpath. We should fix this or note workaround in documentation Key: HBASE-8200 URL: https://issues.apache.org/jira/browse/HBASE-8200 Project: HBase Issue Type: Bug Components: test Environment: ubuntu Reporter: Aleksandr Shulman Expected behavior: Following the instructions here: http://hbase.apache.org/book/hbase.tests.html#integration.tests After pulling down the latest bits from trunk and running, > mvn clean compile test-compile package install assembly:assembly -DskipTests I should be able to run integration tests with the following command: > bin/hbase --config /path/to/configs > org.apache.hadoop.hbase.IntegrationTestsDriver According to the documentation, this should kick off the tests. Observed behavior: Upon executing > bin/hbase --config /path/to/configs > org.apache.hadoop.hbase.IntegrationTestsDriver I see the following: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/IntegrationTestsDriver Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.IntegrationTestsDriver at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) I can fix it by adding the following line to the hbase-env.sh file corresponding to the configs: export HBASE_CLASSPATH=/path/to/project/hbase/hbase-it/target/test-classes/ This produces the correct output. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8049) If a RS cannot use a compression codec, it should have a retry limit on checking results of CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13612292#comment-13612292
]
Aleksandr Shulman commented on HBASE-8049:
--
That solution makes sense to me. There does not seem to be an obvious reason to
have certain RegionServers with the codec and others without. Keeping track of
that is adding complexity without any clear benefit.
We should make sure, when this occurs, to communicate through the stack trace,
logs, and error messages the reason why the assignment failed (codec issue).
> If a RS cannot use a compression codec, it should have a retry limit on
> checking results of CompressionTest
> ---
>
> Key: HBASE-8049
> URL: https://issues.apache.org/jira/browse/HBASE-8049
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.6, 0.92.3, 0.95.0, 0.94.7
> Environment: Including, but not limited to, Centos6_64
>Reporter: Aleksandr Shulman
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.95.0, 0.94.7
>
>
> Observed Behavior:
> When a user attempts to create a table but there is an issue with the codec,
> the attempt continues repeatedly. The shell command times out but the RS and
> Master are both occupied, leading to HBase being down. Further, HBase creates
> the folders for the table in HDFS.
> The only way to restore the service is by disabling and dropping the table.
> Here are the log lines when a table, t8, is created with this definition:
> create 't8', {NAME=>'f1',COMPRESSION=>'lzo'}
> Error from shell:
> hbase(main):003:0> create 't8', {NAME=>'f1',BLOOMFILTER=>'row',
> COMPRESSION=>'lzo'}
> ERROR: org.apache.hadoop.hbase.client.RegionOfflineException: Only 0 of 1
> regions are online; retries exhausted.
> Log lines on Master (repeats a few times/second):
> 2013-03-07 22:55:31,389 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan for
> region t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.;
> plan=hri=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4., src=,
> dest=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,389 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
> t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4. to
> upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,398 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENING,
> server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_FAILED_OPEN,
> server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED
> event for 311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
> was=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4. state=CLOSED,
> ts=1362725731398, server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,406 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x13d47d21483 Creating (or updating) unassigned node for
> 311edabcc1fe52001cb00e7c3e7f75d4 with OFFLINE state
> 2013-03-07 22:55:31,414 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=M_ZK_REGION_OFFLINE, server=upgrade-vm-1.ent.cloudera.com:6,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> Log lines on RS (repeats a few times/second):
> 2013-03-07 22:58:23,323 ERROR
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
> of region=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.
> java.io.IOException: Compression algorithm 'lzo' previously failed test.
> at
> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:78)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2797)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2786)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2774)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:319)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:105)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:163)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPo
[jira] [Commented] (HBASE-8049) If a RS cannot use a compression codec, it should have a retry limit on checking results of CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13611857#comment-13611857
]
Aleksandr Shulman commented on HBASE-8049:
--
Sounds like we have some potential solutions. Does anyone have a patch they'd
like to propose?
> If a RS cannot use a compression codec, it should have a retry limit on
> checking results of CompressionTest
> ---
>
> Key: HBASE-8049
> URL: https://issues.apache.org/jira/browse/HBASE-8049
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.6, 0.92.3, 0.95.0, 0.94.7
> Environment: Including, but not limited to, Centos6_64
>Reporter: Aleksandr Shulman
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.95.0, 0.94.7
>
>
> Observed Behavior:
> When a user attempts to create a table but there is an issue with the codec,
> the attempt continues repeatedly. The shell command times out but the RS and
> Master are both occupied, leading to HBase being down. Further, HBase creates
> the folders for the table in HDFS.
> The only way to restore the service is by disabling and dropping the table.
> Here are the log lines when a table, t8, is created with this definition:
> create 't8', {NAME=>'f1',COMPRESSION=>'lzo'}
> Error from shell:
> hbase(main):003:0> create 't8', {NAME=>'f1',BLOOMFILTER=>'row',
> COMPRESSION=>'lzo'}
> ERROR: org.apache.hadoop.hbase.client.RegionOfflineException: Only 0 of 1
> regions are online; retries exhausted.
> Log lines on Master (repeats a few times/second):
> 2013-03-07 22:55:31,389 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan for
> region t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.;
> plan=hri=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4., src=,
> dest=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,389 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
> t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4. to
> upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,398 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENING,
> server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_FAILED_OPEN,
> server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED
> event for 311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
> was=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4. state=CLOSED,
> ts=1362725731398, server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,406 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x13d47d21483 Creating (or updating) unassigned node for
> 311edabcc1fe52001cb00e7c3e7f75d4 with OFFLINE state
> 2013-03-07 22:55:31,414 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=M_ZK_REGION_OFFLINE, server=upgrade-vm-1.ent.cloudera.com:6,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> Log lines on RS (repeats a few times/second):
> 2013-03-07 22:58:23,323 ERROR
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
> of region=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.
> java.io.IOException: Compression algorithm 'lzo' previously failed test.
> at
> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:78)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2797)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2786)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2774)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:319)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:105)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:163)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Expected behavior:
> We expect to fail fast (after a few retries). This should take <1 sec.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectl
[jira] [Commented] (HBASE-8049) If a RS cannot use a compression codec, it should have a retry limit on checking results of CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13597673#comment-13597673
]
Aleksandr Shulman commented on HBASE-8049:
--
IIUC, the values of the test runs are cached when the RS first starts up, so
retrying won't help.
> If a RS cannot use a compression codec, it should have a retry limit on
> checking results of CompressionTest
> ---
>
> Key: HBASE-8049
> URL: https://issues.apache.org/jira/browse/HBASE-8049
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.6, 0.92.3, 0.95.0, 0.94.7
> Environment: Including, but not limited to, Centos6_64
>Reporter: Aleksandr Shulman
> Fix For: 0.95.0, 0.94.7
>
>
> Observed Behavior:
> When a user attempts to create a table but there is an issue with the codec,
> the attempt continues repeatedly. The shell command times out but the RS and
> Master are both occupied, leading to HBase being down. Further, HBase creates
> the folders for the table in HDFS.
> The only way to restore the service is by disabling and dropping the table.
> Here are the log lines when a table, t8, is created with this definition:
> create 't8', {NAME=>'f1',COMPRESSION=>'lzo'}
> Error from shell:
> hbase(main):003:0> create 't8', {NAME=>'f1',BLOOMFILTER=>'row',
> COMPRESSION=>'lzo'}
> ERROR: org.apache.hadoop.hbase.client.RegionOfflineException: Only 0 of 1
> regions are online; retries exhausted.
> Log lines on Master (repeats a few times/second):
> 2013-03-07 22:55:31,389 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan for
> region t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.;
> plan=hri=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4., src=,
> dest=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,389 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
> t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4. to
> upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,398 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENING,
> server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_FAILED_OPEN,
> server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED
> event for 311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
> was=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4. state=CLOSED,
> ts=1362725731398, server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,406 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x13d47d21483 Creating (or updating) unassigned node for
> 311edabcc1fe52001cb00e7c3e7f75d4 with OFFLINE state
> 2013-03-07 22:55:31,414 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=M_ZK_REGION_OFFLINE, server=upgrade-vm-1.ent.cloudera.com:6,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> Log lines on RS (repeats a few times/second):
> 2013-03-07 22:58:23,323 ERROR
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
> of region=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.
> java.io.IOException: Compression algorithm 'lzo' previously failed test.
> at
> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:78)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2797)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2786)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2774)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:319)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:105)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:163)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Expected behavior:
> We expect to fail fast (after a few retries). This should take <1 sec.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
[jira] [Commented] (HBASE-8049) If a RS cannot use a compression codec, it should have a retry limit on checking results of CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13597520#comment-13597520
]
Aleksandr Shulman commented on HBASE-8049:
--
An additional note -- the reason this occurred is because I installed LZO after
I started the RS. Restarting the RS after intalling LZO causes table creation
to work properly.
> If a RS cannot use a compression codec, it should have a retry limit on
> checking results of CompressionTest
> ---
>
> Key: HBASE-8049
> URL: https://issues.apache.org/jira/browse/HBASE-8049
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.90.6, 0.92.3, 0.95.0, 0.94.7
> Environment: Including, but not limited to, Centos6_64
>Reporter: Aleksandr Shulman
> Fix For: 0.95.0, 0.94.7
>
>
> Observed Behavior:
> When a user attempts to create a table but there is an issue with the codec,
> the attempt continues repeatedly. The shell command times out but the RS and
> Master are both occupied, leading to HBase being down. Further, HBase creates
> the folders for the table in HDFS.
> The only way to restore the service is by disabling and dropping the table.
> Here are the log lines when a table, t8, is created with this definition:
> create 't8', {NAME=>'f1',COMPRESSION=>'lzo'}
> Error from shell:
> hbase(main):003:0> create 't8', {NAME=>'f1',BLOOMFILTER=>'row',
> COMPRESSION=>'lzo'}
> ERROR: org.apache.hadoop.hbase.client.RegionOfflineException: Only 0 of 1
> regions are online; retries exhausted.
> Log lines on Master (repeats a few times/second):
> 2013-03-07 22:55:31,389 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan for
> region t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.;
> plan=hri=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4., src=,
> dest=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,389 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
> t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4. to
> upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,398 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENING,
> server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_FAILED_OPEN,
> server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED
> event for 311edabcc1fe52001cb00e7c3e7f75d4
> 2013-03-07 22:55:31,406 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
> was=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4. state=CLOSED,
> ts=1362725731398, server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
> 2013-03-07 22:55:31,406 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x13d47d21483 Creating (or updating) unassigned node for
> 311edabcc1fe52001cb00e7c3e7f75d4 with OFFLINE state
> 2013-03-07 22:55:31,414 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=M_ZK_REGION_OFFLINE, server=upgrade-vm-1.ent.cloudera.com:6,
> region=311edabcc1fe52001cb00e7c3e7f75d4
> Log lines on RS (repeats a few times/second):
> 2013-03-07 22:58:23,323 ERROR
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
> of region=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.
> java.io.IOException: Compression algorithm 'lzo' previously failed test.
> at
> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:78)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2797)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2786)
> at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2774)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:319)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:105)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:163)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Expected behavior:
> We expect to fail fast (after a few retries). This should take <1 sec.
--
This message is automatically generated by JIRA.
[jira] [Created] (HBASE-8049) If a RS cannot use a compression codec, it should have a retry limit on checking results of CompressionTest
Aleksandr Shulman created HBASE-8049:
Summary: If a RS cannot use a compression codec, it should have a
retry limit on checking results of CompressionTest
Key: HBASE-8049
URL: https://issues.apache.org/jira/browse/HBASE-8049
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.6, 0.92.3, 0.95.0, 0.94.7
Environment: Including, but not limited to, Centos6_64
Reporter: Aleksandr Shulman
Fix For: 0.95.0, 0.94.7
Observed Behavior:
When a user attempts to create a table but there is an issue with the codec,
the attempt continues repeatedly. The shell command times out but the RS and
Master are both occupied, leading to HBase being down. Further, HBase creates
the folders for the table in HDFS.
The only way to restore the service is by disabling and dropping the table.
Here are the log lines when a table, t8, is created with this definition:
create 't8', {NAME=>'f1',COMPRESSION=>'lzo'}
Error from shell:
hbase(main):003:0> create 't8', {NAME=>'f1',BLOOMFILTER=>'row',
COMPRESSION=>'lzo'}
ERROR: org.apache.hadoop.hbase.client.RegionOfflineException: Only 0 of 1
regions are online; retries exhausted.
Log lines on Master (repeats a few times/second):
2013-03-07 22:55:31,389 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Using pre-existing plan for region
t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.;
plan=hri=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4., src=,
dest=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
2013-03-07 22:55:31,389 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Assigning region t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4. to
upgrade-vm-1.ent.cloudera.com,60020,1362709586485
2013-03-07 22:55:31,398 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Handling transition=RS_ZK_REGION_OPENING,
server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
region=311edabcc1fe52001cb00e7c3e7f75d4
2013-03-07 22:55:31,406 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Handling transition=RS_ZK_REGION_FAILED_OPEN,
server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485,
region=311edabcc1fe52001cb00e7c3e7f75d4
2013-03-07 22:55:31,406 DEBUG
org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED
event for 311edabcc1fe52001cb00e7c3e7f75d4
2013-03-07 22:55:31,406 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Forcing OFFLINE; was=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.
state=CLOSED, ts=1362725731398,
server=upgrade-vm-1.ent.cloudera.com,60020,1362709586485
2013-03-07 22:55:31,406 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
master:6-0x13d47d21483 Creating (or updating) unassigned node for
311edabcc1fe52001cb00e7c3e7f75d4 with OFFLINE state
2013-03-07 22:55:31,414 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Handling transition=M_ZK_REGION_OFFLINE,
server=upgrade-vm-1.ent.cloudera.com:6,
region=311edabcc1fe52001cb00e7c3e7f75d4
Log lines on RS (repeats a few times/second):
2013-03-07 22:58:23,323 ERROR
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open of
region=t8,,1362725678436.311edabcc1fe52001cb00e7c3e7f75d4.
java.io.IOException: Compression algorithm 'lzo' previously failed test.
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:78)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2797)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2786)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2774)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:319)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:105)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:163)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Expected behavior:
We expect to fail fast (after a few retries). This should take <1 sec.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7579) HTableDescriptor equals method fails if results are returned in a different order
[ https://issues.apache.org/jira/browse/HBASE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13592539#comment-13592539 ] Aleksandr Shulman commented on HBASE-7579: -- Hi Lars, sorry about the delay in responding. Good point about the TreeMap always returning result in a deterministic ordering. The patch itself I think is still beneficial because it removes a little bit of redundancy in the existing equality code. IMO, the fix makes it easier to understand as well. In addition, the tests are beneficial since we were missing coverage there. I think it'd be better if the fix were in sooner, but it's probably not an issue if we moved it to 0.94.7. > HTableDescriptor equals method fails if results are returned in a different > order > - > > Key: HBASE-7579 > URL: https://issues.apache.org/jira/browse/HBASE-7579 > Project: HBase > Issue Type: Bug > Components: Admin >Affects Versions: 0.95.0, 0.94.6 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.95.0, 0.94.7 > > Attachments: HBASE-7579-v1.patch, HBASE-7579-v2.patch, > HBASE-7579-v3.patch, HBASE-7579-v4.patch > > > HTableDescriptor's compareTo function compares a set of HColumnDescriptors > against another set of HColumnDescriptors. It iterates through both, relying > on the fact that they will be in the same order. > In my testing, I may have seen this issue come up, so I decided to fix it. > It's a straightforward fix. I convert the sets into a hashset for O(1) > lookups (at least in theory), then I check that all items in the first set > are found in the second. > Since the sizes are the same, we know that if all elements showed up in the > second set, then they must be equal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7559) Add additional Snapshots Unit Test Coverage
[ https://issues.apache.org/jira/browse/HBASE-7559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13560160#comment-13560160 ] Aleksandr Shulman commented on HBASE-7559: -- Thanks Ted, Matteo, and Jon for taking the time to review and integrate upstream. -Aleks S. -- Best Regards, Aleks Shulman 847.814.5804 Cloudera > Add additional Snapshots Unit Test Coverage > --- > > Key: HBASE-7559 > URL: https://issues.apache.org/jira/browse/HBASE-7559 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.96.0 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Fix For: 0.96.0 > > Attachments: 7559-v7.txt, 7559-v8.txt, aleks-snapshots.patch > > > Add additional testing for Snapshots. In particular, we should add tests to > verify that operations on cloned tables do not affect the original (and vice > versa). Also, we should do testing on table describes before and after > snapshot/restore operations. Finally, we should add testing for the HBase > shell. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7579) HTableDescriptor equals method fails if results are returned in a different order
[ https://issues.apache.org/jira/browse/HBASE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-7579: - Status: Patch Available (was: Open) Submitting v3 of the patch. Hopefully this will integrate nicely. > HTableDescriptor equals method fails if results are returned in a different > order > - > > Key: HBASE-7579 > URL: https://issues.apache.org/jira/browse/HBASE-7579 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > Attachments: HBASE-7579-v1.patch, HBASE-7579-v2.patch, > HBASE-7579-v3.patch > > > HTableDescriptor's compareTo function compares a set of HColumnDescriptors > against another set of HColumnDescriptors. It iterates through both, relying > on the fact that they will be in the same order. > In my testing, I may have seen this issue come up, so I decided to fix it. > It's a straightforward fix. I convert the sets into a hashset for O(1) > lookups (at least in theory), then I check that all items in the first set > are found in the second. > Since the sizes are the same, we know that if all elements showed up in the > second set, then they must be equal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7579) HTableDescriptor equals method fails if results are returned in a different order
[ https://issues.apache.org/jira/browse/HBASE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-7579: - Status: Open (was: Patch Available) > HTableDescriptor equals method fails if results are returned in a different > order > - > > Key: HBASE-7579 > URL: https://issues.apache.org/jira/browse/HBASE-7579 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > Attachments: HBASE-7579-v1.patch, HBASE-7579-v2.patch, > HBASE-7579-v3.patch > > > HTableDescriptor's compareTo function compares a set of HColumnDescriptors > against another set of HColumnDescriptors. It iterates through both, relying > on the fact that they will be in the same order. > In my testing, I may have seen this issue come up, so I decided to fix it. > It's a straightforward fix. I convert the sets into a hashset for O(1) > lookups (at least in theory), then I check that all items in the first set > are found in the second. > Since the sizes are the same, we know that if all elements showed up in the > second set, then they must be equal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7579) HTableDescriptor equals method fails if results are returned in a different order
[ https://issues.apache.org/jira/browse/HBASE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-7579: - Attachment: HBASE-7579-v3.patch > HTableDescriptor equals method fails if results are returned in a different > order > - > > Key: HBASE-7579 > URL: https://issues.apache.org/jira/browse/HBASE-7579 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > Attachments: HBASE-7579-v1.patch, HBASE-7579-v2.patch, > HBASE-7579-v3.patch > > > HTableDescriptor's compareTo function compares a set of HColumnDescriptors > against another set of HColumnDescriptors. It iterates through both, relying > on the fact that they will be in the same order. > In my testing, I may have seen this issue come up, so I decided to fix it. > It's a straightforward fix. I convert the sets into a hashset for O(1) > lookups (at least in theory), then I check that all items in the first set > are found in the second. > Since the sizes are the same, we know that if all elements showed up in the > second set, then they must be equal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7579) HTableDescriptor equals method fails if results are returned in a different order
[ https://issues.apache.org/jira/browse/HBASE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13555725#comment-13555725 ] Aleksandr Shulman commented on HBASE-7579: -- Okay yea, I see. I have a clean version of the patch. Thanks for taking a look. > HTableDescriptor equals method fails if results are returned in a different > order > - > > Key: HBASE-7579 > URL: https://issues.apache.org/jira/browse/HBASE-7579 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > Attachments: HBASE-7579-v1.patch, HBASE-7579-v2.patch > > > HTableDescriptor's compareTo function compares a set of HColumnDescriptors > against another set of HColumnDescriptors. It iterates through both, relying > on the fact that they will be in the same order. > In my testing, I may have seen this issue come up, so I decided to fix it. > It's a straightforward fix. I convert the sets into a hashset for O(1) > lookups (at least in theory), then I check that all items in the first set > are found in the second. > Since the sizes are the same, we know that if all elements showed up in the > second set, then they must be equal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7559) Add additional Snapshots Unit Test Coverage
[ https://issues.apache.org/jira/browse/HBASE-7559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13555711#comment-13555711 ] Aleksandr Shulman commented on HBASE-7559: -- Actually, Matteo suggested a different fix (retry config change). Hopefully that will fare better. On Wed, Jan 16, 2013 at 4:16 PM, Aleksandr Shulman (JIRA) -- Best Regards, Aleks Shulman 847.814.5804 Cloudera > Add additional Snapshots Unit Test Coverage > --- > > Key: HBASE-7559 > URL: https://issues.apache.org/jira/browse/HBASE-7559 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.96.0 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Fix For: 0.96.0 > > Attachments: aleks-snapshots.patch > > > Add additional testing for Snapshots. In particular, we should add tests to > verify that operations on cloned tables do not affect the original (and vice > versa). Also, we should do testing on table describes before and after > snapshot/restore operations. Finally, we should add testing for the HBase > shell. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7579) HTableDescriptor equals method fails if results are returned in a different order
[ https://issues.apache.org/jira/browse/HBASE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13555702#comment-13555702 ] Aleksandr Shulman commented on HBASE-7579: -- What semantic meaning would the -1 return code have that the 1 wouldn't carry? Are you referring to my other review for snapshots? I'm not 100% sure that snapshots depends on it, but I could potentially see it being an issue. > HTableDescriptor equals method fails if results are returned in a different > order > - > > Key: HBASE-7579 > URL: https://issues.apache.org/jira/browse/HBASE-7579 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > Attachments: HBASE-7579-v1.patch, HBASE-7579-v2.patch > > > HTableDescriptor's compareTo function compares a set of HColumnDescriptors > against another set of HColumnDescriptors. It iterates through both, relying > on the fact that they will be in the same order. > In my testing, I may have seen this issue come up, so I decided to fix it. > It's a straightforward fix. I convert the sets into a hashset for O(1) > lookups (at least in theory), then I check that all items in the first set > are found in the second. > Since the sizes are the same, we know that if all elements showed up in the > second set, then they must be equal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7591) HColumnDescriptor equals method should not rely on HashCode
[ https://issues.apache.org/jira/browse/HBASE-7591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13555696#comment-13555696 ] Aleksandr Shulman commented on HBASE-7591: -- There is a patch for this on the parent JIRA. > HColumnDescriptor equals method should not rely on HashCode > --- > > Key: HBASE-7591 > URL: https://issues.apache.org/jira/browse/HBASE-7591 > Project: HBase > Issue Type: Sub-task > Components: Admin >Affects Versions: 0.96.0, 0.94.4 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > > I wanted to change the way this is implemented so that we can accept > different orderings of attributes as being the same. > This was implemented using HashMap equality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7559) Add additional Snapshots Unit Test Coverage
[ https://issues.apache.org/jira/browse/HBASE-7559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13555649#comment-13555649 ] Aleksandr Shulman commented on HBASE-7559: -- Hmm...that's interesting. Alright let's add it back then. > Add additional Snapshots Unit Test Coverage > --- > > Key: HBASE-7559 > URL: https://issues.apache.org/jira/browse/HBASE-7559 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 0.96.0 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman > Fix For: 0.96.0 > > Attachments: aleks-snapshots.patch > > > Add additional testing for Snapshots. In particular, we should add tests to > verify that operations on cloned tables do not affect the original (and vice > versa). Also, we should do testing on table describes before and after > snapshot/restore operations. Finally, we should add testing for the HBase > shell. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7579) HTableDescriptor equals method fails if results are returned in a different order
[ https://issues.apache.org/jira/browse/HBASE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-7579: - Status: Patch Available (was: Open) > HTableDescriptor equals method fails if results are returned in a different > order > - > > Key: HBASE-7579 > URL: https://issues.apache.org/jira/browse/HBASE-7579 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > Attachments: HBASE-7579-v1.patch, HBASE-7579-v2.patch > > > HTableDescriptor's compareTo function compares a set of HColumnDescriptors > against another set of HColumnDescriptors. It iterates through both, relying > on the fact that they will be in the same order. > In my testing, I may have seen this issue come up, so I decided to fix it. > It's a straightforward fix. I convert the sets into a hashset for O(1) > lookups (at least in theory), then I check that all items in the first set > are found in the second. > Since the sizes are the same, we know that if all elements showed up in the > second set, then they must be equal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7579) HTableDescriptor equals method fails if results are returned in a different order
[ https://issues.apache.org/jira/browse/HBASE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13555636#comment-13555636 ] Aleksandr Shulman commented on HBASE-7579: -- My new patch includes tests and changes for both HTableDescriptor and HColumnDescriptor > HTableDescriptor equals method fails if results are returned in a different > order > - > > Key: HBASE-7579 > URL: https://issues.apache.org/jira/browse/HBASE-7579 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > Attachments: HBASE-7579-v1.patch, HBASE-7579-v2.patch > > > HTableDescriptor's compareTo function compares a set of HColumnDescriptors > against another set of HColumnDescriptors. It iterates through both, relying > on the fact that they will be in the same order. > In my testing, I may have seen this issue come up, so I decided to fix it. > It's a straightforward fix. I convert the sets into a hashset for O(1) > lookups (at least in theory), then I check that all items in the first set > are found in the second. > Since the sizes are the same, we know that if all elements showed up in the > second set, then they must be equal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7579) HTableDescriptor equals method fails if results are returned in a different order
[ https://issues.apache.org/jira/browse/HBASE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-7579: - Attachment: HBASE-7579-v2.patch > HTableDescriptor equals method fails if results are returned in a different > order > - > > Key: HBASE-7579 > URL: https://issues.apache.org/jira/browse/HBASE-7579 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > Attachments: HBASE-7579-v1.patch, HBASE-7579-v2.patch > > > HTableDescriptor's compareTo function compares a set of HColumnDescriptors > against another set of HColumnDescriptors. It iterates through both, relying > on the fact that they will be in the same order. > In my testing, I may have seen this issue come up, so I decided to fix it. > It's a straightforward fix. I convert the sets into a hashset for O(1) > lookups (at least in theory), then I check that all items in the first set > are found in the second. > Since the sizes are the same, we know that if all elements showed up in the > second set, then they must be equal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7579) HTableDescriptor equals method fails if results are returned in a different order
[ https://issues.apache.org/jira/browse/HBASE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Shulman updated HBASE-7579: - Status: Open (was: Patch Available) > HTableDescriptor equals method fails if results are returned in a different > order > - > > Key: HBASE-7579 > URL: https://issues.apache.org/jira/browse/HBASE-7579 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > Attachments: HBASE-7579-v1.patch > > > HTableDescriptor's compareTo function compares a set of HColumnDescriptors > against another set of HColumnDescriptors. It iterates through both, relying > on the fact that they will be in the same order. > In my testing, I may have seen this issue come up, so I decided to fix it. > It's a straightforward fix. I convert the sets into a hashset for O(1) > lookups (at least in theory), then I check that all items in the first set > are found in the second. > Since the sizes are the same, we know that if all elements showed up in the > second set, then they must be equal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
