[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13592385#comment-13592385 ] Nick Dimiduk commented on HBASE-7692: - bq. This is definitely not just client side. Phoenix would use this in the innermost loops of coprocessor region scans and custom filter evaluation. I believe maintainer of RS components tend to view consumption of the coprocessor and filter interfaces as "client" concerns. What I believe [~stack] means here is that the RS implementation would continue to view its whole world as byte[], not make use of this library. bq. I'm not completely against the "RowKey" suffix in the name, but if we want to change it how about "Codec" or "Type" instead? This segues nicely into the related conversation, which is that I believe HBase should provide its own primitive types, just as the Management Systems of RDBMS world provide. Again, nothing that needs baked into the internals of the system, but some set of types that are language-neutral and clearly defined. I think this is the best way to establish interoperability between tools. HBase needs to think more actively about the world outside of Java. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0 > > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13591932#comment-13591932 ] James Taylor commented on HBASE-7692: - Ran some micro perf benchmarks of BigDecimal serialization/deserialization for Phoenix versus Orderly. Both have the same property that their binary representation sort according to their natural order. The test measures the serialization and deserialization time for values from -1M to 1M. Phoenix was 3.8x faster while using 3.5% more memory than Orderly. Probably wouldn't be too difficult to replace the Orderly serialization logic with the Phoenix one for this type. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0 > > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13591580#comment-13591580 ] James Taylor commented on HBASE-7692: - This is definitely not just client side. Phoenix would use this in the innermost loops of coprocessor region scans and custom filter evaluation. We need a bytes pointer mechanism, otherwise you're forced to copy into a byte array. Reuse of the byte arrays is often not possible because you don't know the size in advance. It certainly doesn't have to be the Hadoop ImmutableBytesWritable, though. Any opinion, [~lhofhansl]? To get folks to use the Orderly stuff is a different, bigger question. If Hive and Phoenix adopt it, you'd get those users. If we deprecate the Bytes utils for serializaing/deserializaing and point to the new stuff (explaining why it's better), that might help. You could have a simple cover over the Orderly classes for the common case to make it easier. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0 > > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[
https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13591565#comment-13591565
]
Matt Corgan commented on HBASE-7692:
{quote}Can't we just use Bytes.toBytes and reuse those buffers? Using Writables
for this seems heavy-handed{quote}Or just create a little class that does
what's needed from Writable. Then it won't depend on Bytes.java either. The
benefit of dropping the hadoop dependency and making it an independent library
is well worth a tiny bit of code duplication in my opinion. Imagine if the
Guava libs made you pull in stuff like hadoop jars.
> Add utility class to generate ordered byte[] serialization
> --
>
> Key: HBASE-7692
> URL: https://issues.apache.org/jira/browse/HBASE-7692
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.95.0
>
> Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch,
> HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch
>
>
> The current Bytes utility class works, but produces output that does not
> maintain the native sort ordering of the input value. This results in, for
> example, a negative value that does not necessarily sort before a positive
> value. HBase should provide a canonical implementation of such a
> serialization format so that third-parties can reliably build on top of
> HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or
> HIVE-2903 that is compatible with similar features in Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13591554#comment-13591554 ] stack commented on HBASE-7692: -- bq. It's just a lightweight way to have an offset/length into a byte array. You can also reset the byte[],offset,length on an instance which prevents having to re-instantiate a new one every time (unlike the java.nio.ByteBuffer) Yeah [~giacomotaylor]. [~eclark] was showing me this friday. Can't we just use Bytes.toBytes and reuse those buffers? Using Writables for this seems heavy-handed (Once it may have made sense when Writables were pervasive. I have a distinct aversion though having spent a good few months helping purge them from hbase codebase). More generally, for Orderly to come in, there would have to be a user I'd say else danger it is going to be code that will just sit there, get stuck in a back water -- no one knows it exists -- and it will just rot. And given that we think this a client-side only facility, then it is tougher still pulling this into the general codebase; i.e. +1 on a single serialization but -1 on hbase doing anything but bytes... something like that. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0 > > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13591506#comment-13591506 ] James Taylor commented on HBASE-7692: - I'm not completely against the "RowKey" suffix in the name, but if we want to change it how about "Codec" or "Type" instead? > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0 > > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13591503#comment-13591503 ] James Taylor commented on HBASE-7692: - I'm investigating moving Phoenix to use Orderly. It's very similar to what we're doing today, but has more functionality with a cleaner API. [~saint@gmail.com] The Writable stuff is used as a bytes pointer, not as a client/server data exchange mechanism. Phoenix uses them the same way. It's just a lightweight way to have an offset/length into a byte array. You can also reset the byte[],offset,length on an instance which prevents having to re-instantiate a new one every time (unlike the java.nio.ByteBuffer) The permutations are useful. One set of them instantiate a new object on deserialize (for Java native object types like Integer and Long), while the others reuse a byte buffer to hand you back what was deserialized. Both are useful under different circumstances. We'll try to use the latter. It'd be great if this could be packaged as a separate maven module such that Phoenix can depend on it since it's still on the 0.94 branch. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0 > > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589190#comment-13589190 ] stack commented on HBASE-7692: -- I've put comments over in rb. Highlevel: + This is well-thought through quality stuff. It is great. + Do we need all of it? Can we drop all the Writable stuff in particular (HBase is moving away from Writables so we have no use for it). Do we need all the permuations listed here? Would it even be better to take on a small part of this set first? + The fixation on 'rowkey' is not apt given it won't always be used for rowkey. + Could it be included as a jar rather than as module? If not, module w/ no dependencies would be better (would need to undo the hbase-common current dependency) Thanks Nick. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0 > > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13587414#comment-13587414 ] Nick Dimiduk commented on HBASE-7692: - I've opened a review; fire away! https://reviews.apache.org/r/9623/ > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0 > > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[
https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586563#comment-13586563
]
Nick Dimiduk commented on HBASE-7692:
-
HadoopQA is being pedantic about the line-lengths. It appears to catch the new
pom.xml as culprit:
{noformat}
$ cat HBASE-7692.v5.patch | grep "^+" | grep -v "^@@" | grep -v "^+++" | grep
-v "import" | grep -n ".\{96,\}"
611:+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
{noformat}
> Add utility class to generate ordered byte[] serialization
> --
>
> Key: HBASE-7692
> URL: https://issues.apache.org/jira/browse/HBASE-7692
> Project: HBase
> Issue Type: Improvement
> Components: util
>Affects Versions: 0.96.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch,
> HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch
>
>
> The current Bytes utility class works, but produces output that does not
> maintain the native sort ordering of the input value. This results in, for
> example, a negative value that does not necessarily sort before a positive
> value. HBase should provide a canonical implementation of such a
> serialization format so that third-parties can reliably build on top of
> HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or
> HIVE-2903 that is compatible with similar features in Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[
https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586139#comment-13586139
]
Hadoop QA commented on HBASE-7692:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12570819/HBASE-7692.v5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 68 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4531//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4531//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-orderly.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4531//console
This message is automatically generated.
> Add utility class to generate ordered byte[] serialization
> --
>
> Key: HBASE-7692
> URL: https://issues.apache.org/jira/browse/HBASE-7692
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch,
> HBASE-7692.v3.patch, HBASE-7692.v4.patch, HBASE-7692.v5.patch
>
>
> The current Bytes utility class works, but produces output that does not
> maintain the native sort ordering of the input value. This results in, for
> example, a negative value that does not necessarily sort before a positive
> value. HBase should provide a canonical implementation of such a
> serialization format so that third-parties can reliably build on top of
> HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or
> HIVE-2903 that is compatible with similar features in Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584830#comment-13584830 ] Nick Dimiduk commented on HBASE-7692: - @Ted yikes! My patch is on top of hbase/trunk@1449152 while Jenkins says "At revision 1449215". Git tells me I'm up to date re: origin. I guess there's some lag between SVN and the git mirror. I'll wait and try again later. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[
https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584785#comment-13584785
]
Ted Yu commented on HBASE-7692:
---
@Nick, You need to refresh your workspace:
{code}
patching file
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java
Reversed (or previously applied) patch detected! Assume -R? [n]
Apply anyway? [n]
Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java.rej
patching file
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java
Reversed (or previously applied) patch detected! Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java.rej
{code}
> Add utility class to generate ordered byte[] serialization
> --
>
> Key: HBASE-7692
> URL: https://issues.apache.org/jira/browse/HBASE-7692
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch,
> HBASE-7692.v3.patch, HBASE-7692.v4.patch
>
>
> The current Bytes utility class works, but produces output that does not
> maintain the native sort ordering of the input value. This results in, for
> example, a negative value that does not necessarily sort before a positive
> value. HBase should provide a canonical implementation of such a
> serialization format so that third-parties can reliably build on top of
> HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or
> HIVE-2903 that is compatible with similar features in Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584780#comment-13584780 ] Nick Dimiduk commented on HBASE-7692: - I'm not sure what's going on with HadoopQA here. The patch applies with patch -p1 for me. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch, HBASE-7692.v4.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[
https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584720#comment-13584720
]
Nick Dimiduk commented on HBASE-7692:
-
This implementation is mostly a superset of functionality described/requested
in HBASE-7221. The example provided in that ticket's description is implemented
using orderly like so:
{code}
StructRowKey rowkey = new StructBuilder()
.add(new FixedByteArrayRowKey(16))
.add(new LongRowKey())
.toRowKey();
byte[] bytes = rowkey.serialize(new Object[] { a, b });
{code}
In this case, the hash operation must be supplied by the user.
Consuming the serialized bytes is pretty easy too. Here's an example of basic
iteration:
{code}
rowkey.iterateOver(bytes);
for (Object f : rowkey) {
System.out.println(f);
}
{code}
The attached patches includes a number of
[examples|https://github.com/ndimiduk/hbase/tree/7692-ordered-serialization/hbase-examples/src/main/java/org/apache/hadoop/hbase/orderly].
(cc [~dmeil], [~lhofhansl])
> Add utility class to generate ordered byte[] serialization
> --
>
> Key: HBASE-7692
> URL: https://issues.apache.org/jira/browse/HBASE-7692
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch,
> HBASE-7692.v3.patch, HBASE-7692.v4.patch
>
>
> The current Bytes utility class works, but produces output that does not
> maintain the native sort ordering of the input value. This results in, for
> example, a negative value that does not necessarily sort before a positive
> value. HBase should provide a canonical implementation of such a
> serialization format so that third-parties can reliably build on top of
> HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or
> HIVE-2903 that is compatible with similar features in Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[
https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584701#comment-13584701
]
Hadoop QA commented on HBASE-7692:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12570535/HBASE-7692.v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 74 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4499//console
This message is automatically generated.
> Add utility class to generate ordered byte[] serialization
> --
>
> Key: HBASE-7692
> URL: https://issues.apache.org/jira/browse/HBASE-7692
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch,
> HBASE-7692.v3.patch, HBASE-7692.v4.patch
>
>
> The current Bytes utility class works, but produces output that does not
> maintain the native sort ordering of the input value. This results in, for
> example, a negative value that does not necessarily sort before a positive
> value. HBase should provide a canonical implementation of such a
> serialization format so that third-parties can reliably build on top of
> HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or
> HIVE-2903 that is compatible with similar features in Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583487#comment-13583487 ] Nick Dimiduk commented on HBASE-7692: - I can put the whole mess onto review board, but I think it's pretty easy to digest in the bite-sized [https://github.com/ndimiduk/hbase/commits/7692-ordered-serialization|commits on github}. Can it not be reviewed there? Whichever is easier for reviewers, I'd prefer to keep the comments in one place. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583460#comment-13583460 ] Ted Yu commented on HBASE-7692: --- Is it possible to upload patch v3 onto reviewboard ? > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[
https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583442#comment-13583442
]
Hadoop QA commented on HBASE-7692:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12570338/HBASE-7692.v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 68 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 7
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4484//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4484//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4484//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4484//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4484//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4484//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4484//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4484//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4484//console
This message is automatically generated.
> Add utility class to generate ordered byte[] serialization
> --
>
> Key: HBASE-7692
> URL: https://issues.apache.org/jira/browse/HBASE-7692
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch,
> HBASE-7692.v3.patch
>
>
> The current Bytes utility class works, but produces output that does not
> maintain the native sort ordering of the input value. This results in, for
> example, a negative value that does not necessarily sort before a positive
> value. HBase should provide a canonical implementation of such a
> serialization format so that third-parties can reliably build on top of
> HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or
> HIVE-2903 that is compatible with similar features in Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583397#comment-13583397 ] Nick Dimiduk commented on HBASE-7692: - New ticket for Javadoc warnings: HBASE-7895. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch, > HBASE-7692.v3.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[
https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582851#comment-13582851
]
Hadoop QA commented on HBASE-7692:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12570245/HBASE-7692.v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 68 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 7
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4476//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4476//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4476//console
This message is automatically generated.
> Add utility class to generate ordered byte[] serialization
> --
>
> Key: HBASE-7692
> URL: https://issues.apache.org/jira/browse/HBASE-7692
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HBASE-7692.v1.patch, HBASE-7692.v2.patch
>
>
> The current Bytes utility class works, but produces output that does not
> maintain the native sort ordering of the input value. This results in, for
> example, a negative value that does not necessarily sort before a positive
> value. HBase should provide a canonical implementation of such a
> serialization format so that third-parties can reliably build on top of
> HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or
> HIVE-2903 that is compatible with similar features in Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[
https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568002#comment-13568002
]
Ted Yu commented on HBASE-7692:
---
Please wrap long lines.
{code}
+/** Serializes and deserializes various integer types a sortable,
{code}
Insert 'into' between 'types' and 'a'
{code}
+ * To use the more efficient serialization format employed by
+ * IntWritable while avoiding 32-bit truncation, This class
+ * subclasses IntWritableRowKey to use LongWritable
{code}
The above is not true:
{code}
+public class BigDecimalRowKey extends RowKey
{code}
{code}
+ protected class ExponentRowKey extends IntWritableRowKey {
{code}
The above class can be private.
{code}
+/** Serializes and deserializes IntWritablesWritables into a sortable
{code}
Duplicate Writables in IntWritablesWritables
{code}
+public class StringExample
...
+
+public class StructExample
{code}
Add javadoc for the above classes.
> Add utility class to generate ordered byte[] serialization
> --
>
> Key: HBASE-7692
> URL: https://issues.apache.org/jira/browse/HBASE-7692
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HBASE-7692.v1.patch
>
>
> The current Bytes utility class works, but produces output that does not
> maintain the native sort ordering of the input value. This results in, for
> example, a negative value that does not necessarily sort before a positive
> value. HBase should provide a canonical implementation of such a
> serialization format so that third-parties can reliably build on top of
> HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or
> HIVE-2903 that is compatible with similar features in Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[
https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13567325#comment-13567325
]
Hadoop QA commented on HBASE-7692:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12567283/HBASE-7692.v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 68 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestHLog
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4268//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4268//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4268//console
This message is automatically generated.
> Add utility class to generate ordered byte[] serialization
> --
>
> Key: HBASE-7692
> URL: https://issues.apache.org/jira/browse/HBASE-7692
> Project: HBase
> Issue Type: Improvement
> Components: util
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Attachments: HBASE-7692.v1.patch
>
>
> The current Bytes utility class works, but produces output that does not
> maintain the native sort ordering of the input value. This results in, for
> example, a negative value that does not necessarily sort before a positive
> value. HBase should provide a canonical implementation of such a
> serialization format so that third-parties can reliably build on top of
> HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or
> HIVE-2903 that is compatible with similar features in Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13567284#comment-13567284 ] Nick Dimiduk commented on HBASE-7692: - Please see [hbase/7692-ordered-serialization|https://github.com/ndimiduk/hbase/tree/7692-ordered-serialization]. I've imported the [Orderly|https://github.com/ndimiduk/orderly] library into hbase-common and hadoop-examples. In an attempt to break it down into reviewable chunks, I've split it over 7 commits. The 1st introduces the base classes and support for byte types, the 2nd adds support for String types, the 3rd for integers, the 4th for longs, the 5th for decimals, and the 6th for compound types (or "structs"). The final commit adds additional documentation, cleans up spelling mistakes, javadoc warnings, and generally prepares the code for HadoopQA. I'll also attach the monster-diff to this ticket so that HadoopQA can do it's thing. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: HBASE-7692.v1.patch > > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13567100#comment-13567100 ] Enis Soztutar commented on HBASE-7692: -- We can fork it, and commit to the code base as long as we keep the original copyright attribution. That is my understanding. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13567006#comment-13567006 ] Nick Dimiduk commented on HBASE-7692: - I propose we annex the abandoned [Orderly|https://github.com/zettaset/orderly] library. It's been forked a number of times, notably by [NGDATA|https://github.com/NGDATA/orderly]. I also have my own fork, though I've admittedly not put very much effort into it until recently. It's licensed Apache 2. Can anyone comment about the legal aspect of this? > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13564600#comment-13564600 ] Enis Soztutar commented on HBASE-7692: -- +1, we should rethink our API corresponding to Bytes, and expose a unified API, with utilities for compound row keys. > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement > Components: util >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13564084#comment-13564084 ] Nick Dimiduk commented on HBASE-7692: - I propose we adopt/annex [Orderly|https://github.com/ndimiduk/orderly]. I've been in communication with the original author. He's moved onto Google and is no longer maintaining this project; would love to see someone adopt it. Who better than it's intended use-case? > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7692) Add utility class to generate ordered byte[] serialization
[ https://issues.apache.org/jira/browse/HBASE-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13564070#comment-13564070 ] Anoop Sam John commented on HBASE-7692: --- So here the utility will take some type information of the value right? > Add utility class to generate ordered byte[] serialization > -- > > Key: HBASE-7692 > URL: https://issues.apache.org/jira/browse/HBASE-7692 > Project: HBase > Issue Type: Improvement >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > > The current Bytes utility class works, but produces output that does not > maintain the native sort ordering of the input value. This results in, for > example, a negative value that does not necessarily sort before a positive > value. HBase should provide a canonical implementation of such a > serialization format so that third-parties can reliably build on top of > HBase. This will allow an implementation for HIVE-3634, HIVE-2599, or > HIVE-2903 that is compatible with similar features in Pig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
