[jira] [Commented] (CASSANDRA-4138) Add varint encoding to Serializing Cache
[ https://issues.apache.org/jira/browse/CASSANDRA-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264191#comment-13264191 ] Jonathan Ellis commented on CASSANDRA-4138: --- LGTM, +1. > Add varint encoding to Serializing Cache > > > Key: CASSANDRA-4138 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4138 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Affects Versions: 1.2 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.2 > > Attachments: 0001-CASSANDRA-4138-Take1.patch, > 0001-CASSANDRA-4138-V2.patch, 0001-CASSANDRA-4138-v4.patch, > 0002-sizeof-changes-on-rest-of-the-code.patch, CASSANDRA-4138-v3.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4138) Add varint encoding to Serializing Cache
[ https://issues.apache.org/jira/browse/CASSANDRA-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264118#comment-13264118 ] Pavel Yaskevich commented on CASSANDRA-4138: Yeah, this is my intended clarification of why is that needed but I somehow missed it :( > Add varint encoding to Serializing Cache > > > Key: CASSANDRA-4138 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4138 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Affects Versions: 1.2 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.2 > > Attachments: 0001-CASSANDRA-4138-Take1.patch, > 0001-CASSANDRA-4138-V2.patch, 0001-CASSANDRA-4138-v4.patch, > 0002-sizeof-changes-on-rest-of-the-code.patch, CASSANDRA-4138-v3.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4138) Add varint encoding to Serializing Cache
[ https://issues.apache.org/jira/browse/CASSANDRA-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264115#comment-13264115 ] Vijay commented on CASSANDRA-4138: -- In addition to Pavel's comment, the reason for the change is to make the writeWithShortLength write a varint while using EDOS (will just use a byte instead of 2 for most of the cases). > Add varint encoding to Serializing Cache > > > Key: CASSANDRA-4138 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4138 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Affects Versions: 1.2 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.2 > > Attachments: 0001-CASSANDRA-4138-Take1.patch, > 0001-CASSANDRA-4138-V2.patch, 0001-CASSANDRA-4138-v4.patch, > 0002-sizeof-changes-on-rest-of-the-code.patch, CASSANDRA-4138-v3.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4138) Add varint encoding to Serializing Cache
[ https://issues.apache.org/jira/browse/CASSANDRA-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264110#comment-13264110 ] Vijay commented on CASSANDRA-4138: -- >>> Have you done any smoke tests to see what kind of savings you get on >>> typical cached data? Yes i do see 10% gain in space. >>> is our intuition correct that this is worth the extra complexity? The complexity for the Cache as such is only using EDOS instead of DOS... other complexity are generic to support messaging and SSTable formats. > Add varint encoding to Serializing Cache > > > Key: CASSANDRA-4138 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4138 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Affects Versions: 1.2 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.2 > > Attachments: 0001-CASSANDRA-4138-Take1.patch, > 0001-CASSANDRA-4138-V2.patch, 0001-CASSANDRA-4138-v4.patch, > 0002-sizeof-changes-on-rest-of-the-code.patch, CASSANDRA-4138-v3.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4138) Add varint encoding to Serializing Cache
[ https://issues.apache.org/jira/browse/CASSANDRA-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264108#comment-13264108 ] Pavel Yaskevich commented on CASSANDRA-4138: bq. It's not immediately clear to me what the changes in ByteBufferUtil are doing – EDOS doesn't change writeByte so what is breaking? is this backwards compatible? I can explain ByteBufferUtil changes - instead of doing short write manually (copy code that does it from DO) it just uses appropriate method from the DataOutput which would handle the short encode and write. EDOS don't really need to change the way we write bytes, this is only about encoding integer types compactly. All legacy tests are passing and this doesn't seem to touch code so dramatically to break it as soon as it's all about SerializingCache... bq. Have you done any smoke tests to see what kind of savings you get on typical cached data? In other words: is our intuition correct that this is worth the extra complexity? This is question to Vijay so I won't interfere, just want to note that he pointed out in the first comment that this saves ~10% of memory compared to normal DIS. > Add varint encoding to Serializing Cache > > > Key: CASSANDRA-4138 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4138 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Affects Versions: 1.2 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.2 > > Attachments: 0001-CASSANDRA-4138-Take1.patch, > 0001-CASSANDRA-4138-V2.patch, 0001-CASSANDRA-4138-v4.patch, > 0002-sizeof-changes-on-rest-of-the-code.patch, CASSANDRA-4138-v3.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4138) Add varint encoding to Serializing Cache
[ https://issues.apache.org/jira/browse/CASSANDRA-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264084#comment-13264084 ] Jonathan Ellis commented on CASSANDRA-4138: --- It's not immediately clear to me what the changes in ByteBufferUtil are doing -- EDOS doesn't change writeByte so what is breaking? is this backwards compatible? Have you done any smoke tests to see what kind of savings you get on typical cached data? In other words: is our intuition correct that this is worth the extra complexity? > Add varint encoding to Serializing Cache > > > Key: CASSANDRA-4138 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4138 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Affects Versions: 1.2 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.2 > > Attachments: 0001-CASSANDRA-4138-Take1.patch, > 0001-CASSANDRA-4138-V2.patch, 0001-CASSANDRA-4138-v4.patch, > 0002-sizeof-changes-on-rest-of-the-code.patch, CASSANDRA-4138-v3.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4138) Add varint encoding to Serializing Cache
[ https://issues.apache.org/jira/browse/CASSANDRA-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257089#comment-13257089 ] Pavel Yaskevich commented on CASSANDRA-4138: Ok, let's give a Jonathan chance to make a final look. > Add varint encoding to Serializing Cache > > > Key: CASSANDRA-4138 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4138 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Affects Versions: 1.2 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.2 > > Attachments: 0001-CASSANDRA-4138-Take1.patch, > 0001-CASSANDRA-4138-V2.patch, 0001-CASSANDRA-4138-v4.patch, > 0002-sizeof-changes-on-rest-of-the-code.patch, CASSANDRA-4138-v3.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4138) Add varint encoding to Serializing Cache
[ https://issues.apache.org/jira/browse/CASSANDRA-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255195#comment-13255195 ] Pavel Yaskevich commented on CASSANDRA-4138: You can drop changing *_SIZE constants to private in this one but nothing is keeping us from implementing the later part of the proposal, all your changes would use sizeof(type) methods without need to change the rest of the code. > Add varint encoding to Serializing Cache > > > Key: CASSANDRA-4138 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4138 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Affects Versions: 1.2 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.2 > > Attachments: 0001-CASSANDRA-4138-Take1.patch, > 0001-CASSANDRA-4138-V2.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4138) Add varint encoding to Serializing Cache
[
https://issues.apache.org/jira/browse/CASSANDRA-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255143#comment-13255143
]
Pavel Yaskevich commented on CASSANDRA-4138:
To avoid confusion related to naming of {write, read}VLong methods (as it gives
a feeling that writeInt doesn't really write an int anymore) in the EDIS and
EDOS I propose to rename them to {encode, decode}VInt. Furthermore, we could
give a better feel of the encoding used by adding {VInt} as a prefix to both
classes (as an alternative they could be moved to o.a.c.u.vint package), also I
think the DBContants class is now should be changed to only share sizeof(type)
methods and become something like DBContants.{native, encoded}.sizeof(type)...
> Add varint encoding to Serializing Cache
>
>
> Key: CASSANDRA-4138
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4138
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
>Affects Versions: 1.2
>Reporter: Vijay
>Assignee: Vijay
>Priority: Minor
> Fix For: 1.2
>
> Attachments: 0001-CASSANDRA-4138-Take1.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4138) Add varint encoding to Serializing Cache
[ https://issues.apache.org/jira/browse/CASSANDRA-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253906#comment-13253906 ] Vijay commented on CASSANDRA-4138: -- Hi Jonathan, Nope this ticket was just an extension of the parent, i wanted to get the lhf first :)... Once we have EDOS we can use it for IVersionedSerializer.serialize(T t, EncodedDataOutput dos, int version) and OutboundTcpConnection.write(Message message, String id, EncodedDataOutput out) Which will be easier to deal with the version too (and with SST versioning)... Hope it makes sense. > Add varint encoding to Serializing Cache > > > Key: CASSANDRA-4138 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4138 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Affects Versions: 1.2 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.2 > > Attachments: 0001-CASSANDRA-4138-Take1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4138) Add varint encoding to Serializing Cache
[ https://issues.apache.org/jira/browse/CASSANDRA-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253895#comment-13253895 ] Jonathan Ellis commented on CASSANDRA-4138: --- If we're relying on overriding writeInt etc, does that mean we're giving up being able to use varints over the network? (Not a concern for the cache obviously but I'm thinking ahead to CASSANDRA-3024.) > Add varint encoding to Serializing Cache > > > Key: CASSANDRA-4138 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4138 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Affects Versions: 1.2 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.2 > > Attachments: 0001-CASSANDRA-4138-Take1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4138) Add varint encoding to Serializing Cache
[ https://issues.apache.org/jira/browse/CASSANDRA-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253790#comment-13253790 ] Vijay commented on CASSANDRA-4138: -- I did try it before copying the problem is that we cannot override writeInt() writeLong() and writeShort() because they are marked as final. http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/io/DataOutputStream.java#DataOutputStream.writeInt%28int%29 > Add varint encoding to Serializing Cache > > > Key: CASSANDRA-4138 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4138 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Affects Versions: 1.2 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.2 > > Attachments: 0001-CASSANDRA-4138-Take1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4138) Add varint encoding to Serializing Cache
[ https://issues.apache.org/jira/browse/CASSANDRA-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253761#comment-13253761 ] Jonathan Ellis commented on CASSANDRA-4138: --- Couldn't we just extend DataOutputStream instead of introducing ADOS? (Pretty sure AbstractDataInput was a mistake as well.) > Add varint encoding to Serializing Cache > > > Key: CASSANDRA-4138 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4138 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Affects Versions: 1.2 >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Fix For: 1.2 > > Attachments: 0001-CASSANDRA-4138-Take1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
