[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2017-01-09 Thread Tom van der Woerdt (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812196#comment-15812196
 ] 

Tom van der Woerdt commented on CASSANDRA-11937:


Can we get this backported to 3.0.x?

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Fix For: 3.8
>
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2016-06-24 Thread Alex Petrov (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348447#comment-15348447
 ] 

Alex Petrov commented on CASSANDRA-11937:
-

thank you!

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Fix For: 3.8
>
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2016-06-23 Thread Alex Petrov (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347142#comment-15347142
 ] 

Alex Petrov commented on CASSANDRA-11937:
-

Great, reverted & force-pushed.

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2016-06-23 Thread T Jake Luciani (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346987#comment-15346987
 ] 

T Jake Luciani commented on CASSANDRA-11937:


Sorry, I think I misread the code there is nothing calling reuse() directly. 
the old code was ok.

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2016-06-23 Thread Alex Petrov (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346958#comment-15346958
 ] 

Alex Petrov commented on CASSANDRA-11937:
-

> I think reuse() should call cleanup() too since it previously did both

Added call to `cleanup` from `reuse`. When recycling the object, `cleanup` will 
now be called before we put it to recycler and after we get a recycled object.

Also, re-running all tests just in case:

|[trunk|https://github.com/ifesdjeen/cassandra/tree/11937-trunk]|[utest|http://cassci.datastax.com/job/ifesdjeen-11937-trunk-testall/1/]|[dtest|http://cassci.datastax.com/job/ifesdjeen-11937-trunk-dtest/1/]|[offheap
 memtable 
dtests|http://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-11937-trunk-offheap-dtest/]|


> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2016-06-23 Thread T Jake Luciani (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346897#comment-15346897
 ] 

T Jake Luciani commented on CASSANDRA-11937:


I think reuse() should call cleanup() too since it previously did both

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2016-06-23 Thread Alex Petrov (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346695#comment-15346695
 ] 

Alex Petrov commented on CASSANDRA-11937:
-

Looks like {{dtest failure in largecolumn_test:TestLargeColumn.cleanup_test}} 
[passed|http://cassci.datastax.com/job/ifesdjeen-11937-trunk-offheap-dtest/lastCompletedBuild/testReport/largecolumn_test/TestLargeColumn/cleanup_test/]

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2016-06-23 Thread Philip Thompson (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346527#comment-15346527
 ] 

Philip Thompson commented on CASSANDRA-11937:
-

I kicked off this branch with the offheap memtable dtests
http://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-11937-trunk-offheap-dtest/

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2016-06-22 Thread Alex Petrov (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344435#comment-15344435
 ] 

Alex Petrov commented on CASSANDRA-11937:
-

Yes, I'm currently running multiplexed tests. If they're clean, I'm submitting 
patch shortly.

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2016-06-22 Thread T Jake Luciani (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344430#comment-15344430
 ] 

T Jake Luciani commented on CASSANDRA-11937:


Makes sense. So clear the Builder when build() is called?

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2016-06-22 Thread Alex Petrov (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344305#comment-15344305
 ] 

Alex Petrov commented on CASSANDRA-11937:
-

[~tjake] turns out that same exact non-recycled reference will keep hanging 
because of {{BTree$Builder}} reference that holds on to the same object. 
Screenshot attached. 

So we have to call something similar to {{reuse}} before we actually recycle 
the object, that'd clean it up before it gets recycled, since we discard the 
values on reuse anyways.

The other possible issue (still theoretically possible, although tests are 
passing even without Netty patch, maybe because of major GC) GC nepotism that 
you've previously mentioned. Because in Recycler {{Stack}} object is a hard 
reference, the item that has to get recycled, keeps hanging. That could be 
technically solved by making the reference weak. 

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
> Attachments: Screen Shot 2016-06-22 at 15.24.05.png
>
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2016-06-21 Thread Robert Stupp (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341907#comment-15341907
 ] 

Robert Stupp commented on CASSANDRA-11937:
--

Netty 4.0.37 is now in trunk.

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>  Labels: lhf, netty, reminder
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2016-06-03 Thread Alex Petrov (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314647#comment-15314647
 ] 

Alex Petrov commented on CASSANDRA-11937:
-

Just read my message again, it's a bit wrong. The patch is 
[merged|https://github.com/netty/netty/pull/5333] to both 4.0 and 4.1. We need 
to wait until the next 4.0 release is out, which allows us to upgrade without 
trouble, and remove the buffer trimming from 
[here|https://github.com/apache/cassandra/commit/eb5a59a311a14cc0a3c37a13d10abc8c5a0f6d1b#diff-6aa10752b68c93ed354d642f9bdbe814R92]
 (instead of trimming buffers we'll be able to just throw them away / not 
recycle).

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>  Labels: lhf, netty, reminder
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11937) Clean up buffer trimming large buffers in DataOutputBuffer after the Netty upgrade

2016-06-03 Thread Sylvain Lebresne (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314276#comment-15314276
 ] 

Sylvain Lebresne commented on CASSANDRA-11937:
--

So what is there to do here? Just upgrade to Netty 4.1? Or do we have to do 
something on top of that in C*?

> Clean up buffer trimming large buffers in DataOutputBuffer after the Netty 
> upgrade
> --
>
> Key: CASSANDRA-11937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11937
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Alex Petrov
>  Labels: lhf, netty, reminder
>
> In [https://issues.apache.org/jira/browse/CASSANDRA-11838|11838], we're 
> trimming the large buffers in {{DataOutputBuffer}}. The patch is already 
> submitted and merged in [Netty 
> 4.1|https://github.com/netty/netty/commit/bbed330468b5b82c9e4defa59012d0fcdb70f1aa],
>  we only need to make sure that we throw large buffers away1 alltogether 
> instead of trimming them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)