[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478803#comment-16478803 ] ASF GitHub Bot commented on IO-468: --- Github user berndhopp closed the pull request at:

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14332404#comment-14332404 ] Thomas Neidhart commented on IO-468: Well, my arguments against this patch have certainly

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14332363#comment-14332363 ] Bernd Hopp commented on IO-468: --- as far as I can see, the numbers show two things: - fixed arra

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14325626#comment-14325626 ] Thomas Neidhart commented on IO-468: The Error is the score inside the 99.9% confidence in

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14325308#comment-14325308 ] Sebb commented on IO-468: - I assume the Error column is the margin of error? This increases dramatical

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314993#comment-14314993 ] Thomas Neidhart commented on IO-468: I think it is a pretty easy trap to fall into and man

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314989#comment-14314989 ] Thomas Neidhart commented on IO-468: ok, I will see if I find some time the next days to c

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14313901#comment-14313901 ] Bernd Hopp commented on IO-468: --- @Thomas Neidhart Thank you for your effort, could you share yo

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14313023#comment-14313023 ] Bernd Hopp commented on IO-468: --- this is interesting. Awkward for me of course to not have spott

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312870#comment-14312870 ] Thomas Neidhart commented on IO-468: Some benchmarks I did with my own test harness. The n

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312325#comment-14312325 ] Bernd Hopp commented on IO-468: --- Thomas, I DID read the material. I responded to every point mad

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312252#comment-14312252 ] Thomas Neidhart commented on IO-468: I am sorry, but you got so many pointers to the actua

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312248#comment-14312248 ] Bernd Hopp commented on IO-468: --- So the "compromise" I made was to use WeakReference? And your p

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312219#comment-14312219 ] Joerg Schaible commented on IO-468: --- Sorry, but Sebb is not alone here. We're very defensive

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312215#comment-14312215 ] Bernd Hopp commented on IO-468: --- I will try that > Avoid allocating memory for method internal

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312189#comment-14312189 ] Bernd Hopp commented on IO-468: --- [~s...@apache.org], you are one of the persons responsible for

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312153#comment-14312153 ] Sebb commented on IO-468: - When I said that buffer allocation is not slow, I meant that it is not slow

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312141#comment-14312141 ] Bernd Hopp commented on IO-468: --- allright, thank you > Avoid allocating memory for method inter

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312138#comment-14312138 ] Mark Thomas commented on IO-468: @[~bernd_hopp] Thsi issue with the current implementation is

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312133#comment-14312133 ] Bernd Hopp commented on IO-468: --- @Sebb "think the benchmark is not really the main issue here,

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311673#comment-14311673 ] Sebb commented on IO-468: - I think the benchmark is not really the main issue here, although it will b

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311594#comment-14311594 ] Emmanuel Bourg commented on IO-468: --- Bernd I suggest you rewrite your micro benchmark with J

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311587#comment-14311587 ] Bernd Hopp commented on IO-468: --- [~javaguru] Thank you for your comment. By 'the patch as curren

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311580#comment-14311580 ] Mark Thomas commented on IO-468: The patch as currently would trigger a class loader memory le

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311562#comment-14311562 ] Thomas Neidhart commented on IO-468: You did not fully understand the article: it is not a

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311556#comment-14311556 ] Bernd Hopp commented on IO-468: --- [~tn] please look again, there is a jit-phase at the start of m

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311513#comment-14311513 ] Thomas Neidhart commented on IO-468: additional note: your micro-benchmarking test is also

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311511#comment-14311511 ] Thomas Neidhart commented on IO-468: btw. this Stackoverflow question about ThreadLocals s

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311504#comment-14311504 ] Thomas Neidhart commented on IO-468: The drawbacks of ThreadLocals are that they might lea

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311352#comment-14311352 ] Bernd Hopp commented on IO-468: --- I'm sorry, I attached an outdated 'performancetest.ods'. I have

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311350#comment-14311350 ] Bernd Hopp commented on IO-468: --- [~tn] "ThreadLocals have drawbacks, and I would not use them f

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311337#comment-14311337 ] Bernd Hopp commented on IO-468: --- "Trying to do some internal optimization for the general use ca

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311334#comment-14311334 ] Bernd Hopp commented on IO-468: --- this is my software setup: mvn -version Apache Maven 3.2.2 (NO

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311332#comment-14311332 ] Thomas Neidhart commented on IO-468: if there are already method allowing to pass an array

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311312#comment-14311312 ] Sebb commented on IO-468: - @[~tn] The IOUtils class already provides access to methods that accept a

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311308#comment-14311308 ] Sebb commented on IO-468: - What version of Java did you use for compile/test? As I already wrote, I d

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311300#comment-14311300 ] Thomas Neidhart commented on IO-468: There are use-cases where users want to prevent memor

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311285#comment-14311285 ] Bernd Hopp commented on IO-468: --- testing was more difficult than I expected, but heres some resu

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307092#comment-14307092 ] Bernd Hopp commented on IO-468: --- I'm gonna do some more testing on: 1: Is there a reproducable

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307074#comment-14307074 ] Sebb commented on IO-468: - As to the use of ThreadLocal for the other buffers: I agree that it does s

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307032#comment-14307032 ] Bernd Hopp commented on IO-468: --- you're right, this does not make sense, I'm gonna revert these

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307005#comment-14307005 ] Sebb commented on IO-468: - Regarding the Skip buffer: this is currently a static buffer, and is shared

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306930#comment-14306930 ] Bernd Hopp commented on IO-468: --- I referred to the current PR at https://github.com/apache/comm

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305390#comment-14305390 ] Sebb commented on IO-468: - Where is the new patch? Note that with the current code, the memory is rel

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305183#comment-14305183 ] Bernd Hopp commented on IO-468: --- Hello Sebb, thanks for your comments. Your concerns regarding

[jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead

[ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303254#comment-14303254 ] Sebb commented on IO-468: - AIUI memory allocation via new byte[] is not cpu-intensive. Also the memory