[jira] [Commented] (ACCUMULO-4041) Java reflection eats too much CPU
[ https://issues.apache.org/jira/browse/ACCUMULO-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15004256#comment-15004256 ] Josh Elser commented on ACCUMULO-4041: -- Excellent, thanks to writing it down -- I meant to just write one myself, but never made the time. > Java reflection eats too much CPU > - > > Key: ACCUMULO-4041 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4041 > Project: Accumulo > Issue Type: Bug > Components: tserver >Affects Versions: 1.7.0 > Environment: Ubuntu 15.10, jdk1.8.0_60 >Reporter: Volth >Assignee: Eric Newton > Fix For: 1.7.1, 1.8.0 > > Attachments: ACCUMULO-4041-01.patch, ACCUMULO-4041-experiment.patch, > profiling.jpg > > Time Spent: 50m > Remaining Estimate: 0h > > This is called so many times that it eats too much CPU, even more than the > snappy compressor: http://i.imgur.com/vqX4pFi.png > Perhaps the annotation data should be copied into regular variables which can > be accessed without reflection. > - org.apache.accumulo.core.client.impl.Tables.getZooCache() > - > org.apache.accumulo.server.client.HdfsZooInstance.getZooKeepersSessionTimeOut() > - org.apache.accumulo.core.conf.AccumuloConfiguration.getTimeInMillis() > - org.apache.accumulo.core.conf.SiteConfiguration.get() > - org.apache.accumulo.core.conf.Property.isSensitive() > - org.apache.accumulo.core.conf.Property.hasAnnotation() > - java.lang.Class.getField() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4041) Java reflection eats too much CPU
[ https://issues.apache.org/jira/browse/ACCUMULO-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003941#comment-15003941 ] Eric Newton commented on ACCUMULO-4041: --- I wrote a small test that called the equivalent of the old code 10K times: ~100ms on my box. The improved version (which was basically what [~kturner] suggested), took 2ms. So, roughly 50x faster. > Java reflection eats too much CPU > - > > Key: ACCUMULO-4041 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4041 > Project: Accumulo > Issue Type: Bug > Components: tserver >Affects Versions: 1.7.0 > Environment: Ubuntu 15.10, jdk1.8.0_60 >Reporter: Volth >Assignee: Eric Newton > Fix For: 1.7.1, 1.8.0 > > Attachments: ACCUMULO-4041-01.patch, ACCUMULO-4041-experiment.patch, > profiling.jpg > > > This is called so many times that it eats too much CPU, even more than the > snappy compressor: http://i.imgur.com/vqX4pFi.png > Perhaps the annotation data should be copied into regular variables which can > be accessed without reflection. > - org.apache.accumulo.core.client.impl.Tables.getZooCache() > - > org.apache.accumulo.server.client.HdfsZooInstance.getZooKeepersSessionTimeOut() > - org.apache.accumulo.core.conf.AccumuloConfiguration.getTimeInMillis() > - org.apache.accumulo.core.conf.SiteConfiguration.get() > - org.apache.accumulo.core.conf.Property.isSensitive() > - org.apache.accumulo.core.conf.Property.hasAnnotation() > - java.lang.Class.getField() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4041) Java reflection eats too much CPU
[ https://issues.apache.org/jira/browse/ACCUMULO-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14996763#comment-14996763 ] Volth commented on ACCUMULO-4041: - Maybe it has sense not to cache `Property.isSensitive()` but `SiteConfiguration.get()` It would address both issues ACCUMULO-4041 and ACCUMULO-4048 > Java reflection eats too much CPU > - > > Key: ACCUMULO-4041 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4041 > Project: Accumulo > Issue Type: Bug > Components: tserver >Affects Versions: 1.7.0 > Environment: Ubuntu 15.10, jdk1.8.0_60 >Reporter: Volth >Assignee: Eric Newton > Fix For: 1.7.1, 1.8.0 > > Attachments: ACCUMULO-4041-01.patch, ACCUMULO-4041-experiment.patch, > profiling.jpg > > > This is called so many times that it eats too much CPU, even more than the > snappy compressor: http://i.imgur.com/vqX4pFi.png > Perhaps the annotation data should be copied into regular variables which can > be accessed without reflection. > - org.apache.accumulo.core.client.impl.Tables.getZooCache() > - > org.apache.accumulo.server.client.HdfsZooInstance.getZooKeepersSessionTimeOut() > - org.apache.accumulo.core.conf.AccumuloConfiguration.getTimeInMillis() > - org.apache.accumulo.core.conf.SiteConfiguration.get() > - org.apache.accumulo.core.conf.Property.isSensitive() > - org.apache.accumulo.core.conf.Property.hasAnnotation() > - java.lang.Class.getField() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4041) Java reflection eats too much CPU
[ https://issues.apache.org/jira/browse/ACCUMULO-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14993895#comment-14993895 ] Josh Elser commented on ACCUMULO-4041: -- bq. Proposed patch. Could also cache the lookup in SiteConfiguration, too. I hope I don't have to so this for all the different annotations. Did you happen to write any sort of micro-benchmark for testing this patch? Would be nice to be able to quantify how bad it was previously, and how much the cache approach helps. > Java reflection eats too much CPU > - > > Key: ACCUMULO-4041 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4041 > Project: Accumulo > Issue Type: Bug > Components: tserver >Affects Versions: 1.7.0 > Environment: Ubuntu 15.10, jdk1.8.0_60 >Reporter: Volth >Assignee: Eric Newton > Fix For: 1.7.1, 1.8.0 > > Attachments: ACCUMULO-4041-01.patch, ACCUMULO-4041-experiment.patch, > profiling.jpg > > > This is called so many times that it eats too much CPU, even more than the > snappy compressor: http://i.imgur.com/vqX4pFi.png > Perhaps the annotation data should be copied into regular variables which can > be accessed without reflection. > - org.apache.accumulo.core.client.impl.Tables.getZooCache() > - > org.apache.accumulo.server.client.HdfsZooInstance.getZooKeepersSessionTimeOut() > - org.apache.accumulo.core.conf.AccumuloConfiguration.getTimeInMillis() > - org.apache.accumulo.core.conf.SiteConfiguration.get() > - org.apache.accumulo.core.conf.Property.isSensitive() > - org.apache.accumulo.core.conf.Property.hasAnnotation() > - java.lang.Class.getField() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4041) Java reflection eats too much CPU
[ https://issues.apache.org/jira/browse/ACCUMULO-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992992#comment-14992992 ] Eric Newton commented on ACCUMULO-4041: --- No, that was *not* what I intended! Thanks for catching that. > Java reflection eats too much CPU > - > > Key: ACCUMULO-4041 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4041 > Project: Accumulo > Issue Type: Bug > Components: tserver >Affects Versions: 1.7.0 > Environment: Ubuntu 15.10, jdk1.8.0_60 >Reporter: Volth >Assignee: Eric Newton > Fix For: 1.7.1, 1.8.0 > > Attachments: ACCUMULO-4041-01.patch, ACCUMULO-4041-experiment.patch, > profiling.jpg > > > This is called so many times that it eats too much CPU, even more than the > snappy compressor: http://i.imgur.com/vqX4pFi.png > Perhaps the annotation data should be copied into regular variables which can > be accessed without reflection. > - org.apache.accumulo.core.client.impl.Tables.getZooCache() > - > org.apache.accumulo.server.client.HdfsZooInstance.getZooKeepersSessionTimeOut() > - org.apache.accumulo.core.conf.AccumuloConfiguration.getTimeInMillis() > - org.apache.accumulo.core.conf.SiteConfiguration.get() > - org.apache.accumulo.core.conf.Property.isSensitive() > - org.apache.accumulo.core.conf.Property.hasAnnotation() > - java.lang.Class.getField() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-4041) Java reflection eats too much CPU
[
https://issues.apache.org/jira/browse/ACCUMULO-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978781#comment-14978781
]
Keith Turner commented on ACCUMULO-4041:
Had a similar issue with {{Property.getDefaultValue()}} resolved those once
and cached at [DefaultConfiguration.java line
31|https://github.com/apache/accumulo/blob/2e9643dcc406df3f1a502f9eaf543d1d9bdee560/core/src/main/java/org/apache/accumulo/core/conf/DefaultConfiguration.java#L31].
Could do something similar here, build a hashset of the sensitive props once.
> Java reflection eats too much CPU
> -
>
> Key: ACCUMULO-4041
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4041
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
>Affects Versions: 1.7.0
> Environment: Ubuntu 15.10, jdk1.8.0_60
>Reporter: Volth
>
> This is called so many times that it eats too much CPU, even more than the
> snappy compressor: http://i.imgur.com/vqX4pFi.png
> Perhaps the annotation data should be copied into regular variables which can
> be accessed without reflection.
> - org.apache.accumulo.core.client.impl.Tables.getZooCache()
> -
> org.apache.accumulo.server.client.HdfsZooInstance.getZooKeepersSessionTimeOut()
> - org.apache.accumulo.core.conf.AccumuloConfiguration.getTimeInMillis()
> - org.apache.accumulo.core.conf.SiteConfiguration.get()
> - org.apache.accumulo.core.conf.Property.isSensitive()
> - org.apache.accumulo.core.conf.Property.hasAnnotation()
> - java.lang.Class.getField()
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
