[ https://issues.apache.org/jira/browse/HDFS-13702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16525569#comment-16525569 ]
Todd Lipcon commented on HDFS-13702: ------------------------------------ {quote}I do want a trace layer in there; I do want it broader than just HDFS, and I do want it to be used from the layers above. I don't want stuff taken from DFSClient until there's a better story there. Otherwise: it'll get cut, nobody will replace it, and it'll get lost in folklore. {quote} Sure, I agree tracing is nice. I recently worked on adding OpenTracing support to the HMS (HIVE-19685). I think we could do the same here. That said, regardless of trace framework, I think we can all agree that the current code path is too expensive as written today. I don't have time to do the heavy lifting of moving entirely to some new tracing framework, so in the meantime I think we should fix this perf issue by removing this heavy code path. I didn't remove all the other FS-operation-level trace annotations (eg open/close/listStatus/etc) since those are likely to be less high throughput and therefore overhead not an issue. > HTrace hooks taking 10-15% CPU in DFS client when disabled > ---------------------------------------------------------- > > Key: HDFS-13702 > URL: https://issues.apache.org/jira/browse/HDFS-13702 > Project: Hadoop HDFS > Issue Type: Bug > Components: performance > Affects Versions: 3.0.0 > Reporter: Todd Lipcon > Assignee: Todd Lipcon > Priority: Major > Attachments: hdfs-13702.patch, hdfs-13702.patch, hdfs-13702.patch > > > I am seeing DFSClient.newReaderTraceScope take ~15% CPU in a teravalidate > workload even when HTrace is disabled. This is because it stringifies several > integers. We should avoid all allocation and stringification when htrace is > disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org