[jira] [Comment Edited] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14958698#comment-14958698 ] Ismael Juma edited comment on KAFKA-2419 at 10/15/15 10:27 AM: --- [~aauradkar] [~jjkoshy] We are creating one thread per `Metrics` instance and we create one `Metrics` per consumer and per producer. Do we really want to create one garbage collection thread per consumer/producer? Also, I can't find any usages of `ExpireSensorTask` outside of tests: https://github.com/apache/kafka/search?utf8=%E2%9C%93=ExpireSensorTask What am I missing? was (Author: ijuma): [~aauradkar] [~jjkoshy] We are creating one thread per `Metrics` instance and we create one `Metrics` per consumer and per producer. Do we really one to create one garbage collection thread per consumer/producer? Also, I can't find any usages of `ExpireSensorTask` outside of tests: https://github.com/apache/kafka/search?utf8=%E2%9C%93=ExpireSensorTask What am I missing? > Allow certain Sensors to be garbage collected after inactivity > -- > > Key: KAFKA-2419 > URL: https://issues.apache.org/jira/browse/KAFKA-2419 > Project: Kafka > Issue Type: New Feature >Affects Versions: 0.9.0.0 >Reporter: Aditya Auradkar >Assignee: Aditya Auradkar >Priority: Blocker > Labels: quotas > Fix For: 0.9.0.0 > > > Currently, metrics cannot be removed once registered. > Implement a feature to remove certain sensors after a certain period of > inactivity (perhaps configurable). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14958978#comment-14958978 ] Ismael Juma edited comment on KAFKA-2419 at 10/15/15 2:26 PM: -- [~aauradkar] Thanks for submitting a fix for `ExpireSensorTask` later. Regarding the impact of this change on the new consumer, it seems to me that it's a bit of a shame to create one thread per consumer since we worked so hard to make it really cheap to create new consumers. Others have been more involved with the design of the new consumer so their opinion is more relevant than mine though, what are your thoughts [~hachikuji] [~guozhang] [~ewencp]? was (Author: ijuma): [~aauradkar] Thanks for submitting a fix for `ExpireSensorTask` later. Regarding the impact of this change on the new consumer, it seems to me that it's a bit of a shame to create one thread per consumer since we worked so hard to make it really cheap to create new consumers. Others have been more involved with the design of the new consumer so their opinion is more relevant, what are your thoughts [~hachikuji] [~guozhang] [~ewencp]? > Allow certain Sensors to be garbage collected after inactivity > -- > > Key: KAFKA-2419 > URL: https://issues.apache.org/jira/browse/KAFKA-2419 > Project: Kafka > Issue Type: New Feature >Affects Versions: 0.9.0.0 >Reporter: Aditya Auradkar >Assignee: Aditya Auradkar >Priority: Blocker > Labels: quotas > Fix For: 0.9.0.0 > > > Currently, metrics cannot be removed once registered. > Implement a feature to remove certain sensors after a certain period of > inactivity (perhaps configurable). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959543#comment-14959543 ] Jay Kreps edited comment on KAFKA-2419 at 10/15/15 8:19 PM: It would be preferable to not need additional threads--we put significant effort into designing the framework to work without the need for a background "tick" thread. I agree that this is more a concern on the client than the server. Is it too complex to just expire sensors as part of other work? E.g. check for expiry when registering new sensors (after all you don't care about leaking unless there is growth in the sensor count) was (Author: jkreps): It would be preferable to not need additional threads--we put significant effort into designing it to work without the need for a background "tick" thread. Is it too complex to just expire sensors as part of other work? > Allow certain Sensors to be garbage collected after inactivity > -- > > Key: KAFKA-2419 > URL: https://issues.apache.org/jira/browse/KAFKA-2419 > Project: Kafka > Issue Type: New Feature >Affects Versions: 0.9.0.0 >Reporter: Aditya Auradkar >Assignee: Aditya Auradkar >Priority: Blocker > Labels: quotas > Fix For: 0.9.0.0 > > > Currently, metrics cannot be removed once registered. > Implement a feature to remove certain sensors after a certain period of > inactivity (perhaps configurable). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity
[ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959409#comment-14959409 ] Aditya Auradkar edited comment on KAFKA-2419 at 10/15/15 6:52 PM: -- [~junrao] - I concur that option 2 is simpler. One minor comment is that it may be better to leave the sensor objects as they are. Instead of a boolean, they are configured with a numeric expiration delay. By default, they won't ever expire.. basically the same as not being enabled for time based GC. I can submit a patch for this today. was (Author: aauradkar): [~junrao] - I concur that option 2 is simpler. One minor comment is that it may be better to leave the sensor objects as they are. Instead of a boolean, they are configured with a numeric expiration delay. By default, they won't ever expire.. basically the same as not being enabled for time based GC. > Allow certain Sensors to be garbage collected after inactivity > -- > > Key: KAFKA-2419 > URL: https://issues.apache.org/jira/browse/KAFKA-2419 > Project: Kafka > Issue Type: New Feature >Affects Versions: 0.9.0.0 >Reporter: Aditya Auradkar >Assignee: Aditya Auradkar >Priority: Blocker > Labels: quotas > Fix For: 0.9.0.0 > > > Currently, metrics cannot be removed once registered. > Implement a feature to remove certain sensors after a certain period of > inactivity (perhaps configurable). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
