[jira] [Commented] (KAFKA-7184) Kafka is going down with issue ERROR Failed to clean up log for __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException (kafka.server.LogDirFailureChannel)
[ https://issues.apache.org/jira/browse/KAFKA-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549336#comment-16549336 ] Dhruvil Shah commented on KAFKA-7184: - [~smuddamsetty] after reaching the retention point, Kafka will try to delete segments which are past retention. It is possible that these files no longer exist because the /tmp directory was cleaned out before. Kafka requires a persistent store be used for the log directory so you should test with that. > Kafka is going down with issue ERROR Failed to clean up log for > __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException > (kafka.server.LogDirFailureChannel) > -- > > Key: KAFKA-7184 > URL: https://issues.apache.org/jira/browse/KAFKA-7184 > Project: Kafka > Issue Type: Test > Components: admin, log >Affects Versions: 1.1.0 >Reporter: Sandeep Muddamsetty >Priority: Blocker > Attachments: log-cleaner.log, server.log.2018-07-18-15 > > > Kafka is going down with issue ERROR Failed to clean up log for > __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException > (kafka.server.LogDirFailureChannel). > This error we are seeing very frequently for every 168 hours(7 days) where > the defualt value of log retention in kafka configuration . After modifying > it to 240 hours this thing is happening again after 240 hours . I ahve gone > thorugh some google groups some artiicles and have observed this is happening > in windows system but here i am facing this issue in linux system . Below are > my configuration details . > kafka_2.11-1.0.0 > OS: OEL 7.1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7184) Kafka is going down with issue ERROR Failed to clean up log for __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException (kafka.server.LogDirFailureChannel)
[ https://issues.apache.org/jira/browse/KAFKA-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549286#comment-16549286 ] Sandeep Muddamsetty commented on KAFKA-7184: [~omkreddy] Yes . My log location is /tmp but i guess this error is not occuring due to OS clean up's as i am facing this issue after reaching to retention period hours which was set to 240 hours . If this is from OS level how can we track and get a conclusion to that ?. > Kafka is going down with issue ERROR Failed to clean up log for > __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException > (kafka.server.LogDirFailureChannel) > -- > > Key: KAFKA-7184 > URL: https://issues.apache.org/jira/browse/KAFKA-7184 > Project: Kafka > Issue Type: Test > Components: admin, log >Affects Versions: 1.1.0 >Reporter: Sandeep Muddamsetty >Priority: Blocker > Attachments: log-cleaner.log, server.log.2018-07-18-15 > > > Kafka is going down with issue ERROR Failed to clean up log for > __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException > (kafka.server.LogDirFailureChannel). > This error we are seeing very frequently for every 168 hours(7 days) where > the defualt value of log retention in kafka configuration . After modifying > it to 240 hours this thing is happening again after 240 hours . I ahve gone > thorugh some google groups some artiicles and have observed this is happening > in windows system but here i am facing this issue in linux system . Below are > my configuration details . > kafka_2.11-1.0.0 > OS: OEL 7.1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7184) Kafka is going down with issue ERROR Failed to clean up log for __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException (kafka.server.LogDirFailureChannel)
[ https://issues.apache.org/jira/browse/KAFKA-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549186#comment-16549186 ] Manikumar commented on KAFKA-7184: -- looks like your data directory is in /tmp/ folder. /tmp may get cleared during system OS cleanups etc.. For production deployments, you may need to configure the "log.dirs" property with any other valid directory path. > Kafka is going down with issue ERROR Failed to clean up log for > __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException > (kafka.server.LogDirFailureChannel) > -- > > Key: KAFKA-7184 > URL: https://issues.apache.org/jira/browse/KAFKA-7184 > Project: Kafka > Issue Type: Test > Components: admin, log >Affects Versions: 1.1.0 >Reporter: Sandeep Muddamsetty >Priority: Blocker > Attachments: log-cleaner.log, server.log.2018-07-18-15 > > > Kafka is going down with issue ERROR Failed to clean up log for > __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException > (kafka.server.LogDirFailureChannel). > This error we are seeing very frequently for every 168 hours(7 days) where > the defualt value of log retention in kafka configuration . After modifying > it to 240 hours this thing is happening again after 240 hours . I ahve gone > thorugh some google groups some artiicles and have observed this is happening > in windows system but here i am facing this issue in linux system . Below are > my configuration details . > kafka_2.11-1.0.0 > OS: OEL 7.1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7184) Kafka is going down with issue ERROR Failed to clean up log for __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException (kafka.server.LogDirFailureChannel)
[ https://issues.apache.org/jira/browse/KAFKA-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548927#comment-16548927 ] Sandeep Muddamsetty commented on KAFKA-7184: [~dhruvilshah] Please find the attached the logs > please let me know if any further details required . We are using SSD storage. > Kafka is going down with issue ERROR Failed to clean up log for > __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException > (kafka.server.LogDirFailureChannel) > -- > > Key: KAFKA-7184 > URL: https://issues.apache.org/jira/browse/KAFKA-7184 > Project: Kafka > Issue Type: Test > Components: admin, log >Affects Versions: 1.1.0 >Reporter: Sandeep Muddamsetty >Priority: Blocker > Attachments: log-cleaner.log, server.log.2018-07-18-15 > > > Kafka is going down with issue ERROR Failed to clean up log for > __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException > (kafka.server.LogDirFailureChannel). > This error we are seeing very frequently for every 168 hours(7 days) where > the defualt value of log retention in kafka configuration . After modifying > it to 240 hours this thing is happening again after 240 hours . I ahve gone > thorugh some google groups some artiicles and have observed this is happening > in windows system but here i am facing this issue in linux system . Below are > my configuration details . > kafka_2.11-1.0.0 > OS: OEL 7.1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7184) Kafka is going down with issue ERROR Failed to clean up log for __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException (kafka.server.LogDirFailureChannel)
[ https://issues.apache.org/jira/browse/KAFKA-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548907#comment-16548907 ] Dhruvil Shah commented on KAFKA-7184: - What kind of storage are you using? Could you please upload server.log and log_cleaner.log from around the time of the error. > Kafka is going down with issue ERROR Failed to clean up log for > __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException > (kafka.server.LogDirFailureChannel) > -- > > Key: KAFKA-7184 > URL: https://issues.apache.org/jira/browse/KAFKA-7184 > Project: Kafka > Issue Type: Test > Components: admin, log >Affects Versions: 1.1.0 >Reporter: Sandeep Muddamsetty >Priority: Blocker > > Kafka is going down with issue ERROR Failed to clean up log for > __consumer_offsets-0 in dir /tmp/kafkadev2-logs due to IOException > (kafka.server.LogDirFailureChannel). > This error we are seeing very frequently for every 168 hours(7 days) where > the defualt value of log retention in kafka configuration . After modifying > it to 240 hours this thing is happening again after 240 hours . I ahve gone > thorugh some google groups some artiicles and have observed this is happening > in windows system but here i am facing this issue in linux system . Below are > my configuration details . > kafka_2.11-1.0.0 > OS: OEL 7.1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
