[GitHub] [flink] lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager
lamber-ken commented on issue #10228: URL: https://github.com/apache/flink/pull/10228#issuecomment-618568589 > @lamber-ken @vthinkxie I've created a draft PR #11887 which is based on this one here. The difference is that we don't use the blob cache service to transmit the thread dump from the `TaskExecutor`. Moreover, I changed the return type of the handler from text to JSON. This would allow to extend the handler in the future. Please take a look and let me know what you think. Thanks for doing that, I'm ok closing this đź‘Ť This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager
lamber-ken commented on issue #10228:
URL: https://github.com/apache/flink/pull/10228#issuecomment-618370265
> Hi @lamber-ken
> could the thread dump be part of the log list?
> cc @jinglining
hi @vthinkxie, thread dump is not a real file, we shouldn't place it on log
list
```
// TaskExecutor#requestLogList
@Override
public CompletableFuture> requestLogList(Time timeout) {
return CompletableFuture.supplyAsync(() -> {
final String logDir =
taskManagerConfiguration.getTaskManagerLogDir();
if (logDir != null) {
final File[] logFiles = new File(logDir).listFiles();
if (logFiles == null) {
throw new CompletionException(new
FlinkException(String.format("There isn't a log file in TaskExecutor’s log dir
%s.", logDir)));
}
return Arrays.stream(logFiles)
.filter(File::isFile)
.map(logFile -> new
LogInfo(logFile.getName(), logFile.length()))
.collect(Collectors.toList());
}
return Collections.emptyList();
}, ioExecutor);
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager
lamber-ken commented on issue #10228: URL: https://github.com/apache/flink/pull/10228#issuecomment-616924812 Hi @tillrohrmann @vthinkxie , please review again, thanks  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager
lamber-ken commented on issue #10228: URL: https://github.com/apache/flink/pull/10228#issuecomment-616605289 hi @tillrohrmann @vthinkxie, welcome : ) Thanks for reviewing again, will update the pr. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager
lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager URL: https://github.com/apache/flink/pull/10228#issuecomment-562916680 > Now that the old Web UI is dropped, can we rebase this on the latest master? Done. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager
lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager URL: https://github.com/apache/flink/pull/10228#issuecomment-557055299 > Okay, then let's drop the changes to files from the old UI. But I don't think the changes will affect the deletion of old UI. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager
lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager URL: https://github.com/apache/flink/pull/10228#issuecomment-557052392 > We shouldn't be modifying the old UI. There's no reasonable way for us to review this change. > Given that no issues were found in 1.9 in regards to the new UI, we could even think about removing the old one altogether. I build the project local, the thread dump function works ok on both old ui and new ui. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager
lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager URL: https://github.com/apache/flink/pull/10228#issuecomment-555801286 > > > I'm not sure whether or not it is a desirable feature to implement. Could you share the user case a bit? @lamber-ken > > > > > > It's hard to get the thread dump of tm when job is hanging, because flink tasks are deployed on yarn cluster. > > For example, when fixing the deadlock of elasticserch-connector, it needs to jump to the tm machine > > to dump the jstack. see [elasticsearch-connector-deadlock](https://github.com/elastic/elasticsearch/pull/48013). > > Thanks for your explanation. I think this is a valuable feature! > > @flinkbot approve-until consensus You're welcome. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager
lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager URL: https://github.com/apache/flink/pull/10228#issuecomment-555801129 hi, @StephanEwen. I had update the pr as your suggestion, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager
lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager URL: https://github.com/apache/flink/pull/10228#issuecomment-554885271 > I'm not sure whether or not it is a desirable feature to implement. Could you share the user case a bit? @lamber-ken It's hard to get the thread dump of tm when job is hanging, because flink tasks are deployed on yarn cluster. For example, when fixing the deadlock of elasticserch-connector, it needs to jump to the tm machine to dump the jstack. see [elasticsearch-connector-deadlock](https://github.com/elastic/elasticsearch/pull/48013). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager
lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager URL: https://github.com/apache/flink/pull/10228#issuecomment-554820006 hi, @TisonKun please cc, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager
lamber-ken commented on issue #10228: [FLINK-14816] Add thread dump feature for taskmanager URL: https://github.com/apache/flink/pull/10228#issuecomment-554759306 @flinkbot run travis This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
