[SURVEY] What is the most subtle/hard to catch bug that people have seen?

2019-09-30 Thread Konstantinos Kallas
Hi everyone. I wanted to ask Flink users what are the most subtle Flink bugs that people have witnessed. The cause of the bugs could be anything (e.g. wrong assumptions on data, parallelism of non-parallel operator, simple mistakes). We are developing a testing framework for Flink and it would

Re: [SURVEY] What is the most subtle/hard to catch bug that people have seen?

2019-10-01 Thread Piotr Nowojski
Hi, Are you asking about bugs in Flink, in libraries that Flink is using or bugs in applications that were using Flink? From my perspective/what I have seen: The most problematic bugs while developing features for Flink: Dead locks & data losses caused by concurrency issues in network s

Re: [SURVEY] What is the most subtle/hard to catch bug that people have seen?

2019-10-01 Thread Jan Lukavský
Hi, I'd add another one regarding Java hashCode() and its practical usability for distributed systems [1], although practically all (Java based) data processing systems rely on it. One bug directly related to this I once saw was, that using an Enum inside other object used as partitioning ke

Re: [SURVEY] What is the most subtle/hard to catch bug that people have seen?

2019-10-01 Thread Konstantinos Kallas
Hi Jan, Thanks a lot for that pointer, that is very interesting. Best, Konstantinos On 1/10/19 6:02 π.μ., Jan Lukavský wrote: Hi, I'd add another one regarding Java hashCode() and its practical usability for distributed systems [1], although practically all (Java based) data processing syste

Re: [SURVEY] What is the most subtle/hard to catch bug that people have seen?

2019-10-01 Thread Konstantinos Kallas
Hi Piotrek, Thank you very much for your feedback. I am mostly interested in bugs in Flink applications. Especially ones that are hard to notice -- either because they don't always occur, or because they don't crash the program, but instead they subtly affect its output, or lead to deadlocks.