Dear Igniters, why do I have a hunch that DataStreamer would readily swallow exceptions?
DataStreamerImpl:1756 swallows marshalling error, lines 1774 & 1781 eat deployment errors. Some people are worried they can fill a leaking vessel without noticing anything off. Also in line 2156 fsync() on WAL can throw exceptions, which will be swallowed, and IMP this fsync doesn't belong to DataStreamer code. This question is not purely theoretical, we have also replaced one of these eaters with throw with IGNITE-7519. There was a fault in PDS implementation, which did not propagate to client. A serious issue IMHO. As a data streamer user I will expect that flush()/close() will throw any pending exceptions and will only be silent if all data landed safely in the cluster. I also have this feeling that DataStreamer was written using very internal APIs so that it can compromise guarantees that cache and SQL APIs are bound by. I think I've heard something about not recovering properly in case of node failures. I think it's time to end this, if that was the case. DataStreamer should not be a special case and it should guarantee data safety. WDYT? -- Ilya Kasnacheev
