I'd just like to address the feature creep issue a bit myself.
Maybe I'm being overly cautious but I worry about getting back to a place
where the code base is unnecessarily complex and loaded with features
without paying attention to stabilization, performance, resource control,
and correctness.
We are all cautious and vigilant about issues creeping in. That said,
if you notice we miss out anything, help us by pointing it out.
I know features are exciting, but we're back on to worry
about tail and a Hive sink and that worries me. I'd like to see us
collectively prioritize a stable, fast, correct core first. That's my
opinion.
If the community feels that a Hive sink is necessary and provides a
patch, so be it. I don't see anything wrong with it. That said, which
part of the system do you think the community should be focused on
right now? Make a case for it and open issues as you see fit.
When I first got involved in the project, and actually managed to
familiarise myself with the codebase and outstanding issues somewhat,
one of my greatest concerns was that it seemed like some of the core
initial objectives for FLUME-728 seem to have fallen by the wayside in
favor of feature creep. When I first found that the memory channel was
in no way thread safe I was quite surprised and somewhat concerned.
Further, with the JDBC channel and memory channel we have a hard choice
between a flimsy channel with the potential for high dataloss and a
heavyweight one with only moderate throughput... The FileChannel issue
has been more or less stationary. Where are we going with this? Do we
not consider it particularly important, or is it just stationary because
it is a hard problem? If the latter, hopefully we could kick off some
discussion on how to deal with it.
Finally, it seems like every single issue is reported as major, when it
really isn't the case for many of them. Many issues also do not have a
version number attached. It makes prioritizing anything to work on
awkward, perhaps we should be taking more liberties with recategorizing
the severity of issues? If others also feel this way I would like to
sort through the current open major tickets and recategorize some so we
can focus work on the core issues.
