[GitHub] storm issue #1044: STORM-1464: Support multiple file outputs

2017-11-07 Thread dossett
Github user dossett commented on the issue: https://github.com/apache/storm/pull/1044 @fescandell I don't believe there is. Because I was unfamiliar with the Trident API, I didn't make any similar changes to it. ---

[GitHub] storm issue #1044: STORM-1464: Support multiple file outputs

2017-11-06 Thread fescandell
Github user fescandell commented on the issue: https://github.com/apache/storm/pull/1044 Hello, Is there a similar solution like "withPartitioner(new Partitioner()" by using Trident : `HdfsState.HdfsFileOptions()` ---

[GitHub] storm issue #1044: STORM-1464: Support multiple file outputs

2017-05-10 Thread dossett
Github user dossett commented on the issue: https://github.com/apache/storm/pull/1044 @ryanpersaud Your basic approach looks right to me, thanks for taking the lead on this. Regarding rotation actions, my concern is that there's no absolute guarantee they will get executed. I

[GitHub] storm issue #1044: STORM-1464: Support multiple file outputs

2017-05-04 Thread ryanpersaud
Github user ryanpersaud commented on the issue: https://github.com/apache/storm/pull/1044 Now you've piqued my curiosity - what is it that you don't like about rotation actions? Writers are only stored in the writers map, so once they are removed, there is no way for FSDataOu

[GitHub] storm issue #1044: STORM-1464: Support multiple file outputs

2017-05-03 Thread dossett
Github user dossett commented on the issue: https://github.com/apache/storm/pull/1044 Hi @ryanpersaud -- that is a very good observation, it would seem that doesn't happen. I admit to being biased against rotation actions and would never use them in production, which is probably why

[GitHub] storm issue #1044: STORM-1464: Support multiple file outputs

2017-05-01 Thread ryanpersaud
Github user ryanpersaud commented on the issue: https://github.com/apache/storm/pull/1044 When the eldest entry in the WritersMap gets removed due to the number of writers exceeding maxWriters, how does the writer get closed and any rotation action get executed? Thanks for the inform

[GitHub] storm issue #1044: STORM-1464: Support multiple file outputs

2017-02-09 Thread matthewdfleming
Github user matthewdfleming commented on the issue: https://github.com/apache/storm/pull/1044 @ptgoetz it's an interesting point.. "MINOR version when you add functionality in a backwards-compatible manner" (http://semver.org). I wonder if adding a method to a public interface could e

[GitHub] storm issue #1044: STORM-1464: Support multiple file outputs

2017-02-09 Thread dossett
Github user dossett commented on the issue: https://github.com/apache/storm/pull/1044 @ptgoetz This PR was never, to my knowledge, applied to 1.x. It contained a very small incompatible API change to FileRotation, so users with their own FileRotation implementation would get broken.

[GitHub] storm issue #1044: STORM-1464: Support multiple file outputs

2017-02-08 Thread ptgoetz
Github user ptgoetz commented on the issue: https://github.com/apache/storm/pull/1044 @matthewdfleming This will be available in the 1.1 release, which we are getting closer to. You should be able to use it with 1.0.x, but for now you would have to compile it yourself. --- If your p

[GitHub] storm issue #1044: STORM-1464: Support multiple file outputs

2017-02-08 Thread matthewdfleming
Github user matthewdfleming commented on the issue: https://github.com/apache/storm/pull/1044 Is there any plan to port this to 1.0.x? I saw that there was an api change which I guess makes it more of a 1.1.x kind of change? I guess I'm asking for the best way to add this feature to a