Where is this 1TB dataset living today?
[anup] Resides in a filesystem
- What is the current nature of the dataset? Is it already in large
bundles as files or is it a series of tiny messages, etc..? Does it
need to be split/merged/etc..
[anup] Archived files of size 3MB each collected over
Github user joey commented on the pull request:
https://github.com/apache/incubator-nifi/pull/51#issuecomment-120785272
Alright, docs pushed. This latest version passes the contrib-check so let
me know if you think it needs anything else.
Otherwise, I'll do another rebase and
Github user joewitt commented on the pull request:
https://github.com/apache/incubator-nifi/pull/51#issuecomment-120770118
i'd encourage going with serial only mode for now. Let's learn more and
iterate from there.
---
If your project is set up for it, you can reply to this email an
Github user joey commented on the pull request:
https://github.com/apache/incubator-nifi/pull/51#issuecomment-120766917
If there are specific sources/sinks we want to expose as processors that
support threading, we could probably do that by adding unique processors for
them that wraps
Github user rdblue commented on the pull request:
https://github.com/apache/incubator-nifi/pull/51#issuecomment-120764934
Another way to fix the threading issue is to validate the sinks and sources
are safe and wrap them in first-class processors that allow threading even if
the gener
Github user rdblue commented on the pull request:
https://github.com/apache/incubator-nifi/pull/51#issuecomment-120764330
+1. Thanks Joey!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user joey commented on the pull request:
https://github.com/apache/incubator-nifi/pull/51#issuecomment-120759891
I rebased and pushed a new commit with most of the review feedback. I'm
still working on the extended docs and a template to make testing with a
running instance eas
Github user joewitt commented on the pull request:
https://github.com/apache/incubator-nifi/pull/51#issuecomment-120741524
Joey,
Love it and agree with all the points made.
Ryan, Sean, Mark,
Thanks for the great review effort/input on this.
Thanks
Github user joey commented on the pull request:
https://github.com/apache/incubator-nifi/pull/51#issuecomment-120740843
Sorry, hit comment by accident.
This matches the behavior of Flume so there's no loss in functionality for
the user. In fact a number of sinks/sources have
Github user joey commented on the pull request:
https://github.com/apache/incubator-nifi/pull/51#issuecomment-120740492
Sorry for the delay, I've been looking at the threading model and I'm
convinced that for the first version we should mark the processors as serial
only.
Th
+1
Signatures and Hashes match
build and tests run and pass contrib check
app starts behaves and stops as expected. logs are free of warning or error
messages
Dan Bress
Software Engineer
ONYX Consulting Services
From: Ricky Saltzer
Sent: Friday, July 1
11 matches
Mail list logo
