Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-197294536
@mans2singh Looks great, thanks for getting those changes together and the
contribution. I'll be merging this into master shortly.
---
If your project is set up for it, yo
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/213
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enable
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-197183816
@aldrin @mattyb149 Thanks for your pointers and samples for using data
validator. I've made the changes and updated test cases. Please let me know
if there is anythin
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-197151921
@mattyb149 has it.
@mans2singh the updates look great. Concerning the validator, how does
6bed3bcaad61c516eacf268400cdd03c0cd58dae look to you?
Otherwise,
Github user mattyb149 commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-197125258
@mans2singh I think Aldrin was referring to DATA_SIZE_VALIDATOR in
org.apache.nifi.processor.util.StandardValidators. If you need custom min/max
size (in bytes), you can
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-197122905
@aldrin - I've moved the check for buffer size and rebased as you had
recommended. I could not find the data size validator that you have mentioned
though. Please let
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-197000303
Looking good going to AWS endpoints and services. Good stuff.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-196956296
@mans2singh Overall, the changes look good and appreciate the inclusion of
some additional tests. Going to do some functional testing but think we can
get this merged in.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi/pull/213#discussion_r56212999
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/firehose/PutKinesisFirehose.java
---
@@ -0,0
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi/pull/213#discussion_r56212702
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/firehose/PutKinesisFirehose.java
---
@@ -0,0
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi/pull/213#discussion_r56212500
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/firehose/PutKinesisFirehose.java
---
@@ -0,0
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi/pull/213#discussion_r56212428
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/firehose/AbstractKinesisFirehoseProcessor.java
--
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi/pull/213#discussion_r56201575
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/firehose/PutKinesisFirehose.java
---
@@ -0,0
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-196646278
@aldrin - I've added max buffer size limit check while getting flow files
from the session as you had recommended. I've also kept the batch size limit
in place. I've a
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-196476035
Hey @mans2singh,
Just wanted to see how the progress was going. We are coming up on release
time for 0.6.0 and need to start wrapping things up. If this is somethi
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-195410447
Nope, just send it off to Kinesis and then transfer the files to the
appropriate relationship afterward.
---
If your project is set up for it, you can reply to this email a
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-195409880
@apiri - To make things simple we get rid of the batch size and we just
get one flow file at a time on each invocation of the onTrigger. We store the
flow files in me
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-195367460
@mans2singh I think your logic for collecting the flowfiles should probably
just continuously grab individual files at a time. This continues until one of
a couple scenario
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-195300721
@apiri - Just wanted to clarify - We will grab a batch of flow files and
check that the flow files combined size threshold is reached. Then we send the
batch to the su
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-195173926
@mans2singh Yep. I think we can probably replace the number of items in a
batch size (or make this secondary to the buffer size). The idea would be to
continuously grab Fl
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-195172976
@apiri Your recommendation for using buffer size is great. So, should we
add another property - total batch bytes, and then send them in batches of that
size ? If you h
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-194975737
@mans2singh I had a chance to sit down and revisit this. Overall, it looks
good and I was able to test a flow successfully putting to a Kinesis Firehose
which aggregated an
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/213#issuecomment-188062568
Corrected title of pull request. Please let me know if there is anything
else required.
---
If your project is set up for it, you can reply to this email and have you
23 matches
Mail list logo
