[GitHub] nifi pull request: NIFI-631 Create ListFile and FetchFile processo...

[mpetronic]

Github user mpetronic commented on the pull request:

https://github.com/apache/nifi/pull/112#issuecomment-152697600
  
Joe, thanks for getting this processor going. I need it. :) I've pulled 
this in and am giving it a try. I have some additional thoughts on 
functionality.

1. Should it have a "Recurse sub-directories " option? Reason I 
mention this is because, in my setup, I have to scan files from an NFS share 
and it actually is not so fast, especially if you recurse many levels of 
subdirs that you don't really need to look at. That's special case, I know, but 
it is a valid use case and we could eliminate some latency by not requiring a 
full recursive scan all the time.
2. Should it have the option to specify a seeded last modified time? Say 
there is a directory full of files from days or weeks but you only want to 
start pulling them in from say, one day ago or some specific date/time, and not 
pick up all the previous files
3. If there are empty directories in the path you are scanning, they get 
listed in the "filename", just like an actual file would be listed. I think it 
would be nice to have another attribute that indicated whether the leaf node 
was a file or directory as that could more easily be use by downstream 
processors to decide how to act on that value.
4. Should it expose each files actual last modified timestamp in the 
FlowFile Attribute Map Content? 

I guess for all other types of filtering, like wildcards and such, the 
right 'Nifi' thing to do is use a downstream "UpdateAttribute" processor to 
massage the list. Correct? Maybe this also applies to #2 above, then?

Maybe the following should/work be part of the code review process but I 
will note here just in case. I'm new to this OSS process but, since I see this 
as a pull request, it made me think it was ready to go but seems some stuff is 
missing?

1. There is no description of the processor
2. The 'path' attribute description of "The path on the system from which 
to pull or push files" is misleading, IMO. Maybe "The path on the system where 
this processor will scan files and directories to build the file list."


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: spam to the dev mailing list

[Joe Witt]

rgr - thanks sean.  Do you know of any apache voodoo to help with spam
or is this kind of an open issue?

On Fri, Oct 30, 2015 at 10:08 AM, Sean Busbey  wrote:
> Yes, of course. I meant to address Joe's initial concern that by not
> keeping up with incoming messages spam might leak through.
>
> On Fri, Oct 30, 2015 at 9:04 AM, Tony Kurc  wrote:
>> Sean,
>> The challenging bit is not missing a legit message among the spam.
>>
>> On Fri, Oct 30, 2015 at 10:02 AM, Sean Busbey  wrote:
>>
>>> FWIW, the mailing list FAQ[1] says that any messages moderators don't
>>> respond to are treated as spam and silently dropped after 5 days. By
>>> actively rejecting you send a notice message back to the sender.
>>>
>>>
>>> [1]: http://apache.org/dev/committers.html#mail-moderate
>>>
>>> On Fri, Oct 30, 2015 at 9:00 AM, Joe Witt  wrote:
>>> > Tony Kurc is the other moderator at this time.
>>> >
>>> > On Fri, Oct 30, 2015 at 9:58 AM, Sean Busbey 
>>> wrote:
>>> >> Are the rest of the PMC already moderators?
>>> >>
>>> >> On Fri, Oct 30, 2015 at 8:19 AM, Joe Witt  wrote:
>>> >>> Team,
>>> >>>
>>> >>> As a moderator I'm getting tons of spam emails to dev@nifi.  I cannot
>>> >>> keep up with rejections and am not sure if that means they'll leak
>>> >>> through.  I'm not quite sure how to go about getting it address so if
>>> >>> some come through my apologies.
>>> >>>
>>> >>> About to make a NiFi spam bot!
>>> >>>
>>> >>> Joe
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Sean
>>>
>>>
>>>
>>> --
>>> Sean
>>>
>
>
>
> --
> Sean

spam to the dev mailing list

[Joe Witt]

Team,

As a moderator I'm getting tons of spam emails to dev@nifi.  I cannot
keep up with rejections and am not sure if that means they'll leak
through.  I'm not quite sure how to go about getting it address so if
some come through my apologies.

About to make a NiFi spam bot!

Joe

Re: spam to the dev mailing list

[Tony Kurc]

Sean,
The challenging bit is not missing a legit message among the spam.

On Fri, Oct 30, 2015 at 10:02 AM, Sean Busbey  wrote:

> FWIW, the mailing list FAQ[1] says that any messages moderators don't
> respond to are treated as spam and silently dropped after 5 days. By
> actively rejecting you send a notice message back to the sender.
>
>
> [1]: http://apache.org/dev/committers.html#mail-moderate
>
> On Fri, Oct 30, 2015 at 9:00 AM, Joe Witt  wrote:
> > Tony Kurc is the other moderator at this time.
> >
> > On Fri, Oct 30, 2015 at 9:58 AM, Sean Busbey 
> wrote:
> >> Are the rest of the PMC already moderators?
> >>
> >> On Fri, Oct 30, 2015 at 8:19 AM, Joe Witt  wrote:
> >>> Team,
> >>>
> >>> As a moderator I'm getting tons of spam emails to dev@nifi.  I cannot
> >>> keep up with rejections and am not sure if that means they'll leak
> >>> through.  I'm not quite sure how to go about getting it address so if
> >>> some come through my apologies.
> >>>
> >>> About to make a NiFi spam bot!
> >>>
> >>> Joe
> >>
> >>
> >>
> >> --
> >> Sean
>
>
>
> --
> Sean
>

Re: spam to the dev mailing list

[Sean Busbey]

FWIW, the mailing list FAQ[1] says that any messages moderators don't
respond to are treated as spam and silently dropped after 5 days. By
actively rejecting you send a notice message back to the sender.


[1]: http://apache.org/dev/committers.html#mail-moderate

On Fri, Oct 30, 2015 at 9:00 AM, Joe Witt  wrote:
> Tony Kurc is the other moderator at this time.
>
> On Fri, Oct 30, 2015 at 9:58 AM, Sean Busbey  wrote:
>> Are the rest of the PMC already moderators?
>>
>> On Fri, Oct 30, 2015 at 8:19 AM, Joe Witt  wrote:
>>> Team,
>>>
>>> As a moderator I'm getting tons of spam emails to dev@nifi.  I cannot
>>> keep up with rejections and am not sure if that means they'll leak
>>> through.  I'm not quite sure how to go about getting it address so if
>>> some come through my apologies.
>>>
>>> About to make a NiFi spam bot!
>>>
>>> Joe
>>
>>
>>
>> --
>> Sean



-- 
Sean

Re: spam to the dev mailing list

[Sean Busbey]

Yes, of course. I meant to address Joe's initial concern that by not
keeping up with incoming messages spam might leak through.

On Fri, Oct 30, 2015 at 9:04 AM, Tony Kurc  wrote:
> Sean,
> The challenging bit is not missing a legit message among the spam.
>
> On Fri, Oct 30, 2015 at 10:02 AM, Sean Busbey  wrote:
>
>> FWIW, the mailing list FAQ[1] says that any messages moderators don't
>> respond to are treated as spam and silently dropped after 5 days. By
>> actively rejecting you send a notice message back to the sender.
>>
>>
>> [1]: http://apache.org/dev/committers.html#mail-moderate
>>
>> On Fri, Oct 30, 2015 at 9:00 AM, Joe Witt  wrote:
>> > Tony Kurc is the other moderator at this time.
>> >
>> > On Fri, Oct 30, 2015 at 9:58 AM, Sean Busbey 
>> wrote:
>> >> Are the rest of the PMC already moderators?
>> >>
>> >> On Fri, Oct 30, 2015 at 8:19 AM, Joe Witt  wrote:
>> >>> Team,
>> >>>
>> >>> As a moderator I'm getting tons of spam emails to dev@nifi.  I cannot
>> >>> keep up with rejections and am not sure if that means they'll leak
>> >>> through.  I'm not quite sure how to go about getting it address so if
>> >>> some come through my apologies.
>> >>>
>> >>> About to make a NiFi spam bot!
>> >>>
>> >>> Joe
>> >>
>> >>
>> >>
>> >> --
>> >> Sean
>>
>>
>>
>> --
>> Sean
>>



-- 
Sean

Re: spam to the dev mailing list

[Tony Kurc]

Thanks Alex,
I think Joe was just saying "the workflow for accepting/rejecting makes
mistakes possible, I'm sorry if it breaks down". We both moderate via email
(not sure if there is another way) and this is a problem where adding
people may increase the incidence of errors, because it is a distributed
process.

On Fri, Oct 30, 2015 at 10:45 AM, Alex Moundalexis 
wrote:

> If another moderator is needed, I'm happy to volunteer. I may not write a
> lot of code but I know spam when I see it.
>
> On Fri, Oct 30, 2015 at 10:00 AM, Joe Witt  wrote:
>
> > Tony Kurc is the other moderator at this time.
> >
> > On Fri, Oct 30, 2015 at 9:58 AM, Sean Busbey 
> wrote:
> > > Are the rest of the PMC already moderators?
> > >
> > > On Fri, Oct 30, 2015 at 8:19 AM, Joe Witt  wrote:
> > >> Team,
> > >>
> > >> As a moderator I'm getting tons of spam emails to dev@nifi.  I cannot
> > >> keep up with rejections and am not sure if that means they'll leak
> > >> through.  I'm not quite sure how to go about getting it address so if
> > >> some come through my apologies.
> > >>
> > >> About to make a NiFi spam bot!
> > >>
> > >> Joe
> > >
> > >
> > >
> > > --
> > > Sean
> >
>

Pull Request Comments and JIRA

[Bryan Bende]

Does anyone know why in-line comments on pull-requests don't post back to
the JIRA?

Comments on the overall pull request do post back. I feel like this might
have been something that worked during incubation and no longer works, but
could be wrong.

Re: spam to the dev mailing list

[Alex Moundalexis]

If another moderator is needed, I'm happy to volunteer. I may not write a
lot of code but I know spam when I see it.

On Fri, Oct 30, 2015 at 10:00 AM, Joe Witt  wrote:

> Tony Kurc is the other moderator at this time.
>
> On Fri, Oct 30, 2015 at 9:58 AM, Sean Busbey  wrote:
> > Are the rest of the PMC already moderators?
> >
> > On Fri, Oct 30, 2015 at 8:19 AM, Joe Witt  wrote:
> >> Team,
> >>
> >> As a moderator I'm getting tons of spam emails to dev@nifi.  I cannot
> >> keep up with rejections and am not sure if that means they'll leak
> >> through.  I'm not quite sure how to go about getting it address so if
> >> some come through my apologies.
> >>
> >> About to make a NiFi spam bot!
> >>
> >> Joe
> >
> >
> >
> > --
> > Sean
>

"External" extensions

[Tony Kurc]

All,
I wanted to start a conversation about projects that are good for people
using or developing NiFi, but either can't or don't belong in the source
tree. This could be due to licensing issues (for example not compatible (or
not yet determined if it is compatible (GPL [1])) with the Apache License),
or other thought provoking mild concerns like we're discussing on NIFI-1074
[2].

I'd like to propose either capturing these on the website or on the wiki or
some other approach I didn't think of. I was hoping to find a good
archetype for this type of documentation in another apache project, but
didn't find anything I personally liked. If you have seen something you
like or don't, I'd be interested to hear.


[1] http://www.apache.org/licenses/GPL-compatibility.html
[2] https://issues.apache.org/jira/browse/NIFI-1074

Re: spam to the dev mailing list

[Joe Witt]

That is indeed how it works.  However those items go to moderators to
handle.  This allows folks who might not know about registration or care to
get the emails still be able to communicate.
On Oct 30, 2015 12:52 PM, "Rick Braddy"  wrote:

> How about auto-rejecting emails from all unregistered senders, since one
> must register before sending email this group anyway?
>
> Spammers won't take time to register.
>
> Rick
>
> > On Oct 30, 2015, at 8:19 AM, Joe Witt  wrote:
> >
> > Team,
> >
> > As a moderator I'm getting tons of spam emails to dev@nifi.  I cannot
> > keep up with rejections and am not sure if that means they'll leak
> > through.  I'm not quite sure how to go about getting it address so if
> > some come through my apologies.
> >
> > About to make a NiFi spam bot!
> >
> > Joe
>

Re: spam to the dev mailing list

[Rick Braddy]

How about auto-rejecting emails from all unregistered senders, since one must 
register before sending email this group anyway?

Spammers won't take time to register.

Rick

> On Oct 30, 2015, at 8:19 AM, Joe Witt  wrote:
> 
> Team,
> 
> As a moderator I'm getting tons of spam emails to dev@nifi.  I cannot
> keep up with rejections and am not sure if that means they'll leak
> through.  I'm not quite sure how to go about getting it address so if
> some come through my apologies.
> 
> About to make a NiFi spam bot!
> 
> Joe

[GitHub] nifi pull request: NiFi-1079

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/109


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---