Hi Bryan,
Following is the code snapshot:
try (FileChannel fileChannel = FileChannel.open(file.toPath(),
StandardOpenOption.READ)) {
fileChannel.position(lastKnownLocation);
FlowFile flowFile = session.create();
flowFile = writeToFlowFile(session, flowFile, fileChannel,
positionHolder);
tranferFlowFile(session, flowFile);
} catch (IOException e) {
logger.error("Error while creating/closing file reader", e);
}
session.commit();
context.yield();
It is working even if I set yield to 1 sec(default setting) . However, If I
don't run yield function , it gives the exception.
Regards,
Sourav Gulati
-----Original Message-----
From: Bryan Bende [mailto:bbe...@gmail.com]
Sent: Wednesday, April 27, 2016 7:58 PM
To: dev@nifi.apache.org
Subject: Re: Reg: Issues in Custom Processor
Hi Sourav,
Are you able to share any of the code from your onTrigger method so we can see
how you are opening/closing the file reader?
-Bryan
On Wed, Apr 27, 2016 at 7:43 AM, Sourav Gulati <sourav.gul...@impetus.co.in>
wrote:
> I have introduced yield of 5 secs and after that it started working as
> expected.
>
> Regards,
> Sourav Gulati
>
> -----Original Message-----
> From: Sourav Gulati [mailto:sourav.gul...@impetus.co.in]
> Sent: Wednesday, April 27, 2016 11:34 AM
> To: dev@nifi.apache.org
> Subject: RE: Reg: Issues in Custom Processor
>
> Joe,
>
> Now I have increased number of open files limit.
> ulimit -n
> 50000
>
> How still I am seeing same exception
>
> Regards,
> Sourav Gulati
>
> -----Original Message-----
> From: Sourav Gulati [mailto:sourav.gul...@impetus.co.in]
> Sent: Tuesday, April 26, 2016 11:48 PM
> To: dev@nifi.apache.org
> Subject: Re: Reg: Issues in Custom Processor
>
> Thanks Joe. I will try and update you
>
> Regards,
> Sourav Gulati
> Big Data
>
>
> ________________________________________
> From: Joe Witt <joe.w...@gmail.com>
> Sent: Tuesday, April 26, 2016 11:26 PM
> To: dev@nifi.apache.org
> Subject: Re: Reg: Issues in Custom Processor
>
> Sourav,
>
> On twitter you asked "why is it opening so many file handles"
>
> Your processor is not. It may well be just fine. However, at startup
> nifi loads many files into the classloader and it is at the point now
> where it references more than 1024 objects (files, sockets, etc..) so
> it is important to set a higher default.
>
> I encourage you to go ahead and follow the best practices mentioned in
> the documentation and I suspect you'll find your custom processor is
> doing what you expected.
>
> Thanks
> Joe
>
> On Tue, Apr 26, 2016 at 1:47 PM, Joe Witt <joe.w...@gmail.com> wrote:
> > Sourav,
> >
> > Please make sure you've followed these recommendations
> > https://nifi.apache.org/docs/nifi-docs/html/administration-guide.htm
> > l#
> > configuration-best-practices
> >
> > Specifically make sure you update the max open file handles.
> >
> > Thanks
> > Joe
> >
> > On Tue, Apr 26, 2016 at 9:34 AM, Sourav Gulati
> > <sourav.gul...@impetus.co.in> wrote:
> >> Hi Team,
> >>
> >> I am using "@TriggerSerially" annotation while creating processor
> >> and
> inside "On Trigger" method I am creating a file reader and after
> doing the processing I am closing the reader connection as well. As
> per my assumption, if I am using @TriggerSerially, then "On trigger"
> method runs in serial fashion.
> >>
> >> However, while executing processor I am getting "too many open files"
> error . I check with lsof command as well, it has opened lot of reader
> connections to file.
> >>
> >> What should I change ? Need urgent help
> >>
> >>
> >>
> >> Regards,
> >> Sourav Gulati
> >>
> >>
> >> ________________________________
> >>
> >>
> >>
> >>
> >>
> >>
> >> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited
> when received in error. Impetus does not represent, warrant and/or
> guarantee, that the integrity of this communication has been
> maintained nor that the communication is free of errors, virus, interception
> or interference.
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited
> when received in error. Impetus does not represent, warrant and/or
> guarantee, that the integrity of this communication has been
> maintained nor that the communication is free of errors, virus, interception
> or interference.
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited
> when received in error. Impetus does not represent, warrant and/or
> guarantee, that the integrity of this communication has been
> maintained nor that the communication is free of errors, virus, interception
> or interference.
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited
> when received in error. Impetus does not represent, warrant and/or
> guarantee, that the integrity of this communication has been
> maintained nor that the communication is free of errors, virus, interception
> or interference.
>
________________________________
NOTE: This message may contain information that is confidential, proprietary,
privileged or otherwise protected by law. The message is intended solely for
the named addressee. If received in error, please destroy and notify the
sender. Any use of this email is prohibited when received in error. Impetus
does not represent, warrant and/or guarantee, that the integrity of this
communication has been maintained nor that the communication is free of errors,
virus, interception or interference.
