File filter option using genericFileFilter is not working properly
We have a running SFTP route using the &filter option to accept files using a
regular expression. This is working fine in Camel 2.25.0.
Migrating to Camel 3.5.0 it is not working at all. Every file in the directory
is downloaded, disregarding the filter option.
I can't find any documentation about what might have changed EXCEPT that the
&filter option has changed so that the bean reference should be without the "#"
character prefix.
Bean definition
@Bean
public static StatusFileFilter shuttleStatusFileFilter() {
return new StatusFileFilter<>();
}
Class definition
@Component
public class StatusFileFilter implements GenericFileFilter {
/**
* The Shuttle file.
*/
@Value("${shuttle.status-filename.regexp}")
private String shuttleFile;
@Override
public boolean accept(final GenericFile genericFile) {
final long fileLength = genericFile.getFileLength();
final String fileName = genericFile.getFileName();
final boolean matches = fileName.matches(shuttleFile);
return (fileLength > 0 && matches);
}
}
The route:
final String fromStatusFilesStr = "{{shuttle.ftp.incoming-status-files.url}}" +
"&username={{shuttle.ftp.username}}" +
"&password={{shuttle.ftp.password}}" +
"&allowNullBody=false" +
"&filter=shuttleStatusFileFilter" +
"&bridgeErrorHandler=true" +
"&throwExceptionOnConnectFailed=true";
from(fromStatusFilesStr)
.to("log:StatusFiles?showHeaders=true&showProperties=true&multiline=true")
.choice()
.when(body().isNotNull())
.split(bodyAs(String.class).tokenize("\n")) //<.>
.parallelProcessing(true)
.streaming()
.to("{{jms.queue.statefiles}}")
.to("log:StateFiles?level=INFO&groupInterval=1&groupActiveOnly=true")
.choice()
.when(simple("${header.CamelSplitComplete} == true"))
.log("Number of records split: ${header.CamelSplitSize}")
.log("Importing complete: ${header.CamelFileName}")
.endChoice()
.end();
I have tried to log whatever is going on in the bean but no log statements are
present so that's why I think it is not working properly, the bean is not
activated.
The documentation still points out the hash-char as prefix so that must still
be wrong:
https://camel.apache.org/components/latest/file-component.html#_filter_using_org_apache_camel_component_file_genericfilefilter
Pls advice what could be wrong and/or point me to some documentation
Thx
/M
Re: File filter option using genericFileFilter is not working properly
Hi
No you should use the # syntax. See the FromFtpFilterTest.
On Wed, Oct 21, 2020 at 10:00 AM Mikael Andersson Wigander
wrote:
>
> We have a running SFTP route using the &filter option to accept files using a
> regular expression. This is working fine in Camel 2.25.0.
>
> Migrating to Camel 3.5.0 it is not working at all. Every file in the
> directory is downloaded, disregarding the filter option.
>
> I can't find any documentation about what might have changed EXCEPT that the
> &filter option has changed so that the bean reference should be without the
> "#" character prefix.
>
> Bean definition
>
>
> @Bean
> public static StatusFileFilter shuttleStatusFileFilter() {
> return new StatusFileFilter<>();
> }
>
> Class definition
>
>
> @Component
> public class StatusFileFilter implements GenericFileFilter {
> /**
> * The Shuttle file.
> */
> @Value("${shuttle.status-filename.regexp}")
> private String shuttleFile;
>
> @Override
> public boolean accept(final GenericFile genericFile) {
> final long fileLength = genericFile.getFileLength();
> final String fileName = genericFile.getFileName();
> final boolean matches = fileName.matches(shuttleFile);
> return (fileLength > 0 && matches);
> }
> }
>
> The route:
>
>
> final String fromStatusFilesStr = "{{shuttle.ftp.incoming-status-files.url}}"
> +
> "&username={{shuttle.ftp.username}}" +
> "&password={{shuttle.ftp.password}}" +
> "&allowNullBody=false" +
> "&filter=shuttleStatusFileFilter" +
> "&bridgeErrorHandler=true" +
> "&throwExceptionOnConnectFailed=true";
>
> from(fromStatusFilesStr)
> .to("log:StatusFiles?showHeaders=true&showProperties=true&multiline=true")
> .choice()
> .when(body().isNotNull())
> .split(bodyAs(String.class).tokenize("\n")) //<.>
> .parallelProcessing(true)
> .streaming()
> .to("{{jms.queue.statefiles}}")
> .to("log:StateFiles?level=INFO&groupInterval=1&groupActiveOnly=true")
> .choice()
> .when(simple("${header.CamelSplitComplete} == true"))
> .log("Number of records split: ${header.CamelSplitSize}")
> .log("Importing complete: ${header.CamelFileName}")
> .endChoice()
> .end();
>
> I have tried to log whatever is going on in the bean but no log statements
> are present so that's why I think it is not working properly, the bean is not
> activated.
>
> The documentation still points out the hash-char as prefix so that must still
> be wrong:
> https://camel.apache.org/components/latest/file-component.html#_filter_using_org_apache_camel_component_file_genericfilefilter
>
> Pls advice what could be wrong and/or point me to some documentation
>
> Thx
>
> /M
--
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Camel documentation question
Hi Yeah you can use those options if you prefix with destination. As in the CamelDestinationExclusiveConsumerTest unit test. You are welcome to create a JIRA and if you want to add a section in the docs about this. On Tue, Oct 20, 2020 at 3:39 PM Иванов Григорий - wrote: > > > Hello! > > As I could remember there was a section on the old camel website about using > activeMQ destination options > > Is there a reason for not to have it now? > > Camel AMQ component - > https://camel.apache.org/components/latest/activemq-component.html > > AMQ destination options - https://activemq.apache.org/destination-options > > Thanks! -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
Re: File filter option using genericFileFilter is not working properly
Thanks
Using the # sign gives me this error
Caused by: org.apache.camel.NoSuchBeanException: No bean could be found in the
registry for: shuttleReconFileFilter of type:
org.apache.camel.component.file.GenericFileFilter
What am I missing, really?
/M
‐‐‐ Original Message ‐‐‐
On Wednesday 21 October 2020 kl. 10:14, Claus Ibsen
wrote:
> Hi
>
> No you should use the # syntax. See the FromFtpFilterTest.
>
> On Wed, Oct 21, 2020 at 10:00 AM Mikael Andersson Wigander
>
> mikael.andersson.wigan...@pm.me.invalid wrote:
>
> > We have a running SFTP route using the &filter option to accept files using
> > a regular expression. This is working fine in Camel 2.25.0.
> >
> > Migrating to Camel 3.5.0 it is not working at all. Every file in the
> > directory is downloaded, disregarding the filter option.
> >
> > I can't find any documentation about what might have changed EXCEPT that
> > the &filter option has changed so that the bean reference should be without
> > the "#" character prefix.
> >
> > Bean definition
> > ===
> >
> > @Bean
> >
> > public static StatusFileFilter shuttleStatusFileFilter() {
> >
> > return new StatusFileFilter<>();
> >
> > }
> >
> > Class definition
> >
> >
> > @Component
> >
> > public class StatusFileFilter implements GenericFileFilter {
> >
> > /**
> >
> > - The Shuttle file.
> >
> > */
> >
> > @Value("${shuttle.status-filename.regexp}")
> >
> > private String shuttleFile;
> >
> > @Override
> >
> > public boolean accept(final GenericFile genericFile) {
> >
> > final long fileLength = genericFile.getFileLength();
> >
> > final String fileName = genericFile.getFileName();
> >
> > final boolean matches = fileName.matches(shuttleFile);
> >
> > return (fileLength > 0 && matches);
> >
> > }
> >
> > }
> >
> > The route:
> > ==
> >
> > final String fromStatusFilesStr =
> > "{{shuttle.ftp.incoming-status-files.url}}" +
> >
> > "&username={{shuttle.ftp.username}}" +
> >
> > "&password={{shuttle.ftp.password}}" +
> >
> > "&allowNullBody=false" +
> >
> > "&filter=shuttleStatusFileFilter" +
> >
> > "&bridgeErrorHandler=true" +
> >
> > "&throwExceptionOnConnectFailed=true";
> >
> > from(fromStatusFilesStr)
> >
> > .to("log:StatusFiles?showHeaders=true&showProperties=true&multiline=true")
> >
> > .choice()
> >
> > .when(body().isNotNull())
> >
> > .split(bodyAs(String.class).tokenize("\n")) //<.>
> >
> > .parallelProcessing(true)
> >
> > .streaming()
> >
> > .to("{{jms.queue.statefiles}}")
> >
> > .to("log:StateFiles?level=INFO&groupInterval=1&groupActiveOnly=true")
> >
> > .choice()
> >
> > .when(simple("${header.CamelSplitComplete} == true"))
> >
> > .log("Number of records split: ${header.CamelSplitSize}")
> >
> > .log("Importing complete: ${header.CamelFileName}")
> >
> > .endChoice()
> >
> > .end();
> >
> > I have tried to log whatever is going on in the bean but no log statements
> > are present so that's why I think it is not working properly, the bean is
> > not activated.
> >
> > The documentation still points out the hash-char as prefix so that must
> > still be wrong:
> >
> > https://camel.apache.org/components/latest/file-component.html#_filter_using_org_apache_camel_component_file_genericfilefilter
> >
> > Pls advice what could be wrong and/or point me to some documentation
> >
> > Thx
> >
> > /M
>
> --
>
> Claus Ibsen
> ---
>
> http://davsclaus.com @davsclaus
>
> Camel in Action 2: https://www.manning.com/ibsen2
Re: File filter option using genericFileFilter is not working properly
Some spring issue that it does not register your bean so Camel cant find it.
On Wed, Oct 21, 2020 at 10:31 AM Mikael Andersson Wigander
wrote:
>
> Thanks
>
> Using the # sign gives me this error
>
> Caused by: org.apache.camel.NoSuchBeanException: No bean could be found in
> the registry for: shuttleReconFileFilter of type:
> org.apache.camel.component.file.GenericFileFilter
>
> What am I missing, really?
>
>
> /M
>
> ‐‐‐ Original Message ‐‐‐
>
> On Wednesday 21 October 2020 kl. 10:14, Claus Ibsen
> wrote:
>
> > Hi
> >
> > No you should use the # syntax. See the FromFtpFilterTest.
> >
> > On Wed, Oct 21, 2020 at 10:00 AM Mikael Andersson Wigander
> >
> > mikael.andersson.wigan...@pm.me.invalid wrote:
> >
> > > We have a running SFTP route using the &filter option to accept files
> > > using a regular expression. This is working fine in Camel 2.25.0.
> > >
> > > Migrating to Camel 3.5.0 it is not working at all. Every file in the
> > > directory is downloaded, disregarding the filter option.
> > >
> > > I can't find any documentation about what might have changed EXCEPT that
> > > the &filter option has changed so that the bean reference should be
> > > without the "#" character prefix.
> > >
> > > Bean definition
> > > ===
> > >
> > > @Bean
> > >
> > > public static StatusFileFilter shuttleStatusFileFilter() {
> > >
> > > return new StatusFileFilter<>();
> > >
> > > }
> > >
> > > Class definition
> > >
> > >
> > > @Component
> > >
> > > public class StatusFileFilter implements GenericFileFilter {
> > >
> > > /**
> > >
> > > - The Shuttle file.
> > >
> > > */
> > >
> > > @Value("${shuttle.status-filename.regexp}")
> > >
> > > private String shuttleFile;
> > >
> > > @Override
> > >
> > > public boolean accept(final GenericFile genericFile) {
> > >
> > > final long fileLength = genericFile.getFileLength();
> > >
> > > final String fileName = genericFile.getFileName();
> > >
> > > final boolean matches = fileName.matches(shuttleFile);
> > >
> > > return (fileLength > 0 && matches);
> > >
> > > }
> > >
> > > }
> > >
> > > The route:
> > > ==
> > >
> > > final String fromStatusFilesStr =
> > > "{{shuttle.ftp.incoming-status-files.url}}" +
> > >
> > > "&username={{shuttle.ftp.username}}" +
> > >
> > > "&password={{shuttle.ftp.password}}" +
> > >
> > > "&allowNullBody=false" +
> > >
> > > "&filter=shuttleStatusFileFilter" +
> > >
> > > "&bridgeErrorHandler=true" +
> > >
> > > "&throwExceptionOnConnectFailed=true";
> > >
> > > from(fromStatusFilesStr)
> > >
> > > .to("log:StatusFiles?showHeaders=true&showProperties=true&multiline=true")
> > >
> > > .choice()
> > >
> > > .when(body().isNotNull())
> > >
> > > .split(bodyAs(String.class).tokenize("\n")) //<.>
> > >
> > > .parallelProcessing(true)
> > >
> > > .streaming()
> > >
> > > .to("{{jms.queue.statefiles}}")
> > >
> > > .to("log:StateFiles?level=INFO&groupInterval=1&groupActiveOnly=true")
> > >
> > > .choice()
> > >
> > > .when(simple("${header.CamelSplitComplete} == true"))
> > >
> > > .log("Number of records split: ${header.CamelSplitSize}")
> > >
> > > .log("Importing complete: ${header.CamelFileName}")
> > >
> > > .endChoice()
> > >
> > > .end();
> > >
> > > I have tried to log whatever is going on in the bean but no log
> > > statements are present so that's why I think it is not working properly,
> > > the bean is not activated.
> > >
> > > The documentation still points out the hash-char as prefix so that must
> > > still be wrong:
> > >
> > > https://camel.apache.org/components/latest/file-component.html#_filter_using_org_apache_camel_component_file_genericfilefilter
> > >
> > > Pls advice what could be wrong and/or point me to some documentation
> > >
> > > Thx
> > >
> > > /M
> >
> > --
> >
> > Claus Ibsen
> > ---
> >
> > http://davsclaus.com @davsclaus
> >
> > Camel in Action 2: https://www.manning.com/ibsen2
--
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
