Re: Camel lock gets acquired on files but not picked sometime

[Hemant Pekhale]

Thanks for the replies guys.

Actually processing of files is realtime and there is one more process
other than vb script which creates files in source folder so not feasible
with done filename.
Need to try with file consumer options only.



On Mon, Aug 6, 2018 at 9:10 PM, Trilok Agarwal 
wrote:

> You can make the processes mutually exclusive through setting up a
> scheduler by giving a cron expression so that your FTP or File component
> would not read files when they are being written, just in case
> if processing of your file not real time and copying of file via VB script
> is also scheduled.
>
> Otherwise readLock=changed would help, try it out-
> From camel documentation-
>
> *changed* is using file length/modification timestamp to detect whether the
> file is currently being copied or not. Will at least use 1 sec. to
> determine this, so this option cannot consume files as fast as the others,
> but can be more reliable as the JDK IO API cannot always determine whether
> a file is currently being used by another process. The option
> *readLockCheckInterval* can be used to set the check frequency. This option
> is *only* avail for the FTP  component
> from *Camel 2.8* onward. Note: from *Camel 2.10.1* onward the FTP
>  option *fastExistsCheck* can be
> enabled
> to speedup this *readLock* strategy, if the FTP server support the LIST
> operation with a full file name (some servers may not)
>
> On Mon, Aug 6, 2018 at 7:26 PM, Claus Ibsen  wrote:
>
> > Also you can use "done" files, eg in the VB script you write a 2nd
> > empty file with a special name, that tells Camel that the other file
> > is done being copied and ready.
> > There is some options on Camel file endpoint you can configure for that
> > also.
> >
> > On Mon, Aug 6, 2018 at 3:54 PM, Claus Ibsen 
> wrote:
> > > Hi
> > >
> > > You can configure Camel to only pickup files with a certain file
> > > pattern, or to exclude names with temporary names, etc.
> > > There is a number of different filter options, see its doc.
> > >
> > > On Mon, Aug 6, 2018 at 3:52 PM, Hemant Pekhale <
> hemant.pekh...@gmail.com>
> > wrote:
> > >> Hi Claus,
> > >>
> > >> Thanks for the reply.
> > >>
> > >> Yes, I am able to modify the script. But if we do copy the file with
> > >> temporary name and then rename, I doubt that there may be lock
> acquired
> > on
> > >> temporary file name already. because my route has no waiting
> parameter.
> > So
> > >> I think it might not be helpful for our case. Please correct me if I
> am
> > >> wrong.
> > >>
> > >> Regarding the file lock strategies, I think  readLock think
> > >> readLockCheckInterval think can be tried. Do you have any other
> > suggestion?
> > >>
> > >> FYI : Whatever I can try I need to try on Production server so just
> > wanted
> > >> to tackle this with minimum and required trials.
> > >>
> > >> Thanks!
> > >>
> > >>
> > >> On Mon, Aug 6, 2018 at 7:10 PM, Claus Ibsen 
> > wrote:
> > >>
> > >>> Hi
> > >>>
> > >>> Are you able to modify the VB script so it copies the file using a
> > >>> temporary name and then rename the file when the copy is done.
> > >>>
> > >>> It sounds a bit like the problem is that the VB script is currently
> > >>> still copying the file, while Camel gets a lock on the file, and
> > >>> attempt to process it.
> > >>> File locks is not as easy and it depends a bit what file system, OS,
> > >>> and how big the files are and other things.
> > >>>
> > >>> A good strategy is to copy the files using temporary names or in
> > >>> another folder, and then do a file rename when the copy is over, that
> > >>> latter then is an atomic file operation which should not cause file
> > >>> locking issues.
> > >>>
> > >>> You can also try to use some of the other Camel file lock strategies
> > >>> instead of marker files.
> > >>> You can see more in the file documentation about these.
> > >>>
> > >>>
> > >>> On Mon, Aug 6, 2018 at 3:19 PM, Hemant Pekhale <
> > hemant.pekh...@gmail.com>
> > >>> wrote:
> > >>> > Hi All,
> > >>> >
> > >>> > I have a route which picks files from a folder and do some
> > processing on
> > >>> it
> > >>> > and store it in other folder. I have used parameter "delete=true"
> for
> > >>> > deleting file from source folder.
> > >>> >
> > >>> > Sometime it happens that files are stuck in the source folder and
> > camel
> > >>> > lock files are also created for all the files but files are neither
> > >>> process
> > >>> > nor gets deleted.
> > >>> >
> > >>> > There is a VB script which copied files in the source folder. When
> I
> > >>> > manually copied the files in the source folder then it never gives
> > issue
> > >>> > but when the files are copied from VB script then only issue
> occurs.
> > Also
> > >>> > the issue does not occur for the first file but it occurs randomly
> > after
> > >>> > processing number of files.
> > >>> >
> > >>> > This happens on server. I tried to reproduce this issue 

Re: Camel lock gets acquired on files but not picked sometime

[Trilok Agarwal]

You can make the processes mutually exclusive through setting up a
scheduler by giving a cron expression so that your FTP or File component
would not read files when they are being written, just in case
if processing of your file not real time and copying of file via VB script
is also scheduled.

Otherwise readLock=changed would help, try it out-
>From camel documentation-

*changed* is using file length/modification timestamp to detect whether the
file is currently being copied or not. Will at least use 1 sec. to
determine this, so this option cannot consume files as fast as the others,
but can be more reliable as the JDK IO API cannot always determine whether
a file is currently being used by another process. The option
*readLockCheckInterval* can be used to set the check frequency. This option
is *only* avail for the FTP  component
from *Camel 2.8* onward. Note: from *Camel 2.10.1* onward the FTP
 option *fastExistsCheck* can be enabled
to speedup this *readLock* strategy, if the FTP server support the LIST
operation with a full file name (some servers may not)

On Mon, Aug 6, 2018 at 7:26 PM, Claus Ibsen  wrote:

> Also you can use "done" files, eg in the VB script you write a 2nd
> empty file with a special name, that tells Camel that the other file
> is done being copied and ready.
> There is some options on Camel file endpoint you can configure for that
> also.
>
> On Mon, Aug 6, 2018 at 3:54 PM, Claus Ibsen  wrote:
> > Hi
> >
> > You can configure Camel to only pickup files with a certain file
> > pattern, or to exclude names with temporary names, etc.
> > There is a number of different filter options, see its doc.
> >
> > On Mon, Aug 6, 2018 at 3:52 PM, Hemant Pekhale 
> wrote:
> >> Hi Claus,
> >>
> >> Thanks for the reply.
> >>
> >> Yes, I am able to modify the script. But if we do copy the file with
> >> temporary name and then rename, I doubt that there may be lock acquired
> on
> >> temporary file name already. because my route has no waiting parameter.
> So
> >> I think it might not be helpful for our case. Please correct me if I am
> >> wrong.
> >>
> >> Regarding the file lock strategies, I think  readLock think
> >> readLockCheckInterval think can be tried. Do you have any other
> suggestion?
> >>
> >> FYI : Whatever I can try I need to try on Production server so just
> wanted
> >> to tackle this with minimum and required trials.
> >>
> >> Thanks!
> >>
> >>
> >> On Mon, Aug 6, 2018 at 7:10 PM, Claus Ibsen 
> wrote:
> >>
> >>> Hi
> >>>
> >>> Are you able to modify the VB script so it copies the file using a
> >>> temporary name and then rename the file when the copy is done.
> >>>
> >>> It sounds a bit like the problem is that the VB script is currently
> >>> still copying the file, while Camel gets a lock on the file, and
> >>> attempt to process it.
> >>> File locks is not as easy and it depends a bit what file system, OS,
> >>> and how big the files are and other things.
> >>>
> >>> A good strategy is to copy the files using temporary names or in
> >>> another folder, and then do a file rename when the copy is over, that
> >>> latter then is an atomic file operation which should not cause file
> >>> locking issues.
> >>>
> >>> You can also try to use some of the other Camel file lock strategies
> >>> instead of marker files.
> >>> You can see more in the file documentation about these.
> >>>
> >>>
> >>> On Mon, Aug 6, 2018 at 3:19 PM, Hemant Pekhale <
> hemant.pekh...@gmail.com>
> >>> wrote:
> >>> > Hi All,
> >>> >
> >>> > I have a route which picks files from a folder and do some
> processing on
> >>> it
> >>> > and store it in other folder. I have used parameter "delete=true" for
> >>> > deleting file from source folder.
> >>> >
> >>> > Sometime it happens that files are stuck in the source folder and
> camel
> >>> > lock files are also created for all the files but files are neither
> >>> process
> >>> > nor gets deleted.
> >>> >
> >>> > There is a VB script which copied files in the source folder. When I
> >>> > manually copied the files in the source folder then it never gives
> issue
> >>> > but when the files are copied from VB script then only issue occurs.
> Also
> >>> > the issue does not occur for the first file but it occurs randomly
> after
> >>> > processing number of files.
> >>> >
> >>> > This happens on server. I tried to reproduce this issue locally but
> no
> >>> luck.
> >>> >
> >>> > Does anybody have idea about it?
> >>> >
> >>> > Thanks!
> >>>
> >>>
> >>>
> >>> --
> >>> Claus Ibsen
> >>> -
> >>> http://davsclaus.com @davsclaus
> >>> Camel in Action 2: https://www.manning.com/ibsen2
> >>>
> >
> >
> >
> > --
> > Claus Ibsen
> > -
> > http://davsclaus.com @davsclaus
> > Camel in Action 2: https://www.manning.com/ibsen2
>
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>

Re: Camel lock gets acquired on files but not picked sometime

[Claus Ibsen]

Also you can use "done" files, eg in the VB script you write a 2nd
empty file with a special name, that tells Camel that the other file
is done being copied and ready.
There is some options on Camel file endpoint you can configure for that also.

On Mon, Aug 6, 2018 at 3:54 PM, Claus Ibsen  wrote:
> Hi
>
> You can configure Camel to only pickup files with a certain file
> pattern, or to exclude names with temporary names, etc.
> There is a number of different filter options, see its doc.
>
> On Mon, Aug 6, 2018 at 3:52 PM, Hemant Pekhale  
> wrote:
>> Hi Claus,
>>
>> Thanks for the reply.
>>
>> Yes, I am able to modify the script. But if we do copy the file with
>> temporary name and then rename, I doubt that there may be lock acquired on
>> temporary file name already. because my route has no waiting parameter. So
>> I think it might not be helpful for our case. Please correct me if I am
>> wrong.
>>
>> Regarding the file lock strategies, I think  readLock think
>> readLockCheckInterval think can be tried. Do you have any other suggestion?
>>
>> FYI : Whatever I can try I need to try on Production server so just wanted
>> to tackle this with minimum and required trials.
>>
>> Thanks!
>>
>>
>> On Mon, Aug 6, 2018 at 7:10 PM, Claus Ibsen  wrote:
>>
>>> Hi
>>>
>>> Are you able to modify the VB script so it copies the file using a
>>> temporary name and then rename the file when the copy is done.
>>>
>>> It sounds a bit like the problem is that the VB script is currently
>>> still copying the file, while Camel gets a lock on the file, and
>>> attempt to process it.
>>> File locks is not as easy and it depends a bit what file system, OS,
>>> and how big the files are and other things.
>>>
>>> A good strategy is to copy the files using temporary names or in
>>> another folder, and then do a file rename when the copy is over, that
>>> latter then is an atomic file operation which should not cause file
>>> locking issues.
>>>
>>> You can also try to use some of the other Camel file lock strategies
>>> instead of marker files.
>>> You can see more in the file documentation about these.
>>>
>>>
>>> On Mon, Aug 6, 2018 at 3:19 PM, Hemant Pekhale 
>>> wrote:
>>> > Hi All,
>>> >
>>> > I have a route which picks files from a folder and do some processing on
>>> it
>>> > and store it in other folder. I have used parameter "delete=true" for
>>> > deleting file from source folder.
>>> >
>>> > Sometime it happens that files are stuck in the source folder and camel
>>> > lock files are also created for all the files but files are neither
>>> process
>>> > nor gets deleted.
>>> >
>>> > There is a VB script which copied files in the source folder. When I
>>> > manually copied the files in the source folder then it never gives issue
>>> > but when the files are copied from VB script then only issue occurs. Also
>>> > the issue does not occur for the first file but it occurs randomly after
>>> > processing number of files.
>>> >
>>> > This happens on server. I tried to reproduce this issue locally but no
>>> luck.
>>> >
>>> > Does anybody have idea about it?
>>> >
>>> > Thanks!
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> -
>>> http://davsclaus.com @davsclaus
>>> Camel in Action 2: https://www.manning.com/ibsen2
>>>
>
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Re: Camel lock gets acquired on files but not picked sometime

[Claus Ibsen]

Hi

You can configure Camel to only pickup files with a certain file
pattern, or to exclude names with temporary names, etc.
There is a number of different filter options, see its doc.

On Mon, Aug 6, 2018 at 3:52 PM, Hemant Pekhale  wrote:
> Hi Claus,
>
> Thanks for the reply.
>
> Yes, I am able to modify the script. But if we do copy the file with
> temporary name and then rename, I doubt that there may be lock acquired on
> temporary file name already. because my route has no waiting parameter. So
> I think it might not be helpful for our case. Please correct me if I am
> wrong.
>
> Regarding the file lock strategies, I think  readLock think
> readLockCheckInterval think can be tried. Do you have any other suggestion?
>
> FYI : Whatever I can try I need to try on Production server so just wanted
> to tackle this with minimum and required trials.
>
> Thanks!
>
>
> On Mon, Aug 6, 2018 at 7:10 PM, Claus Ibsen  wrote:
>
>> Hi
>>
>> Are you able to modify the VB script so it copies the file using a
>> temporary name and then rename the file when the copy is done.
>>
>> It sounds a bit like the problem is that the VB script is currently
>> still copying the file, while Camel gets a lock on the file, and
>> attempt to process it.
>> File locks is not as easy and it depends a bit what file system, OS,
>> and how big the files are and other things.
>>
>> A good strategy is to copy the files using temporary names or in
>> another folder, and then do a file rename when the copy is over, that
>> latter then is an atomic file operation which should not cause file
>> locking issues.
>>
>> You can also try to use some of the other Camel file lock strategies
>> instead of marker files.
>> You can see more in the file documentation about these.
>>
>>
>> On Mon, Aug 6, 2018 at 3:19 PM, Hemant Pekhale 
>> wrote:
>> > Hi All,
>> >
>> > I have a route which picks files from a folder and do some processing on
>> it
>> > and store it in other folder. I have used parameter "delete=true" for
>> > deleting file from source folder.
>> >
>> > Sometime it happens that files are stuck in the source folder and camel
>> > lock files are also created for all the files but files are neither
>> process
>> > nor gets deleted.
>> >
>> > There is a VB script which copied files in the source folder. When I
>> > manually copied the files in the source folder then it never gives issue
>> > but when the files are copied from VB script then only issue occurs. Also
>> > the issue does not occur for the first file but it occurs randomly after
>> > processing number of files.
>> >
>> > This happens on server. I tried to reproduce this issue locally but no
>> luck.
>> >
>> > Does anybody have idea about it?
>> >
>> > Thanks!
>>
>>
>>
>> --
>> Claus Ibsen
>> -
>> http://davsclaus.com @davsclaus
>> Camel in Action 2: https://www.manning.com/ibsen2
>>



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Re: Camel lock gets acquired on files but not picked sometime

[Hemant Pekhale]

Hi Claus,

Thanks for the reply.

Yes, I am able to modify the script. But if we do copy the file with
temporary name and then rename, I doubt that there may be lock acquired on
temporary file name already. because my route has no waiting parameter. So
I think it might not be helpful for our case. Please correct me if I am
wrong.

Regarding the file lock strategies, I think  readLock think
readLockCheckInterval think can be tried. Do you have any other suggestion?

FYI : Whatever I can try I need to try on Production server so just wanted
to tackle this with minimum and required trials.

Thanks!


On Mon, Aug 6, 2018 at 7:10 PM, Claus Ibsen  wrote:

> Hi
>
> Are you able to modify the VB script so it copies the file using a
> temporary name and then rename the file when the copy is done.
>
> It sounds a bit like the problem is that the VB script is currently
> still copying the file, while Camel gets a lock on the file, and
> attempt to process it.
> File locks is not as easy and it depends a bit what file system, OS,
> and how big the files are and other things.
>
> A good strategy is to copy the files using temporary names or in
> another folder, and then do a file rename when the copy is over, that
> latter then is an atomic file operation which should not cause file
> locking issues.
>
> You can also try to use some of the other Camel file lock strategies
> instead of marker files.
> You can see more in the file documentation about these.
>
>
> On Mon, Aug 6, 2018 at 3:19 PM, Hemant Pekhale 
> wrote:
> > Hi All,
> >
> > I have a route which picks files from a folder and do some processing on
> it
> > and store it in other folder. I have used parameter "delete=true" for
> > deleting file from source folder.
> >
> > Sometime it happens that files are stuck in the source folder and camel
> > lock files are also created for all the files but files are neither
> process
> > nor gets deleted.
> >
> > There is a VB script which copied files in the source folder. When I
> > manually copied the files in the source folder then it never gives issue
> > but when the files are copied from VB script then only issue occurs. Also
> > the issue does not occur for the first file but it occurs randomly after
> > processing number of files.
> >
> > This happens on server. I tried to reproduce this issue locally but no
> luck.
> >
> > Does anybody have idea about it?
> >
> > Thanks!
>
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>

Re: Problem writing camel logs with log4j in a spring-boot application

[Claus Ibsen]

On Mon, Aug 6, 2018 at 3:40 PM, RavinderReddy k  wrote:
> Simple solution is use logback.xml file include in your spring boot
> application that's all.
>

Yeah SB uses logback as the logger impl by default.

> On Mon, Aug 6, 2018, 9:37 AM Claus Ibsen  wrote:
>
>> Hi
>>
>> Its probably more of a Spring Boot question how to configure its logging.
>> I suggest to study their documentation.
>>
>> Camel uses the slf4j-api and that ought to work fine with SB.
>>
>> On Mon, Aug 6, 2018 at 2:33 PM, Ranabroto Ghosh
>>  wrote:
>> > Hi
>> > I have a Spring-boot application with Apache Camel. All I am trying to
>> > achieve here writing all logs(java log and also camel route level logs
>> > using camel 'log' EIP) in a log file using log4j.
>> >
>> >
>> >
>> > *My POM: *
>> >
>> >   
>> > org.springframework.boot
>> > spring-boot-starter-web
>> >  
>> > 
>> > org.springframework.boot
>> > spring-boot-starter-logging
>> > 
>> > 
>> > 
>> >
>> > 
>> > org.apache.camel
>> > camel-spring-boot
>> > ${camel.version}
>> > 
>> >
>> > 
>> >commons-logging
>> >commons-logging
>> >1.2
>> > 
>> > 
>> > log4j
>> > log4j
>> > 1.2.16
>> >
>> >
>> >
>> > *LOG4J properties file: *
>> > log4j.rootLogger=info, rollinglog, rollingerrorlog
>> >
>> > log4j.logger.org.apache.camel=INFO
>> >
>> > # File output configuration
>> > log4j.appender.rollinglog=org.apache.log4j.DailyRollingFileAppender
>> > log4j.appender.rollinglog.File=C://logs/testgateway.log
>> > log4j.appender.rollinglog.Threshold=INFO
>> >
>> > # roll over the file at the end of each day and append the date to the
>> end
>> > of the file
>> > log4j.appender.rollinglog.DatePattern='.'-MM-dd
>> >
>> > # Configure the layout for the log file
>> > log4j.appender.rollinglog.layout=org.apache.log4j.PatternLayout
>> > log4j.appender.rollinglog.layout.ConversionPattern=(%d) %p %t %c - %m%n
>> >
>> > # File output configuration
>> > log4j.appender.rollingerrorlog=org.apache.log4j.DailyRollingFileAppender
>> > log4j.appender.rollingerrorlog.File=C://logs/ testgateway.log
>> > log4j.appender.rollingerrorlog.Threshold=ERROR
>> >
>> > # roll over the file at the end of each day and append the date to the
>> end
>> > of the file
>> > log4j.appender.rollingerrorlog.DatePattern='.'-MM-dd
>> >
>> > # Configure the layout for the log file
>> > log4j.appender.rollingerrorlog.layout=org.apache.log4j.PatternLayout
>> > log4j.appender.rollingerrorlog.layout.ConversionPattern=(%d) %p %t %c -
>> %m%n
>> >
>> >
>> > *My Camel route: *
>> >
>> > 
>> > 
>> >
>> > 
>> > 
>> >
>> > 
>> >
>> > 
>> > 
>> >
>> > So I am confused why camel log not getting printed from camel contexts
>> > wjere java logs are working fine, could you please guide me what I am
>> > missing here
>> >
>> > --
>> > Kind Regards
>> > Rana
>>
>>
>>
>> --
>> Claus Ibsen
>> -
>> http://davsclaus.com @davsclaus
>> Camel in Action 2: https://www.manning.com/ibsen2
>>



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Re: Camel lock gets acquired on files but not picked sometime

[Claus Ibsen]

Hi

Are you able to modify the VB script so it copies the file using a
temporary name and then rename the file when the copy is done.

It sounds a bit like the problem is that the VB script is currently
still copying the file, while Camel gets a lock on the file, and
attempt to process it.
File locks is not as easy and it depends a bit what file system, OS,
and how big the files are and other things.

A good strategy is to copy the files using temporary names or in
another folder, and then do a file rename when the copy is over, that
latter then is an atomic file operation which should not cause file
locking issues.

You can also try to use some of the other Camel file lock strategies
instead of marker files.
You can see more in the file documentation about these.


On Mon, Aug 6, 2018 at 3:19 PM, Hemant Pekhale  wrote:
> Hi All,
>
> I have a route which picks files from a folder and do some processing on it
> and store it in other folder. I have used parameter "delete=true" for
> deleting file from source folder.
>
> Sometime it happens that files are stuck in the source folder and camel
> lock files are also created for all the files but files are neither process
> nor gets deleted.
>
> There is a VB script which copied files in the source folder. When I
> manually copied the files in the source folder then it never gives issue
> but when the files are copied from VB script then only issue occurs. Also
> the issue does not occur for the first file but it occurs randomly after
> processing number of files.
>
> This happens on server. I tried to reproduce this issue locally but no luck.
>
> Does anybody have idea about it?
>
> Thanks!



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Re: Problem writing camel logs with log4j in a spring-boot application

[RavinderReddy k]

Simple solution is use logback.xml file include in your spring boot
application that's all.

On Mon, Aug 6, 2018, 9:37 AM Claus Ibsen  wrote:

> Hi
>
> Its probably more of a Spring Boot question how to configure its logging.
> I suggest to study their documentation.
>
> Camel uses the slf4j-api and that ought to work fine with SB.
>
> On Mon, Aug 6, 2018 at 2:33 PM, Ranabroto Ghosh
>  wrote:
> > Hi
> > I have a Spring-boot application with Apache Camel. All I am trying to
> > achieve here writing all logs(java log and also camel route level logs
> > using camel 'log' EIP) in a log file using log4j.
> >
> >
> >
> > *My POM: *
> >
> >   
> > org.springframework.boot
> > spring-boot-starter-web
> >  
> > 
> > org.springframework.boot
> > spring-boot-starter-logging
> > 
> > 
> > 
> >
> > 
> > org.apache.camel
> > camel-spring-boot
> > ${camel.version}
> > 
> >
> > 
> >commons-logging
> >commons-logging
> >1.2
> > 
> > 
> > log4j
> > log4j
> > 1.2.16
> >
> >
> >
> > *LOG4J properties file: *
> > log4j.rootLogger=info, rollinglog, rollingerrorlog
> >
> > log4j.logger.org.apache.camel=INFO
> >
> > # File output configuration
> > log4j.appender.rollinglog=org.apache.log4j.DailyRollingFileAppender
> > log4j.appender.rollinglog.File=C://logs/testgateway.log
> > log4j.appender.rollinglog.Threshold=INFO
> >
> > # roll over the file at the end of each day and append the date to the
> end
> > of the file
> > log4j.appender.rollinglog.DatePattern='.'-MM-dd
> >
> > # Configure the layout for the log file
> > log4j.appender.rollinglog.layout=org.apache.log4j.PatternLayout
> > log4j.appender.rollinglog.layout.ConversionPattern=(%d) %p %t %c - %m%n
> >
> > # File output configuration
> > log4j.appender.rollingerrorlog=org.apache.log4j.DailyRollingFileAppender
> > log4j.appender.rollingerrorlog.File=C://logs/ testgateway.log
> > log4j.appender.rollingerrorlog.Threshold=ERROR
> >
> > # roll over the file at the end of each day and append the date to the
> end
> > of the file
> > log4j.appender.rollingerrorlog.DatePattern='.'-MM-dd
> >
> > # Configure the layout for the log file
> > log4j.appender.rollingerrorlog.layout=org.apache.log4j.PatternLayout
> > log4j.appender.rollingerrorlog.layout.ConversionPattern=(%d) %p %t %c -
> %m%n
> >
> >
> > *My Camel route: *
> >
> > 
> > 
> >
> > 
> > 
> >
> > 
> >
> > 
> > 
> >
> > So I am confused why camel log not getting printed from camel contexts
> > wjere java logs are working fine, could you please guide me what I am
> > missing here
> >
> > --
> > Kind Regards
> > Rana
>
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>

Re: Problem writing camel logs with log4j in a spring-boot application

[Claus Ibsen]

Hi

Its probably more of a Spring Boot question how to configure its logging.
I suggest to study their documentation.

Camel uses the slf4j-api and that ought to work fine with SB.

On Mon, Aug 6, 2018 at 2:33 PM, Ranabroto Ghosh
 wrote:
> Hi
> I have a Spring-boot application with Apache Camel. All I am trying to
> achieve here writing all logs(java log and also camel route level logs
> using camel 'log' EIP) in a log file using log4j.
>
>
>
> *My POM: *
>
>   
> org.springframework.boot
> spring-boot-starter-web
>  
> 
> org.springframework.boot
> spring-boot-starter-logging
> 
> 
> 
>
> 
> org.apache.camel
> camel-spring-boot
> ${camel.version}
> 
>
> 
>commons-logging
>commons-logging
>1.2
> 
> 
> log4j
> log4j
> 1.2.16
>
>
>
> *LOG4J properties file: *
> log4j.rootLogger=info, rollinglog, rollingerrorlog
>
> log4j.logger.org.apache.camel=INFO
>
> # File output configuration
> log4j.appender.rollinglog=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.rollinglog.File=C://logs/testgateway.log
> log4j.appender.rollinglog.Threshold=INFO
>
> # roll over the file at the end of each day and append the date to the end
> of the file
> log4j.appender.rollinglog.DatePattern='.'-MM-dd
>
> # Configure the layout for the log file
> log4j.appender.rollinglog.layout=org.apache.log4j.PatternLayout
> log4j.appender.rollinglog.layout.ConversionPattern=(%d) %p %t %c - %m%n
>
> # File output configuration
> log4j.appender.rollingerrorlog=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.rollingerrorlog.File=C://logs/ testgateway.log
> log4j.appender.rollingerrorlog.Threshold=ERROR
>
> # roll over the file at the end of each day and append the date to the end
> of the file
> log4j.appender.rollingerrorlog.DatePattern='.'-MM-dd
>
> # Configure the layout for the log file
> log4j.appender.rollingerrorlog.layout=org.apache.log4j.PatternLayout
> log4j.appender.rollingerrorlog.layout.ConversionPattern=(%d) %p %t %c - %m%n
>
>
> *My Camel route: *
>
> 
> 
>
> 
> 
>
> 
>
> 
> 
>
> So I am confused why camel log not getting printed from camel contexts
> wjere java logs are working fine, could you please guide me what I am
> missing here
>
> --
> Kind Regards
> Rana



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Camel lock gets acquired on files but not picked sometime

[Hemant Pekhale]

Hi All,

I have a route which picks files from a folder and do some processing on it
and store it in other folder. I have used parameter "delete=true" for
deleting file from source folder.

Sometime it happens that files are stuck in the source folder and camel
lock files are also created for all the files but files are neither process
nor gets deleted.

There is a VB script which copied files in the source folder. When I
manually copied the files in the source folder then it never gives issue
but when the files are copied from VB script then only issue occurs. Also
the issue does not occur for the first file but it occurs randomly after
processing number of files.

This happens on server. I tried to reproduce this issue locally but no luck.

Does anybody have idea about it?

Thanks!

Problem with camel route with sql endpoint and jpa

[Manish.Shukla]

I have an Order and it has two type of todo tasks. There is as table where new 
todo entries are inserted. Camel route read todo entries and process it and 
with help of jpa(hibernate) and processing result is stored in database and 
todo entries are also deleted.

This looks working in general except one case which just happend once since 
last month, that when jpa go to store processing result to database , it took 
more than 10 minute and I got transaction rollback exception. The problem is 
that the log does not tell what was wrong there with jpa and what need to fixed?

Tools used : Apache karaf, Camel, Hibernate (Jpa), Mariadb

-- Camel route in blueprint file









-- Domain entities
@Entity
@Table(name = "order")
Class Order {
@Id
@Column(name = "id", unique = true, nullable = false)
private Long id;

@OneToMany(mappedBy = "order_id", cascade = CascadeType.ALL, fetch = 
FetchType.EAGER, orphanRemoval = true)
private Set tasks;

@OneToMany(mappedBy = "order_id", cascade = CascadeType.ALL, fetch = 
FetchType.EAGER)
private Set todoProcessingLogs;

}
-- class for processing of todo type-1 
class ProcessorOrderTodoType1 {

OrderService service;//inject as osgi service and use jpa entity manager


public void process(Map data) {
String orderId = data.get("order_id").toString();
Order order = em.find(Order.class, orderId);
//do business processing step - 1
order.getTodoProcessingLogs.add(new TodoProcessingLogs());
service.merge(order);//save partial processing status
//do business processing step - 2
order.removeSpecifiedTodoType("TYPE-1");
...
service.merge(order);
}

}
-- Error logs
 | ERROR | _fk%20=%20tft.id | DefaultErrorHandler  | 78 - 
org.apache.camel.camel-core - 2.18.0 | Failed delivery for (MessageId: 
ID-d8f4bff62acc-41735-1531810001157-0-15905 on ExchangeId: 
ID-d8f4bff62acc-41735-1531810001157-0-15906). Exhausted after delivery attempt: 
1 caught: javax.transaction.RollbackException

Message History
---
RouteId   ProcessorId  Processor

  Elapsed (ms)
[order-todo-type1-processing   ] [order-todo-type1-processing   ] [sql://SELECT 
tdt.order_id FROM order_todo_task tdt where tdt.type='TYPE-1'] [804724]
[order-todo-type1-processing   ] [to1] 
[bean:processorOrderTodoType1?method=process
] [804722]

Stacktrace
---
javax.transaction.RollbackException
at 
org.apache.aries.transaction.TxInterceptorImpl.postCallWithReturn(TxInterceptorImpl.java:109)
at 
org.apache.aries.blueprint.proxy.Collaborator.postInvoke(Collaborator.java:98)
at Proxy8af2e5ee_c7cb_4f0f_b75b_dacb4d121b4f.merge(Unknown Source)
at Proxy22cba4f1_c863_4172_bb5c_72387fe65471.merge(Unknown Source)
at 
com.mycompany.ProcessorOrderTodoType1.save(ProcessorOrderTodoType1.java:39)
at Proxy7bce399e_051e_45f0_a58a_2bbec26edb59.save(Unknown Source)
at 
com.mycompany.ProcessorOrderTodoType1.process(ProcessorOrderTodoType1.java:5)[11:com.mycompany.order-application:1.0.6]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.8.0_162]
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:1.8.0_162]
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_162]
at java.lang.reflect.Method.invoke(Method.java:498)[:1.8.0_162]
at 
org.apache.camel.component.bean.MethodInfo.invoke(MethodInfo.java:435)[78:org.apache.camel.camel-core:2.18.0]
at 
org.apache.camel.component.bean.MethodInfo$1.doProceed(MethodInfo.java:286)[78:org.apache.camel.camel-core:2.18.0]
at 
org.apache.camel.component.bean.MethodInfo$1.proceed(MethodInfo.java:259)[78:org.apache.camel.camel-core:2.18.0]
at 
org.apache.camel.component.bean.BeanProcessor.process(BeanProcessor.java:178)[78:org.apache.camel.camel-core:2.18.0]
at 
org.apache.camel.component.bean.BeanProducer.process(BeanProducer.java:41)[78:org.apache.camel.camel-core:2.18.0]
at 
org.apache.camel.processor.SendProcessor.process(SendProcessor.java:145)[78:org.apache.camel.camel-core:2.18.0]
at 
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[78:org.apache.camel.camel-core:2.18.0]
at 
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:542)[78:org.apache.camel.camel-core:2.18.0]
at 

Problem writing camel logs with log4j in a spring-boot application

[Ranabroto Ghosh]

Hi
I have a Spring-boot application with Apache Camel. All I am trying to
achieve here writing all logs(java log and also camel route level logs
using camel 'log' EIP) in a log file using log4j.



*My POM: *

  
org.springframework.boot
spring-boot-starter-web
 

org.springframework.boot
spring-boot-starter-logging





org.apache.camel
camel-spring-boot
${camel.version}



   commons-logging
   commons-logging
   1.2


log4j
log4j
1.2.16
   


*LOG4J properties file: *
log4j.rootLogger=info, rollinglog, rollingerrorlog

log4j.logger.org.apache.camel=INFO

# File output configuration
log4j.appender.rollinglog=org.apache.log4j.DailyRollingFileAppender
log4j.appender.rollinglog.File=C://logs/testgateway.log
log4j.appender.rollinglog.Threshold=INFO

# roll over the file at the end of each day and append the date to the end
of the file
log4j.appender.rollinglog.DatePattern='.'-MM-dd

# Configure the layout for the log file
log4j.appender.rollinglog.layout=org.apache.log4j.PatternLayout
log4j.appender.rollinglog.layout.ConversionPattern=(%d) %p %t %c - %m%n

# File output configuration
log4j.appender.rollingerrorlog=org.apache.log4j.DailyRollingFileAppender
log4j.appender.rollingerrorlog.File=C://logs/ testgateway.log
log4j.appender.rollingerrorlog.Threshold=ERROR

# roll over the file at the end of each day and append the date to the end
of the file
log4j.appender.rollingerrorlog.DatePattern='.'-MM-dd

# Configure the layout for the log file
log4j.appender.rollingerrorlog.layout=org.apache.log4j.PatternLayout
log4j.appender.rollingerrorlog.layout.ConversionPattern=(%d) %p %t %c - %m%n


*My Camel route: *












So I am confused why camel log not getting printed from camel contexts
wjere java logs are working fine, could you please guide me what I am
missing here

-- 
Kind Regards
Rana