Re: Support for Jakarta Namespace
Hallo Peter. This plans are very concrete (CAMEL-17136 <https://issues.apache.org/jira/browse/CAMEL-17136>) and work already goes <https://github.com/apache/camel/pull/7116>! :) Thanks, Vladimir Bychkov On 06.04.2022 12:13, peter.kuenz...@bluewin.ch wrote: Hi, are there plans for the Apache Camel framework to support in the future the Jakarta namespace for applications running on a Jakarta EE 9.x application server? Thanks, Peter OpenPGP_signature Description: OpenPGP digital signature
Replace Mule with Apache Camel-cmis
Hi all, I am working on CRM project with alfresco repository. I am trying to replace Mule with Camel-cmis so I can create communication between my application and alfresco repository trough Camel-cmis component. I am trying to create routes for : * new folder * remove folder * move folder * copy folder * new file * delete file * move file * replace file * copy file * check in * check out After creating new folder route, i continued with remove folder route and got stuck. I saw in camel-cmis description that with camel I can add/read files and folders. The other operations like delete,move, copy.. are not mentioned. Is there any chance to achieve my goal with camel-cmis ?
Re: ProducerTemaplate with direct or vm component, no consumer error
As you can see in the code below, I am using
producerTemplate.requestBody("direct:createFolderQueue", item); and
from("direct:createFolderQueue").
It says there is no consumer for "direct"createFolderQueue"
From: Claus Ibsen
Sent: Friday, April 12, 2019 12:16 PM
To: users@camel.apache.org
Subject: Re: ProducerTemaplate with direct or vm component, no consumer error
Hi
The ref component refers to an endpoint in the bean registry, eg if you
setup a bean of endpoint type etc,
so what you should do is just to use "direct:xxx" instead.
On Fri, Apr 12, 2019 at 11:44 AM Vladimir Cherepnalkovski <
cherepnalkov...@hotmail.com> wrote:
> Hi all,
> I got org.apache.camel.NoSuchEndpointException: No endpoint could be found
> for: ref:direct:createFolderQueue .
>
> I am using this bean to create one camel context, and use this context in
> all routes:
>
> @Component
> public class ArkCaseCamelContext
> {
> private CamelContext context;
>
> public ArkCaseCamelContext() throws Exception
> {
> context = new DefaultCamelContext();
> context.start();
> }
>
> I have producer class :
>
> @Component
> public class CMISQueue
> {
> private ProducerTemplate producerTemplate;
>
> @Autowired
> private ArkCaseCamelContext arkCaseCamelContext;
>
> public void createFolder(Item item) throws Exception
> {
> CamelContext context = arkCaseCamelContext.getContext();
> producerTemplate = context.createProducerTemplate();
> producerTemplate.start();
>
> producerTemplate.requestBody("direct:createFolderQueue", item);
> }
>
> and consumer route
>
> @Component
> public class CMISRoute
> {
> private CamelContext camelContext;
>
> @Autowired
> private ArkCaseCamelContext arkCaseCamelContext;
>
> public void test() throws Exception
> {
> camelContext = arkCaseCamelContext.getContext();
> camelContext.addRoutes(new RouteBuilder()
> {
> @Override
> public void configure() throws Exception
> {
> from("direct:createFolderQueue")
> .startupOrder(1)
>
> .log("***Executing
> CMISRoute***")
> .process(exchange -> {
>
> exchange.getIn().getHeaders().put(PropertyIds.OBJECT_TYPE_ID,
> "cmis:folder");
>
> exchange.getIn().getHeaders().put(PropertyIds.NAME, "New folder from Camel
> - adfasdfasdf");
>
> exchange.getIn().getHeaders().put(CamelCMISConstants.CMIS_FOLDER_PATH,
> "/User Homes/ann-acm");
> })
> // arkcase-cmis is the file under META-INF which
> points to ArkCaseCMISComponent
> .to(
> "arkcase-cmis:..");
> }
> });
> }
>
> So, I have direct producer and consumer. Why camel says that there is no
> consumer for direct:createFolderQueue ?
> I was trying with direct, seda, vm...
> I need synchronous communication, because of that I am using direct.
> Any suggestions are welcomed
>
--
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
ProducerTemaplate with direct or vm component, no consumer error
Hi all,
I got org.apache.camel.NoSuchEndpointException: No endpoint could be found for:
ref:direct:createFolderQueue .
I am using this bean to create one camel context, and use this context in all
routes:
@Component
public class ArkCaseCamelContext
{
private CamelContext context;
public ArkCaseCamelContext() throws Exception
{
context = new DefaultCamelContext();
context.start();
}
I have producer class :
@Component
public class CMISQueue
{
private ProducerTemplate producerTemplate;
@Autowired
private ArkCaseCamelContext arkCaseCamelContext;
public void createFolder(Item item) throws Exception
{
CamelContext context = arkCaseCamelContext.getContext();
producerTemplate = context.createProducerTemplate();
producerTemplate.start();
producerTemplate.requestBody("direct:createFolderQueue", item);
}
and consumer route
@Component
public class CMISRoute
{
private CamelContext camelContext;
@Autowired
private ArkCaseCamelContext arkCaseCamelContext;
public void test() throws Exception
{
camelContext = arkCaseCamelContext.getContext();
camelContext.addRoutes(new RouteBuilder()
{
@Override
public void configure() throws Exception
{
from("direct:createFolderQueue")
.startupOrder(1)
.log("***Executing
CMISRoute***")
.process(exchange -> {
exchange.getIn().getHeaders().put(PropertyIds.OBJECT_TYPE_ID, "cmis:folder");
exchange.getIn().getHeaders().put(PropertyIds.NAME,
"New folder from Camel - adfasdfasdf");
exchange.getIn().getHeaders().put(CamelCMISConstants.CMIS_FOLDER_PATH, "/User
Homes/ann-acm");
})
// arkcase-cmis is the file under META-INF which points
to ArkCaseCMISComponent
.to(
"arkcase-cmis:..");
}
});
}
So, I have direct producer and consumer. Why camel says that there is no
consumer for direct:createFolderQueue ?
I was trying with direct, seda, vm...
I need synchronous communication, because of that I am using direct.
Any suggestions are welcomed
RE: Camel-box download file version
I will test this, and reply. Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10 From: Andrea Cosentino Sent: Wednesday, November 28, 2018 11:44:00 AM To: users@camel.apache.org; Vladimir Cherepnalkovski Subject: Re: Camel-box download file version That class is generated. You just need to change the methodPattern here: https://github.com/apache/camel/blob/master/components/camel-box/camel-box-component/pom.xml#L278 If you are fancy opening a PR about that after testing the fix, you're welcome to do it -- Andrea Cosentino -- Apache Camel PMC Chair Apache Karaf Committer Apache Servicemix PMC Member Email: ancosen1...@yahoo.com Twitter: @oscerd2 Github: oscerd On Wednesday, November 28, 2018, 11:35:48 AM GMT+1, Vladimir Cherepnalkovski wrote: Done. I hope that someone will change the mapping on backend, so the shorthand alias will be available for use. From: Andrea Cosentino Sent: Wednesday, November 28, 2018 11:22 AM To: users@camel.apache.org Subject: Re: Camel-box download file version You're welcome to work on a PR.The documentation for this component is herehttps://github.com/apache/camel/blob/master/components/camel-box/camel-box-component/src/main/docs/box-component.adoc camel/box-component.adoc at master · apache/camel · GitHub github.com Box.com requires the use of OAuth2.0 for all client application authentication. In order to use camel-box with your account, you’ll need to create a new application within Box.com at https://developer.box.com.The Box application’s client id and secret will allow access to Box APIs which require a current user. --Andrea Cosentino --Apache Camel PMC ChairApache Karaf CommitterApache Servicemix PMC MemberEmail: ancosen1985@yahoo.comTwitter: @oscerd2Github: oscerdOn Wednesday, November 28, 2018, 11:19:12 AM GMT+1, Vladimir Cherepnalkovski wrote: I found the issue, and a solution.First there is a mistake in camel - box documentation.Endpoint "downloadPreviousFileVersions" should be "downloadPreviousFileVersion".If we use full endpoint name downloadPreviousFileVersion, the route will work.Also, there is a problem in camel - box endpoint mapping.I saw that shorthand alias name is mapped to "downloadPreviousFileVersions" which not exists.
Re: Camel-box download file version
Done. I hope that someone will change the mapping on backend, so the shorthand alias will be available for use. From: Andrea Cosentino Sent: Wednesday, November 28, 2018 11:22 AM To: users@camel.apache.org Subject: Re: Camel-box download file version You're welcome to work on a PR. The documentation for this component is here https://github.com/apache/camel/blob/master/components/camel-box/camel-box-component/src/main/docs/box-component.adoc [https://avatars3.githubusercontent.com/u/47359?s=400=4]<https://github.com/apache/camel/blob/master/components/camel-box/camel-box-component/src/main/docs/box-component.adoc> camel/box-component.adoc at master · apache/camel · GitHub<https://github.com/apache/camel/blob/master/components/camel-box/camel-box-component/src/main/docs/box-component.adoc> github.com Box.com requires the use of OAuth2.0 for all client application authentication. In order to use camel-box with your account, you’ll need to create a new application within Box.com at https://developer.box.com.The Box application’s client id and secret will allow access to Box APIs which require a current user. -- Andrea Cosentino -- Apache Camel PMC Chair Apache Karaf Committer Apache Servicemix PMC Member Email: ancosen1...@yahoo.com Twitter: @oscerd2 Github: oscerd On Wednesday, November 28, 2018, 11:19:12 AM GMT+1, Vladimir Cherepnalkovski wrote: I found the issue, and a solution. First there is a mistake in camel - box documentation. Endpoint "downloadPreviousFileVersions" should be "downloadPreviousFileVersion". If we use full endpoint name downloadPreviousFileVersion, the route will work. Also, there is a problem in camel - box endpoint mapping. I saw that shorthand alias name is mapped to "downloadPreviousFileVersions" which not exists.
Re: Camel-box download file version
I found the issue, and a solution. First there is a mistake in camel - box documentation. Endpoint "downloadPreviousFileVersions" should be "downloadPreviousFileVersion". If we use full endpoint name downloadPreviousFileVersion, the route will work. Also, there is a problem in camel - box endpoint mapping. I saw that shorthand alias name is mapped to "downloadPreviousFileVersions" which not exists.
Camel-Box : Get BoxItem insted of GetFolder or GetFile
Hi,
I am using Camel-box and I have a situation where I have some id, but i don't
know the object. I don't know if this id belongs to a File or Folder.
So I want to create a route to get BoxItem with the given itemId.
How do you solve this ?
I need something like this but for BoxItem.
from("seda:getFolderQueue").setExchangePattern(ExchangePattern.InOut)
.log("* GETTING FOLDER *")
.setHeader("CamelBox.path", method(folderService, "getPath"))
.to("box://folders/folder");
Regards,
Vladimir
RE: Camel-box download file version
Yes, I tried that too.
I tried two options :
.toD("box://files/downloadVersion?version=1");
Exception : Failed to resolve endpoint: box://files/downloadVersion?version=1
due to: No matching method for files/downloadVersion, with arguments [version]
version id received from “BoxFile.Info” is 374220338428
.toD("box://files/downloadVersion?version=374220338428");
Exception : Failed to resolve endpoint:
box://files/downloadVersion?version=374220338428 due to:
org.apache.camel.TypeConversionException: Error during type conversion from
type: java.lang.String to the required type: java.lang.Integer with value
374220338428 due For input string: "374220338428"
I this this is some bug. How to open a ticket ?
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
From: Andrea Cosentino
Sent: Wednesday, November 21, 2018 2:32:13 PM
To: users@camel.apache.org
Subject: Re: Camel-box download file version
Did you try by specifying option directly in the URI by using a toD?
If you think this is a bug, please raise a JIRA in the Apache Camel JIRA.
--
Andrea Cosentino
--
Apache Camel PMC Chair
Apache Karaf Committer
Apache Servicemix PMC Member
Email: ancosen1...@yahoo.com
Twitter: @oscerd2
Github: oscerd
On Wednesday, November 21, 2018, 2:12:23 PM GMT+1, Vladimir Cherepnalkovski
wrote:
Hi All,
I am trying to create Camel-Box route for downloading older file version.
I am using Camel version 2.22.1
This Is the code
from("seda:downloadFileQueue").setExchangePattern(ExchangePattern.InOut)
.log("* Downloading file version *")
.setHeader("CamelBox.fileId", method(service, "getFileId"))
.setHeader("CamelBox.output", method(service, "getFileOutputStream"))
.setHeader("CamelBox.version", method(service, "getFileVersion"))
.to("box://files/downloadVersion");
The exception is :
Failed to resolve endpoint: box://files/downloadVersion due to: No matching
method for files/downloadVersion, with arguments [].
I have tried with “downloadPreviousFileVersions” and “downloadVersion”
endpoints, but with no success.
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
Camel-box download file version
Hi All,
I am trying to create Camel-Box route for downloading older file version.
I am using Camel version 2.22.1
This Is the code
from("seda:downloadFileQueue").setExchangePattern(ExchangePattern.InOut)
.log("* Downloading file version *")
.setHeader("CamelBox.fileId", method(service, "getFileId"))
.setHeader("CamelBox.output", method(service, "getFileOutputStream"))
.setHeader("CamelBox.version", method(service, "getFileVersion"))
.to("box://files/downloadVersion");
The exception is :
Failed to resolve endpoint: box://files/downloadVersion due to: No matching
method for files/downloadVersion, with arguments [].
I have tried with “downloadPreviousFileVersions” and “downloadVersion”
endpoints, but with no success.
Sent from Mail for Windows 10
Re: Camel-fop does not work properly with ServiceMix 5.4.0 - error creating XSL-FO
Resolved. Used bean class=org.apache.xalan.processor.TransformerFactoryImpl -- View this message in context: http://camel.465427.n5.nabble.com/Camel-fop-does-not-work-properly-with-ServiceMix-5-4-0-error-creating-XSL-FO-tp5767810p5767876.html Sent from the Camel - Users mailing list archive at Nabble.com.
Camel-fop does not work properly with ServiceMix 5.4.0 - error creating XSL-FO
Hi! I try to use camel-fop with ServiceMix. I made a route in Eclipse, test it in Eclipse - everything is Ok. But after deploying it in ServiceMix I've got the error: *javax.xml.transform.TransformerException: org.apache.fop.fo.ValidationException: Element fo:simple-page-master is missing required property master-name * *Here is the part of xsl-fo (from ServiceMix log) - incorrect one:* ?xml version=1.0 encoding=UTF-8? fo:root xmlns:fo=http://www.w3.org/1999/XSL/Format; xmlns:fox=http://xml.apache.org/fop/extensions; fo:layout-master-set fo:simple-page-master fo:region-body/ /fo:simple-page-master fo:simple-page-master fo:region-body/ /fo:simple-page-master fo:page-sequence-master fo:repeatable-page-master-alternatives fo:conditional-page-master-reference/ fo:conditional-page-master-reference/ /fo:repeatable-page-master-alternatives /fo:page-sequence-master /fo:layout-master-set fo:page-sequence fo:flow fo:block fo:block fo:inline/ /fo:block /fo:block fo:block . fo:table fo:table-column/ fo:table-column/ fo:table-column/ fo:table-column/ fo:table-column/ fo:table-column/ fo:table-column/ fo:table-body fo:table-row fo:table-cell fo:block fo:inline/ /fo:block /fo:table-cell fo:table-cell fo:blockDoc №/fo:block /fo:table-cell fo:table-cell fo:block12-5789/fo:block /fo:table-cell fo:table-cell fo:block fo:inline/ /fo:block /fo:table-cell fo:table-cell fo:blockCod/fo:block /fo:table-cell /fo:table-row fo:table-row fo:table-cell fo:block/ /fo:table-cell As you see - many is lost. *And this is the correct one, when I try and get it by Eclipse:* ?xml version=1.0 encoding=UTF-8?fo:root xmlns:fox=http://xml.apache.org/fop/extensions; xmlns:fo=http://www.w3.org/1999/XSL/Format; fo:layout-master-set fo:simple-page-master master-name=first margin-right=0.8cm margin-left=0.8cm margin-bottom=1.0cm margin-top=1.5cm page-width=29.7cm page-height=21cm fo:region-body margin-bottom=0.5cm margin-top=0cm/ /fo:simple-page-master fo:simple-page-master master-name=rest margin-right=0.8cm margin-left=0.8cm margin-bottom=1.0cm margin-top=1.5cm page-width=29.7cm page-height=21cm fo:region-body margin-bottom=0.5cm margin-top=0cm/ /fo:simple-page-master fo:page-sequence-master master-name=document fo:repeatable-page-master-alternatives fo:conditional-page-master-reference page-position=first master-reference=first/ fo:conditional-page-master-reference page-position=rest master-reference=rest/ /fo:repeatable-page-master-alternatives /fo:page-sequence-master /fo:layout-master-set fo:page-sequence master-reference=document language=ru hyphenate=true fo:flow flow-name=xsl-region-body fo:block fo:block space-before.optimum=1.5pt space-after.optimum=1.5pt keep-together=always line-height=100% margin-bottom=0cm fo:inline white-space-collapse=false/ /fo:block /fo:block ... Any help is very appreciated! -- View this message in context: http://camel.465427.n5.nabble.com/Camel-fop-does-not-work-properly-with-ServiceMix-5-4-0-error-creating-XSL-FO-tp5767810.html Sent from the Camel - Users mailing list archive at Nabble.com.
What is recommended way of running many route sets in one application?
Hi, in our setup we would like to package different set of routes for different tasks into one java-based windows service. For example, we have one task to import files from some ftp to one clients database, another to import data from web-based repotys into another app db and so on. The question is - what is the recommended way of doing this in the Camel2? One option is to just include multiple RouteBuilder subclasses into the project and be done with it. This however raises some questions: am I correct to assume that global definitions like interceptors and error handler would be shared by all routes? Also, there is a chance of name clashes for direct endpoints name - for example I could accidentally use direct:initiateImport endpoint in 2 separate route sets, which would not be good - isn't there? Another option is to use multiple camel contexts in a single spring context - would this resolve the global definitions and naming clash issues? I tried to search the list (and the net in general) but there turned up little information about this. Does anybody use the camel this way? Are there any performance/scalability/whatever problems anticipated, if we expect up to dozen of camel contexts and more? Perhaps there are some other options I do not see? Regards, Vladimir Okhotnikov -- View this message in context: http://www.nabble.com/What-is-recommended-way-of-running-many-route-sets-in-one-application--tp25892016p25892016.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File component blocked by existing files
Sorry for the poor English - I know about the timeout option now - I saw it immediately AFTER I found the reason. I obviously saw the option before, among others, but did not pay enough attention to realize I should not use the default. And I did have the camel DEBUG level enabled - was the first thing I have done to debug the problem - and it did not help me at all - there were no relevant messages at all - and the DEBUG logging did work - I get a log of all routes on camel start etc. Claus, I'm not trying to blame or rant or something. I still think the framework is awesome. I apologize if I somehow made you think I'm feeling different. I just truly believe that infinite waits by default should be ruthlessly eliminated, and doing this will benefit the Camel and its users a great deal. Claus Ibsen-2 wrote: On Mon, Oct 12, 2009 at 10:15 AM, Vladimir Okhotnikov vokhotni...@gmail.com wrote: Yes, I known about the readLockTimeout option. The problem with this option is that by a) default it waits forever and b) while it waits for one file it does not process others. That's what made me spend 2 days wondering about the behavior described in the thread almost without clues, and it will made others. You did not mention that you knew this option and why have you not tried using it? Camel does TRACE / DEBUG logging which you could have enabled. Which is also written on the file component wiki page http://camel.apache.org/file2.html Debug logging This component has log level TRACE that can be helpful if you have problems. So if you have done that you would NOT have been in the dark for 2 days. The file consumer is NOT concurrent. So yes it will wait. If you want concurrency then define multiple routes with a file consumer. Which btw is not fun as file system is not transactional and having multiple consumers racing for the same files is not fun. Good idea about the default timeout which I have created a ticket for https://issues.apache.org/activemq/browse/CAMEL-2062 As is pretty well explained in the Release It book, infinite waits are bad. Infinite waits by default are very bad, especially in integration software - it virtually guarantees that at least one will slip into your live system, then it's just a matter of time until it blocks critical function or critical amount of resources - and you're waken in the middle of the night to deal with it. What do you think? Claus Ibsen-2 wrote: You can already do this by the readLockTimeout option. See more details here: http://camel.apache.org/file2.html For example you can set it to 1 as 10 seconds and if Camel cannot acquire the lock within that period it will skip this file and try the next one. -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25851978.html Sent from the Camel - Users mailing list archive at Nabble.com. -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25853488.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File component blocked by existing files
Thanks for the changes and suggestions - for my case just setting the timeout to 5 or 10 seconds should suffice, but at least I got some insight about async options and the reason file component does work sequentially. Claus Ibsen-2 wrote: BTW if you want to speed up the file routing you can use the async concurrency. A bit on this blog http://davsclaus.blogspot.com/2009/05/on-road-to-camel-20-concurrency-with.html Note the DSL was renamed from async to threads And then use a single file consumer to avoid race conditions for grabbing new files. On Mon, Oct 12, 2009 at 12:56 PM, Claus Ibsen claus.ib...@gmail.com wrote: On Mon, Oct 12, 2009 at 12:27 PM, Vladimir Okhotnikov vokhotni...@gmail.com wrote: Sorry for the poor English - I know about the timeout option now - I saw it immediately AFTER I found the reason. I obviously saw the option before, among others, but did not pay enough attention to realize I should not use the default. You english is good. Just as poor/good as mine :) You have a good point about the default timeout not being good. I have just changed it to 10 seconds and also Camel will now log at WARN level if the lock could not be acquired within that period. Before it was just DEBUG level. And I did have the camel DEBUG level enabled - was the first thing I have done to debug the problem - and it did not help me at all - there were no relevant messages at all - and the DEBUG logging did work - I get a log of all routes on camel start etc. You can enable DEBUG for only the file component org.apache.camel.component.file=DEBUG Claus, I'm not trying to blame or rant or something. I still think the framework is awesome. I apologize if I somehow made you think I'm feeling different. I just truly believe that infinite waits by default should be ruthlessly eliminated, and doing this will benefit the Camel and its users a great deal. None taken. I may have well started this, sorry as well. Its just that there is alot of free based support which does take much time. But also gives great feedback on the Camel. All together helps improve the community, which is ultimate the best for us and Apache. In you case you can set the timeout to a low value such as 1, 100, 500, 1000 or 2000 etc to let Camel quickly try the next file so you get it to process files quicker. Claus Ibsen-2 wrote: On Mon, Oct 12, 2009 at 10:15 AM, Vladimir Okhotnikov vokhotni...@gmail.com wrote: Yes, I known about the readLockTimeout option. The problem with this option is that by a) default it waits forever and b) while it waits for one file it does not process others. That's what made me spend 2 days wondering about the behavior described in the thread almost without clues, and it will made others. You did not mention that you knew this option and why have you not tried using it? Camel does TRACE / DEBUG logging which you could have enabled. Which is also written on the file component wiki page http://camel.apache.org/file2.html Debug logging This component has log level TRACE that can be helpful if you have problems. So if you have done that you would NOT have been in the dark for 2 days. The file consumer is NOT concurrent. So yes it will wait. If you want concurrency then define multiple routes with a file consumer. Which btw is not fun as file system is not transactional and having multiple consumers racing for the same files is not fun. Good idea about the default timeout which I have created a ticket for https://issues.apache.org/activemq/browse/CAMEL-2062 As is pretty well explained in the Release It book, infinite waits are bad. Infinite waits by default are very bad, especially in integration software - it virtually guarantees that at least one will slip into your live system, then it's just a matter of time until it blocks critical function or critical amount of resources - and you're waken in the middle of the night to deal with it. What do you think? Claus Ibsen-2 wrote: You can already do this by the readLockTimeout option. See more details here: http://camel.apache.org/file2.html For example you can set it to 1 as 10 seconds and if Camel cannot acquire the lock within that period it will skip this file and try the next one. -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25851978.html Sent from the Camel - Users mailing list archive at Nabble.com. -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25853488
Re: File component blocked by existing files
Got it! The bitness (and probably OS flavor) is not relevant - the problem is because the service account does not have Modify permission on the existing file. I strongly suspect that it cannot rename the offending file to satisfy readLock=rename and hangs forever. When I manually gave the Modify permission on one of the offending files, the corresponding consumer worked after service start. Nevertherless, I think it is still a bug - having such a poison file should not prevent the consumer from processing other files. At the very least, an indication of the problem in the log and skipping the problematic file are needed. What do you think? Claus Ibsen-2 wrote: Hi It should work, at least that is the goal. Is it always the same consumer that does not work? Can you try with only 2 and 1 to see if the issue is there as well? And can you create a ticket in JIRA about this? I will get time next week to take a peak, but would appreciate if you had time to track down more details where the problem may lie. BTW Which OS do you use? On Thu, Oct 8, 2009 at 9:41 PM, Vladimir Okhotnikov vokhotni...@gmail.com wrote: Sorry, Camel 2.0 release 3 routes like: file:///home/javadev/workspace/camelroutes/testdir/testdata/testimport?delete=trueinclude=%5E.%2Bmapping_warnings%5C.txt%24readLock=rename the only difference between all 3 is the include pattern. Claus Ibsen-2 wrote: Hi Always state which version of Camel you are using. And post how you route is configured. On Thu, Oct 8, 2009 at 4:57 PM, Vladimir Okhotnikov vokhotni...@gmail.com wrote: Hi I'm observing quite a strange behavior at the moment. In my app, I have several routes which all start by reading files in the same directory, deleting them after processing. Obviously, each rule uses its own include pattern to process separate subset of files. Now, the problem is that if the camel context is started when there are some existing files in the directory, the matching routes do not seem to work. I cannot see any related activity in the log file (except for the first 1 file to consume message), and the newly added files are being piled up in the directory. The other routes, which had no matching files on camel start do work. Is it a bug or am I just missing something? -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25803233.html Sent from the Camel - Users mailing list archive at Nabble.com. -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25806158.html Sent from the Camel - Users mailing list archive at Nabble.com. -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25833119.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File component blocked by existing files
Turned out that the OS question is the most important one. The effect happens at Windows 2003 x64, which is our live environment. So far I was not able to reproduce it on WinXP 32 VM. Sadly, I think I have no available Win2003x64 machines I could use for java development/debugging. To complicate things more - the live routes app is deployed as a windows service, using the 'wrapper' application as packaged by the appassembler maven plugin. As such, it actually runs the camel context in a 32-bit java VM there: Java Executable: C:\WINDOWS\system32\java.exe Windows version: 5.2.3790 Java Version : 1.6.0_11-b03 Java HotSpot(TM) Client VM Java VM Vendor : Sun Microsystems Inc. I think I'll try putting together a minimum sample which illustrates the error. Judging by the absence of any messages in the log relevant to those routes, the culprit likely happens in the FileComponent itself. As for other question: I tried restarting the camel context with different existing files in the pickup directory, and the rule is just that - if there is a matching file for a consumer on start - it hangs. If there are no matching files on start - the consumer works. I've added https://issues.apache.org/activemq/browse/CAMEL-2057 with Minor priority. Claus Ibsen-2 wrote: Hi It should work, at least that is the goal. Is it always the same consumer that does not work? Can you try with only 2 and 1 to see if the issue is there as well? And can you create a ticket in JIRA about this? I will get time next week to take a peak, but would appreciate if you had time to track down more details where the problem may lie. BTW Which OS do you use? On Thu, Oct 8, 2009 at 9:41 PM, Vladimir Okhotnikov vokhotni...@gmail.com wrote: Sorry, Camel 2.0 release 3 routes like: file:///home/javadev/workspace/camelroutes/testdir/testdata/testimport?delete=trueinclude=%5E.%2Bmapping_warnings%5C.txt%24readLock=rename the only difference between all 3 is the include pattern. Claus Ibsen-2 wrote: Hi Always state which version of Camel you are using. And post how you route is configured. On Thu, Oct 8, 2009 at 4:57 PM, Vladimir Okhotnikov vokhotni...@gmail.com wrote: Hi I'm observing quite a strange behavior at the moment. In my app, I have several routes which all start by reading files in the same directory, deleting them after processing. Obviously, each rule uses its own include pattern to process separate subset of files. Now, the problem is that if the camel context is started when there are some existing files in the directory, the matching routes do not seem to work. I cannot see any related activity in the log file (except for the first 1 file to consume message), and the newly added files are being piled up in the directory. The other routes, which had no matching files on camel start do work. Is it a bug or am I just missing something? -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25803233.html Sent from the Camel - Users mailing list archive at Nabble.com. -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25806158.html Sent from the Camel - Users mailing list archive at Nabble.com. -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25825859.html Sent from the Camel - Users mailing list archive at Nabble.com.
File component blocked by existing files
Hi I'm observing quite a strange behavior at the moment. In my app, I have several routes which all start by reading files in the same directory, deleting them after processing. Obviously, each rule uses its own include pattern to process separate subset of files. Now, the problem is that if the camel context is started when there are some existing files in the directory, the matching routes do not seem to work. I cannot see any related activity in the log file (except for the first 1 file to consume message), and the newly added files are being piled up in the directory. The other routes, which had no matching files on camel start do work. Is it a bug or am I just missing something? -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25803233.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File component blocked by existing files
Sorry, Camel 2.0 release 3 routes like: file:///home/javadev/workspace/camelroutes/testdir/testdata/testimport?delete=trueinclude=%5E.%2Bmapping_warnings%5C.txt%24readLock=rename the only difference between all 3 is the include pattern. Claus Ibsen-2 wrote: Hi Always state which version of Camel you are using. And post how you route is configured. On Thu, Oct 8, 2009 at 4:57 PM, Vladimir Okhotnikov vokhotni...@gmail.com wrote: Hi I'm observing quite a strange behavior at the moment. In my app, I have several routes which all start by reading files in the same directory, deleting them after processing. Obviously, each rule uses its own include pattern to process separate subset of files. Now, the problem is that if the camel context is started when there are some existing files in the directory, the matching routes do not seem to work. I cannot see any related activity in the log file (except for the first 1 file to consume message), and the newly added files are being piled up in the directory. The other routes, which had no matching files on camel start do work. Is it a bug or am I just missing something? -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25803233.html Sent from the Camel - Users mailing list archive at Nabble.com. -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25806158.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Zip format problem
There are a couple of options I think of
First, you can wire tap (multicast) between uncompress and
to(file://tempDir), extract the file names, aggregate and sort them, then
move files according to this list one by one (with a more or less trivial
processor) into source dir for the second route.
Second, latest entry in Claus's blog suggests that chosen routes can now be
started manually (do not known details though, so not sure whether it will
suffice).
In any case, I'd say it would be a good idea if uncompress component added
following headers:
- Source archive name and/or path
- CamelFileName
- Path in archive (for recursive extractions)
- no. of extracted file in archive and total number of files in archive
(i.e. MessageSequence pattern headers)
Christian Schneider wrote:
If just a normal move would be used it could still happen that the
second route starts while half of the files have been moved. I think it
could work if the files are moved in the sort order. If this moving is
done directly after the unpacking it should work. I have no idea though
if this could be expressed in Camel. On the other hand including this
move logic in uncompress is probably not such a good idea.
Greetings
Christian
Hadrian Zbarcea schrieb:
What about unpack in a temp dir and after unpacking is complete mv the
temp dir to the destination (tempCacheFolder in this case)?
My $0.02,
Hadrian
On Oct 5, 2009, at 6:03 PM, Christian Schneider wrote:
Hi Vladimir,
looks almost good. The problem is though that the second route
already starts processing while the first route still unpacks the
zip. We had to use a special marker file and a special
filter for the files to make sure it waits till the zip is fully
unpacked. Our solution was quite special though.
I think a general solution could be to somehow block the second route
while the first route is active. I have no idea though if this can be
done with camel. Any ideas?
Greetings
Christian
Vladimir Okhotnikov schrieb:
After having slept on it, I think that since you will have to resort
to use
disk as a kind of cache anyway in general case, you can just as well do
from(...).uncompress(Zip).to(file://tempCacheFolder?...);
from(file://tempCacheFolder?sort=${file:name}).process(...
Which means it is actually not worth it to create more elaborate
solution
for extracting files from zip archives in particular order, given the
considerations that it a) would not work without penalty on solid
archives
and b) is probably kind of rare requirement - in some cases sorting of
unarchived files is irrelevant, in other it is possible to reorder
processing results before aggregation instead.
What do you think?
Christian Schneider wrote:
Vladimir Okhotnikov schrieb:
Christian Schneider wrote:
Btw. In our scenario we had the requirement that the files from
the zip had to be processed in a certain order. In our case the
processing should be done in order of the filenames.
Any idea how this could be expressed?
decompress(Zip).resequencer()... ?
Logically this would work but I fear it could consume much memory.
Resequencer can´t be made streaming but perhaps it can use a disk.
Greetings
Christian
--
Christian Schneider
---
http://www.liquid-reality.de
--
Christian Schneider
---
http://www.liquid-reality.de
--
Christian Schneider
---
http://www.liquid-reality.de
--
View this message in context:
http://www.nabble.com/Zip-format-problem-tp25723682p25765958.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Zip format problem
Actually, not even multicast - just something like uncompress(Zip).to(file://tempDir).body(header(CamelFileName)).aggregate().sort().process(MoveFilesProcessor); Vladimir Okhotnikov wrote: First, you can wire tap (multicast) between uncompress and to(file://tempDir), extract the file names, aggregate and sort them, then move files according to this list one by one (with a more or less trivial processor) into source dir for the second route. -- View this message in context: http://www.nabble.com/Zip-format-problem-tp25723682p25766010.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Zip format problem
I'd rather prefer git than bzr (at least have some limited experience with git). Mercurial is not an option I assume? :-) Christian Grobmeier wrote: I hoped the sandbox was open to other apache committers. However, we'll use GIT or Bazaar for preparing the patch then You can also have projects hosted at the FUSE forge. http://forge.fusesource.com/ But it may be overkill as its meant for new projects, and not for minor patches. No problem, bzr oder git will be easier than the forge Cheers Christian -- View this message in context: http://www.nabble.com/Zip-format-problem-tp25723682p25747315.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Zip format problem
Will do tomorrow (need to print it out and visit a post office to send it) Christian Grobmeier wrote: Btw, Vladimir can you please sign an ICLA? http://www.apache.org/licenses/icla.txt Thanks -- View this message in context: http://www.nabble.com/Zip-format-problem-tp25723682p25748236.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Zip format problem
After having slept on it, I think that since you will have to resort to use
disk as a kind of cache anyway in general case, you can just as well do
from(...).uncompress(Zip).to(file://tempCacheFolder?...);
from(file://tempCacheFolder?sort=${file:name}).process(...
Which means it is actually not worth it to create more elaborate solution
for extracting files from zip archives in particular order, given the
considerations that it a) would not work without penalty on solid archives
and b) is probably kind of rare requirement - in some cases sorting of
unarchived files is irrelevant, in other it is possible to reorder
processing results before aggregation instead.
What do you think?
Christian Schneider wrote:
Vladimir Okhotnikov schrieb:
Christian Schneider wrote:
Btw. In our scenario we had the requirement that the files from the zip
had to be processed in a certain order. In our case the processing
should be done in order of the filenames.
Any idea how this could be expressed?
decompress(Zip).resequencer()... ?
Logically this would work but I fear it could consume much memory.
Resequencer can´t be made streaming but perhaps it can use a disk.
Greetings
Christian
--
Christian Schneider
---
http://www.liquid-reality.de
--
View this message in context:
http://www.nabble.com/Zip-format-problem-tp25723682p25748655.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Zip format problem
This would work nice for tar.gz, but not so nice for zip and other formats which tend to combine aggregation and compression in one operation. The main question is, what exactly should do split(Zip) and what decompress(Zip) in your scenario? Christian Schneider wrote: How about using .compress to compress and .decompress to extract something? Could perhaps the notion of extracting several parts from a container be handled with the existing splitting component? I imagine a 2 phase extraction like in tar.gz: .decompress(Gzip).split(Tar) or for a zip file: .decompress(Zip).split(Zip) -- Christian Schneider --- http://www.liquid-reality.de -- View this message in context: http://www.nabble.com/Zip-format-problem-tp25723682p25738016.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Zip format problem
Christian Schneider wrote: Hmm, I think you are right as the Zip API is not separate it makes sense to use only one DSL function. The question is then of course how would you name it? Would you rather name the operation split(Zip) or decompress(Zip)? If the operation was named decompress the tar.gz example could be done like this: decompress(GZip).decompress(Tar) and zip would simply look like decompress(Zip) decompress is surely way better than split. The approach seems okay. The only thing that bothers me is that I'm still not sure the split/aggregate behavior is specific to compression, and will not be useful for other data formats. Christian Schneider wrote: Btw. In our scenario we had the requirement that the files from the zip had to be processed in a certain order. In our case the processing should be done in order of the filenames. Any idea how this could be expressed? decompress(Zip).resequencer()... ? -- View this message in context: http://www.nabble.com/Zip-format-problem-tp25723682p25740336.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Zip format problem
Yep, didn't think about memory consumption.
It actually just occurred to me that with such requirements it would be much
more logical to use something like a File component (i.e. a file-based
component, like FtpComponent) - it already addresses sorting, split needs
etc. (Perhaps this is what Claus referred to as 'VFS' in his original
reply?)
So you could do something like:
from(ftp://...;).to(file://downloadedFiles).to(decompress://zip?sort=${file:name}).process(...
but I'm not sure such usage is possible at all.
Alternatively we could extract 2 separate operations on the composite
archives - list included file names and extract file with given name from
the archive. Then we could use existing patterns/custom processors to
transform the list of files according to requirements, and extract files by
one. Only this would perform bad for solid archives I suppose :-(
Christian Schneider wrote:
Vladimir Okhotnikov schrieb:
The approach seems okay. The only thing that bothers me is that I'm still
not sure the split/aggregate behavior is specific to compression, and
will
not be useful for other data formats.
split and aggregate are already present in the DSL. That is why I
first thought about using them. The question is if they can be used for
this case. I guess
the extraction could be done in an expression like in the example
split(body(String.class).tokenize(\n))
split(body(String.class).decompress(Zip))
The only thing is that I think decompress should work in a streaming way
to handle large archives. I guess the above example would read the whole
archive into a string.
Christian Schneider wrote:
Btw. In our scenario we had the requirement that the files from the zip
had to be processed in a certain order. In our case the processing
should be done in order of the filenames.
Any idea how this could be expressed?
decompress(Zip).resequencer()... ?
Logically this would work but I fear it could consume much memory.
Resequencer can´t be made streaming but perhaps it can use a disk.
Greetings
Christian
--
Christian Schneider
---
http://www.liquid-reality.de
--
View this message in context:
http://www.nabble.com/Zip-format-problem-tp25723682p25741749.html
Sent from the Camel - Users mailing list archive at Nabble.com.
