[jira] [Updated] (CAMEL-16953) Unmarshalling large tar.gz files fails
[ https://issues.apache.org/jira/browse/CAMEL-16953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Vottner updated CAMEL-16953: -- Description: ZulipChat-Discussion: https://camel.zulipchat.com/#narrow/stream/257298-camel/topic/tar.2Egz.20unmarshalling In a very simple route setup that just reads any tar.gz archives found in the provided directory and prints the names of the files within that archive to the log, this code fails on processing larger tar.gz archives. {code:title=PreProcessingRoute.java|borderStyle=solid} from(file("archiveFile")) .routeId("pre-processing") .process(exchange -> { LOG.info("Processing archive: {}", exchange.getIn().getHeader(Exchange.FILE_NAME, String.class)); }) .unmarshal().gzipDeflater() .split(new TarSplitter()).streaming() .process(exchange -> { final String name = exchange.getIn().getHeader(Exchange.FILE_NAME, String.class); LOG.debug("name: {}", name); }) .end(); {code} The JVM quickly runs out of memory as it probably copies over the bytes from the original stream to a stream that should take care of decompressing the files, as indicated by the exception being thrown in the `hugeCapacity(...)` method of the `ByteArrayOutputStream`class: {code:title=StackTrace|borderStyle=solid} org.apache.camel.CamelExecutionException: Exception occurred during execution on the exchange: Exchange[] at org.apache.camel.CamelExecutionException.wrapCamelExecutionException(CamelExecutionException.java:45) at org.apache.camel.support.AbstractExchange.setException(AbstractExchange.java:589) at org.apache.camel.support.DefaultExchange.setException(DefaultExchange.java:27) at org.apache.camel.support.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:81) at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:463) at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:179) at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:64) at org.apache.camel.processor.Pipeline.process(Pipeline.java:184) at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:398) at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:492) at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:245) at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:206) at org.apache.camel.support.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:190) at org.apache.camel.support.ScheduledPollConsumer.run(ScheduledPollConsumer.java:107) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:308) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: null at java.io.ByteArrayOutputStream.hugeCapacity(ByteArrayOutputStream.java:123) at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:117) at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153) at org.apache.camel.support.builder.OutputStreamBuilder.write(OutputStreamBuilder.java:58) at org.apache.camel.util.IOHelper.copy(IOHelper.java:193) at org.apache.camel.util.IOHelper.copy(IOHelper.java:148) at org.apache.camel.util.IOHelper.copy(IOHelper.java:143) at org.apache.camel.util.IOHelper.copy(IOHelper.java:139) at org.apache.camel.dataformat.deflater.GzipDeflaterDataFormat.unmarshal(GzipDeflaterDataFormat.java:63) at org.apache.camel.support.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:64) ... 18 common frames omitted {code} If I instead use a custom DataFormat class that looks like this: {code:title=PreProcessingRoute.java|borderStyle=solid} import org.apache.camel.Exchange; import org.apache.camel.spi.DataFormat; import org.apache.camel.util.IOHelper; import org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream; import org.apache.commons.compress.compressors.gzip.GzipCompressorOutputStream; import java.io.InputStream; import java.io.OutputStream; public class GZipDataFormat implements DataFormat { @Override public void marshal(Exchange exchange, Object graph, OutputStream stream) throws Exception { InputStream is = exc
[jira] [Created] (CAMEL-16953) Unmarshalling large tar.gz files fails
Roman Vottner created CAMEL-16953: - Summary: Unmarshalling large tar.gz files fails Key: CAMEL-16953 URL: https://issues.apache.org/jira/browse/CAMEL-16953 Project: Camel Issue Type: Bug Components: came-core Affects Versions: 3.x Environment: Ubuntu 19.04 openjdk 11.0.11 2021-04-20 OpenJDK Runtime Environment (build 11.0.11+9-Ubuntu-0ubuntu2.20.04) OpenJDK 64-Bit Server VM (build 11.0.11+9-Ubuntu-0ubuntu2.20.04, mixed mode, sharing) Camel 3.11.1 Reporter: Roman Vottner ZulipChat-Discussion: https://camel.zulipchat.com/#narrow/stream/257298-camel/topic/tar.2Egz.20unmarshalling In a very simple route setup that just reads any tar.gz archives found in the provided directory and prints the names of the files within that archive to the log, this code fails on processing larger tar.gz archives. from(file({color:#6a8759}"\{{archiveFile}}"{color})) .routeId({color:#6a8759}"pre-processing"{color}) .process(exchange -> { {color:#9876aa}LOG{color}.info({color:#6a8759}"Processing archive: {}"{color}{color:#cc7832}, {color}exchange.getIn().getHeader(Exchange.{color:#9876aa}FILE_NAME{color}{color:#cc7832}, {color}String.{color:#cc7832}class{color})){color:#cc7832}; {color} }){color:#808080} {color} .unmarshal().gzipDeflater() .split({color:#cc7832}new {color}TarSplitter()).streaming() .process(exchange -> { {color:#cc7832}final {color}String name = exchange.getIn().getHeader(Exchange.{color:#9876aa}FILE_NAME{color}{color:#cc7832}, {color}String.{color:#cc7832}class{color}){color:#cc7832}; {color} {color:#9876aa}LOG{color}.debug({color:#6a8759}"name: {}"{color}{color:#cc7832}, {color}name){color:#cc7832}; {color} }){color:#808080} {color} .end(){color:#cc7832};{color} The JVM quickly runs out of memory as it probably copies over the bytes from the original stream to a stream that should take care of decompressing the files, as indicated by the exception being thrown in the `hugeCapacity(...)` method of the `ByteArrayOutputStream`class: {{org.apache.camel.CamelExecutionException: Exception occurred during execution on the exchange: Exchange[]at org.apache.camel.CamelExecutionException.wrapCamelExecutionException(CamelExecutionException.java:45) at org.apache.camel.support.AbstractExchange.setException(AbstractExchange.java:589) at org.apache.camel.support.DefaultExchange.setException(DefaultExchange.java:27) at org.apache.camel.support.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:81) at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:463) at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:179) at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:64) at org.apache.camel.processor.Pipeline.process(Pipeline.java:184)at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:398) at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:492) at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:245) at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:206) at org.apache.camel.support.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:190) at org.apache.camel.support.ScheduledPollConsumer.run(ScheduledPollConsumer.java:107) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:308) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java)at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)Caused by: java.lang.OutOfMemoryError: nullat java.io.ByteArrayOutputStream.hugeCapacity(ByteArrayOutputStream.java:123) at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:117)at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)at org.apache.camel.support.builder.OutputStreamBuilder.write(OutputStreamBuilder.java:58) at org.apache.camel.util.IOHelper.copy(IOHelper.java:193)at org.apache.camel.util.IOHelper.cop
[jira] [Comment Edited] (CAMEL-13162) Unknown parameter issue on weaving from-with on a REST-DSL route
[ https://issues.apache.org/jira/browse/CAMEL-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761011#comment-16761011 ] Roman Vottner edited comment on CAMEL-13162 at 2/5/19 4:47 PM: --- Code runs with Camel 2.21.4 and 2.22.3 if I add a configuration such as {code:java} restConfiguration().component("servlet");{code} before the rest route definition, though with 2.23.0 and 2.23.1 the same test fails even with the restConfiguration in place. It might be related to https://issues.apache.org/jira/browse/CAMEL-12943 was (Author: rovo): Code runs with Camel 2.21.4 and 2.22.3 if I add a configuration such as {code:java} restConfiguration().component("servlet");{code} before the rest route definition, though with 2.23.0 and 2.23.1 the same test fails even with the restConfiguration in place. > Unknown parameter issue on weaving from-with on a REST-DSL route > > > Key: CAMEL-13162 > URL: https://issues.apache.org/jira/browse/CAMEL-13162 > Project: Camel > Issue Type: Bug > Components: camel-test >Affects Versions: 2.23.0, 2.23.1 >Reporter: Roman Vottner >Priority: Critical > > On attempting to weave a REST-DSL route and replace the route with something > different, i.e. {color:#d04437}direct:start{color} seems to be fine in > regards to the produced debug XML ... > XML before weaving: > {code:java} > > http://camel.apache.org/schema/spring"; > customId="true" > id="apiHeartbeat" > rest="true"> > uri="rest://post:/heartbeat:/%7Bsender%7D?description=Test+description&routeId=apiHeartbeat"/> > message="Received request with payload ${body} and headers: > ${headers}"/> > > {code} > XML after weaving: > {code:java} > > http://camel.apache.org/schema/spring"; > customId="true" > id="apiHeartbeat" > rest="true"> > > message="Received request with payload ${body} and headers: > ${headers}"/> > > {code} > ... though on sending a message via a producerTemplate will result in the > following exception: > {code:java} > org.apache.camel.FailedToCreateRouteException: Failed to create route > apiHeartbeat: Route(apiHeartbeat)[[From[direct://start?routeId=apiHeartbea... > because of Failed to resolve endpoint: direct://start?routeId=apiHeartbeat > due to: Failed to resolve endpoint: direct://start?routeId=apiHeartbeat due > to: There are 1 parameters that couldn't be set on the endpoint. Check the > uri if the parameters are spelt correctly and that they are properties of the > endpoint. Unknown parameters=[{routeId=apiHeartbeat}] > at > org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:217) > at > org.apache.camel.impl.DefaultCamelContext.startRoute(DefaultCamelContext.java:1140) > at > org.apache.camel.model.RouteDefinition.adviceWith(RouteDefinition.java:318) > at > at.erpel.messaginghub.services.unit.routes.api.CamelRouteAdviceTest.testRouteWeaving(CamelRouteAdviceTest.java:69) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:74) > at > org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:84) > at > org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75) > at > org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86) > at > org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > at > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251) > at > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > at org.junit.runne
[jira] [Commented] (CAMEL-13162) Unknown parameter issue on weaving from-with on a REST-DSL route
[ https://issues.apache.org/jira/browse/CAMEL-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761011#comment-16761011 ] Roman Vottner commented on CAMEL-13162: --- Code runs with Camel 2.21.4 and 2.22.3 if I add a configuration such as {code:java} restConfiguration().component("servlet");{code} before the rest route definition, though with 2.23.0 and 2.23.1 the same test fails even with the restConfiguration in place. > Unknown parameter issue on weaving from-with on a REST-DSL route > > > Key: CAMEL-13162 > URL: https://issues.apache.org/jira/browse/CAMEL-13162 > Project: Camel > Issue Type: Bug > Components: camel-test >Affects Versions: 2.23.0, 2.23.1 >Reporter: Roman Vottner >Priority: Critical > > On attempting to weave a REST-DSL route and replace the route with something > different, i.e. {color:#d04437}direct:start{color} seems to be fine in > regards to the produced debug XML ... > XML before weaving: > {code:java} > > http://camel.apache.org/schema/spring"; > customId="true" > id="apiHeartbeat" > rest="true"> > uri="rest://post:/heartbeat:/%7Bsender%7D?description=Test+description&routeId=apiHeartbeat"/> > message="Received request with payload ${body} and headers: > ${headers}"/> > > {code} > XML after weaving: > {code:java} > > http://camel.apache.org/schema/spring"; > customId="true" > id="apiHeartbeat" > rest="true"> > > message="Received request with payload ${body} and headers: > ${headers}"/> > > {code} > ... though on sending a message via a producerTemplate will result in the > following exception: > {code:java} > org.apache.camel.FailedToCreateRouteException: Failed to create route > apiHeartbeat: Route(apiHeartbeat)[[From[direct://start?routeId=apiHeartbea... > because of Failed to resolve endpoint: direct://start?routeId=apiHeartbeat > due to: Failed to resolve endpoint: direct://start?routeId=apiHeartbeat due > to: There are 1 parameters that couldn't be set on the endpoint. Check the > uri if the parameters are spelt correctly and that they are properties of the > endpoint. Unknown parameters=[{routeId=apiHeartbeat}] > at > org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:217) > at > org.apache.camel.impl.DefaultCamelContext.startRoute(DefaultCamelContext.java:1140) > at > org.apache.camel.model.RouteDefinition.adviceWith(RouteDefinition.java:318) > at > at.erpel.messaginghub.services.unit.routes.api.CamelRouteAdviceTest.testRouteWeaving(CamelRouteAdviceTest.java:69) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:74) > at > org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:84) > at > org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75) > at > org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86) > at > org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > at > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251) > at > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > at > org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.jav
[jira] [Created] (CAMEL-13162) Unknown parameter issue on weaving from-with on a REST-DSL route
Roman Vottner created CAMEL-13162: - Summary: Unknown parameter issue on weaving from-with on a REST-DSL route Key: CAMEL-13162 URL: https://issues.apache.org/jira/browse/CAMEL-13162 Project: Camel Issue Type: Bug Components: camel-test Affects Versions: 2.23.1, 2.23.0 Reporter: Roman Vottner On attempting to weave a REST-DSL route and replace the route with something different, i.e. {color:#d04437}direct:start{color} seems to be fine in regards to the produced debug XML ... XML before weaving: {code:java} http://camel.apache.org/schema/spring"; customId="true" id="apiHeartbeat" rest="true"> {code} XML after weaving: {code:java} http://camel.apache.org/schema/spring"; customId="true" id="apiHeartbeat" rest="true"> {code} ... though on sending a message via a producerTemplate will result in the following exception: {code:java} org.apache.camel.FailedToCreateRouteException: Failed to create route apiHeartbeat: Route(apiHeartbeat)[[From[direct://start?routeId=apiHeartbea... because of Failed to resolve endpoint: direct://start?routeId=apiHeartbeat due to: Failed to resolve endpoint: direct://start?routeId=apiHeartbeat due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{routeId=apiHeartbeat}] at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:217) at org.apache.camel.impl.DefaultCamelContext.startRoute(DefaultCamelContext.java:1140) at org.apache.camel.model.RouteDefinition.adviceWith(RouteDefinition.java:318) at at.erpel.messaginghub.services.unit.routes.api.CamelRouteAdviceTest.testRouteWeaving(CamelRouteAdviceTest.java:69) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:74) at org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:84) at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75) at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86) at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: direct://start?routeId=apiHeartbeat due to: Failed to resolve endpoint: direct://start?routeId=apiHeartbeat
[jira] [Commented] (CAMEL-12166) FTPConsumer - no disconnect if polling files fails
[ https://issues.apache.org/jira/browse/CAMEL-12166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16343735#comment-16343735 ] Roman Vottner commented on CAMEL-12166: --- We ran into a similar issue today on an old AX400 system where we have to up- and download files from via FTP. While we hadn't issues before (used a camel-driven client application on their system since Camel 2.14.1 (and before) with regular version updates of Camel during this period, in the early morning suddenly the same behavior as mentioned here occurred and we roughly needed half a day to fix this issue even though the AX400 server maintainer assured that he didn't change anything in regards to general server and FTP server configurations. Note that the most recent versions deployed contained Camel 2.20.1 and for testing even 2.21.0-SNAPSHOT. Our issue was caused by Camel connecting in active mode to the server by default. On analyzing the Apache FTP client log we noticed that after successfully interacting with the server after the second time listing (or the previous PORT command) the mentioned exception occurred in our logs. INFO | jvm 1 | 2018/01/29 17:51:47 | CWD tmp INFO | jvm 1 | 2018/01/29 17:51:47 | 250 Befehl CWD erfolgreich. INFO | jvm 1 | 2018/01/29 17:51:47 | CWD some_subdir INFO | jvm 1 | 2018/01/29 17:51:47 | 250 Befehl CWD erfolgreich. INFO | jvm 1 | 2018/01/29 17:51:47 | SYST INFO | jvm 1 | 2018/01/29 17:51:47 | 215 UNIX-Typ: L8 Version: BSD-44 INFO | jvm 1 | 2018/01/29 17:51:47 | PORT 1,0,0,130,196,47 INFO | jvm 1 | 2018/01/29 17:51:47 | 200 PORT-Befehl erfolgreich. INFO | jvm 1 | 2018/01/29 17:51:47 | LIST INFO | jvm 1 | 2018/01/29 17:53:02 | 425 Keine Datenverbindung Connecting to the FTP server via FileZilla worked and after studying the produced logs from FileZilla we noticed that it communicates in passiveMode which we gave a try. After adding passiveMode=true we managed to read and write files to the server again. Why the errors occurred without any interactions on our or the admins behalf is a different storry. > FTPConsumer - no disconnect if polling files fails > -- > > Key: CAMEL-12166 > URL: https://issues.apache.org/jira/browse/CAMEL-12166 > Project: Camel > Issue Type: Bug > Components: camel-ftp >Affects Versions: 2.16.2 > Environment: We are using Camel 2.16.2 as part of ServiceMix 6.1.1 >Reporter: Oliver Limberg >Assignee: Claus Ibsen >Priority: Major > Fix For: 2.21.0 > > > We encountered an issue downoading files from various different ftp servers. > If listing the files to consume fails for whatever reason, the FTP client > does not disconnect from the server, even if disconnect=true is specified in > the endpoint URL. > This leads to the problem, that upon the next poll, the connection ist still > open and the component starts to navigate stepwise to the target folder which > will fail, because we are not in the home folder of the FTP user anymore. > This leads to an GenericFileOperationFailedException, which will again cause > the poll to fail without a disconnect. From now on, polling this endpoint > will fail until restarting the containing route. > > Adding stepwise=false as an workaround did not work in our case because one > of teh external FTP servers will return absolute paths when calling the list > command, resulting in an error when trying to download the files. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CAMEL-11998) Configuring SSL via endpointProperty in restConfiguration for an Undertow component will cause a BindException for multiple endpoints
[ https://issues.apache.org/jira/browse/CAMEL-11998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244215#comment-16244215 ] Roman Vottner commented on CAMEL-11998: --- Note that this also occurs if an Undertow component is created, configured and added to the context manually via {code:java} @Override public void setupCamelContext(CamelContext camelContext) throws Exception { super.setupCamelContext(camelContext); ... UndertowComponent undertow = new UndertowComponent(); undertow.setSslContextParameters(sslContextParameters()); undertow.setUndertowHttpBinding(undertowHttpBinding()); undertow.setUseGlobalSslContextParameters(true); camelContext.addComponent("undertow", undertow); } {code} Here the HTTP binding will be replaced with the default RestUndertowHttpBinding if no endpoint property is specified in the restConfiguration itself, maybe this also applies for other parameters as well. > Configuring SSL via endpointProperty in restConfiguration for an Undertow > component will cause a BindException for multiple endpoints > - > > Key: CAMEL-11998 > URL: https://issues.apache.org/jira/browse/CAMEL-11998 > Project: Camel > Issue Type: Bug > Components: camel-undertow >Affects Versions: 2.19.0, 2.20.0 > Environment: MacBookPro 15' with Mac OS X (10.12.6), Oracle Java > 1.8.144 >Reporter: Roman Vottner > > On recreating the test case depicted in > https://github.com/apache/camel/blob/master/components/camel-undertow/src/test/java/org/apache/camel/component/undertow/rest/RestApiUndertowTest.java > and activating SSL configuration, I noticed that multiple endpoints exposed > will configure multiple Undertow consumers which all try to bind onto the > same port and thus throw a BindException. > The full Test-Code looks as follows: > {code:java} > import static org.hamcrest.CoreMatchers.equalTo; > import static org.hamcrest.CoreMatchers.is; > import at.rovo.utils.TestHttpClient; > import io.undertow.server.HttpServerExchange; > import java.net.InetSocketAddress; > import java.net.URL; > import java.util.ArrayList; > import java.util.List; > import java.util.Map; > import javax.annotation.Resource; > import javax.net.ssl.SSLSession; > import org.apache.camel.CamelContext; > import org.apache.camel.Exchange; > import org.apache.camel.builder.RouteBuilder; > import org.apache.camel.component.properties.PropertiesComponent; > import org.apache.camel.component.undertow.RestUndertowHttpBinding; > import org.apache.camel.component.undertow.UndertowHttpBinding; > import org.apache.camel.model.rest.RestParamType; > import org.apache.camel.spring.javaconfig.CamelConfiguration; > import org.apache.camel.test.spring.CamelSpringRunner; > import org.apache.camel.test.spring.CamelTestContextBootstrapper; > import org.apache.camel.util.jsse.KeyManagersParameters; > import org.apache.camel.util.jsse.KeyStoreParameters; > import org.apache.camel.util.jsse.SSLContextParameters; > import org.apache.camel.util.jsse.TrustManagersParameters; > import org.junit.Assert; > import org.junit.Test; > import org.junit.runner.RunWith; > import org.springframework.context.annotation.Bean; > import org.springframework.context.annotation.Configuration; > import org.springframework.context.annotation.PropertySource; > import org.springframework.core.env.Environment; > import org.springframework.test.context.BootstrapWith; > import org.springframework.test.context.ContextConfiguration; > import org.springframework.test.context.support.AnnotationConfigContextLoader; > @RunWith(CamelSpringRunner.class) > @BootstrapWith(CamelTestContextBootstrapper.class) > @ContextConfiguration(loader = AnnotationConfigContextLoader.class, > classes = {UndertowRouteTest.ContextConfig.class}) > public class UndertowRouteTest { > private static final String REMOTE_IP = "REMOTE_IP"; > private static final String REMOTE_PORT = "REMOTE_PORT"; > private static final String USER_AGENT = "USER_AGENT"; > private static final String TLS_PROTOCOL_VERSION = "TLS_PROTOCOL_VERSION"; > private static final String TLS_CIPHER_SUITE = "TLS_CIPHER_SUITE"; > @Configuration > @PropertySource({"classpath:test.properties"}) > public static class ContextConfig extends CamelConfiguration { > @Resource > private Environment env; > @Bean(name = "undertowHttpBinding") > public UndertowHttpBinding undertowHttpBinding() { > return new IPResolvableRestUndertowHttpBinding(); > } > public static class IPResolvableRestUndertowHttpBinding extends > RestUndertowHttpBinding { > @Override > public void populateCamelHeaders(HttpServerExchange httpExchange, > Map headersMap, Exc
[jira] [Updated] (CAMEL-11998) Configuring SSL via endpointProperty in restConfiguration for an Undertow component will cause a BindException for multiple endpoints
[ https://issues.apache.org/jira/browse/CAMEL-11998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Vottner updated CAMEL-11998: -- Description: On recreating the test case depicted in https://github.com/apache/camel/blob/master/components/camel-undertow/src/test/java/org/apache/camel/component/undertow/rest/RestApiUndertowTest.java and activating SSL configuration, I noticed that multiple endpoints exposed will configure multiple Undertow consumers which all try to bind onto the same port and thus throw a BindException. The full Test-Code looks as follows: {code:java} import static org.hamcrest.CoreMatchers.equalTo; import static org.hamcrest.CoreMatchers.is; import at.rovo.utils.TestHttpClient; import io.undertow.server.HttpServerExchange; import java.net.InetSocketAddress; import java.net.URL; import java.util.ArrayList; import java.util.List; import java.util.Map; import javax.annotation.Resource; import javax.net.ssl.SSLSession; import org.apache.camel.CamelContext; import org.apache.camel.Exchange; import org.apache.camel.builder.RouteBuilder; import org.apache.camel.component.properties.PropertiesComponent; import org.apache.camel.component.undertow.RestUndertowHttpBinding; import org.apache.camel.component.undertow.UndertowHttpBinding; import org.apache.camel.model.rest.RestParamType; import org.apache.camel.spring.javaconfig.CamelConfiguration; import org.apache.camel.test.spring.CamelSpringRunner; import org.apache.camel.test.spring.CamelTestContextBootstrapper; import org.apache.camel.util.jsse.KeyManagersParameters; import org.apache.camel.util.jsse.KeyStoreParameters; import org.apache.camel.util.jsse.SSLContextParameters; import org.apache.camel.util.jsse.TrustManagersParameters; import org.junit.Assert; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.PropertySource; import org.springframework.core.env.Environment; import org.springframework.test.context.BootstrapWith; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.support.AnnotationConfigContextLoader; @RunWith(CamelSpringRunner.class) @BootstrapWith(CamelTestContextBootstrapper.class) @ContextConfiguration(loader = AnnotationConfigContextLoader.class, classes = {UndertowRouteTest.ContextConfig.class}) public class UndertowRouteTest { private static final String REMOTE_IP = "REMOTE_IP"; private static final String REMOTE_PORT = "REMOTE_PORT"; private static final String USER_AGENT = "USER_AGENT"; private static final String TLS_PROTOCOL_VERSION = "TLS_PROTOCOL_VERSION"; private static final String TLS_CIPHER_SUITE = "TLS_CIPHER_SUITE"; @Configuration @PropertySource({"classpath:test.properties"}) public static class ContextConfig extends CamelConfiguration { @Resource private Environment env; @Bean(name = "undertowHttpBinding") public UndertowHttpBinding undertowHttpBinding() { return new IPResolvableRestUndertowHttpBinding(); } public static class IPResolvableRestUndertowHttpBinding extends RestUndertowHttpBinding { @Override public void populateCamelHeaders(HttpServerExchange httpExchange, Map headersMap, Exchange exchange) throws Exception { super.populateCamelHeaders(httpExchange, headersMap, exchange); InetSocketAddress peer = httpExchange.getSourceAddress(); headersMap.put(REMOTE_IP, peer.getAddress().getHostAddress()); headersMap.put(REMOTE_PORT, peer.getPort()); for (String key : headersMap.keySet()) { if (key.toLowerCase().equals("user-agent")) { headersMap.put(USER_AGENT, headersMap.get(key)); } } if (null != httpExchange.getConnection().getSslSessionInfo() && null != httpExchange.getConnection().getSslSessionInfo().getSSLSession()) { SSLSession sslSession = httpExchange.getConnection().getSslSessionInfo().getSSLSession(); exchange.getIn().setHeader(TLS_PROTOCOL_VERSION, sslSession.getProtocol()); exchange.getIn().setHeader(TLS_CIPHER_SUITE, sslSession.getCipherSuite()); } } } @Bean(name = "sslContextParameters") public SSLContextParameters sslContextParameters() { String keyStore = env.getProperty("ssl.keyStore.resource"); URL keyStoreUrl = this.getClass().getResource(keyStore); KeyStoreParameters ksp = new KeyStoreParameters(); ksp.setResource(keyStoreUrl.getPath()); ksp.setPassword(env.getProperty("ssl.keyStore.password")); KeyManagersParameters kmp = new KeyManagersParameters(); kmp.setKeyStore(ksp); kmp.setKeyPassword(env.getProperty("ssl.key.password")); String trustStore = env.getProperty("ssl.trustStore.resource"); URL trustStoreUrl = th
[jira] [Created] (CAMEL-11998) Configuring SSL via endpointProperty in restConfiguration for an Undertow component will cause a BindException for multiple endpoints
Roman Vottner created CAMEL-11998: - Summary: Configuring SSL via endpointProperty in restConfiguration for an Undertow component will cause a BindException for multiple endpoints Key: CAMEL-11998 URL: https://issues.apache.org/jira/browse/CAMEL-11998 Project: Camel Issue Type: Bug Components: camel-undertow Affects Versions: 2.20.0, 2.19.0 Environment: MacBookPro 15' with Mac OS X (10.12.6), Oracle Java 1.8.144 Reporter: Roman Vottner On recreating the test case depicted in https://github.com/apache/camel/blob/master/components/camel-undertow/src/test/java/org/apache/camel/component/undertow/rest/RestApiUndertowTest.java and activating SSL configuration, I noticed that multiple endpoints exposed will configure multiple Undertow consumers which all try to bind onto the same port and thus throw a BindException. The full Test-Code looks as follows: {code:java} import static org.hamcrest.CoreMatchers.equalTo; import static org.hamcrest.CoreMatchers.is; import at.rovo.utils.TestHttpClient; import io.undertow.server.HttpServerExchange; import java.net.InetSocketAddress; import java.net.URL; import java.util.ArrayList; import java.util.List; import java.util.Map; import javax.annotation.Resource; import javax.net.ssl.SSLSession; import org.apache.camel.CamelContext; import org.apache.camel.Exchange; import org.apache.camel.builder.RouteBuilder; import org.apache.camel.component.properties.PropertiesComponent; import org.apache.camel.component.undertow.RestUndertowHttpBinding; import org.apache.camel.component.undertow.UndertowHttpBinding; import org.apache.camel.model.rest.RestParamType; import org.apache.camel.spring.javaconfig.CamelConfiguration; import org.apache.camel.test.spring.CamelSpringRunner; import org.apache.camel.test.spring.CamelTestContextBootstrapper; import org.apache.camel.util.jsse.KeyManagersParameters; import org.apache.camel.util.jsse.KeyStoreParameters; import org.apache.camel.util.jsse.SSLContextParameters; import org.apache.camel.util.jsse.TrustManagersParameters; import org.junit.Assert; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.PropertySource; import org.springframework.core.env.Environment; import org.springframework.test.context.BootstrapWith; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.support.AnnotationConfigContextLoader; @RunWith(CamelSpringRunner.class) @BootstrapWith(CamelTestContextBootstrapper.class) @ContextConfiguration(loader = AnnotationConfigContextLoader.class, classes = {UndertowRouteTest.ContextConfig.class}) public class UndertowRouteTest { private static final String REMOTE_IP = "REMOTE_IP"; private static final String REMOTE_PORT = "REMOTE_PORT"; private static final String USER_AGENT = "USER_AGENT"; private static final String TLS_PROTOCOL_VERSION = "TLS_PROTOCOL_VERSION"; private static final String TLS_CIPHER_SUITE = "TLS_CIPHER_SUITE"; @Configuration @PropertySource({"classpath:services-test.properties"}) public static class ContextConfig extends CamelConfiguration { @Resource private Environment env; @Bean(name = "undertowHttpBinding") public UndertowHttpBinding undertowHttpBinding() { return new IPResolvableRestUndertowHttpBinding(); } public static class IPResolvableRestUndertowHttpBinding extends RestUndertowHttpBinding { @Override public void populateCamelHeaders(HttpServerExchange httpExchange, Map headersMap, Exchange exchange) throws Exception { super.populateCamelHeaders(httpExchange, headersMap, exchange); InetSocketAddress peer = httpExchange.getSourceAddress(); headersMap.put(REMOTE_IP, peer.getAddress().getHostAddress()); headersMap.put(REMOTE_PORT, peer.getPort()); for (String key : headersMap.keySet()) { if (key.toLowerCase().equals("user-agent")) { headersMap.put(USER_AGENT, headersMap.get(key)); } } if (null != httpExchange.getConnection().getSslSessionInfo() && null != httpExchange.getConnection().getSslSessionInfo().getSSLSession()) { SSLSession sslSession = httpExchange.getConnection().getSslSessionInfo().getSSLSession(); exchange.getIn().setHeader(TLS_PROTOCOL_VERSION, sslSession.getProtocol()); exchange.getIn().setHeader(TLS_CIPHER_SUITE, sslSession.getCipherSuite()); } } } @Bean(name = "sslContextParameters") public SSLContextParameters sslContextParameters() { String keyStore = env.getProperty("ssl.keyStore.resource"); URL keyStoreUrl = this.getClass().getResource(keyStore); KeyStoreParameters ksp = new KeyStoreP
[jira] [Commented] (CAMEL-11482) SSLContextParameters settings are not properly copied to SslContextFactory
[ https://issues.apache.org/jira/browse/CAMEL-11482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16180661#comment-16180661 ] Roman Vottner commented on CAMEL-11482: --- Test for exclusion pattern. Both tests are green though you might notice a stacktrace on executing testHelloEndpoint as no server certificate is obtainable: https://github.com/RovoMe/camel/blob/fix/CAMEL-11482_SSLContextParameters_settings_are_not_properly_copied_to_SslContextFactory/components/camel-jetty9/src/test/java/org/apache/camel/component/jetty/HttpsRouteSslContextParametersInUriTest_TLSv1SupportWithExclusionPattern.java Test for defining explicite ciphers to support. Though as it turns out Jettys exclusion patterns seem to kick in before and hence remove all of the TLSv1/1.1 ciphers: https://github.com/RovoMe/camel/blob/fix/CAMEL-11482_SSLContextParameters_settings_are_not_properly_copied_to_SslContextFactory/components/camel-jetty9/src/test/java/org/apache/camel/component/jetty/HttpsRouteSslContextParametersInUriTest_TLSv1SupportWithInclusionPattern.java HTH > SSLContextParameters settings are not properly copied to SslContextFactory > -- > > Key: CAMEL-11482 > URL: https://issues.apache.org/jira/browse/CAMEL-11482 > Project: Camel > Issue Type: Bug > Components: camel-jetty >Affects Versions: 2.19.0, 2.19.1 > Environment: Max OS X, Java 8 Update 131 > Ubuntu 14.04 LTS, Java 8 Update 111 > Camel 2.19.0 > Jetty9 9.4.5v20170502 and 9.3.14.v20161028 >Reporter: Roman Vottner >Assignee: Claus Ibsen > Fix For: 2.19.4, 2.20.0 > > > Jetty 9.3+ excludes unsecure ciphers which end on either MD5, SHA or SHA1 by > default now. This will however remove all ciphers that are used by either > TLSv1 or TLSv1.1 and thus no ciphers remain in order to agree on a cipher for > TLSv1 or TLSv1.1 connection attempts. (Further reading: > https://github.com/eclipse/jetty.project/issues/860) > The Jetty 9 SSL configuration documentation > (https://www.eclipse.org/jetty/documentation/9.3.x/configuring-ssl.html) > states that this exclusion cipher suites can be customized by providing an > own exclusion list. On specifying SSLContextParameters like below however > will not correctly propagate this exclution cipher suites to the > SslContextFactory of Jetty and thus use the default setting which prevents > TLSv1 and TLSv1.1 connections. > {code:title=SSLContextParameters Spring Config|borderStyle=solid} > @Bean(name = "sslContextParameters") > public SSLContextParameters sslContextParameters() { > String keyStore = env.getProperty("ssl.keyStore.resource"); > URL keyStoreUrl = this.getClass().getResource(keyStore); > // http://camel.apache.org/jetty.html > KeyStoreParameters ksp = new KeyStoreParameters(); > ksp.setResource(keyStoreUrl.getPath()); > ksp.setPassword(env.getProperty("ssl.keyStore.password")); > KeyManagersParameters kmp = new KeyManagersParameters(); > kmp.setKeyStore(ksp); > kmp.setKeyPassword(env.getProperty("ssl.key.password")); > SSLContextParameters scp = new SSLContextParameters(); > scp.setKeyManagers(kmp); > // Jetty 9.3+ support only TLSv1.2 by default hence clients not > supporting this protocol will fail > List supportedSslProtocols = Arrays.asList("TLSv1", "TLSv1.1", > "TLSv1.2"); > SecureSocketProtocolsParameters protocolsParameters = new > SecureSocketProtocolsParameters(); > protocolsParameters.setSecureSocketProtocol(supportedSslProtocols); > scp.setSecureSocketProtocols(protocolsParameters); > // TLS 1.0 / 1.1 have been disabled by jetty 9.3 > // this is a first attempt to re-enable them > // see > // - > https://www.eclipse.org/jetty/documentation/9.3.x/configuring-ssl.html > // - https://github.com/eclipse/jetty.project/issues/860 > // - http://camel.apache.org/camel-configuration-utilities.html > FilterParameters cipherParameters = new FilterParameters(); > cipherParameters.getInclude().add(".*"); > cipherParameters.getExclude().add("^.*_(MD5|SHA1)$"); > scp.setCipherSuitesFilter(cipherParameters); > return scp; > } > {code} > A workaround is to use a custom JettyHttpComponent9 implementation that sets > the excludedCipherSuites manually like depicted below: > {code:title=Workaround|borderStyle=solid} > /** >* A custom jetty http component which explicitly sets the > excludedCipherSuites during creation of >* the jetty connector. >* >* Why? It seems camel does not push included/excluded cipherSuites from > {@link >* SSLContextParameters} to the {@link SslContextFactory} nor does push > explicitly listed cipher >* suites (i.e. like TLS_RSA_WITH_AES_256_CBC_SHA) to the Jetty > SSL context factory. >
[jira] [Commented] (CAMEL-11482) SSLContextParameters settings are not properly copied to SslContextFactory
[ https://issues.apache.org/jira/browse/CAMEL-11482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16180616#comment-16180616 ] Roman Vottner commented on CAMEL-11482: --- Thanks for taking over as I'm currently not having enough time lately. I've some (WIP) tests for the changes I had in my branch if you are interested. They basically test that TLSv1 or TLSv1.1 connections work as they should via Java's HttpUrlConnection. The only thing that bugs me, is, that I am not able to retrieve the server certificate but I'm not sure if this is related to self-signed certificates. > SSLContextParameters settings are not properly copied to SslContextFactory > -- > > Key: CAMEL-11482 > URL: https://issues.apache.org/jira/browse/CAMEL-11482 > Project: Camel > Issue Type: Bug > Components: camel-jetty >Affects Versions: 2.19.0, 2.19.1 > Environment: Max OS X, Java 8 Update 131 > Ubuntu 14.04 LTS, Java 8 Update 111 > Camel 2.19.0 > Jetty9 9.4.5v20170502 and 9.3.14.v20161028 >Reporter: Roman Vottner >Assignee: Claus Ibsen > Fix For: 2.19.4, 2.20.0 > > > Jetty 9.3+ excludes unsecure ciphers which end on either MD5, SHA or SHA1 by > default now. This will however remove all ciphers that are used by either > TLSv1 or TLSv1.1 and thus no ciphers remain in order to agree on a cipher for > TLSv1 or TLSv1.1 connection attempts. (Further reading: > https://github.com/eclipse/jetty.project/issues/860) > The Jetty 9 SSL configuration documentation > (https://www.eclipse.org/jetty/documentation/9.3.x/configuring-ssl.html) > states that this exclusion cipher suites can be customized by providing an > own exclusion list. On specifying SSLContextParameters like below however > will not correctly propagate this exclution cipher suites to the > SslContextFactory of Jetty and thus use the default setting which prevents > TLSv1 and TLSv1.1 connections. > {code:title=SSLContextParameters Spring Config|borderStyle=solid} > @Bean(name = "sslContextParameters") > public SSLContextParameters sslContextParameters() { > String keyStore = env.getProperty("ssl.keyStore.resource"); > URL keyStoreUrl = this.getClass().getResource(keyStore); > // http://camel.apache.org/jetty.html > KeyStoreParameters ksp = new KeyStoreParameters(); > ksp.setResource(keyStoreUrl.getPath()); > ksp.setPassword(env.getProperty("ssl.keyStore.password")); > KeyManagersParameters kmp = new KeyManagersParameters(); > kmp.setKeyStore(ksp); > kmp.setKeyPassword(env.getProperty("ssl.key.password")); > SSLContextParameters scp = new SSLContextParameters(); > scp.setKeyManagers(kmp); > // Jetty 9.3+ support only TLSv1.2 by default hence clients not > supporting this protocol will fail > List supportedSslProtocols = Arrays.asList("TLSv1", "TLSv1.1", > "TLSv1.2"); > SecureSocketProtocolsParameters protocolsParameters = new > SecureSocketProtocolsParameters(); > protocolsParameters.setSecureSocketProtocol(supportedSslProtocols); > scp.setSecureSocketProtocols(protocolsParameters); > // TLS 1.0 / 1.1 have been disabled by jetty 9.3 > // this is a first attempt to re-enable them > // see > // - > https://www.eclipse.org/jetty/documentation/9.3.x/configuring-ssl.html > // - https://github.com/eclipse/jetty.project/issues/860 > // - http://camel.apache.org/camel-configuration-utilities.html > FilterParameters cipherParameters = new FilterParameters(); > cipherParameters.getInclude().add(".*"); > cipherParameters.getExclude().add("^.*_(MD5|SHA1)$"); > scp.setCipherSuitesFilter(cipherParameters); > return scp; > } > {code} > A workaround is to use a custom JettyHttpComponent9 implementation that sets > the excludedCipherSuites manually like depicted below: > {code:title=Workaround|borderStyle=solid} > /** >* A custom jetty http component which explicitly sets the > excludedCipherSuites during creation of >* the jetty connector. >* >* Why? It seems camel does not push included/excluded cipherSuites from > {@link >* SSLContextParameters} to the {@link SslContextFactory} nor does push > explicitly listed cipher >* suites (i.e. like TLS_RSA_WITH_AES_256_CBC_SHA) to the Jetty > SSL context factory. >*/ > public static class HackedJettyHttpComponent extends JettyHttpComponent9 { > @Override > protected AbstractConnector createConnectorJettyInternal(Server server, > > JettyHttpEndpoint endpoint, > > SslContextFactory sslcf) { > sslcf.setExcludeCipherSuites("^.*_(MD5|SHA1)$"); > return super.createConnectorJettyInternal(serv
[jira] [Commented] (CAMEL-11482) SSLContextParameters settings are not properly copied to SslContextFactory
[ https://issues.apache.org/jira/browse/CAMEL-11482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073965#comment-16073965 ] Roman Vottner commented on CAMEL-11482: --- Just a quick headsup on this issue. I've started to work on a fix (https://github.com/RovoMe/camel/commit/952175559110babf0ee7224f8045c1270dad5aa7) though I'm not sure if the code has to support Java 7 (or even 6). Also, there are probably a couple other settings that aren't copied to the SslContextFactory which I have not yet included either. What should be the strategy on these? Continue work on that issue and copy over all settings or leave them to those who need them and ask them to provide a fix/PR? Will also check how to setup unit-tests therefore, though as the method is private I guess I have to test it within createConnector(Server, JettyHttpEndpoint) and/or createHttpClient(JettyHttpEndpoint, Integer, SSLContextParameters) which use the updated method. > SSLContextParameters settings are not properly copied to SslContextFactory > -- > > Key: CAMEL-11482 > URL: https://issues.apache.org/jira/browse/CAMEL-11482 > Project: Camel > Issue Type: Bug > Components: camel-jetty >Affects Versions: 2.19.0, 2.19.1 > Environment: Max OS X, Java 8 Update 131 > Ubuntu 14.04 LTS, Java 8 Update 111 > Camel 2.19.0 > Jetty9 9.4.5v20170502 and 9.3.14.v20161028 >Reporter: Roman Vottner > Fix For: 2.19.2, 2.20.0 > > > Jetty 9.3+ excludes unsecure ciphers which end on either MD5, SHA or SHA1 by > default now. This will however remove all ciphers that are used by either > TLSv1 or TLSv1.1 and thus no ciphers remain in order to agree on a cipher for > TLSv1 or TLSv1.1 connection attempts. (Further reading: > https://github.com/eclipse/jetty.project/issues/860) > The Jetty 9 SSL configuration documentation > (https://www.eclipse.org/jetty/documentation/9.3.x/configuring-ssl.html) > states that this exclusion cipher suites can be customized by providing an > own exclusion list. On specifying SSLContextParameters like below however > will not correctly propagate this exclution cipher suites to the > SslContextFactory of Jetty and thus use the default setting which prevents > TLSv1 and TLSv1.1 connections. > {code:title=SSLContextParameters Spring Config|borderStyle=solid} > @Bean(name = "sslContextParameters") > public SSLContextParameters sslContextParameters() { > String keyStore = env.getProperty("ssl.keyStore.resource"); > URL keyStoreUrl = this.getClass().getResource(keyStore); > // http://camel.apache.org/jetty.html > KeyStoreParameters ksp = new KeyStoreParameters(); > ksp.setResource(keyStoreUrl.getPath()); > ksp.setPassword(env.getProperty("ssl.keyStore.password")); > KeyManagersParameters kmp = new KeyManagersParameters(); > kmp.setKeyStore(ksp); > kmp.setKeyPassword(env.getProperty("ssl.key.password")); > SSLContextParameters scp = new SSLContextParameters(); > scp.setKeyManagers(kmp); > // Jetty 9.3+ support only TLSv1.2 by default hence clients not > supporting this protocol will fail > List supportedSslProtocols = Arrays.asList("TLSv1", "TLSv1.1", > "TLSv1.2"); > SecureSocketProtocolsParameters protocolsParameters = new > SecureSocketProtocolsParameters(); > protocolsParameters.setSecureSocketProtocol(supportedSslProtocols); > scp.setSecureSocketProtocols(protocolsParameters); > // TLS 1.0 / 1.1 have been disabled by jetty 9.3 > // this is a first attempt to re-enable them > // see > // - > https://www.eclipse.org/jetty/documentation/9.3.x/configuring-ssl.html > // - https://github.com/eclipse/jetty.project/issues/860 > // - http://camel.apache.org/camel-configuration-utilities.html > FilterParameters cipherParameters = new FilterParameters(); > cipherParameters.getInclude().add(".*"); > cipherParameters.getExclude().add("^.*_(MD5|SHA1)$"); > scp.setCipherSuitesFilter(cipherParameters); > return scp; > } > {code} > A workaround is to use a custom JettyHttpComponent9 implementation that sets > the excludedCipherSuites manually like depicted below: > {code:title=Workaround|borderStyle=solid} > /** >* A custom jetty http component which explicitly sets the > excludedCipherSuites during creation of >* the jetty connector. >* >* Why? It seems camel does not push included/excluded cipherSuites from > {@link >* SSLContextParameters} to the {@link SslContextFactory} nor does push > explicitly listed cipher >* suites (i.e. like TLS_RSA_WITH_AES_256_CBC_SHA) to the Jetty > SSL context factory. >*/ > public static class HackedJettyHttpComponent extends JettyHttpComponent9 { > @Override > protected AbstractConnector create
[jira] [Commented] (CAMEL-11482) SSLContextParameters settings are not properly copied to SslContextFactory
[ https://issues.apache.org/jira/browse/CAMEL-11482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072151#comment-16072151 ] Roman Vottner commented on CAMEL-11482: --- Have to see if I am able to find some time, but our upcoming release has higher priorities. > SSLContextParameters settings are not properly copied to SslContextFactory > -- > > Key: CAMEL-11482 > URL: https://issues.apache.org/jira/browse/CAMEL-11482 > Project: Camel > Issue Type: Bug > Components: camel-jetty >Affects Versions: 2.19.0, 2.19.1 > Environment: Max OS X, Java 8 Update 131 > Ubuntu 14.04 LTS, Java 8 Update 111 > Camel 2.19.0 > Jetty9 9.4.5v20170502 and 9.3.14.v20161028 >Reporter: Roman Vottner > Fix For: 2.19.2, 2.20.0 > > > Jetty 9.3+ excludes unsecure ciphers which end on either MD5, SHA or SHA1 by > default now. This will however remove all ciphers that are used by either > TLSv1 or TLSv1.1 and thus no ciphers remain in order to agree on a cipher for > TLSv1 or TLSv1.1 connection attempts. (Further reading: > https://github.com/eclipse/jetty.project/issues/860) > The Jetty 9 SSL configuration documentation > (https://www.eclipse.org/jetty/documentation/9.3.x/configuring-ssl.html) > states that this exclusion cipher suites can be customized by providing an > own exclusion list. On specifying SSLContextParameters like below however > will not correctly propagate this exclution cipher suites to the > SslContextFactory of Jetty and thus use the default setting which prevents > TLSv1 and TLSv1.1 connections. > {code:title=SSLContextParameters Spring Config|borderStyle=solid} > @Bean(name = "sslContextParameters") > public SSLContextParameters sslContextParameters() { > String keyStore = env.getProperty("ssl.keyStore.resource"); > URL keyStoreUrl = this.getClass().getResource(keyStore); > // http://camel.apache.org/jetty.html > KeyStoreParameters ksp = new KeyStoreParameters(); > ksp.setResource(keyStoreUrl.getPath()); > ksp.setPassword(env.getProperty("ssl.keyStore.password")); > KeyManagersParameters kmp = new KeyManagersParameters(); > kmp.setKeyStore(ksp); > kmp.setKeyPassword(env.getProperty("ssl.key.password")); > SSLContextParameters scp = new SSLContextParameters(); > scp.setKeyManagers(kmp); > // Jetty 9.3+ support only TLSv1.2 by default hence clients not > supporting this protocol will fail > List supportedSslProtocols = Arrays.asList("TLSv1", "TLSv1.1", > "TLSv1.2"); > SecureSocketProtocolsParameters protocolsParameters = new > SecureSocketProtocolsParameters(); > protocolsParameters.setSecureSocketProtocol(supportedSslProtocols); > scp.setSecureSocketProtocols(protocolsParameters); > // TLS 1.0 / 1.1 have been disabled by jetty 9.3 > // this is a first attempt to re-enable them > // see > // - > https://www.eclipse.org/jetty/documentation/9.3.x/configuring-ssl.html > // - https://github.com/eclipse/jetty.project/issues/860 > // - http://camel.apache.org/camel-configuration-utilities.html > FilterParameters cipherParameters = new FilterParameters(); > cipherParameters.getInclude().add(".*"); > cipherParameters.getExclude().add("^.*_(MD5|SHA1)$"); > scp.setCipherSuitesFilter(cipherParameters); > return scp; > } > {code} > A workaround is to use a custom JettyHttpComponent9 implementation that sets > the excludedCipherSuites manually like depicted below: > {code:title=Workaround|borderStyle=solid} > /** >* A custom jetty http component which explicitly sets the > excludedCipherSuites during creation of >* the jetty connector. >* >* Why? It seems camel does not push included/excluded cipherSuites from > {@link >* SSLContextParameters} to the {@link SslContextFactory} nor does push > explicitly listed cipher >* suites (i.e. like TLS_RSA_WITH_AES_256_CBC_SHA) to the Jetty > SSL context factory. >*/ > public static class HackedJettyHttpComponent extends JettyHttpComponent9 { > @Override > protected AbstractConnector createConnectorJettyInternal(Server server, > > JettyHttpEndpoint endpoint, > > SslContextFactory sslcf) { > sslcf.setExcludeCipherSuites("^.*_(MD5|SHA1)$"); > return super.createConnectorJettyInternal(server, endpoint, sslcf); > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CAMEL-11482) SSLContextParameters settings are not properly copied to SslContextFactory
Roman Vottner created CAMEL-11482: - Summary: SSLContextParameters settings are not properly copied to SslContextFactory Key: CAMEL-11482 URL: https://issues.apache.org/jira/browse/CAMEL-11482 Project: Camel Issue Type: Bug Components: camel-jetty Affects Versions: 2.19.1, 2.19.0 Environment: Max OS X, Java 8 Update 131 Ubuntu 14.04 LTS, Java 8 Update 111 Camel 2.19.0 Jetty9 9.4.5v20170502 and 9.3.14.v20161028 Reporter: Roman Vottner Priority: Critical Jetty 9.3+ excludes unsecure ciphers which end on either MD5, SHA or SHA1 by default now. This will however remove all ciphers that are used by either TLSv1 or TLSv1.1 and thus no ciphers remain in order to agree on a cipher for TLSv1 or TLSv1.1 connection attempts. (Further reading: https://github.com/eclipse/jetty.project/issues/860) The Jetty 9 SSL configuration documentation (https://www.eclipse.org/jetty/documentation/9.3.x/configuring-ssl.html) states that this exclusion cipher suites can be customized by providing an own exclusion list. On specifying SSLContextParameters like below however will not correctly propagate this exclution cipher suites to the SslContextFactory of Jetty and thus use the default setting which prevents TLSv1 and TLSv1.1 connections. {code:title=SSLContextParameters Spring Config|borderStyle=solid} @Bean(name = "sslContextParameters") public SSLContextParameters sslContextParameters() { String keyStore = env.getProperty("ssl.keyStore.resource"); URL keyStoreUrl = this.getClass().getResource(keyStore); // http://camel.apache.org/jetty.html KeyStoreParameters ksp = new KeyStoreParameters(); ksp.setResource(keyStoreUrl.getPath()); ksp.setPassword(env.getProperty("ssl.keyStore.password")); KeyManagersParameters kmp = new KeyManagersParameters(); kmp.setKeyStore(ksp); kmp.setKeyPassword(env.getProperty("ssl.key.password")); SSLContextParameters scp = new SSLContextParameters(); scp.setKeyManagers(kmp); // Jetty 9.3+ support only TLSv1.2 by default hence clients not supporting this protocol will fail List supportedSslProtocols = Arrays.asList("TLSv1", "TLSv1.1", "TLSv1.2"); SecureSocketProtocolsParameters protocolsParameters = new SecureSocketProtocolsParameters(); protocolsParameters.setSecureSocketProtocol(supportedSslProtocols); scp.setSecureSocketProtocols(protocolsParameters); // TLS 1.0 / 1.1 have been disabled by jetty 9.3 // this is a first attempt to re-enable them // see // - https://www.eclipse.org/jetty/documentation/9.3.x/configuring-ssl.html // - https://github.com/eclipse/jetty.project/issues/860 // - http://camel.apache.org/camel-configuration-utilities.html FilterParameters cipherParameters = new FilterParameters(); cipherParameters.getInclude().add(".*"); cipherParameters.getExclude().add("^.*_(MD5|SHA1)$"); scp.setCipherSuitesFilter(cipherParameters); return scp; } {code} A workaround is to use a custom JettyHttpComponent9 implementation that sets the excludedCipherSuites manually like depicted below: {code:title=Workaround|borderStyle=solid} /** * A custom jetty http component which explicitly sets the excludedCipherSuites during creation of * the jetty connector. * * Why? It seems camel does not push included/excluded cipherSuites from {@link * SSLContextParameters} to the {@link SslContextFactory} nor does push explicitly listed cipher * suites (i.e. like TLS_RSA_WITH_AES_256_CBC_SHA) to the Jetty SSL context factory. */ public static class HackedJettyHttpComponent extends JettyHttpComponent9 { @Override protected AbstractConnector createConnectorJettyInternal(Server server, JettyHttpEndpoint endpoint, SslContextFactory sslcf) { sslcf.setExcludeCipherSuites("^.*_(MD5|SHA1)$"); return super.createConnectorJettyInternal(server, endpoint, sslcf); } } {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CAMEL-6848) Enable vararg support for beaninvocation with Simple
[ https://issues.apache.org/jira/browse/CAMEL-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13831496#comment-13831496 ] Roman Vottner commented on CAMEL-6848: -- If you have a predefined class changing the internals will be hard to achieve unless you employ a byte-manipulation framework like f.e javassist. An other solution would be to create a new class which has access to the exchange and invokes the preexisting class from within the specified method - but this is just more boilerplate code. As far as I have understood the concept, the .bean(class) or .bean(class, String) methods are intended to make use of already existing POJOs so modifying the internals of that class to inject @Headers Map headers or Exchange exchange is just a misuse of the bean concept - processors are there therefore IMHO. If, f.e. I'd like to store received REST path and/or query parameters into a DB, your suggestion would be to use a bean that injects the exchange and extracts the received parameters from the header - but what if not all but only a specified selection of parameters are really of interest? Extend the class for every defined route and adept it to your needs when all there would be necessary IMHO is to have one single bean that could receive a vararg via SIMPLE language (f.e. .bean(ParamExtractor.class, "${ queryParams(new String[] {'param1', 'param3', 'param5'})}"). Of course you could write the method to only accept one parameter and invoke the bean for every parameter of interest - but this would also create a SELECT and/or UPDATE statement per invocation when actually there might only be one necessary. If I'm not seeing the forest due to all the trees around please point me to a source where I can see how to specify a single bean invocation that achieves the above task. The example presented might not be the best case, but as I mentioned in the OP, when dealing with resource-intensive operations (like DB f.e) a single bean invocation would be superior to multiple invocations. And I prefer a generic approach to a lot of boilerplate code. > Enable vararg support for beaninvocation with Simple > - > > Key: CAMEL-6848 > URL: https://issues.apache.org/jira/browse/CAMEL-6848 > Project: Camel > Issue Type: Improvement > Components: bean-integration >Affects Versions: 2.11.1 >Reporter: Roman Vottner >Priority: Minor > Labels: Simple, beaninvocation, vararg > > Having a simple bean method that accepts multiple String parameters declared > as varargs, I'm having issues sending values from .bean(MyBean.class, "${ > foo('a','b') }") or .bean(MyBean.class, "${ foo(new String[] {'a', 'b'}) }") > to my bean directly. The error states that it can't convert from String to > String[] - but even with declaring a String[] (like the second bean > invocation) it is not able to execute the bean. > The bean class simply looks like this: > public MyBean > { > public void foo(String ... param) > { > // do some stuff ... > } > } > I'd need to specify multiple parameters and to avoid creating multiple > classes or methods which are simply copy&paste classes with small adaptions > (inheritance is in place) I'd love to have a more generic approach in place. > If resource intensive calculations are done in the back running the method > multiple times with changed parameters may not be the best solution imho. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CAMEL-6848) Enable vararg support for beaninvocation with Simple
[ https://issues.apache.org/jira/browse/CAMEL-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Vottner updated CAMEL-6848: - Description: Having a simple bean method that accepts multiple String parameters declared as varargs, I'm having issues sending values from .bean(MyBean.class, "${ foo('a','b') }") or .bean(MyBean.class, "${ foo(new String[] {'a', 'b'}) }") to my bean directly. The error states that it can't convert from String to String[] - but even with declaring a String[] (like the second bean invocation) it is not able to execute the bean. The bean class simply looks like this: public MyBean { public void foo(String ... param) { // do some stuff ... } } I'd need to specify multiple parameters and to avoid creating multiple classes or methods which are simply copy&paste classes with small adaptions (inheritance is in place) I'd love to have a more generic approach in place. If resource intensive calculations are done in the back running the method multiple times with changed parameters may not be the best solution imho. was: Having a simple bean method that accepts multiple String parameters declared as varargs, I'm having issues sending values from .bean(MyBean.class, "${ foo('a','b') }") or .bean(MyBean.class, "${ foo(new String[] {'a', 'b'}) }) to my bean directly. The error states that it can't convert from String to String[] - but even with declaring a String[] (like the second bean invocation) it is not able to execute the bean. The bean class simply looks like this: public MyBean { public void foo(String ... param) { // do some stuff ... } } I'd need to specify multiple parameters and to avoid creating multiple classes or methods which are simply copy&paste classes with small adaptions (inheritance is in place) I'd love to have a more generic approach in place. If resource intensive calculations are done in the back running the method multiple times with changed parameters may not be the best solution imho. > Enable vararg support for beaninvocation with Simple > - > > Key: CAMEL-6848 > URL: https://issues.apache.org/jira/browse/CAMEL-6848 > Project: Camel > Issue Type: Improvement > Components: bean-integration >Affects Versions: 2.11.1 >Reporter: Roman Vottner >Priority: Minor > Labels: Simple, beaninvocation, vararg > > Having a simple bean method that accepts multiple String parameters declared > as varargs, I'm having issues sending values from .bean(MyBean.class, "${ > foo('a','b') }") or .bean(MyBean.class, "${ foo(new String[] {'a', 'b'}) }") > to my bean directly. The error states that it can't convert from String to > String[] - but even with declaring a String[] (like the second bean > invocation) it is not able to execute the bean. > The bean class simply looks like this: > public MyBean > { > public void foo(String ... param) > { > // do some stuff ... > } > } > I'd need to specify multiple parameters and to avoid creating multiple > classes or methods which are simply copy&paste classes with small adaptions > (inheritance is in place) I'd love to have a more generic approach in place. > If resource intensive calculations are done in the back running the method > multiple times with changed parameters may not be the best solution imho. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (CAMEL-6848) Enable vararg support for beaninvocation with Simple
Roman Vottner created CAMEL-6848: Summary: Enable vararg support for beaninvocation with Simple Key: CAMEL-6848 URL: https://issues.apache.org/jira/browse/CAMEL-6848 Project: Camel Issue Type: Improvement Components: bean-integration Affects Versions: 2.11.1 Reporter: Roman Vottner Priority: Minor Having a simple bean method that accepts multiple String parameters declared as varargs, I'm having issues sending values from .bean(MyBean.class, "${ foo('a','b') }") or .bean(MyBean.class, "${ foo(new String[] {'a', 'b'}) }) to my bean directly. The error states that it can't convert from String to String[] - but even with declaring a String[] (like the second bean invocation) it is not able to execute the bean. The bean class simply looks like this: public MyBean { public void foo(String ... param) { // do some stuff ... } } I'd need to specify multiple parameters and to avoid creating multiple classes or methods which are simply copy&paste classes with small adaptions (inheritance is in place) I'd love to have a more generic approach in place. If resource intensive calculations are done in the back running the method multiple times with changed parameters may not be the best solution imho. -- This message was sent by Atlassian JIRA (v6.1#6144)