[jira] [Commented] (CAMEL-16912) camel-jackson - Make it easy to configure property naming strategy
[ https://issues.apache.org/jira/browse/CAMEL-16912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408544#comment-17408544 ] Andrea Cosentino commented on CAMEL-16912: -- Not sure this property will go into the generic JsonDataFormat, because I don't think the other implementations support this directly > camel-jackson - Make it easy to configure property naming strategy > -- > > Key: CAMEL-16912 > URL: https://issues.apache.org/jira/browse/CAMEL-16912 > Project: Camel > Issue Type: Improvement > Components: camel-jackson >Reporter: Claus Ibsen >Assignee: Andrea Cosentino >Priority: Major > Fix For: 3.x > > > There are a number of cases out of the box: snake, Camel, upper, etc > https://fasterxml.github.io/jackson-databind/javadoc/2.12/com/fasterxml/jackson/databind/PropertyNamingStrategies.SnakeCaseStrategy.html > It may be nice to have an option to more easily configure this. Otherwise you > need to setup your own ObjectMapper instance. > https://camel.apache.org/components/latest/dataformats/json-jackson-dataformat.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (CAMEL-16912) camel-jackson - Make it easy to configure property naming strategy
[ https://issues.apache.org/jira/browse/CAMEL-16912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrea Cosentino reassigned CAMEL-16912: Assignee: Andrea Cosentino > camel-jackson - Make it easy to configure property naming strategy > -- > > Key: CAMEL-16912 > URL: https://issues.apache.org/jira/browse/CAMEL-16912 > Project: Camel > Issue Type: Improvement > Components: camel-jackson >Reporter: Claus Ibsen >Assignee: Andrea Cosentino >Priority: Major > Fix For: 3.x > > > There are a number of cases out of the box: snake, Camel, upper, etc > https://fasterxml.github.io/jackson-databind/javadoc/2.12/com/fasterxml/jackson/databind/PropertyNamingStrategies.SnakeCaseStrategy.html > It may be nice to have an option to more easily configure this. Otherwise you > need to setup your own ObjectMapper instance. > https://camel.apache.org/components/latest/dataformats/json-jackson-dataformat.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CAMEL-16912) camel-jackson - Make it easy to configure property naming strategy
Claus Ibsen created CAMEL-16912: --- Summary: camel-jackson - Make it easy to configure property naming strategy Key: CAMEL-16912 URL: https://issues.apache.org/jira/browse/CAMEL-16912 Project: Camel Issue Type: Improvement Components: camel-jackson Reporter: Claus Ibsen Fix For: 3.x There are a number of cases out of the box: snake, Camel, upper, etc https://fasterxml.github.io/jackson-databind/javadoc/2.12/com/fasterxml/jackson/databind/PropertyNamingStrategies.SnakeCaseStrategy.html It may be nice to have an option to more easily configure this. Otherwise you need to setup your own ObjectMapper instance. https://camel.apache.org/components/latest/dataformats/json-jackson-dataformat.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CAMEL-16911) The test case SpringJacksonJsonDataFormatTest fails
[ https://issues.apache.org/jira/browse/CAMEL-16911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Claus Ibsen resolved CAMEL-16911. - Fix Version/s: 3.12.0 Resolution: Fixed > The test case SpringJacksonJsonDataFormatTest fails > --- > > Key: CAMEL-16911 > URL: https://issues.apache.org/jira/browse/CAMEL-16911 > Project: Camel > Issue Type: Test > Components: camel-jacksonxml >Affects Versions: 3.11.0 >Reporter: Karen Lease >Assignee: Karen Lease >Priority: Minor > Fix For: 3.12.0 > > > I noticed that the jacksonxml test SpringJacksonJsonDataFormatTest is failing. > [https://ci-builds.apache.org/job/Camel/job/Camel%20JDK11/job/main/319/testReport/junit/org.apache.camel.component.jacksonxml/SpringJacksonJsonDataFormatTest/.|https://ci-builds.apache.org/job/Camel/job/Camel%20JDK11/job/main/319/testReport/junit/org.apache.camel.component.jacksonxml/SpringJacksonJsonDataFormatTest/] > This is due to incorrect configuration and test assumption. > The test should also be renamed to SpringJacksonXmlDataFormatTest. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CAMEL-16911) The test case SpringJacksonJsonDataFormatTest fails
[ https://issues.apache.org/jira/browse/CAMEL-16911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Claus Ibsen updated CAMEL-16911: Issue Type: Test (was: Bug) > The test case SpringJacksonJsonDataFormatTest fails > --- > > Key: CAMEL-16911 > URL: https://issues.apache.org/jira/browse/CAMEL-16911 > Project: Camel > Issue Type: Test > Components: camel-jacksonxml >Affects Versions: 3.11.0 >Reporter: Karen Lease >Assignee: Karen Lease >Priority: Minor > > I noticed that the jacksonxml test SpringJacksonJsonDataFormatTest is failing. > [https://ci-builds.apache.org/job/Camel/job/Camel%20JDK11/job/main/319/testReport/junit/org.apache.camel.component.jacksonxml/SpringJacksonJsonDataFormatTest/.|https://ci-builds.apache.org/job/Camel/job/Camel%20JDK11/job/main/319/testReport/junit/org.apache.camel.component.jacksonxml/SpringJacksonJsonDataFormatTest/] > This is due to incorrect configuration and test assumption. > The test should also be renamed to SpringJacksonXmlDataFormatTest. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CAMEL-16910) Update Commons Compress to 1.21
[ https://issues.apache.org/jira/browse/CAMEL-16910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Claus Ibsen updated CAMEL-16910: Fix Version/s: 3.12.0 > Update Commons Compress to 1.21 > --- > > Key: CAMEL-16910 > URL: https://issues.apache.org/jira/browse/CAMEL-16910 > Project: Camel > Issue Type: Dependency upgrade >Reporter: Colm O hEigeartaigh >Assignee: Andrea Cosentino >Priority: Major > Fix For: 3.7.6, 3.11.2, 3.12.0 > > > The main branch update Commons Compress to 1.21: > ommit 0d26c3124ceb66c508e3d0cfbb4fb0400e5417b2 > Author: Andrea Cosentino > Date: Wed Jul 14 08:01:21 2021 +0200 > > Upgrade Commons-compress to version 1.21 > We should backport it for 3.11.x + 3.7.x as it fixes several high CVEs, e.g. > https://nvd.nist.gov/vuln/detail/CVE-2021-35517 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CAMEL-16907) camel-jt400 - Program Call usability concerns
[
https://issues.apache.org/jira/browse/CAMEL-16907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen resolved CAMEL-16907.
-
Resolution: Fixed
> camel-jt400 - Program Call usability concerns
> -
>
> Key: CAMEL-16907
> URL: https://issues.apache.org/jira/browse/CAMEL-16907
> Project: Camel
> Issue Type: Improvement
> Components: camel-jt400
>Affects Versions: 3.11.0
>Reporter: Jesse Gorzinski
>Assignee: Jesse Gorzinski
>Priority: Minor
> Fix For: 3.12.0
>
>
> The program call abilities are not very usable.
> When running in binary mode, all parameters must be converted to byte arrays,
> like so:
> {code:java}
> final int usrSpcSize = 16;//16776704;
> final String pgmCallUri =
> String.format("jt400://*CURRENT:*CURRENt@localhost/qsys.lib/QUSRTVUS.PGM?fieldsLength=20,4,4,%d=3=binary",
> usrSpcSize);
> context.addRoutes(new RouteBuilder() {
> @Override
> public void configure() {
> from("timer://foo?period=5000")
> .process( exchange -> {
> String usrSpc = "MYSPACE JESSEG";
> Object[] parms = new Object[] {
> usrSpc.getBytes("Cp037"), // Qualified user space
> name
> intToBytes(1), // starting position
> intToBytes(usrSpcSize),// length of data
> new byte[usrSpcSize] // output
> };
> exchange.getIn().setBody(parms);
> })
> .to(pgmCallUri)
> .setBody(simple("${body[3]}"))
> .to("stream:out");
> }
> });
> {code}
> Note, any character output must be explicitly converted from an EBCDIC byte
> array to a String (but the ccsid is unknown, so the "Cp037" in the above
> example is just a guess)
> In non-binary mode, all input parameters must be text, and I can't figure out
> how to actually send common data types, most notably integers.
> I will work on and likely send a PR shortly
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CAMEL-16899) camel-jt400 - Documentation or ability to handle parameters returned from RPG program
[ https://issues.apache.org/jira/browse/CAMEL-16899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Claus Ibsen resolved CAMEL-16899. - Resolution: Fixed > camel-jt400 - Documentation or ability to handle parameters returned from RPG > program > - > > Key: CAMEL-16899 > URL: https://issues.apache.org/jira/browse/CAMEL-16899 > Project: Camel > Issue Type: Improvement > Components: camel-jt400 >Affects Versions: 3.11.0 >Reporter: King Harrison >Assignee: Jesse Gorzinski >Priority: Minor > Fix For: 3.12.0 > > > When calling an RPG program there is no way to handle or description of how > to handle values returned by the RPG program -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CAMEL-16910) Update Commons Compress to 1.21
[ https://issues.apache.org/jira/browse/CAMEL-16910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Claus Ibsen updated CAMEL-16910: Issue Type: Dependency upgrade (was: Improvement) > Update Commons Compress to 1.21 > --- > > Key: CAMEL-16910 > URL: https://issues.apache.org/jira/browse/CAMEL-16910 > Project: Camel > Issue Type: Dependency upgrade >Reporter: Colm O hEigeartaigh >Assignee: Andrea Cosentino >Priority: Major > Fix For: 3.7.6, 3.11.2 > > > The main branch update Commons Compress to 1.21: > ommit 0d26c3124ceb66c508e3d0cfbb4fb0400e5417b2 > Author: Andrea Cosentino > Date: Wed Jul 14 08:01:21 2021 +0200 > > Upgrade Commons-compress to version 1.21 > We should backport it for 3.11.x + 3.7.x as it fixes several high CVEs, e.g. > https://nvd.nist.gov/vuln/detail/CVE-2021-35517 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CAMEL-16905) camel-mock - Remove old code that checks for header values as Expression/Predicate
[
https://issues.apache.org/jira/browse/CAMEL-16905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-16905:
Fix Version/s: (was: 3.x)
3.12.0
> camel-mock - Remove old code that checks for header values as
> Expression/Predicate
> --
>
> Key: CAMEL-16905
> URL: https://issues.apache.org/jira/browse/CAMEL-16905
> Project: Camel
> Issue Type: Improvement
> Components: camel-core
>Reporter: Claus Ibsen
>Assignee: Karen Lease
>Priority: Major
> Fix For: 3.12.0
>
>
> 1. For consistency in testing, it seems like
> MockEndpoint.extractActualValue() should also not evaluate the Expression.
> Also it looks like CAMEL-1848 may be added before we had a better way
> for mock endpoints to do assertions.
> The use case seemed to be to provide an XPathBuilder (expression) as a
> value that then is used to match against the real value.
> Today you can setup
> mock.message(0).header("foo").matches(xpath("...")) (something like
> that)
> So I would assume we could look at remove that ancient code from the
> mock component.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CAMEL-16871) Support camel-aws2-s3 multipart with remote files
[
https://issues.apache.org/jira/browse/CAMEL-16871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brendan updated CAMEL-16871:
Description:
Camel aws2-s3 mutlipart upload only supports Files.
There are at least two approaches to solving this:
The hard, general case and a simple specific case.
1) The harder but general case: use something like Spring's aws
SimpleStorageInputStream:
[https://github.com/spring-cloud/spring-cloud-aws/blob/main/spring-cloud-aws-core/src/main/java/org/springframework/cloud/aws/core/io/s3/SimpleStorageResource.java|http://example.com/]
and translate any exchange body to an Inputstream for mulitpart uploads.
2) The simpler but specific case: for remote files in particular, camel often
makes use of a localwork directory to download files. In fact,
{code:java}
e.getIn().getBody(File.class)
{code}
will resolve the file in the local work directory when polling from an FTP site
with a local work directory specified.
The aws2-s3 producer (AWS2S3Producer.class) on lines 134 to 145 would need to
be modified in both cases.
was:
Camel aws2-s3 mutlipart upload only supports Files.
There are at least two approaches to solving this:
The hard, general case and a simple specific case.
1) The harder but general case: use something like Spring's aws
SimpleStorageInputStream:
[https://github.com/spring-cloud/spring-cloud-aws/blob/main/spring-cloud-aws-core/src/main/java/org/springframework/cloud/aws/core/io/s3/SimpleStorageResource.java|http://example.com/]
and translate any exchange body to an Inputstream for mulitpart
uploads.|https://github.com/spring-cloud/spring-cloud-aws/blob/main/spring-cloud-aws-core/src/main/java/org/springframework/cloud/aws/core/io/s3/SimpleStorageResource.java]to]
2) The simpler but specific case: for remote files in particular, camel often
makes use of a localwork directory to download files. In fact,
{code:java}
e.getIn().getBody(File.class)
{code}
will resolve the file in the local work directory when polling from an FTP site
with a local work directory specified.
The aws2-s3 producer (AWS2S3Producer.class) on lines 134 to 145 would need to
be modified in both cases.
> Support camel-aws2-s3 multipart with remote files
> -
>
> Key: CAMEL-16871
> URL: https://issues.apache.org/jira/browse/CAMEL-16871
> Project: Camel
> Issue Type: Wish
> Components: camel-aws2-s3
>Reporter: Brendan
>Priority: Minor
>
> Camel aws2-s3 mutlipart upload only supports Files.
>
> There are at least two approaches to solving this:
> The hard, general case and a simple specific case.
>
> 1) The harder but general case: use something like Spring's aws
> SimpleStorageInputStream:
> [https://github.com/spring-cloud/spring-cloud-aws/blob/main/spring-cloud-aws-core/src/main/java/org/springframework/cloud/aws/core/io/s3/SimpleStorageResource.java|http://example.com/]
> and translate any exchange body to an Inputstream for mulitpart uploads.
>
> 2) The simpler but specific case: for remote files in particular, camel often
> makes use of a localwork directory to download files. In fact,
> {code:java}
> e.getIn().getBody(File.class)
> {code}
> will resolve the file in the local work directory when polling from an FTP
> site with a local work directory specified.
>
> The aws2-s3 producer (AWS2S3Producer.class) on lines 134 to 145 would need to
> be modified in both cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CAMEL-16911) The test case SpringJacksonJsonDataFormatTest fails
Karen Lease created CAMEL-16911: --- Summary: The test case SpringJacksonJsonDataFormatTest fails Key: CAMEL-16911 URL: https://issues.apache.org/jira/browse/CAMEL-16911 Project: Camel Issue Type: Bug Components: camel-jacksonxml Affects Versions: 3.11.0 Reporter: Karen Lease Assignee: Karen Lease I noticed that the jacksonxml test SpringJacksonJsonDataFormatTest is failing. [https://ci-builds.apache.org/job/Camel/job/Camel%20JDK11/job/main/319/testReport/junit/org.apache.camel.component.jacksonxml/SpringJacksonJsonDataFormatTest/.|https://ci-builds.apache.org/job/Camel/job/Camel%20JDK11/job/main/319/testReport/junit/org.apache.camel.component.jacksonxml/SpringJacksonJsonDataFormatTest/] This is due to incorrect configuration and test assumption. The test should also be renamed to SpringJacksonXmlDataFormatTest. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (CAMEL-16910) Update Commons Compress to 1.21
[ https://issues.apache.org/jira/browse/CAMEL-16910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colm O hEigeartaigh reassigned CAMEL-16910: --- Assignee: Andrea Cosentino > Update Commons Compress to 1.21 > --- > > Key: CAMEL-16910 > URL: https://issues.apache.org/jira/browse/CAMEL-16910 > Project: Camel > Issue Type: Improvement >Reporter: Colm O hEigeartaigh >Assignee: Andrea Cosentino >Priority: Major > Fix For: 3.7.6, 3.11.2 > > > The main branch update Commons Compress to 1.21: > ommit 0d26c3124ceb66c508e3d0cfbb4fb0400e5417b2 > Author: Andrea Cosentino > Date: Wed Jul 14 08:01:21 2021 +0200 > > Upgrade Commons-compress to version 1.21 > We should backport it for 3.11.x + 3.7.x as it fixes several high CVEs, e.g. > https://nvd.nist.gov/vuln/detail/CVE-2021-35517 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CAMEL-16910) Update Commons Compress to 1.21
Colm O hEigeartaigh created CAMEL-16910: --- Summary: Update Commons Compress to 1.21 Key: CAMEL-16910 URL: https://issues.apache.org/jira/browse/CAMEL-16910 Project: Camel Issue Type: Improvement Reporter: Colm O hEigeartaigh Fix For: 3.7.6, 3.11.2 The main branch update Commons Compress to 1.21: ommit 0d26c3124ceb66c508e3d0cfbb4fb0400e5417b2 Author: Andrea Cosentino Date: Wed Jul 14 08:01:21 2021 +0200 Upgrade Commons-compress to version 1.21 We should backport it for 3.11.x + 3.7.x as it fixes several high CVEs, e.g. https://nvd.nist.gov/vuln/detail/CVE-2021-35517 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CAMEL-16909) Camel-Kubernetes: Added support for Event Resources
Andrea Cosentino created CAMEL-16909: Summary: Camel-Kubernetes: Added support for Event Resources Key: CAMEL-16909 URL: https://issues.apache.org/jira/browse/CAMEL-16909 Project: Camel Issue Type: New Feature Components: camel-kubernetes Reporter: Andrea Cosentino Assignee: Andrea Cosentino Fix For: 3.12.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CAMEL-16895) Log masking results in stackoverflow for large inputs
[
https://issues.apache.org/jira/browse/CAMEL-16895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407921#comment-17407921
]
Claus Ibsen commented on CAMEL-16895:
-
Ah so you are low on memory, yeah I just ran a test with GBs of heap
> Log masking results in stackoverflow for large inputs
> -
>
> Key: CAMEL-16895
> URL: https://issues.apache.org/jira/browse/CAMEL-16895
> Project: Camel
> Issue Type: Bug
> Components: camel-core
>Affects Versions: 3.11.0
>Reporter: Julius Krah
>Priority: Minor
> Fix For: 3.12.0
>
>
> When a fairly large input string is submitted for logging and masking is
> enabled, this results in a StackoverflowError
>
> Using camel with Spring Boot
> pom.xml
> {noformat}
>
>
>
> org.apache.camel.springboot
> camel-spring-boot-bom
> 3.11.1
> pom
> import
>
>
>
>
>
> org.springframework.boot
> spring-boot-starter-validation
>
>
> org.apache.camel.springboot
> camel-spring-boot-starter
>
> {noformat}
> Route.java
> {noformat}
> @Component
> public class Route extends RouteBuilder {
> @Override
> public void configure() throws Exception {
> from("rabbitmq:{{core.consumer.rabbitmq.exchange}}" +
> "?exchangeType={{core.consumer.rabbitmq.exchange-type}}" +
> "={{core.consumer.rabbitmq.queue}}" +
> "={{core.consumer.rabbitmq.routing-key}}")
> .routeId("queue route")
> .log("Received payload from rabbitMQ: ${body}")
> .unmarshal(new JacksonDataFormat(Dto.class))
> .process(aProcessor)
> .log("Data received ${body}")
>
> .to("bean:requestProcessor?method=handleAckRequest").id("handle-ack-request")
>
> .to("direct:anotherRoute")
> .end();
> }
> }
> {noformat}
> In the application.properties
> *camel.springboot.log-mask=true*
>
> Now if I should post the following JSON payload through RabbitMQ
>
> {noformat}
> {
> "qwjabrlanfssaPaymentID": 93974430,
> "apiPortNumber": 0,
> "pollerFrequency": 0,
> "serviceCode": "QWERETEMKLJLANDD",
> "extraData": {
> "97280991": {
> "pushToOriginator": true,
> "uidID": 41189716,
> "metaData": [
> {
> "Value": 1070,
> "Name": "Amount"
> },
> {
> "Value": "PHP82TK6MY",
>"Name": "MpesaQwasdrtjknsNumber"
> },
> {
> "Name": "Balance"
> },
> {
> "Value": 20210825224512,
> "Name": "TransactionDate"
> },
> {
> "Value": 599726145860,
> "Name": "PhoneNumber"
> }],
> "pokjytsAmountCharged": 1070,
> "hardwareRequestId": null,
> "ProductCode": "UJHANSNLWII",
> "isTillNumber": true,
> "uniqueID": "83859103",
> "vouchercode": " ",
> "paybillNumber": "431553"
> }
> },
> "oiusdftrPaymentStatus": 139,
> "serviceUrl": "",
> "lkjmnbsdcvfdertCurrencyCode": "YUM",
> "wrapperUrl": "",
> "wsdlFile": "",
> "protocol": "default",
> "sslEnabled": 0,
> "numberOfSends": 0,
> "invoiceNumber": "",
> "erdfghytIsdgftgDate": "2021-08-25 22:45:20",
> "lkjditgfJhsgruaCustomerName": "Customer",
> "apiUserName": "",
> "ojudhfgKaswsxsUUID": "1102967588",
> "iuytredfHsdededAccountNumber": "4622785548",
> "tyghfderrMode": [
> "TUND_PUDF"
> ],
> "zxcvfgtrNsdfdfdMSISDN": "254726173860",
> "apiFunctionName": "",
> "kjhxcTransactionID": [
> "PHP82TK6MY"
> ],
> "apiPassword": "x",
> "kjhcvbnUUID": "1102967588",
> "dfcxvbnClientCode": [
> "SAFKE"
> ],
> "sslCertificatePath": "",
> "isTokenService": 0,
> "wsazxcvgbgthyujAmount": 1070,
> "plsjdhfhPusherProcessorClass": "",
> "requestOriginIDs": [
> "21"
> ],
> "firstSend": "",
> "maxNumberOfSends": 0,
> "narration": "The service request is processed successfully.",
> "hubCreationDate": "2021-08-25 22:45:20",
> "allowsPolling": 1,
> "serviceID": 3004,
> "lastSend":
[jira] [Created] (CAMEL-16908) camel-core - Log Mask should use the known sensitive keys
Claus Ibsen created CAMEL-16908: --- Summary: camel-core - Log Mask should use the known sensitive keys Key: CAMEL-16908 URL: https://issues.apache.org/jira/browse/CAMEL-16908 Project: Camel Issue Type: Improvement Components: camel-core Reporter: Claus Ibsen Assignee: Claus Ibsen Fix For: 3.12.0 It is currently hardcoded to only 3 keys, but we have SensitiveHelper that has all known keys (there are 60+) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CAMEL-16895) Log masking results in stackoverflow for large inputs
[
https://issues.apache.org/jira/browse/CAMEL-16895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407914#comment-17407914
]
Julius Krah commented on CAMEL-16895:
-
Great, I actually don't need the JSON to be parsed and masked, this is already
done by the upstream system. It is being masked as a side effect of a global
masking setting
> Log masking results in stackoverflow for large inputs
> -
>
> Key: CAMEL-16895
> URL: https://issues.apache.org/jira/browse/CAMEL-16895
> Project: Camel
> Issue Type: Bug
> Components: camel-core
>Affects Versions: 3.11.0
>Reporter: Julius Krah
>Priority: Minor
> Fix For: 3.12.0
>
>
> When a fairly large input string is submitted for logging and masking is
> enabled, this results in a StackoverflowError
>
> Using camel with Spring Boot
> pom.xml
> {noformat}
>
>
>
> org.apache.camel.springboot
> camel-spring-boot-bom
> 3.11.1
> pom
> import
>
>
>
>
>
> org.springframework.boot
> spring-boot-starter-validation
>
>
> org.apache.camel.springboot
> camel-spring-boot-starter
>
> {noformat}
> Route.java
> {noformat}
> @Component
> public class Route extends RouteBuilder {
> @Override
> public void configure() throws Exception {
> from("rabbitmq:{{core.consumer.rabbitmq.exchange}}" +
> "?exchangeType={{core.consumer.rabbitmq.exchange-type}}" +
> "={{core.consumer.rabbitmq.queue}}" +
> "={{core.consumer.rabbitmq.routing-key}}")
> .routeId("queue route")
> .log("Received payload from rabbitMQ: ${body}")
> .unmarshal(new JacksonDataFormat(Dto.class))
> .process(aProcessor)
> .log("Data received ${body}")
>
> .to("bean:requestProcessor?method=handleAckRequest").id("handle-ack-request")
>
> .to("direct:anotherRoute")
> .end();
> }
> }
> {noformat}
> In the application.properties
> *camel.springboot.log-mask=true*
>
> Now if I should post the following JSON payload through RabbitMQ
>
> {noformat}
> {
> "qwjabrlanfssaPaymentID": 93974430,
> "apiPortNumber": 0,
> "pollerFrequency": 0,
> "serviceCode": "QWERETEMKLJLANDD",
> "extraData": {
> "97280991": {
> "pushToOriginator": true,
> "uidID": 41189716,
> "metaData": [
> {
> "Value": 1070,
> "Name": "Amount"
> },
> {
> "Value": "PHP82TK6MY",
>"Name": "MpesaQwasdrtjknsNumber"
> },
> {
> "Name": "Balance"
> },
> {
> "Value": 20210825224512,
> "Name": "TransactionDate"
> },
> {
> "Value": 599726145860,
> "Name": "PhoneNumber"
> }],
> "pokjytsAmountCharged": 1070,
> "hardwareRequestId": null,
> "ProductCode": "UJHANSNLWII",
> "isTillNumber": true,
> "uniqueID": "83859103",
> "vouchercode": " ",
> "paybillNumber": "431553"
> }
> },
> "oiusdftrPaymentStatus": 139,
> "serviceUrl": "",
> "lkjmnbsdcvfdertCurrencyCode": "YUM",
> "wrapperUrl": "",
> "wsdlFile": "",
> "protocol": "default",
> "sslEnabled": 0,
> "numberOfSends": 0,
> "invoiceNumber": "",
> "erdfghytIsdgftgDate": "2021-08-25 22:45:20",
> "lkjditgfJhsgruaCustomerName": "Customer",
> "apiUserName": "",
> "ojudhfgKaswsxsUUID": "1102967588",
> "iuytredfHsdededAccountNumber": "4622785548",
> "tyghfderrMode": [
> "TUND_PUDF"
> ],
> "zxcvfgtrNsdfdfdMSISDN": "254726173860",
> "apiFunctionName": "",
> "kjhxcTransactionID": [
> "PHP82TK6MY"
> ],
> "apiPassword": "x",
> "kjhcvbnUUID": "1102967588",
> "dfcxvbnClientCode": [
> "SAFKE"
> ],
> "sslCertificatePath": "",
> "isTokenService": 0,
> "wsazxcvgbgthyujAmount": 1070,
> "plsjdhfhPusherProcessorClass": "",
> "requestOriginIDs": [
> "21"
> ],
> "firstSend": "",
> "maxNumberOfSends": 0,
> "narration": "The service request is processed successfully.",
>
[jira] [Commented] (CAMEL-16895) Log masking results in stackoverflow for large inputs
[
https://issues.apache.org/jira/browse/CAMEL-16895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407908#comment-17407908
]
Julius Krah commented on CAMEL-16895:
-
I obfuscated it a little to remove sensitive information, but it's generally
the same structure. It fails on my system and in our k8s environment, below is
the memory configuration
{noformat}
resources:
limits:
cpu: 500m
memory: 800Mi
requests:
cpu: 150m
memory: 250Mi
{noformat}
> Log masking results in stackoverflow for large inputs
> -
>
> Key: CAMEL-16895
> URL: https://issues.apache.org/jira/browse/CAMEL-16895
> Project: Camel
> Issue Type: Bug
> Components: camel-core
>Affects Versions: 3.11.0
>Reporter: Julius Krah
>Priority: Minor
> Fix For: 3.12.0
>
>
> When a fairly large input string is submitted for logging and masking is
> enabled, this results in a StackoverflowError
>
> Using camel with Spring Boot
> pom.xml
> {noformat}
>
>
>
> org.apache.camel.springboot
> camel-spring-boot-bom
> 3.11.1
> pom
> import
>
>
>
>
>
> org.springframework.boot
> spring-boot-starter-validation
>
>
> org.apache.camel.springboot
> camel-spring-boot-starter
>
> {noformat}
> Route.java
> {noformat}
> @Component
> public class Route extends RouteBuilder {
> @Override
> public void configure() throws Exception {
> from("rabbitmq:{{core.consumer.rabbitmq.exchange}}" +
> "?exchangeType={{core.consumer.rabbitmq.exchange-type}}" +
> "={{core.consumer.rabbitmq.queue}}" +
> "={{core.consumer.rabbitmq.routing-key}}")
> .routeId("queue route")
> .log("Received payload from rabbitMQ: ${body}")
> .unmarshal(new JacksonDataFormat(Dto.class))
> .process(aProcessor)
> .log("Data received ${body}")
>
> .to("bean:requestProcessor?method=handleAckRequest").id("handle-ack-request")
>
> .to("direct:anotherRoute")
> .end();
> }
> }
> {noformat}
> In the application.properties
> *camel.springboot.log-mask=true*
>
> Now if I should post the following JSON payload through RabbitMQ
>
> {noformat}
> {
> "qwjabrlanfssaPaymentID": 93974430,
> "apiPortNumber": 0,
> "pollerFrequency": 0,
> "serviceCode": "QWERETEMKLJLANDD",
> "extraData": {
> "97280991": {
> "pushToOriginator": true,
> "uidID": 41189716,
> "metaData": [
> {
> "Value": 1070,
> "Name": "Amount"
> },
> {
> "Value": "PHP82TK6MY",
>"Name": "MpesaQwasdrtjknsNumber"
> },
> {
> "Name": "Balance"
> },
> {
> "Value": 20210825224512,
> "Name": "TransactionDate"
> },
> {
> "Value": 599726145860,
> "Name": "PhoneNumber"
> }],
> "pokjytsAmountCharged": 1070,
> "hardwareRequestId": null,
> "ProductCode": "UJHANSNLWII",
> "isTillNumber": true,
> "uniqueID": "83859103",
> "vouchercode": " ",
> "paybillNumber": "431553"
> }
> },
> "oiusdftrPaymentStatus": 139,
> "serviceUrl": "",
> "lkjmnbsdcvfdertCurrencyCode": "YUM",
> "wrapperUrl": "",
> "wsdlFile": "",
> "protocol": "default",
> "sslEnabled": 0,
> "numberOfSends": 0,
> "invoiceNumber": "",
> "erdfghytIsdgftgDate": "2021-08-25 22:45:20",
> "lkjditgfJhsgruaCustomerName": "Customer",
> "apiUserName": "",
> "ojudhfgKaswsxsUUID": "1102967588",
> "iuytredfHsdededAccountNumber": "4622785548",
> "tyghfderrMode": [
> "TUND_PUDF"
> ],
> "zxcvfgtrNsdfdfdMSISDN": "254726173860",
> "apiFunctionName": "",
> "kjhxcTransactionID": [
> "PHP82TK6MY"
> ],
> "apiPassword": "x",
> "kjhcvbnUUID": "1102967588",
> "dfcxvbnClientCode": [
> "SAFKE"
> ],
> "sslCertificatePath": "",
> "isTokenService": 0,
> "wsazxcvgbgthyujAmount": 1070,
> "plsjdhfhPusherProcessorClass": "",
> "requestOriginIDs": [
> "21"
> ],
>
[jira] [Commented] (CAMEL-16895) Log masking results in stackoverflow for large inputs
[
https://issues.apache.org/jira/browse/CAMEL-16895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407906#comment-17407906
]
Claus Ibsen commented on CAMEL-16895:
-
We can optimize the log-mask to not do all 3 kind of masking if we know the
structure is json or xml, that reduces the regexp parsing overhead *DONE*
> Log masking results in stackoverflow for large inputs
> -
>
> Key: CAMEL-16895
> URL: https://issues.apache.org/jira/browse/CAMEL-16895
> Project: Camel
> Issue Type: Bug
> Components: camel-core
>Affects Versions: 3.11.0
>Reporter: Julius Krah
>Priority: Minor
> Fix For: 3.12.0
>
>
> When a fairly large input string is submitted for logging and masking is
> enabled, this results in a StackoverflowError
>
> Using camel with Spring Boot
> pom.xml
> {noformat}
>
>
>
> org.apache.camel.springboot
> camel-spring-boot-bom
> 3.11.1
> pom
> import
>
>
>
>
>
> org.springframework.boot
> spring-boot-starter-validation
>
>
> org.apache.camel.springboot
> camel-spring-boot-starter
>
> {noformat}
> Route.java
> {noformat}
> @Component
> public class Route extends RouteBuilder {
> @Override
> public void configure() throws Exception {
> from("rabbitmq:{{core.consumer.rabbitmq.exchange}}" +
> "?exchangeType={{core.consumer.rabbitmq.exchange-type}}" +
> "={{core.consumer.rabbitmq.queue}}" +
> "={{core.consumer.rabbitmq.routing-key}}")
> .routeId("queue route")
> .log("Received payload from rabbitMQ: ${body}")
> .unmarshal(new JacksonDataFormat(Dto.class))
> .process(aProcessor)
> .log("Data received ${body}")
>
> .to("bean:requestProcessor?method=handleAckRequest").id("handle-ack-request")
>
> .to("direct:anotherRoute")
> .end();
> }
> }
> {noformat}
> In the application.properties
> *camel.springboot.log-mask=true*
>
> Now if I should post the following JSON payload through RabbitMQ
>
> {noformat}
> {
> "qwjabrlanfssaPaymentID": 93974430,
> "apiPortNumber": 0,
> "pollerFrequency": 0,
> "serviceCode": "QWERETEMKLJLANDD",
> "extraData": {
> "97280991": {
> "pushToOriginator": true,
> "uidID": 41189716,
> "metaData": [
> {
> "Value": 1070,
> "Name": "Amount"
> },
> {
> "Value": "PHP82TK6MY",
>"Name": "MpesaQwasdrtjknsNumber"
> },
> {
> "Name": "Balance"
> },
> {
> "Value": 20210825224512,
> "Name": "TransactionDate"
> },
> {
> "Value": 599726145860,
> "Name": "PhoneNumber"
> }],
> "pokjytsAmountCharged": 1070,
> "hardwareRequestId": null,
> "ProductCode": "UJHANSNLWII",
> "isTillNumber": true,
> "uniqueID": "83859103",
> "vouchercode": " ",
> "paybillNumber": "431553"
> }
> },
> "oiusdftrPaymentStatus": 139,
> "serviceUrl": "",
> "lkjmnbsdcvfdertCurrencyCode": "YUM",
> "wrapperUrl": "",
> "wsdlFile": "",
> "protocol": "default",
> "sslEnabled": 0,
> "numberOfSends": 0,
> "invoiceNumber": "",
> "erdfghytIsdgftgDate": "2021-08-25 22:45:20",
> "lkjditgfJhsgruaCustomerName": "Customer",
> "apiUserName": "",
> "ojudhfgKaswsxsUUID": "1102967588",
> "iuytredfHsdededAccountNumber": "4622785548",
> "tyghfderrMode": [
> "TUND_PUDF"
> ],
> "zxcvfgtrNsdfdfdMSISDN": "254726173860",
> "apiFunctionName": "",
> "kjhxcTransactionID": [
> "PHP82TK6MY"
> ],
> "apiPassword": "x",
> "kjhcvbnUUID": "1102967588",
> "dfcxvbnClientCode": [
> "SAFKE"
> ],
> "sslCertificatePath": "",
> "isTokenService": 0,
> "wsazxcvgbgthyujAmount": 1070,
> "plsjdhfhPusherProcessorClass": "",
> "requestOriginIDs": [
> "21"
> ],
> "firstSend": "",
> "maxNumberOfSends": 0,
> "narration": "The service request is processed successfully.",
> "hubCreationDate": "2021-08-25
[jira] [Commented] (CAMEL-16895) Log masking results in stackoverflow for large inputs
[
https://issues.apache.org/jira/browse/CAMEL-16895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407898#comment-17407898
]
Claus Ibsen commented on CAMEL-16895:
-
Is that the original json that fails for you? Or did you change something as I
can parse/mask it on my system.
> Log masking results in stackoverflow for large inputs
> -
>
> Key: CAMEL-16895
> URL: https://issues.apache.org/jira/browse/CAMEL-16895
> Project: Camel
> Issue Type: Bug
> Components: camel-core
>Affects Versions: 3.11.0
>Reporter: Julius Krah
>Priority: Minor
> Fix For: 3.12.0
>
>
> When a fairly large input string is submitted for logging and masking is
> enabled, this results in a StackoverflowError
>
> Using camel with Spring Boot
> pom.xml
> {noformat}
>
>
>
> org.apache.camel.springboot
> camel-spring-boot-bom
> 3.11.1
> pom
> import
>
>
>
>
>
> org.springframework.boot
> spring-boot-starter-validation
>
>
> org.apache.camel.springboot
> camel-spring-boot-starter
>
> {noformat}
> Route.java
> {noformat}
> @Component
> public class Route extends RouteBuilder {
> @Override
> public void configure() throws Exception {
> from("rabbitmq:{{core.consumer.rabbitmq.exchange}}" +
> "?exchangeType={{core.consumer.rabbitmq.exchange-type}}" +
> "={{core.consumer.rabbitmq.queue}}" +
> "={{core.consumer.rabbitmq.routing-key}}")
> .routeId("queue route")
> .log("Received payload from rabbitMQ: ${body}")
> .unmarshal(new JacksonDataFormat(Dto.class))
> .process(aProcessor)
> .log("Data received ${body}")
>
> .to("bean:requestProcessor?method=handleAckRequest").id("handle-ack-request")
>
> .to("direct:anotherRoute")
> .end();
> }
> }
> {noformat}
> In the application.properties
> *camel.springboot.log-mask=true*
>
> Now if I should post the following JSON payload through RabbitMQ
>
> {noformat}
> {
> "qwjabrlanfssaPaymentID": 93974430,
> "apiPortNumber": 0,
> "pollerFrequency": 0,
> "serviceCode": "QWERETEMKLJLANDD",
> "extraData": {
> "97280991": {
> "pushToOriginator": true,
> "uidID": 41189716,
> "metaData": [
> {
> "Value": 1070,
> "Name": "Amount"
> },
> {
> "Value": "PHP82TK6MY",
>"Name": "MpesaQwasdrtjknsNumber"
> },
> {
> "Name": "Balance"
> },
> {
> "Value": 20210825224512,
> "Name": "TransactionDate"
> },
> {
> "Value": 599726145860,
> "Name": "PhoneNumber"
> }],
> "pokjytsAmountCharged": 1070,
> "hardwareRequestId": null,
> "ProductCode": "UJHANSNLWII",
> "isTillNumber": true,
> "uniqueID": "83859103",
> "vouchercode": " ",
> "paybillNumber": "431553"
> }
> },
> "oiusdftrPaymentStatus": 139,
> "serviceUrl": "",
> "lkjmnbsdcvfdertCurrencyCode": "YUM",
> "wrapperUrl": "",
> "wsdlFile": "",
> "protocol": "default",
> "sslEnabled": 0,
> "numberOfSends": 0,
> "invoiceNumber": "",
> "erdfghytIsdgftgDate": "2021-08-25 22:45:20",
> "lkjditgfJhsgruaCustomerName": "Customer",
> "apiUserName": "",
> "ojudhfgKaswsxsUUID": "1102967588",
> "iuytredfHsdededAccountNumber": "4622785548",
> "tyghfderrMode": [
> "TUND_PUDF"
> ],
> "zxcvfgtrNsdfdfdMSISDN": "254726173860",
> "apiFunctionName": "",
> "kjhxcTransactionID": [
> "PHP82TK6MY"
> ],
> "apiPassword": "x",
> "kjhcvbnUUID": "1102967588",
> "dfcxvbnClientCode": [
> "SAFKE"
> ],
> "sslCertificatePath": "",
> "isTokenService": 0,
> "wsazxcvgbgthyujAmount": 1070,
> "plsjdhfhPusherProcessorClass": "",
> "requestOriginIDs": [
> "21"
> ],
> "firstSend": "",
> "maxNumberOfSends": 0,
> "narration": "The service request is processed successfully.",
> "hubCreationDate": "2021-08-25 22:45:20",
> "allowsPolling": 1,
>
