[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162:
URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1032228729
##
library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/format/DefaultDataTypeRegistry.java:
##
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kamelets.utils.format;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+
+import org.apache.camel.CamelContext;
+import org.apache.camel.CamelContextAware;
+import org.apache.camel.ExtendedCamelContext;
+import org.apache.camel.RuntimeCamelException;
+import org.apache.camel.impl.engine.DefaultInjector;
+import org.apache.camel.impl.engine.DefaultPackageScanClassResolver;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeConverter;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeLoader;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeRegistry;
+import org.apache.camel.spi.PackageScanClassResolver;
+import org.apache.camel.support.service.ServiceSupport;
+
+/**
+ * Default data type registry able to resolve data types converters in the
project. Data types may be defined at the component level
+ * via {@link org.apache.camel.kamelets.utils.format.spi.annotations.DataType}
annotations. Also, users can add data types directly
+ * to the Camel context or manually to the registry.
+ *
+ * The registry is able to retrieve converters for a given data type based on
the component scheme and the given data type name.
+ */
+public class DefaultDataTypeRegistry extends ServiceSupport implements
DataTypeRegistry, CamelContextAware {
Review Comment:
Another option would be to create a quarkus extension which would register
the service and i.e. also the factory finder without the need to have properties
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162: URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1032534497 ## library/camel-kamelets-utils/pom.xml: ## @@ -71,12 +71,22 @@ camel-kafka - + org.apache.camel camel-aws2-ddb provided + +org.apache.camel +camel-aws2-s3 +provided + + +org.apache.camel +camel-cloudevents + Review Comment: there is a camel-cloudevents extension https://github.com/apache/camel-quarkus/tree/main/extensions/cloudevents -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162:
URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1034727829
##
library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/format/converter/aws2/s3/AWS2S3CloudEventOutputType.java:
##
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kamelets.utils.format.converter.aws2.s3;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.Map;
+
+import org.apache.camel.Exchange;
+import org.apache.camel.component.aws2.s3.AWS2S3Constants;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeConverter;
+import org.apache.camel.kamelets.utils.format.spi.annotations.DataType;
+
+/**
+ * Output data type represents AWS S3 get object response as CloudEvent V1.
The data type sets Camel specific
+ * CloudEvent headers on the exchange.
+ */
+@DataType(scheme = "aws2-s3", name = "cloudevents", mediaType =
"application/octet-stream")
+public class AWS2S3CloudEventOutputType implements DataTypeConverter {
+
+static final String CAMEL_CLOUD_EVENT_TYPE = "CamelCloudEventType";
+static final String CAMEL_CLOUD_EVENT_SOURCE = "CamelCloudEventSource";
+static final String CAMEL_CLOUD_EVENT_SUBJECT = "CamelCloudEventSubject";
+static final String CAMEL_CLOUD_EVENT_TIME = "CamelCloudEventTime";
+
+@Override
+public void convert(Exchange exchange) {
+final Map headers = exchange.getMessage().getHeaders();
+
+headers.put(CAMEL_CLOUD_EVENT_TYPE, "kamelet.aws.s3.source");
Review Comment:
should the `kamelet.` prefix be removed ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162:
URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1034732779
##
library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/format/converter/standard/JsonModelDataType.java:
##
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kamelets.utils.format.converter.standard;
+
+import java.io.ByteArrayInputStream;
+import java.io.InputStream;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.camel.CamelContext;
+import org.apache.camel.CamelContextAware;
+import org.apache.camel.CamelExecutionException;
+import org.apache.camel.Exchange;
+import org.apache.camel.InvalidPayloadException;
+import org.apache.camel.component.jackson.JacksonDataFormat;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeConverter;
+import org.apache.camel.kamelets.utils.format.spi.annotations.DataType;
+import org.apache.camel.util.ObjectHelper;
+
+/**
+ * Data type converter able to unmarshal to given unmarshalType using jackson
data format.
+ *
+ * Unmarshal type should be given as a fully qualified class name in the
exchange properties.
+ */
+@DataType(name = "jsonObject", mediaType = "application/json")
+public class JsonModelDataType implements DataTypeConverter, CamelContextAware
{
+
+public static final String DATA_TYPE_MODEL_PROPERTY = "CamelDataTypeModel";
+
+private CamelContext camelContext;
+
+private static final ObjectMapper mapper = new ObjectMapper();
+
+@Override
+public void convert(Exchange exchange) {
+if (!exchange.hasProperties() ||
!exchange.getProperties().containsKey(DATA_TYPE_MODEL_PROPERTY)) {
+return;
+}
+
+ObjectHelper.notNull(camelContext, "camelContext");
+
+String type = exchange.getProperty(DATA_TYPE_MODEL_PROPERTY,
String.class);
+try (JacksonDataFormat dataFormat = new JacksonDataFormat(mapper,
camelContext.getClassResolver().resolveMandatoryClass(type))) {
Review Comment:
This seems quite heavy as you need to create a class from a string for each
message
IMHO, as a followup there should be a way to configure the converter
properties i.e. in this case I think the type is likely to be a fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162:
URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1034732779
##
library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/format/converter/standard/JsonModelDataType.java:
##
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kamelets.utils.format.converter.standard;
+
+import java.io.ByteArrayInputStream;
+import java.io.InputStream;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.camel.CamelContext;
+import org.apache.camel.CamelContextAware;
+import org.apache.camel.CamelExecutionException;
+import org.apache.camel.Exchange;
+import org.apache.camel.InvalidPayloadException;
+import org.apache.camel.component.jackson.JacksonDataFormat;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeConverter;
+import org.apache.camel.kamelets.utils.format.spi.annotations.DataType;
+import org.apache.camel.util.ObjectHelper;
+
+/**
+ * Data type converter able to unmarshal to given unmarshalType using jackson
data format.
+ *
+ * Unmarshal type should be given as a fully qualified class name in the
exchange properties.
+ */
+@DataType(name = "jsonObject", mediaType = "application/json")
+public class JsonModelDataType implements DataTypeConverter, CamelContextAware
{
+
+public static final String DATA_TYPE_MODEL_PROPERTY = "CamelDataTypeModel";
+
+private CamelContext camelContext;
+
+private static final ObjectMapper mapper = new ObjectMapper();
+
+@Override
+public void convert(Exchange exchange) {
+if (!exchange.hasProperties() ||
!exchange.getProperties().containsKey(DATA_TYPE_MODEL_PROPERTY)) {
+return;
+}
+
+ObjectHelper.notNull(camelContext, "camelContext");
+
+String type = exchange.getProperty(DATA_TYPE_MODEL_PROPERTY,
String.class);
+try (JacksonDataFormat dataFormat = new JacksonDataFormat(mapper,
camelContext.getClassResolver().resolveMandatoryClass(type))) {
Review Comment:
This seems quite heavy as you need to create a class from a string for each
message
IMHO, as a followup there should be a way to configure the converter
properties i.e. in this case I think the type is likely to be a fixed in the
majority of the cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162:
URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1034732779
##
library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/format/converter/standard/JsonModelDataType.java:
##
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kamelets.utils.format.converter.standard;
+
+import java.io.ByteArrayInputStream;
+import java.io.InputStream;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.camel.CamelContext;
+import org.apache.camel.CamelContextAware;
+import org.apache.camel.CamelExecutionException;
+import org.apache.camel.Exchange;
+import org.apache.camel.InvalidPayloadException;
+import org.apache.camel.component.jackson.JacksonDataFormat;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeConverter;
+import org.apache.camel.kamelets.utils.format.spi.annotations.DataType;
+import org.apache.camel.util.ObjectHelper;
+
+/**
+ * Data type converter able to unmarshal to given unmarshalType using jackson
data format.
+ *
+ * Unmarshal type should be given as a fully qualified class name in the
exchange properties.
+ */
+@DataType(name = "jsonObject", mediaType = "application/json")
+public class JsonModelDataType implements DataTypeConverter, CamelContextAware
{
+
+public static final String DATA_TYPE_MODEL_PROPERTY = "CamelDataTypeModel";
+
+private CamelContext camelContext;
+
+private static final ObjectMapper mapper = new ObjectMapper();
+
+@Override
+public void convert(Exchange exchange) {
+if (!exchange.hasProperties() ||
!exchange.getProperties().containsKey(DATA_TYPE_MODEL_PROPERTY)) {
+return;
+}
+
+ObjectHelper.notNull(camelContext, "camelContext");
+
+String type = exchange.getProperty(DATA_TYPE_MODEL_PROPERTY,
String.class);
+try (JacksonDataFormat dataFormat = new JacksonDataFormat(mapper,
camelContext.getClassResolver().resolveMandatoryClass(type))) {
Review Comment:
This seems quite heavy as you need to create a class from a string and a
JacksonDataFormat for each message.
IMHO, as a followup there should be a way to configure the converter
properties i.e. in this case I think the type is likely to be a fixed in the
majority of the cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162:
URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1034740367
##
library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/format/DefaultDataTypeRegistry.java:
##
@@ -0,0 +1,208 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kamelets.utils.format;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+
+import org.apache.camel.CamelContext;
+import org.apache.camel.CamelContextAware;
+import org.apache.camel.RuntimeCamelException;
+import
org.apache.camel.kamelets.utils.format.converter.standard.BinaryDataType;
+import
org.apache.camel.kamelets.utils.format.converter.standard.JsonModelDataType;
+import
org.apache.camel.kamelets.utils.format.converter.standard.StringDataType;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeConverter;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeConverterResolver;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeLoader;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeRegistry;
+import org.apache.camel.kamelets.utils.format.spi.annotations.DataType;
+import org.apache.camel.support.CamelContextHelper;
+import org.apache.camel.support.service.ServiceSupport;
+import org.apache.camel.util.ObjectHelper;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Default data type registry able to resolve data types converters in the
project. Data types may be defined at the component level
+ * via {@link org.apache.camel.kamelets.utils.format.spi.annotations.DataType}
annotations. Also, users can add data types directly
+ * to the Camel context or manually to the registry.
+ *
+ * The registry is able to retrieve converters for a given data type based on
the component scheme and the given data type name.
+ */
+public class DefaultDataTypeRegistry extends ServiceSupport implements
DataTypeRegistry, CamelContextAware {
+
+private static final Logger LOG =
LoggerFactory.getLogger(DefaultDataTypeRegistry.class);
+
+private CamelContext camelContext;
+
+protected final List dataTypeLoaders = new ArrayList<>();
+
+private DataTypeConverterResolver dataTypeConverterResolver;
+
+private boolean classpathScan = true;
Review Comment:
must this be the default ?
given that each kamelet is gogin to have it's private registry, this may
have some impact on the footprint and startup time of integrations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162:
URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1034946699
##
library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/format/converter/aws2/s3/AWS2S3CloudEventOutputType.java:
##
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kamelets.utils.format.converter.aws2.s3;
+
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.Map;
+
+import org.apache.camel.Exchange;
+import org.apache.camel.component.aws2.s3.AWS2S3Constants;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeConverter;
+import org.apache.camel.kamelets.utils.format.spi.annotations.DataType;
+
+/**
+ * Output data type represents AWS S3 get object response as CloudEvent V1.
The data type sets Camel specific
+ * CloudEvent headers on the exchange.
+ */
+@DataType(scheme = "aws2-s3", name = "cloudevents", mediaType =
"application/octet-stream")
+public class AWS2S3CloudEventOutputType implements DataTypeConverter {
+
+static final String CAMEL_CLOUD_EVENT_TYPE = "CamelCloudEventType";
+static final String CAMEL_CLOUD_EVENT_SOURCE = "CamelCloudEventSource";
+static final String CAMEL_CLOUD_EVENT_SUBJECT = "CamelCloudEventSubject";
+static final String CAMEL_CLOUD_EVENT_TIME = "CamelCloudEventTime";
+
+@Override
+public void convert(Exchange exchange) {
+final Map headers = exchange.getMessage().getHeaders();
+
+headers.put(CAMEL_CLOUD_EVENT_TYPE, "kamelet.aws.s3.source");
Review Comment:
I was more thinking on the line of the suggestion made here:
https://github.com/apache/camel-k/issues/3668 so i.e:
`org.apache.camel.event.aws.s3.bucket_create`, so i.e. you can distinguish what
the event is about
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162:
URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r103538
##
library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/format/converter/standard/JsonModelDataType.java:
##
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kamelets.utils.format.converter.standard;
+
+import java.io.ByteArrayInputStream;
+import java.io.InputStream;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.camel.CamelContext;
+import org.apache.camel.CamelContextAware;
+import org.apache.camel.CamelExecutionException;
+import org.apache.camel.Exchange;
+import org.apache.camel.InvalidPayloadException;
+import org.apache.camel.component.jackson.JacksonDataFormat;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeConverter;
+import org.apache.camel.kamelets.utils.format.spi.annotations.DataType;
+import org.apache.camel.util.ObjectHelper;
+
+/**
+ * Data type converter able to unmarshal to given unmarshalType using jackson
data format.
+ *
+ * Unmarshal type should be given as a fully qualified class name in the
exchange properties.
+ */
+@DataType(name = "jsonObject", mediaType = "application/json")
+public class JsonModelDataType implements DataTypeConverter, CamelContextAware
{
+
+public static final String DATA_TYPE_MODEL_PROPERTY = "CamelDataTypeModel";
+
+private CamelContext camelContext;
+
+private static final ObjectMapper mapper = new ObjectMapper();
+
+@Override
+public void convert(Exchange exchange) {
+if (!exchange.hasProperties() ||
!exchange.getProperties().containsKey(DATA_TYPE_MODEL_PROPERTY)) {
+return;
+}
+
+ObjectHelper.notNull(camelContext, "camelContext");
+
+String type = exchange.getProperty(DATA_TYPE_MODEL_PROPERTY,
String.class);
+try (JacksonDataFormat dataFormat = new JacksonDataFormat(mapper,
camelContext.getClassResolver().resolveMandatoryClass(type))) {
Review Comment:
Also note that the a data format it likely needed to be[ initialized/started
](https://github.com/apache/camel/blob/bf98522eb05c8e477c182f11d15c302e69d0bc0d/components/camel-jackson/src/main/java/org/apache/camel/component/jackson/AbstractJacksonDataFormat.java#L531)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162:
URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1024570694
##
library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/format/converter/standard/JsonModelDataType.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kamelets.utils.format.converter.standard;
+
+import java.io.ByteArrayInputStream;
+import java.io.InputStream;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.camel.CamelExecutionException;
+import org.apache.camel.Exchange;
+import org.apache.camel.InvalidPayloadException;
+import org.apache.camel.component.jackson.JacksonDataFormat;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeConverter;
+import org.apache.camel.kamelets.utils.format.spi.annotations.DataType;
+
+/**
+ * Data type converter able to unmarshal to given unmarshalType using jackson
data format.
+ *
+ * Unmarshal type should be given as a fully qualified class name in the
exchange properties.
+ */
+@DataType(name = "jsonObject")
+public class JsonModelDataType implements DataTypeConverter {
+
+public static final String JSON_DATA_TYPE_KEY = "CamelJsonModelDataType";
+
+@Override
+public void convert(Exchange exchange) {
+if (!exchange.hasProperties() ||
!exchange.getProperties().containsKey(JSON_DATA_TYPE_KEY)) {
+return;
+}
+
+String type = exchange.getProperty(JSON_DATA_TYPE_KEY, String.class);
+try (JacksonDataFormat dataFormat = new JacksonDataFormat(new
ObjectMapper(), Class.forName(type))) {
Review Comment:
Creating an ObjectMapper per invocation seems quite expensive.
Ideally an object mapper from the registry should be used if present (i.e.
you may want to tweak how the object mapper works), if not a local but cached
one can be used.
##
library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/format/converter/aws2/s3/AWS2S3JsonOutputType.java:
##
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kamelets.utils.format.converter.aws2.s3;
+
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+
+import org.apache.camel.CamelExecutionException;
+import org.apache.camel.Exchange;
+import org.apache.camel.component.aws2.s3.AWS2S3Constants;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeConverter;
+import org.apache.camel.kamelets.utils.format.spi.annotations.DataType;
+import software.amazon.awssdk.core.ResponseInputStream;
+import software.amazon.awssdk.utils.IoUtils;
+
+/**
+ * Json output data type represents file name as key and file content as Json
structure.
+ *
+ * Example Json structure: { "key": "myFile.txt", "content": "Hello", }
+ */
+@DataType(scheme = "aws2-s3", name = "json")
+public class AWS2S3JsonOutputType implements DataTypeConverter {
+
+private static final String TEMPLATE = "{" +
+"\"key\": \"%s\", " +
Review Comment:
I'm not sure to understand the use case for this converter as it would mean
that the body would need to be unwrapped every time to access the actual
content and what happens if the actual payload is already a json or another
maybe binary format ?
##
kamelets/aws-ddb-sink.kamelet.yaml:
##
@@ -107,17 +113,24 @@ spec:
- "camel:aws2-ddb"
- "camel:kamelet"
templat
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162:
URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1024620655
##
library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/format/DefaultDataTypeRegistry.java:
##
@@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kamelets.utils.format;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+
+import org.apache.camel.CamelContext;
+import org.apache.camel.CamelContextAware;
+import org.apache.camel.ExtendedCamelContext;
+import org.apache.camel.RuntimeCamelException;
+import org.apache.camel.impl.engine.DefaultInjector;
+import org.apache.camel.impl.engine.DefaultPackageScanClassResolver;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeConverter;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeLoader;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeRegistry;
+import org.apache.camel.spi.PackageScanClassResolver;
+import org.apache.camel.support.service.ServiceSupport;
+
+/**
+ * Default data type registry able to resolve data types converters in the
project. Data types may be defined at the component level
+ * via {@link org.apache.camel.kamelets.utils.format.spi.annotations.DataType}
annotations. Also, users can add data types directly
+ * to the Camel context or manually to the registry.
+ *
+ * The registry is able to retrieve converters for a given data type based on
the component scheme and the given data type name.
+ */
+public class DefaultDataTypeRegistry extends ServiceSupport implements
DataTypeRegistry, CamelContextAware {
Review Comment:
I wonder if we need a registry or we could lazy load the converter when the
processor is created and initialized using the standard camel factory finder
mechanism which would work out of the box also on quarkus
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162:
URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1024896815
##
library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/format/converter/aws2/s3/AWS2S3JsonOutputType.java:
##
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.camel.kamelets.utils.format.converter.aws2.s3;
+
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+
+import org.apache.camel.CamelExecutionException;
+import org.apache.camel.Exchange;
+import org.apache.camel.component.aws2.s3.AWS2S3Constants;
+import org.apache.camel.kamelets.utils.format.spi.DataTypeConverter;
+import org.apache.camel.kamelets.utils.format.spi.annotations.DataType;
+import software.amazon.awssdk.core.ResponseInputStream;
+import software.amazon.awssdk.utils.IoUtils;
+
+/**
+ * Json output data type represents file name as key and file content as Json
structure.
+ *
+ * Example Json structure: { "key": "myFile.txt", "content": "Hello", }
+ */
+@DataType(scheme = "aws2-s3", name = "json")
+public class AWS2S3JsonOutputType implements DataTypeConverter {
+
+private static final String TEMPLATE = "{" +
+"\"key\": \"%s\", " +
Review Comment:
I think the main issue here is that we gonna have yet another envelope,
which may be ok for sync kamelets but I would be careful using it on a source.
I heard some similar discussion for i.e. debezium where the current payload
is a little bit like this one but it make it things very complicated and users
are often adding a transformation to unwrap the payload to be able to do
further transformations.
Food for thought
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162: URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1024898835 ## kamelets/aws-ddb-sink.kamelet.yaml: ## @@ -107,17 +113,24 @@ spec: - "camel:aws2-ddb" - "camel:kamelet" template: +beans: +- name: dataTypeRegistry Review Comment: it must be done outside kamelets but eventually you can explore about not needing a registry at all ad lazy loading converters. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162: URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1024899592 ## kamelets/aws-s3-source.kamelet.yaml: ## @@ -107,13 +107,28 @@ spec: description: The number of milliseconds before the next poll of the selected bucket. type: integer default: 500 + outputFormat: Review Comment: the problem is: how the user/tooling know what converters are available and what are supported ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [camel-kamelets] lburgazzoli commented on a diff in pull request #1162: chore: Introduce Kamelet input/output data types
lburgazzoli commented on code in PR #1162: URL: https://github.com/apache/camel-kamelets/pull/1162#discussion_r1025043230 ## kamelets/aws-s3-source.kamelet.yaml: ## @@ -107,13 +107,28 @@ spec: description: The number of milliseconds before the next poll of the selected bucket. type: integer default: 500 + outputFormat: Review Comment: yes but if we expect the user to provide custom converter, then the only way to validate it is to wait for runtime failures. I'm thinking about how tooling would provide support/validation for that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
