Need an advice how to poll enrich from Kafka topic aggregating on content property
Hi!
I need to implement Request-Reply EIP on Kafka.
But aggregation strategy have to be done on content's property.
Simple example
from(QUEUE_PROCESS_FAST_RISK_SCORING)
.setProperty(PROP_PAYMENT_DETAILS, body())
.bean(self,"assembleKycRequest") // request contains one property
"walletId"
.marshal().json()
.to(propertiesProvider.getKycRequestEndpoint()) // KAFKA REQUEST TOPIC
.pollEnrich(propertiesProvider.getWalletKycResponseEndpoint(),// KAFKA
RESPONSE TOPIC 3,
(oldExchange, newExchange) -> {
return oldExchange;
},false)
I need to aggregate response basing on property "walletId".
How do I deal with it?
--
Vyacheslav Boyko
mailto:mail4...@gmail.com
Re: Adding something new into Camel DSL -- but how?
Hi Adding EIPs or changes to the DSL is not a Camel end user task, and is not documented thoroughly. Adding new EIPs happens more rarely and there is maybe a little hesitation to add because when its in the DSL it stays for a long time. Also the DSL can become bloated - there are some mistakes from Camel v1 and v2 that we have today. That said to add a new EIP there are 2 ways - The EIP is implemented fully in the core, so all its code are in core. - The EIP is implemeted in a component and are located in camel-xxx in the components folder. Examples are circuit breakers, saga, kamelet and maybe others I forgot. So first you need to find out if its the former or the latter. However for both cases, the the DSL model are in the core and there is a fair bit of code to write to add a new model. Therefore its a good idea to find a simple EIP and see where its used in the code - can be ConvertBodyDefinition (are in core). Or if its outside then maybe KameletDefinition which is the latest EIP added. The reifer is the "guy" that sits between the model (DSL) and the processor (implementation), so all 3 are separated in their own JARs. And on top of that then there is also the new YAML DSL, so adding a new EIP can require some coding there too, but that can be done afterwards. At first is to get the regular Java and XML DSLs to work. The XML DSL is source generated with a maven plugin, and it can "have weird problems" if the DSL model is wrong. So if you have a very complex EIP then it can be tricky sometimes. On Mon, Nov 22, 2021 at 10:16 PM Steve973 wrote: > > Hi, all. I am working on a new version of the Dynamic Router processor > that implements the pattern spec much more closely, and I have the initial > implementation complete, but I want to test it (among other ways) by using > it in the DSL. I haven't seen any documentation on how to do this, or any > tutorials. Have I missed something? Could anyone give some kind of > overview? I could probably use a different processor and trace up through > the classes to see what all is involved, but I don't want to miss anything, > and this would be great documentation to have. If it would be helpful for > other contributors, I would not mind putting it together for inclusion in > the overall documentation. > > Thanks, > Steve -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
RE: Read HTML mails
Hi,
It's a little bit more complicated.
First, you have to know if the body mail is string or multipart or Nested
Because
if body is string, the mail doesn’t contain HTML content !
if body is multi-part, it can contains NestedMessage ( mail message in
attachment )
I didn't recommend working with HTML mail, it is very complicated.
We have been reading e-mail with Camel-mail (imaps) since 4 years now, and we
gave up HTML processing after 1 year.
We changed our approach, extracting the original message first and processed it
like a file attachment.
Then we look if body is com.sun.mail.imap.IMAPNestedMessage or java.lang.string
or java.mail.internet.MimeMultipart
and extract the TEXT of the body,because some times only have HTML without
alternative-text.
Here is a small part taken from our email processing, but enough to understand
how it works
${body} is "com.sun.mail.imap.IMAPNestedMessage"
def MimeUtility = new javax.mail.internet.MimeUtility();
// result =
request.body.getContent().getBodyPart(0).getContent()
result = "( Impossible d'extraire le Corps du message,
voir pièce jointe msg822 )"
${body} is "java.lang.String"
def MimeUtility = new javax.mail.internet.MimeUtility();
def Jsoup = new org.jsoup.Jsoup();
if (
request.getOriginalMessage().getContentType().toLowerCase().contains('html')) {
zresult =
MimeUtility.decodeText(request.getOriginalMessage().getContent());
result = Jsoup.parse(zresult).wholeText(); // preserve CRLF
} else {
result =
MimeUtility.decodeText(request.getOriginalMessage().getContent());
}
${body} is "javax.mail.internet.MimeMultipart"
zPjTemp = new File( request.headers['zzTEMPPJ'] )
zzPj = zPjTemp.listFiles();
if ( zzPj.length > 0 ) { result="OUI"}
else { result="NON" }
${header.zzSwitchlirePj} == "OUI"
Permet d'ecrire le courriel Original complet dans un fichier
def FileUtils = new org.apache.commons.io.FileUtils()
zRootTemp = new File( request.headers['zzTEMP'] )
z822Temp= new File( request.headers['zzTEMP822'] )
zPjTemp = new File( request.headers['zzTEMPPJ'] )
FileUtils.forceMkdir(z822Temp);
FileUtils.forceMkdir(zPjTemp);
FileUtils.cleanDirectory(z822Temp);
FileUtils.cleanDirectory(zPjTemp);
result ="\n- Extraction du Mail Originale \n"
zHeaders=""
crlf="\r\n"
headerEnum = request.getOriginalMessage().getAllHeaders();
while (headerEnum.hasMoreElements()) {
header = headerEnum.nextElement();
name = header.getName();
value = header.getValue().replaceAll('10.100.','192.168.');
zHeaders = zHeaders + name +": " + value + crlf
}
mailfilename = request.headers['zzTEMP822']+"Courriel.eml"
mailoriginal = new File(mailfilename);
mailoriginal.write(zHeaders+crlf);
mailoriginal.append(request.getOriginalMessage().getContentStream().getText()+crlf);
result = result + mailfilename + "\n- Extraction du Mail Originale - Complété
\n"
part= request.body.getBodyPart(0)
//part=
request.getOriginalMessage().getContent().getBodyPart(0)
def Jsoup = new org.jsoup.Jsoup();
def wlist(org.jsoup.safety.Whitelist basic ) {
return new org.jsoup.safety.Whitelist()
.addTags(
"a", "b", "blockquote", "br", "cite", "code",
"dd", "dl", "dt", "em",
"i", "li", "ol", "p", "pre", "q", "small",
"span", "strike", "strong", "sub",
"sup", "u", "ul")
.addAttributes("a", "href")
.addAttributes("blockquote", "cite")
.addAttributes("q", "cite")
.addProtocols("a", "href", "ftp", "http", "https",
"mailto")
.addProtocols("blockquote", "cite", "http", "https")
.addProtocols("cite", "cite", "http", "https")
}
if (
part.getContent().getClass().equals(com.sun.mail.imap.IMAPNestedMessage)) {
result = "( PAS DE CORPS DE MESSAGE / voir piece jointe ) ";
}
else if ( part.getContentType().toLowerCase().contains('html')) {
zbody = part.getContent();
zbody = Jsoup.clean(zbody,wlist()); // conserve
seulement certains tags de bases
zbody
[ANNOUNCE] Apache Camel 3.11.4 (LTS) Released
The Camel PMC is pleased to announce the release of Apache Camel 3.11.4 (LTS). Apache Camel is an open source integration framework that empowers you to quickly and easily integrate various systems consuming or producing data. This patch release contains 21 bug fixes and improvements. The release is available for immediate download at: https://camel.apache.org/download/ For more details please take a look at the release notes at: https://camel.apache.org/releases/release-3.11.4/
Read HTML mails
Hello, I am using camel-mail to poll a mailbox (HTML mails). I want to retrieve the HTML source of the mail, i didn't find how i can do this... I tried exchange.getIn().getBody() but it contains only text, tags are lost. Do i have to configure/declare something to keep it ? Can you help me ? Thanks you for informations...
AW: Azure Cosmo with MongoAPI migration
Hi Raymond,
thank you for response.
We use apache camel since version 2 with spring-boot and docker.
Now we are using apache camel 3.7 in aks (k8s) from azure
The Challenge:
When i call cosmosdb with mongoapi at azure:
@Configration
public static final String MONGODB_FIND_BY_ID = MONGODB_BASE + "&collection=" +
COLLECTION_COVER_PRODUCT_IN + "&operation=" + MongoDbOperation.findById;
@DSL
from(QUEUE_PRODUCT_IN).routeId("splitProduct")
.setProperty(IMPORTED_PRODUCT_ID).body()
.log(INFO, "Received Product to split with ID '${exchangeProperty." +
IMPORTED_PRODUCT_ID + "}'")
.log(DEBUG, "Received body: ${body}")
.log(INFO, "${body}")
.to(MONGODB_FIND_BY_ID)
It raised an exeception that $where is not supported from cosmosdb
Is there an example (github?)how to use the new component for azure cosmos.
best regards
Tobias
Von: ski n
Gesendet: Dienstag, 23. November 2021 18:49
An: users@camel.apache.org
Betreff: Re: Azure Cosmo with MongoAPI migration
Hi Tobias,
To get better response I advise to tell a bit more about the setup, the
challenges and the goals you are facing and where it relates to Camel. I
think Cosmos provide a compatibility API for MongoDB mapping Containers
(Collections at MongoDB) and Items (Documents at MongboDB) with each other.
Since version Camel 3.10 there is also a specific Cosmos DB component (
https://deu01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcamel.apache.org%2Fcomponents%2Fnext%2Fazure-cosmosdb-component.html&data=04%7C01%7Ctle%40covernet.de%7C7c173839f4304a55fc6408d9aea99d4b%7C208430bc6bc34416b38372f0236b7581%7C0%7C0%7C637732865774483097%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=tOhjLo2nwMz%2FQva%2FZn0nn1cDkf%2FlS5%2BqrBNeoARX1D4%3D&reserved=0).
Raymond
Op di 23 nov. 2021 om 16:12 schreef Tobias Letschka :
> Hi all,
>
> we use apache camel in ms azure + microservices (Version 3.7, AKS, k8s)and
> have legacy code.
>
> In legacy code (docker-compose) we used mongoDB.
>
> What could the best way to migrate to cosmodb with mongoAPI
>
> best regards
>
> Tobias
>
