Using StaX to parse xml document into database tables

[richie.rivi...@gmail.com]

Hi All,

I've just spent a few hours trying to implement a solution and am a bit
confused. My use case is that I have an XML document full of items. i.e.

.

There could be about 100 individual items in the document.

I am currently using StAXXPathSplitChoicePerformanceTest.java and
StAXJAXBIteratorExpressionTest.java as my guides to try and come up with a
good solution. The pattern I want to use is the message router pattern.

So far I have something like this...

MainApp.java   
ToStagingRouteBuilder.java
  
AbstractFileRouteBuilder.java
 
 
Item.java   
application-context.xml
  
camel-context.xml
  

The bit that reads the XML file from the input directory works. And I know
when my route runs and my processor does a String xml =
exchange.getIn().getBody(String.class); the xml in the file gets read
correctly.

Where I am stuck is that jaxb doesn't seem to have split the items into
individual item records.
i.e. This code returns empty and I get an array Index out of bounds.

/   protected void getObjects() {
recordsEndpoint.expectedMessageCount(10);
recordsEndpoint.allMessages().body().isInstanceOf(Item.class);
Item item =
recordsEndpoint.getReceivedExchanges().get(0).getIn().getBody(Item.class);
System.out.println(item.getItemNo());
}/  

Does what I am doing in my code make sense. Can anyone help me with what I
am missing to get the parsing to work? 


Also if I do get the parsing of the xml file working can anyone provide me
some advice on a nice design pattern for me to use to write to the db. 


thank you!



--
View this message in context: 
http://camel.465427.n5.nabble.com/Using-StaX-to-parse-xml-document-into-database-tables-tp5741319.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Help to re-write route

[richie.rivi...@gmail.com]

Hi guys,

I was wondering if someone could give me some help out how to re-write this
camel test into a proper camel program. The issue is I know how to write a
simple route and then execute it. But after the route is executed and I want
to grab the parsed xml in a fashion similar to testStaxExpression how would
I do this in a real Camel application. I know the configure method in the
route is not the right place to do it.

Where is the right place. Sorry for the easy questions. I am a beginner.



package org.apache.camel.component.stax;

import org.apache.camel.EndpointInject;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.mock.MockEndpoint;
import org.apache.camel.component.stax.model.Record;
import org.apache.camel.component.stax.model.RecordsUtil;
import org.apache.camel.test.junit4.CamelTestSupport;
import org.junit.BeforeClass;
import org.junit.Test;

import static org.apache.camel.component.stax.StAXBuilder.stax;

public class StAXJAXBIteratorExpressionTest extends CamelTestSupport {
@EndpointInject(uri = "mock:records")
private MockEndpoint recordsEndpoint;

@BeforeClass
public static void initRouteExample() {
RecordsUtil.createXMLFile();
}

@Override
public RouteBuilder createRouteBuilder() {
return new RouteBuilder() {
@Override
public void configure() throws Exception {
log.info("Doing route");
// START SNIPPET: e1
from("file:target/in")
// split the file using StAX (the stax method is from
StAXBuilder)
// and use streaming mode in the splitter
.split(stax(Record.class)).streaming()
.to("mock:records");
// END SNIPPET: e1
}
};
}

@Test
public void testStaxExpression() throws InterruptedException {
recordsEndpoint.expectedMessageCount(10);
recordsEndpoint.allMessages().body().isInstanceOf(Record.class);

recordsEndpoint.assertIsSatisfied();

Record five =
recordsEndpoint.getReceivedExchanges().get(4).getIn().getBody(Record.class);
assertEquals("4", five.getKey());
assertEquals("#4", five.getValue());
}
}




--
View this message in context: 
http://camel.465427.n5.nabble.com/Help-to-re-write-route-tp5741339.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Help to re-write route

[richie.rivi...@gmail.com]

No quite. I know that If I want to create a separate class for the route i
can do this...

public class MyRouteBuilder extends RouteBuilder {  
@Override
public void configure() throws Exception {
from("file:src/data?noop=true") 
.split(stax(Record.class)).streaming()
.to("mock:records");
}
}

And then I can call it from a Main...

public class MainApp {
public static void main(String... args) throws Exception {
Main main = new Main();
main.enableHangupSupport();
main.addRouteBuilder(new MyRouteBuilder());
main.run(args);
}
}

But what I am confused about is where would I place the bit where the
endpoint is injected in etc...
@EndpointInject(uri = "mock:records")
public static MockEndpoint recordsEndpoint;

and then...

recordsEndpoint.expectedMessageCount(10);
recordsEndpoint.allMessages().body().isInstanceOf(Record.class);
recordsEndpoint.assertIsSatisfied();
Record five =
recordsEndpoint.getReceivedExchanges().get(4).getIn().getBody(Record.class);

I'm asking because everywhere I put that code I can't get it to work. Yet in
the unit test it works perfectly!

appreciate your advice.






--
View this message in context: 
http://camel.465427.n5.nabble.com/Help-to-re-write-route-tp5741339p5741381.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Help to re-write route

[richie.rivi...@gmail.com]

scrap this question. I'm following the camel-example-etl code from the camel
examples instead.

thanks



--
View this message in context: 
http://camel.465427.n5.nabble.com/Help-to-re-write-route-tp5741339p5741389.html
Sent from the Camel - Users mailing list archive at Nabble.com.

How to write a type converter?

[richie.rivi...@gmail.com]

Hi all (sorry in advance for the long post),

I have been tweaking the camel-example-etl project to what I require for my
project. My requirements are very similar. I need to parse an xml document
and then insert the parsed entities as records into the database.

The structure of my xml is different to the example. Here is what my xml
looks like.


   A
   
 93501250080
 
   
   0
   0
   .00
 
 
   0
   .00
   0
   .00
   0
   189.95
 
 Y
   
 

So far I have succesfully managed to get jaxb to parse my xml using four
beans (hooray!).

1. ItemDocuments (holds a list of ItemDocument)
2. ItemDocument
3. PromotionDocument
4. SellPriceDocument

So after parsing the document I end up with a single ItemDocuments class
holding an ArrayList of ItemDocument which in turn has a PromotionDocument
and a SellPriceDocument.

Now I need to insert my List of ItemDocument into the database as records. I
have three entity beans (ItemEntity, PromotionEntity, SellPriceEntity). 

I am confused about how to change the Type Converter to do this for me. In
the original camel-example-etl project there was not a collection of parsed
records, it was only parsing 1 record at a time and only using 1 entity.

So I have a few questions...

1. How do I re-write my route below to convert to the three separate entity
beans? i.e. One parsed record for me should lead to three db inserts into my
three entities.

public class ItemEtl extends SpringRouteBuilder {
public void configure() throws Exception {
System.err.println("CONVERT STARTED");  
from("file:src/data?noop=true") 
.convertBodyTo(ItemDocuments.class)  
.to("jpa:org.apache.camel.example.etl.ItemEntity");
System.err.println("CONVERT FINISHED"); 
}
}

2. How do I write my Type Converter to handle my ItemDocuments and my list
of ItemDocument. In t he original example there is just a single record not
a list of them. I tried something like below but very unsure about it. 

@Converter
public class ItemsTransformer {
private static final transient Logger LOG =
LoggerFactory.getLogger(ItemsTransformer.class);
@Converter
public List toItem(ItemDocuments docs, Exchange exchange)
throws Exception {
JpaTemplate template =
exchange.getIn().getHeader("CamelJpaTemplate", JpaTemplate.class);
List itemEntities = new ArrayList();
List itemDocumentList = docs.getItemDocumentList();
for (ItemDocument doc : itemDocumentList) {

String itemNo = doc.getItemNo();   
ItemEntity item = findItemByItemNumber(template, itemNo);

item.setItemNo(itemNo);
item.setClassNo(item.getClassNo());
item.setDescription(item.getDescription());
item.setGstCode(item.getGstCode()); 
LOG.debug("Created object item: " + itemNo);
itemEntities.add(item);
}
return itemEntities;
}
protected ItemEntity findItemByItemNumber(JpaTemplate template, String
itemNo) throws Exception {
List list = CastUtils.cast(template.find("select x from
item"
 + " x where
x.item_no = ?1", itemNo));
if (list.isEmpty()) {
ItemEntity answer = new ItemEntity();
answer.setItemNo(itemNo);
return answer;
} else {
return list.get(0);
}
}
}

Of course it does not work. My error is ...

No type converter available to convert from type:
org.apache.camel.example.etl.ItemDocuments to the required type:
org.apache.camel.example.etl.ItemEntity with value
org.apache.camel.example.etl.ItemDocuments@535a4838

I would appreciate any help that anyone could give me. 

thank you



--
View this message in context: 
http://camel.465427.n5.nabble.com/How-to-write-a-type-converter-tp5741592.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: How to write a type converter?

[richie.rivi...@gmail.com]

thanks. This is exactly the type of suggestion I was looking for. Since I'm
new to Camel I'm still learning the concepts and different patterns. 

I'm going to try that right now. Will let you know how I go.



--
View this message in context: 
http://camel.465427.n5.nabble.com/How-to-write-a-type-converter-tp5741592p5741624.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: How to write a type converter?

[richie.rivi...@gmail.com]

I liked your original idea of doing a split so I had a look around and found
that chapter 8 in the Camel in Action book had a good example.

To split the ItemDocuments what I did was this

public void configure() throws Exception {
System.err.println("CONVERT STARTED");  
from("file:src/data?noop=true") 
.convertBodyTo(ItemDocuments.class)  
.split().method(ItemDocumentService.class,
"splitItemsDocuments")
.log("Split line ${body}")
.to("jpa:org.apache.camel.example.etl.ItemEntity"); 
System.err.println("CONVERT FINISHED"); 
}

public class ItemDocumentService {
public List splitItemsDocuments(ItemDocuments docs) {
return docs.getItemDocumentList();
}
}


That has worked. It has split the itemDocuments into ItemDocument objects
and I am now entering my TypeConverter code.

I have another point of confusion now though. In my type converter if only
only persist my item object the item records get inserted into the database
(success!). However, if I try to persist the promotion and sell price
documents with the item document my application starts to spit errors.

Caused by: 
org.apache.openjpa.persistence.PersistenceException: Incorrect integer
value:
'\xAC\xED\x00\x05sr\x00,org.apache.camel.example.etl.PromotionEntity$\x0C\xF5\xF1\x08\x0B\xA2\x81\x02\x00\x05L\x00\x02idt\x00\x10'
for column 'ITEM_PROMOTION_ID' at row 1 {prepstmnt 1042322838 INSERT INTO
item (id, ATTRIBUTE_1, ATTRIBUTE_3, ATTRIBUTE_2, BRAND_LOGO_FILE_NAME,
BRAND_NAME, CLASS_NO, DEFAULT_MARGIN, DESCRIPTION, EXTENDED_DESCRIPTION,
EXTENDED_DESCRIPTION_2, GST_CODE, IMAGE_FILE_NAME, ITEM_NO,
OUT_OF_STOCK_IND, PACK_QTY, ITEM_PROMOTION_ID, ITEM_SELL_PRICE_ID,
SELLING_UNIT, SIZE_APPLICABLE, STOCK_AVAILABLE, SPPLR_NO, VOLUME,
WEB_AGE_GROUP, WEB_COLOR_DESCRIPTION, WEB_DESCRIPTION, WEB_SIZE_DESCRIPTION,
WEIGHT) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?)} [code=1366, state=HY000]
FailedObject: org.apache.camel.example.etl.ItemEntity@2d8bfa3f
at 
org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4958)
at
org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4918)
at
org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:136)
at
org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:78)
at
org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:143)
at
org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:79)
at
org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInternal(PreparedStatementManagerImpl.java:99)
at
org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(PreparedStatementManagerImpl.java:87)
at
org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:550)
at
org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:106)
at
org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:59)
at
org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:105)
at
org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:78)
at
org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:735)
at
org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:131)
... 101 more

What I was thinking was that I could have two things wrong here.

1. In my route I have only specified
.to("jpa:org.apache.camel.example.etl.ItemEntity"); Do I need to specify the
other entities too? Or because there is a relationship between the entities
will JPA be smart enough to figure it out.

2. Perhaps I have the JPA entities set up incorrectly. In my JPA item entity
i have the relationships set up like this...

@OneToOne(cascade = CascadeType.ALL)
@JoinColumn(name="ITEM_PROMOTION_ID")
private PromotionEntity promotion;

@OneToOne(cascade = CascadeType.ALL)
@JoinColumn(name="ITEM_SELL_PRICE_ID")
private SellPriceEntity sellPrice;

I'm thinking it is the 1st one. My route is not defined properly. Do you
guys have any thoughts?

thanks



--
View this message in context: 
http://camel.465427.n5.nabble.com/How-to-write-a-type-converter-tp5741592p5741628.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: JPA type converter for multiple entities?

[richie.rivi...@gmail.com]

Just bumping this question as I'm struggling with a solution. If I am trying
to persist to three separate entities should my routes specify all three
entities?

Or is it enough for only the first entity to be specified in the route and
then to rely on the relationships defined to the other two beans to allow
for inserts into the other tables?

thanks again



--
View this message in context: 
http://camel.465427.n5.nabble.com/How-to-write-a-type-converter-tp5741592p5741645.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: JPA type converter for multiple entities?

[richie.rivi...@gmail.com]

okay thanks. My main concern is that I am defining my route properly. At the
moment I am only defining my route to the item entity and I was thinking
maybe because I had not specified the other entities it was using null or
something like that.

So it seems like from the response my problem is not my route but more a
problem with how I have configured JPA. I will take a look at my JPA
definitions and post my resolution... assuming I find it.

thanks again for your help.



--
View this message in context: 
http://camel.465427.n5.nabble.com/How-to-write-a-type-converter-tp5741592p5741711.html
Sent from the Camel - Users mailing list archive at Nabble.com.

JPA component - Persist 1 record working but how to persist many records?

[richie.rivi...@gmail.com]

Hi guys,

Fairly new to Apache Camel. I have a Camel program which parses some XML
using Jaxb and then inserts the parsed objects into the database using open
JPA. The example is very similar to the Camel ETL example. At the moment I
have the application persisting records from the XML however it is only
persisting the first record! 

I'm looking for some advice on how to change the application so that it
inserts all parsed objects (all items in my case).

My item XML looks like this...



 
   
 93501250080
 KHOMBU APFOOTA KOKO HIGH
   
 
 
   
 93501250090
 TEST
   
 


Here is what my route looks like...

from("file:src/data?fileName=FIT.XML&noop=true")
.convertBodyTo(ItemRootDocument.class)  
.split().method(ItemDocumentService.class,
"splitItemsDocuments")
.log("Split line ${body}")
.to("jpa:org.apache.camel.example.etl.ItemEntity");

Here is my Split...

public List splitItemsDocuments(ItemRootDocument doc) {
ItemDocuments itemDocuments = doc.getItemDocumentsList().get(0);
return itemDocuments.getItemDocumentList(); 
}

And my ItemTransformer to convert to an ItemEntity looks like this...

 public class ItemTransformer {

@Converter
public ItemEntity toItem(ItemDocument  doc, Exchange exchange)
throws Exception { 
  JpaTemplate template =
exchange.getIn().getHeader("CamelJpaTemplate", JpaTemplate.class);
  String itemNo = doc.getItemNo(); 
  ItemEntity item = new ItemEntity();
  item.setItemNo(itemNo);
  item.setDescription(doc.getDescription());
  return item;
}

So it splits the Items in the XML document and persists but only persists
the first one. How would I change this so that all parsed items are split?
Thanks







--
View this message in context: 
http://camel.465427.n5.nabble.com/JPA-component-Persist-1-record-working-but-how-to-persist-many-records-tp5742267.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: JPA component - Persist 1 record working but how to persist many records?

[richie.rivi...@gmail.com]

Hi All,

Well I spent a few hours on this today trying to understand how to do this.
What I did was to change my splitter to do this...

public List splitItemsRootDocuments(ItemRootDocument 
doc) {
List itemDocuments = doc.getItemDocumentsList();
return itemDocuments;
}

It returns both items 93501250080 and 93501250090.

Then I changed my converter...

 public class ItemTransformer {
@Converter
public ItemEntity toItem(ItemDocuments docs, Exchange exchange)
throws Exception {
  JpaTemplate template =
exchange.getIn().getHeader("CamelJpaTemplate", JpaTemplate.class);
  List items = new ArrayList();
  List itemDocs = docs.getItemDocumentList();
  for (Iterator i = itemDocs.iterator(); i.hasNext();){
  ItemDocument doc = (ItemDocument)i.next();
  String itemNo = doc.getItemNo(); 
  ItemEntity item = new ItemEntity();
  item.setItemNo(itemNo);
  item.setDescription(doc.getDescription());
  }
  return items;
} 

to return a list of ItemEntity's. (i.e. not just the one). Does this seem
like the right approach. I thinking it is. 

The only error I have now is ...

No type converter available to convert from type:
org.apache.camel.example.etl.ItemDocuments to the required type:
org.apache.camel.example.etl.ItemEntity

Because I'm now returning a list of ItemEntity objects instead of just the
single one. I don't know how to fix this yet but will try to find out.

Comments to tell me if I am heading down the right track would be much
appreciated.

thanks



--
View this message in context: 
http://camel.465427.n5.nabble.com/JPA-component-Persist-1-record-working-but-how-to-persist-many-records-tp5742267p5742269.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: JPA component - Persist 1 record working but how to persist many records?

[richie.rivi...@gmail.com]

Thank you so much Taariq,

Your advice was exactly what I needed to give me enough information to
resolve my issue.

public List splitItemsRootDocuments(ItemRootDocument doc) 
{
List items = new ArrayList();
List itemDocumentsList = 
doc.getItemDocumentsList();
for (ItemDocuments itemDocuments : itemDocumentsList) {
items.addAll(itemDocuments.getItemDocumentList());
  }
return items;
}


regards



--
View this message in context: 
http://camel.465427.n5.nabble.com/JPA-component-Persist-1-record-working-but-how-to-persist-many-records-tp5742267p5742278.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Springws component - beginner

[richie.rivi...@gmail.com]

Hi,

I am about to embark on setting up my project to call soap web services
(magento soap services) and process the responses using spring ws. I am
thinking spring ws only because I am very comfortable with spring.

I know that there is an example camel-example-spring-ws which says that it
exposes a soap based web service. I don't think this is what I'm after. I
need an example of just calling and processing the response of a soap based
web service.

Am I right? Can anyone point me out to a simple example where I can find
something like that?

thanks



--
View this message in context: 
http://camel.465427.n5.nabble.com/Springws-component-beginner-tp5743283.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Ideas on how to structure JPA endpoint application

[richie.rivi...@gmail.com]

Hi guys,

I am looking for some general guidance on how I should structure my JPA
endpoint route. i.e. what would be a standard approach to my issue.

My camel route parses an XML document containing many items then inserts the
items into a db using a JPA enpoint. The way I did this was to structure the
route something like this

from("file:src/data?fileName=FIT32.XML&noop=true")  
.convertBodyTo(
ItemDocument.class) 
.split().method(
ItemDocumentService.class,
"splitItemDocument")
.to("jpa:org.apache.camel.auski.etl.entity.ItemEntity");

ItemDocumentService > splitItemDocument...

public List splitItemDocument(ItemDocument doc) {
List items = new ArrayList();
List itemDocumentsList = doc.getItems();
for (Items itemDocuments : itemDocumentsList) {
  items.addAll(itemDocuments.getItems());
 }
return items;
}

And then my Type Converter would run through each individual item returned
by the split and return an ItemEntity. JPA would then take care of the rest
and insert into the item table.

Now I have decided to change the structure of my db and have a table which
records batch import runs. For example the batch table would have...

id,LoadDateTime,Filename
1,10:50am,items_20131116.xml

And the item table would have...
id,item_price,item_code,import_id
9,$5,9,1

To achieve try and achieve this I've restructured my route to look like...

from("file:src/data?fileName=FIT32.XML&noop=true")  
.process(new Processor() {
public void process(Exchange exchange) throws 
Exception {   
String filename = (String)
exchange.getIn().getHeader("CamelFileName");
String body = 
exchange.getIn().getBody(String.class);

ImportPayloadEntity importPayloadEntity = new
ImportPayloadEntity();
importPayloadEntity.setFilename(filename);
importPayloadEntity.setLoadDateTime(new Date());

exchange.getIn().setHeader("importPayloadEntity",
importPayloadEntity);
}
})
.convertBodyTo(
ItemDocument.class) 
.split().method(
ItemDocumentService.class,
"splitItemDocument")

.to("jpa:org.apache.camel.auski.etl.entity.ImportPayloadEntity");
}

So what I have done is to add a new object inside the header which I can
access in my TypeConverter. And my TypeConverter definitely gets the new
header object. 

Now here comes the question

How do I best persist the ImportPayloadEntity and the items from here. If I
keep the route as per what I have changed it to, in my type converter I
would need to loop through all the items that JaxB has parsed (could be as
many as 200) and then associate them to the batch table entity
(ImportPayloadEntity). I can't help thinking this is a bad thing to do...

Or perhaps there is a better way to persist the items and the batch record.
Could I please get your thoughts?

I hope I have explained the problem well. Please let me know if more info is
required.

thanks


So what that would mean is that if my xml document has 120 individual items
in it the type converter would have tun
 



--
View this message in context: 
http://camel.465427.n5.nabble.com/Ideas-on-how-to-structure-JPA-endpoint-application-tp5743372.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Ideas on how to structure JPA endpoint application

[richie.rivi...@gmail.com]

Forget this question. I solved it myself. Instead of splitting the
ItemDocument into individual items I chose not to split and then in the
TypeConverter I just convert everything that I have parsed using JaxB. It
seems to be pretty permanent. 

But I'm still not sure if it's a great idea to just loop through my entire
xml document in the type converter. Especially if it is quite large.

Perhaps someone could chip in with a comment concerning this. Whether or not
there is a less memory intensive way... or does Camel handle it fine.

regards



--
View this message in context: 
http://camel.465427.n5.nabble.com/Ideas-on-how-to-structure-JPA-endpoint-application-tp5743372p5743390.html
Sent from the Camel - Users mailing list archive at Nabble.com.

JPA enpoint to CSV enpoint working but only 1 row

[richie.rivi...@gmail.com]

Hi all,

I have written a route that executes a JPA named query and then marshall's
the object into csv...

private void configureCsvExportRoute () throws Exception {   
from(END_POINT_JPA_ITEM_EXPORT)   
.marshal()
.csv() 
.to(END_POINT_FILE_ITEM_EXPORT);  
}

It works fine however the query picks up about 100 rows but I'm only left
with 1 row in the csv. When having a look at the output in the console I can
see that the csv is being overwritten for each row.

How can I achieve all the rows being created in the csv? Do I have to use an
aggregator? If so what would it look like.

thanks



--
View this message in context: 
http://camel.465427.n5.nabble.com/JPA-enpoint-to-CSV-enpoint-working-but-only-1-row-tp5743483.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Execute a spedific route from unix cron

[richie.rivi...@gmail.com]

Hi All,

I have written a camel application that I would like to execute from the
crontab. My application has a main class which looks like ...

public class Main extends org.apache.camel.spring.Main {

public static void main(String... args) throws Exception {
new Main().run(args);
}
}

And it has several routes. So it's easy enough to execute camel and tell it
to run all the routes but how would I tell it to only run a specific route
from the unix cron or just any other unix command line for that matter?

thanks



--
View this message in context: 
http://camel.465427.n5.nabble.com/Execute-a-spedific-route-from-unix-cron-tp5744010.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Execute a spedific route from unix cron

[richie.rivi...@gmail.com]

Yes your right letting cron right files out is not very nice. I'm not sure
about quartz because we didn't really want the application to live. The
original idea for us what to just startup our camel app when the routes
needed to run. I would have been fine if I only had a single route in my
application however with multiple routs what options do I have?

Is the standard out there for the camel application to live for the entire
day and for jobs to be scheduled via quartz?



--
View this message in context: 
http://camel.465427.n5.nabble.com/Execute-a-spedific-route-from-unix-cron-tp5744010p5744060.html
Sent from the Camel - Users mailing list archive at Nabble.com.

CamelLock file being created and not sure why

[richie.rivi...@gmail.com]

Hi,

I have a route that grabs files from an input directory, does some
operations and then moves files to another directory.

The route works well and my file is moved post processing. My issue is that
a camelLock file gets left in the source directory after processing. I read
somewhere in the camel forums that you need to set the unit of work is done
to complete this and I've tried this (see route below where I add a
processor after the last operation in my route) but it didn't work and it
didn't make sense to do that either. 

Can anyone help me out please.

thanks

Here is what the route looks like.

from("file:src/inbox/items/?preMove=processed&move=done") 
.process(new Processor() {
public void process(Exchange exchange) throws 
Exception {   
String filename = (String)
exchange.getIn().getHeader(CamelConstants.HEADER_KEY_CAMEL_FILE_NAME);
String body = 
exchange.getIn().getBody(String.class);
ImportPayloadEntity importPayloadEntity = new
ImportPayloadEntity();

importPayloadEntity.setPayloadType(DaoConstants.ITEM_PAYLOAD);
importPayloadEntity.setPayloadBody(body);
importPayloadEntity.setFilename(filename);
importPayloadEntity.setLoadDateTime(new Date());

importPayloadEntity.setProcessedInd(DaoConstants.NOT_PROCESSED);
importPayloadEntity.setProcessedDatetime(null);
   
exchange.getIn().setHeader(CamelConstants.HEADER_KEY_IMPORT_PAYLOAD,
importPayloadEntity);
}
}) 
.convertBodyTo(
ItemDocument.class) 

.to("jpa:org.apache.camel.etl.entity.ImportPayloadEntity")
.process(new Processor() {
public void process(Exchange exchange) throws 
Exception {   
exchange.getUnitOfWork().done(exchange);
}
});



--
View this message in context: 
http://camel.465427.n5.nabble.com/CamelLock-file-being-created-and-not-sure-why-tp5744115.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Success but still confused about how to shut down my application

[richie.rivi...@gmail.com]

Hi All,

I've been reading through Camel in action today but am a bit confused about
how to shut down my application. I have it working but ultimately my
understanding is lacking.

My entry point into the application originally looked like below. It loads a
spring application context.

public class Main extends org.apache.camel.spring.Main {  
public static void main(String... args) throws Exception {
new Main().run(args);
}
}
Using this approach the application just lives indefinitely. 

The first thing I tried was to add the following into my application
context...



 

http://camel.apache.org/schema/spring";>
org.apache.camel.etl.route


This did not work and my application still lived after executing the route.

The next thing I tried was to change my main
public class Main extends org.apache.camel.spring.Main {  
Main main = new Main();
main.enableHangupSupport();
main.start();
main.run(args);
main.stop();
}
}

This did not work and my application still died straight away before
executing the route.

The next thing I tried was to change my main again
public class Main extends org.apache.camel.spring.Main {  
String filename = "META-INF/spring/camel-context.xml";
AbstractXmlApplicationContext spring =
   new ClassPathXmlApplicationContext(filename);
spring.start();
Thread.sleep(3);
spring.stop();
spring.destroy();   
}
}

Success, because of the sleep the application runs and then before it shuts
down it waits to check what routes have not completed because in my routes I
have specified .shutdownRoute(ShutdownRoute.Defer).


So I guess I can keep my application like this as it does what i want. But
appreciate if someone could respond in regards to why the application didn't
work properly for first two attempts. I am in particular interested as to
why the bean injection method in the spring application context files did
not work. And also why I have to allow for a sleep time in camel before
trying to shut down.

thanks for your time









--
View this message in context: 
http://camel.465427.n5.nabble.com/Success-but-still-confused-about-how-to-shut-down-my-application-tp5744116.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: CamelLock file being created and not sure why

[richie.rivi...@gmail.com]

Well I don't know what I did but the problem seems to be gone now. I didn't
do anything to fix it. Maybe it was a resource thing relating to me having
it open somewhere else?



--
View this message in context: 
http://camel.465427.n5.nabble.com/CamelLock-file-being-created-and-not-sure-why-tp5744115p5744117.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: CamelLock file being created and not sure why

[richie.rivi...@gmail.com]

actually scrap that. the problem is back...lol



--
View this message in context: 
http://camel.465427.n5.nabble.com/CamelLock-file-being-created-and-not-sure-why-tp5744115p5744118.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: CamelLock file being created and not sure why

[richie.rivi...@gmail.com]

I'm using 2.11. Its strange. Sometime it's happening and other times it
doesn't.

thanks



--
View this message in context: 
http://camel.465427.n5.nabble.com/CamelLock-file-being-created-and-not-sure-why-tp5744115p5744121.html
Sent from the Camel - Users mailing list archive at Nabble.com.

error handling design pattern advice

[richie.rivi...@gmail.com]

Hi All,

I've got a question about error handling. Here is my scenario...

I have a route which calls a bean..

from(...JPA...)
.errorHandler(deadLetterChannel("log:dead")
.maximumRedeliveries(5)
.retryAttemptedLogLevel(LoggingLevel.ERROR))
.to("bean:serviceBean?method=login")
.to("bean:serviceBean?method=createSimpleProduct")


Where my service bean looks like this...

public void login(Exchange exchange) {
SharedModel model = new SharedModel();
exchange.getOut().setHeaders(exchange.getIn().getHeaders());
exchange.getOut().setBody(model);
}

public void createSimpleProduct(Exchange exchange) {
ImportPayloadEntity importPayloadEntity =
(ImportPayloadEntity)exchange.getIn().getHeader(CamelConstants.HEADER_KEY_IMPORT_PAYLOAD);
Collection itemEntities = 
importPayloadEntity.getItems();
for (ItemEntity item : itemEntities) {  
 
  SharedModel model = ((SharedModel)
(exchange.getIn().getBody()));
  
  call soap call for each item in loop and use
attributes from SharedModel in calls
}
}


So what happens currently is that when there is an error in create simple
product, because the model is being used as the message, the route will fail
(after unsuccessful retries) when it comes across the first bad item.

I have been wondering how could I change this so that it sidelines only the
bad item and keep going through the rest?

I had some tries at changing my route to use a Splitter to split my route
into individual item messages (which might work nicely) but then I realized
that I would lose my sharedModel as the message body which i need that to
make the service calls. 

What is the best design pattern for this situation?

much appreciated
 



--
View this message in context: 
http://camel.465427.n5.nabble.com/error-handling-design-pattern-advice-tp5744364.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: error handling design pattern advice

[richie.rivi...@gmail.com]

Thanks all for your replies. I have taken Kraythe's advice. Rather than just
putting a try catch in createSimpleProduct, in my route I did some splitting
so that the createSimpleProduct gets called for each item.

I guess I could have just put a try catch in but this way seems to be more
of a camel way.





--
View this message in context: 
http://camel.465427.n5.nabble.com/error-handling-design-pattern-advice-tp5744364p5744591.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Apache Camel File component preMove, move and moveFailed relative rather than absolute

[richie.rivi...@gmail.com]

Hi guys,

I've got a question about preMove, move and moveFailed. PreMove seems to be
relative to the input directory and move seems to be relative to the preMove
directory. For example if I do a..

from("file:{{dir.item.inbox}}?"
+ "doneFileName={{filename.item.trigger}}&"
+ "preMove={{dir.item.staging}}&"
+ "move={{dir.item.done}}&"
+ "moveFailed={{dir.item.error}}")   

where 

dir.item.inbox=src/inbox/items/
dir.item.staging=staging
dir.item.done=.done
dir.item.error=.error
filename.item.trigger=NEWFILES.TRG

when the file is processed successfully I get the original file moved into a
done directory in the following folder structure...

src>inbox>items>staging>.done

I assume it is the same for error although I haven't been able to test this
yet.

Is there anyway to make the paths not relative to each other. For example is
t here someway I could specify
dir.item.inbox=src/inbox/items/
dir.item.staging=src/inbox/items/staging
dir.item.done=src/inbox/items/done
dir.item.error=src/inbox/items/error
filename.item.trigger=NEWFILES.TRG

I've actually tried this but the directory structure that was created was
nasty. lol

thanks for your help.




--
View this message in context: 
http://camel.465427.n5.nabble.com/Apache-Camel-File-component-preMove-move-and-moveFailed-relative-rather-than-absolute-tp5744592.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Inject an endpoint into my route for testing purposes

[richie.rivi...@gmail.com]

Hi,

Sorry to ask a few questions. I know i have asked a few recently. I have
been looking up bean and endpoint injection today but can't find a blurb on
my scenario.

I have a route which uses a file component as the from endpoint...

from("file:{{dir.item.inbox}}?"
+ "doneFileName={{filename.item.trigger}}&"
+ "preMove={{dir.item.staging}}&"
+ "move={{dir.item.done}}&"
+ "moveFailed={{dir.item.error}}")

All my routes get loaded from spring via

http://camel.apache.org/schema/spring";>

org.apache.camel.etl.route


For testing purposes it is annoying when the file gets moved all the time on
processing. I want to just change it to noop for testing purposes. What is
the best way to do this. Can I do some sort of endpoint injection from
Spring?

thanks




--
View this message in context: 
http://camel.465427.n5.nabble.com/Inject-an-endpoint-into-my-route-for-testing-purposes-tp5744594.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Apache Camel File component preMove, move and moveFailed relative rather than absolute

[richie.rivi...@gmail.com]

ok. As long as this is the way it is meant to work. thank you. yes making
your change worked.



--
View this message in context: 
http://camel.465427.n5.nabble.com/Apache-Camel-File-component-preMove-move-and-moveFailed-relative-rather-than-absolute-tp5744592p5744632.html
Sent from the Camel - Users mailing list archive at Nabble.com.

JPA @Consumed

[richie.rivi...@gmail.com]

Hi guys,

I have a route which selects from the database using a JPA entity bean. I
then do some processing on the bean in the route and would like to mark the
bean as processed when the route is complete.

Seems like a pretty typical scenario for me to use @Consumed in my entity
bean. The select works, the processing works but for some reason I don't get
the updates i do in @consumed reflected in the database.

Can someone give me a tip as to why this is not happening.

thanks

Sorry for the long code extract but my route looks like this...

from("direct:itemExport")

//route generics
.routeId("itemExportMagento")
.autoStartup("{{item.export.magento}}") 
.startupOrder(2) 
.shutdownRoute(ShutdownRoute.Defer)
.errorHandler(deadLetterChannel("log:dead")
.maximumRedeliveries(5)
.retryAttemptedLogLevel(LoggingLevel.ERROR))

//route specifics

.to("jpa://org.apache.camel.auski.etl.entity.ImportPayloadEntity?consumer.namedQuery=queryImportPayloads&consumeDelete=false&delay=3000&consumeLockEntity=false")
.to("itemExportMagentoFromSourceHeaderProcessor")   
.to("bean:serviceBean?method=login")
.to("itemExportMagentoFromLoginBodyProcessor")  
.split().method("splitterService","magentoUowSplit")
.to("itemExportMagentoActivityCodeProcessor")   

// check activity code
.choice()
.when(header(CamelConstants.HEADER_KEY_ITEM_ACTIVITY_CODE)
.endsWith(MinderConstants.ITEM_ADD))
.to("bean:serviceBean?method=createSimpleProduct")
.when(header(CamelConstants.HEADER_KEY_ITEM_ACTIVITY_CODE)
.endsWith(MinderConstants.ITEM_UPDATE))
.to("bean:serviceBean?method=updateSimpleProduct")
.when(header(CamelConstants.HEADER_KEY_ITEM_ACTIVITY_CODE)
.endsWith(MinderConstants.ITEM_DELETE))
.to("bean:serviceBean?method=deleteSimpleProduct")
 .otherwise()
.to("log:xml?level=ERROR").stop();  

My org.apache.camel.auski.etl.entity.ImportPayloadEntity bean actually
contains another entity bean named
org.apache.camel.auski.etl.entity.ItemEntity. So what I am doing in the
route is selecting import payload entities and then splitting each one up to
process item entities one by one.

Inside createSimpleProduct, updateSimpleProduct and deleteSimpleProduct I
then call markConsumed on the item entity bean and the import entity bean
which are both marked with @Consumed to execute.

@Consumed
public void markConsumed() {
this.setProcessedInd("Y");
this.setProcessedDatetime(new Date());
}   

These fields are not getting set in my db. I also know that it is not trying
to run any sql to update these fields as I have a log of all the sql
executed during execution.

If you have any ideas as to why this is not working I would appreciate it.



--
View this message in context: 
http://camel.465427.n5.nabble.com/JPA-Consumed-tp5744640.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: JPA @Consumed

[richie.rivi...@gmail.com]

I just had a thought whilst walking my dog. In the past when I have got JPA
updating the database it has been in a TypeConverter class and what I had to
do was .

JpaTemplate template =
exchange.getIn().getHeader(CamelConstants.HEADER_KEY_JPA_TEMPLATE,
JpaTemplate.class);
template.persist(item); 


So having a think about that I went into my type converter and tried to grab
this header but to my surprise it wasn't there. 

Will be thinking about this more... ideas are welcome.

thanks



--
View this message in context: 
http://camel.465427.n5.nabble.com/JPA-Consumed-tp5744640p5744641.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: JPA @Consumed

[richie.rivi...@gmail.com]

Thanks Williem,

Changing my route to ...

.pollEnrich("jpa://org.apache.camel.auski.etl.entity.ImportPayloadEntity?consumer.namedQuery=queryImportPayloads&consumeDelete=false&delay=3000&consumeLockEntity=false")

worked!!!

But still a bit of a problem.. The ImportPayloadEntity contains a list of
ItemEntity beans (one to many jpa relationship) and they also have a flag on
them (processed_ind) which I am trying to update. Is there some setting you
can use to do something similar for the nested ItemEntity beans? 

I'm going to have a read of the documentation on this setting now to see if
I can find anything but any help is welcome.

thanks again



--
View this message in context: 
http://camel.465427.n5.nabble.com/JPA-Consumed-tp5744640p5744646.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: JPA @Consumed

[richie.rivi...@gmail.com]

Maybe the pattern is to just have ImportPayloadEntity @Consumed loop through
it's collection of ItemEntity classes and mark them all as processed. I just
tried this and it worked (although I'm not sure it's the right pattern).

thanks



--
View this message in context: 
http://camel.465427.n5.nabble.com/JPA-Consumed-tp5744640p5744650.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: JPA @Consumed

[richie.rivi...@gmail.com]

Abother question about @Consumed. Using the strategy as described in my last
post, what I am doing in selecting an ImportPayloadEntity, splitting this
entity which has a list of ItemEntities into individual ItemEntity objects
and processing each ItemEntity object.

During the processing of each ItemEntity object I call...

item.setWorkingProcessedInd("Y");
item.setWorkingProcessedDatetime(new Date());  

Then once the route is finish in the ImportPayloadEntity bean I do something
like this...

@Consumed
public void markConsumed() {
boolean errors = false;
Collection itemEntities = this.getItems();
for (ItemEntity itemEntity: itemEntities){
if (itemEntity.getProcessedInd().equals(DaoConstants.ERROR)){
errors = true;
}
itemEntity.setProcessedInd(itemEntity.getWorkingProcessedInd());

itemEntity.setProcessedDatetime(itemEntity.getWorkingProcessedDatetime());
}
this.setProcessedInd(errors ? DaoConstants.ERROR :
DaoConstants.PROCESSED);
this.setProcessedDatetime(new Date());
}


This leads to the ImportPayloadEntity record in the db being updated. But
where this strategy fails is that when i do...
itemEntity.setProcessedInd(itemEntity.getWorkingProcessedInd());

itemEntity.setProcessedDatetime(itemEntity.getWorkingProcessedDatetime());

itemEntity.getWorkingProcessedInd() and
itemEntity.getWorkingProcessedDatetime() are both null. In other words it
has not remembered the values I set on the objects in my route.

It seems somewhere along the line my ItemEntity objects are not being passed
by reference. Am I right?
What would be a good way to fix this?

thanks






--
View this message in context: 
http://camel.465427.n5.nabble.com/JPA-Consumed-tp5744640p5744651.html
Sent from the Camel - Users mailing list archive at Nabble.com.

First CamelSpringTestSupport unit test - AdviceWith - No consumers available on endpoint

[richie.rivi...@gmail.com]

Hi guys,

Thanks for the advice on a previous post asking about how to change the
route of an application for the purposes of testing. Since that time I have
gone and done some research into using adviceWith and given it a go. It
seems to be working for me but I'm getting an error whilst executing the
unit test and I'm not sure what it means. 

I am trying to use adviceWith to replace the JPA select from the database
and just give camel an entity bean that I have created.

The error is 

Caused by:
org.apache.camel.component.direct.DirectConsumerNotAvailableException: No
consumers available on endpoint: Endpoint[direct://itemExport].
Exchange[Message: ImportPayloadEntity[ id:999 ]

The original route in my application looks like this...

from("direct:itemExport")
.routeId("itemExportMagento")
.autoStartup("{{item.export.magento}}") 
.startupOrder(2) 
.shutdownRoute(ShutdownRoute.Defer)
.errorHandler(deadLetterChannel("log:dead")
.maximumRedeliveries(5)
.retryAttemptedLogLevel(LoggingLevel.ERROR))
.pollEnrich("jpa://org.apache.camel.auski.etl.entity.ImportPayloadEntity?consumer.namedQuery=queryImportPayloads&consumeDelete=false&delay=3000&consumeLockEntity=false")
.to("itemExportMagentoFromSourceHeaderProcessor")   
.to("bean:serviceBean?method=login")
.to("itemExportMagentoFromLoginBodyProcessor")  
.split().method("splitterService","magentoUowSplit")
.to("itemExportMagentoActivityCodeProcessor")   

and my unit test looks like this...

public class ItemExportMagentoRouteTest extends CamelSpringTestSupport {
@Override
protected AbstractApplicationContext createApplicationContext() {
return new
ClassPathXmlApplicationContext("META-INF/spring/camel-context.xml");
}
@Override
public boolean isUseAdviceWith() {
return true;
}
@Test
public void testExportRoute() throws Exception {
context.getRouteDefinition("itemExportMagento").adviceWith(context, new
AdviceWithRouteBuilder() {
@Override
public void configure() throws Exception {
weaveByType(PollEnrichDefinition.class).remove();
}
});
context.startRoute("itemExportMagento");

ImportPayloadEntity importPayloadEntity = new ImportPayloadEntity();
importPayloadEntity.setId(new Long(999));
importPayloadEntity.setFilename("MinderTest.xml");

Collection items = new ArrayList();
ItemEntity itemEntity = createItem(importPayloadEntity);
items.add(itemEntity);
importPayloadEntity.setItems(items);

template.sendBody("direct:itemExport", importPayloadEntity);
}

I am using Camel 2.11.0. I'm thinking maybe it is something to do with the
way I'm doing template.sendBody("direct:itemExport", importPayloadEntity); ?

Has anyone got any thoughts?

thanks



--
View this message in context: 
http://camel.465427.n5.nabble.com/First-CamelSpringTestSupport-unit-test-AdviceWith-No-consumers-available-on-endpoint-tp5744656.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: First CamelSpringTestSupport unit test - AdviceWith - No consumers available on endpoint

[richie.rivi...@gmail.com]

Hi,

Yes i think you may have missed the first part of the post where I describe
the original route. It definitely has a from(“direct:itemExport”).

So you think it can't find the route?



--
View this message in context: 
http://camel.465427.n5.nabble.com/First-CamelSpringTestSupport-unit-test-AdviceWith-No-consumers-available-on-endpoint-tp5744656p5744661.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: JPA when is @Consumed called - timing of it

[richie.rivi...@gmail.com]

Actually I think I know the issue here. I'm guessing is probably related to
the timing that the @consume method is called.

In my route I don't set the workingProcessedIndicator on the ItemEntity
until the very end of the route using my Bean createSimpleItem method.
Perhaps when the @consume method is called that part of the route has not
executed yet.

In the documentation it says...

@Consumed which will be invoked on your entity bean when the entity bean
when it has been processed (and when routing is done)

Does this mean it executes the entire route before running this method?



--
View this message in context: 
http://camel.465427.n5.nabble.com/JPA-Consumed-tp5744640p5744699.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Ideas on how to structure route (alternatives to direct:...)

[richie.rivi...@gmail.com]

Hi All,

I've got two main routes in my application. An import route and an export
route.

At the end of my import route i say... .to("direct:import");

Then at the start of my export i say ... .from("direct:import")

Pretty standard stuff.

Yesterday I had to change my application to run about 5 post processing
routes that execute sql queries to run after the import and before the
export. I did this using the sql component. This worked fine. But i had to
link them all up using the direct component.

i.e.
to(direct)>from(direct),(direct)>from(direct),(direct)>from(direct),(direct)>from(direct)
etc

It's starting to get ugly now

Can anyone give me some ideas on how to structure my routes to make it a bit
nicer. Really there are three parts to my application now..

An import route, some post processing and an export route. Is there a way
that I can have some sort of container route for all my post processing
routes? They can't all run parallel because some of the sql queries are
dependent on each other.

My original thought was just to have one routeBuilder with many from and to
definitions...

from("direct:import").to("sql:query1")

from("direct:import").to("sql:query2")

from("direct:import").to("sql:query3")

But I found out yesterday you can't do this. I get a camel error saying
multiple routes are linked to "direct:import".

thanks for your help





--
View this message in context: 
http://camel.465427.n5.nabble.com/Ideas-on-how-to-structure-route-alternatives-to-direct-tp5744748.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Ideas on how to structure route (alternatives to direct:...)

[richie.rivi...@gmail.com]

the mullticast example you posted looks good. I thought multi cast was just
for parallel. Now that I know it's not i'll look into it.

So with the multicast example you posted does it do them in order? I'll have
a read of doco. Here are the post processing routes...

public void configure() throws Exception {
LOG.debug("Executing 
ItemImportPostProcessingConfigurableItemsRoute ...");


from("direct:itemImportRoute")
.routeId("itemImportPostProcessingConfigurableItemsRoute")
.autoStartup(true)  
.startupOrder(2) 
.shutdownRoute(ShutdownRoute.Defer)
.errorHandler(deadLetterChannel("log:dead")
.retryAttemptedLogLevel(LoggingLevel.ERROR))
.setHeader("notProcessed", constant(DaoConstants.NOT_PROCESSED))
.to("sql:{{sql.notProcessed}}?dataSourceRef=dataSource")
.to("sqlComponentSelectInputProcessor")
.to("sql:{{sql.configurableScenarioOne}}")
.to("sqlComponentSelectInputProcessor")
.to("sql:{{sql.updateConfigurableScenarioOne}}")
  
.to("direct:itemImportPostProcessingConfigurableItemsRoute");
}
}




public void configure() throws Exception {
LOG.debug("Executing ItemImportPostProcessingSimpleItemsRoute 
...");
from("direct:itemImportPostProcessingConfigurableItemsRoute")
.routeId("itemImportPostProcessingSimpleItemsRoute")
.autoStartup(true)  
.startupOrder(3) 
.shutdownRoute(ShutdownRoute.Defer)
.errorHandler(deadLetterChannel("log:dead")
.retryAttemptedLogLevel(LoggingLevel.ERROR))
.setHeader("notProcessed", constant(DaoConstants.NOT_PROCESSED))
.to("sql:{{sql.notProcessed}}?dataSourceRef=dataSource")
.to("sqlComponentSelectInputProcessor")
.to("sql:{{sql.simpleScenarioOne}}")
.to("sqlComponentSelectInputProcessor")
.to("sql:{{sql.updateSimpleScenarioOne}}")
.to("direct:itemImportPostProcessingSimpleItemsRoute");
}


public void configure() throws Exception {
LOG.debug("Executing
ItemImportPostProcessingVisibleInCatAndSearchItemsRoute ...");
from("direct:itemImportPostProcessingSimpleItemsRoute")

.routeId("itemImportPostProcessingVisibleInCatAndSearchItemsRoute")
.autoStartup(true)  
.startupOrder(4) 
.shutdownRoute(ShutdownRoute.Defer)
.errorHandler(deadLetterChannel("log:dead")
.retryAttemptedLogLevel(LoggingLevel.ERROR))
.setHeader("notProcessed", constant(DaoConstants.NOT_PROCESSED))
.to("sql:{{sql.notProcessed}}?dataSourceRef=dataSource")
.to("sqlComponentSelectInputProcessor")
.to("sql:{{sql.visibleInSearchAndCatalog}}")
.to("sqlComponentSelectInputProcessor")
.to("sql:{{sql.updateVisibleInSearchAndCatalog}}")  
  

.to("direct:itemImportPostProcessingVisibleInCatAndSearchItemsRoute");

}


public void configure() throws Exception {
LOG.debug("Executing 
ItemImportPostProcessingNotVisibleItemsRoute ...");

from("direct:itemImportPostProcessingVisibleInCatAndSearchItemsRoute")
.routeId("itemImportPostProcessingNotVisibleItemsRoute")
.autoStartup(true)  
.startupOrder(5) 
.shutdownRoute(ShutdownRoute.Defer)
.errorHandler(deadLetterChannel("log:dead")
.retryAttemptedLogLevel(LoggingLevel.ERROR))
.setHeader("notProcessed", constant(DaoConstants.NOT_PROCESSED))
.to("sql:{{sql.notProcessed}}?dataSourceRef=dataSource")
.to("sqlComponentSelectInputProcessor")
.to("sql:{{sql.visibleInSearchAndCatalog}}")
.to("sqlComponentSelectInputProcessor")
.to("sql:{{sql.updateVisibleInSearchAndCatalog}}")  
  
.to("direct:itemImportPostProcessingNotVisibleItemsRoute");

}



--
View this message in context: 
http://camel.465427.n5.nabble.com/Ideas-on-how-to-structure-route-alternatives-to-direct-tp5744748p5744752.html
Sent from the Camel - Users mailing list archive at Nabble.com.

SQL component issue - infinite loop

[richie.rivi...@gmail.com]

Hi All,

I currently having issues with the SQL component in my camel project. I was
using the stack overflow forum to try and get some help but haven't been
able to get any yet and then I discovered these forums.

My camel project uses Spring DSL. It has multiple routes specified in a
route context. 

What I am finding is that when I have two routes in my routeContext both
routes only execute once but as soon as I add a third (no matter what route
that third is), camel starts to execute all the routes infinitely. All of my
routes lead to sql inserts so having the same route run more than once
causes a duplicate key error.

I've played around with things a bit to try and get around it but not luck
yet. Here is the link to the original post which describes the issue in
depth. 

http://stackoverflow.com/questions/18738832/camel-route-inside-routecontext-executing-infinitely

  

Thanks in advance for your help !



--
View this message in context: 
http://camel.465427.n5.nabble.com/SQL-component-issue-infinite-loop-tp5739486.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Cam JDBC component help

[richie.rivi...@gmail.com]

Hi all,

Apologies for the beginner question but I'm trying to get a new project
going from the examples I have seen and read (Spring DSL).

I'm trying to use the camel jdbc component to do a select from database 1
and insert into db2. I've got my select working as per below

 

   


   ${properties:sql.selectProduct}



${body}






I'm not sure how to change this route to now insert into database 2. Can
someone please help me with the syntax.

thanks



--
View this message in context: 
http://camel.465427.n5.nabble.com/Cam-JDBC-component-help-tp5739528.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: SQL component issue - infinite loop

[richie.rivi...@gmail.com]

woo hoo. Thanks for helping me with this... I'm so glad. 

Would it be better if I make my project available on github so you can grab
it as a zip?

regards



--
View this message in context: 
http://camel.465427.n5.nabble.com/SQL-component-issue-infinite-loop-tp5739486p5739583.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: SQL component issue - infinite loop

[richie.rivi...@gmail.com]

Hi,

I've posted the small project to github and you can grab a zip of it at...

https://github.com/rriviere/camel-example-sql

Really appreciate your help.

regards



--
View this message in context: 
http://camel.465427.n5.nabble.com/SQL-component-issue-infinite-loop-tp5739486p5739620.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: SQL component issue - infinite loop

[richie.rivi...@gmail.com]

Hi Costa,

Yes you are correct, what you are looking at is what I have described in my
post where I say "Tonight I also tried to separate the routes out into
different route contexts but it didn't change anything". 

I've found the way I have structured it with multiple route contexts a bit
more convenient because I can easily comment out a route context in the
camel context.

Nevertheless the behavior between what you are looking at and what I posted
on stack overflow is exactly the same. The application works well if I have
two routes (i.e. two sets of select inserts) but as soon as I add a third
the behavior changes and rather than the first route executing and then
finishing and tries to execute it multiple times leaving naturally me with
errors on a primary key.

Does the way I have defined the root look okay? 

I will try with Derby but it's not realistic for me. I need to be connecting
to sql server and mysql.

thanks



--
View this message in context: 
http://camel.465427.n5.nabble.com/SQL-component-issue-infinite-loop-tp5739486p5739788.html
Sent from the Camel - Users mailing list archive at Nabble.com.