Hello
I had similar use case for one of my customers. It was a bulk dump of
electricity consumption data of subscribers which was to be loaded into
charging engine. The records ran to millions.
I doubt if there is something out of box for json to json transformation. I
did following
1) Source Json ->Equalent Source XML ( json unmarshalling component in
camel )
2) Equalent Source XML->Equalent Target XML ( transformation using
xslt,xquery etc)
3) Equalent Target XML -> Equalent Target Json ( xml-json marshalling )
Note - Don't attempt step 1 with huge payloads. It kills the memory and i
had a crash.
I did some java coding to chunk out the huge json into smaller chunks and
processed each of them in parallel. You can adopt some cool algo's for
chunking. I adopted stack approach. a small snippet of the code below using
java.util.Stack class.
Read json char by char and pass to this function. When it returns a boolean
true, that means you now have the point where you have a valid json. Pls
manage the outer envelops / roots if any.
try {
char current;
int counter = 0;
while (stream.available() > 0) {
jsonProcessed=false;
current = (char) stream.read();
validJson.append(current);
if (current == '{' || current == '(' || current == ')'
|| current == '}') {
if (IsValidJsonRecord(current) && validJson.length()>500000) {
/* System.out.println((++counter) + "This is valid"
+ validJson.toString());*/
producer.sendBody("vm:SplittedRecords",
"{\"CollectDataList\":{\"CollectData\":["+validJson.toString()+"]}}");
//The outer envelopes being attached here to the chunks
jsonProcessed=true;
validJson.delete(0, validJson.length()); //Clearing the buffer variable
stream.read(); // To skip the comma
}
else if (stream.available() < 1 && validJson.length()>0 &&
jsonProcessed==false)
{
System.out.println("PROCESSINGLASTRECORD- "+ validJson.toString());
producer.sendBody("vm:SplittedRecords",
"{\"CollectDataList\":{\"CollectData\":["+validJson.toString()+"]}}");
jsonProcessed=true;
}
}
}
} catch (java.util.EmptyStackException e){}
private static boolean IsValidJsonRecord(char c) {
// TODO Auto-generated method stub
if (c == '(')
stack.push(c);
else if (c == '{')
stack.push(c);
else if (c == ')') {
if (stack.peek() == '(') {
stack.pop();
return stack.empty();
} else {
stack.push(c);
return false;
}
}
else if (c == '}') {
if (stack.peek() == '{') {
stack.pop();
return stack.empty();
} else {
stack.push(c);
return false;
}
}
return stack.empty();
}
}
Cheers
Reji
On Thu, Apr 23, 2015 at 10:23 AM, erd <evanday...@gmail.com> wrote:
> Hello,
>
> I am building a camel route and I am looking to manipulate an inputStream
> formatted as JSON into a different "hierarchy".
>
> The endgame is to turn the service's response
>
> {"foo":
> {"bar":{"type": "motor","status":"spinning"}},
> {"baz":{"type": "calculator","status":"crunching"}}
> }
> into a series of JSON strings with name and properties a la:
> {"foo":
> {"name":"bar", properties:{"type": "motor","status":"spinning"}}
> {"name":"baz", properties:{"type": "calculator","status":"crunching"}}
> }
>
> What's the best way to approach this?
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Manipulating-JSON-from-inputStream-tp5766240.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
