Interesting that we aren't discussing a tiered off the shelf solution. We were discussing the processes of mapping incoming events by classification into some table of tariffs, applying any found cost, then aggregating such events periodically into a pipeline ending up in some single line for the account's monthly activity.
Then we decided - this must have been done before. Why reinvent the wheel and introduce our own bugs and technical debt? I guess we are looking for a set of components that we can feed standard formatted data into. I realise each use case will involve slightly different data sets but is there really no common ground to be found for a billing pipeline out there already? To give you some impression of the background, we do a lot of messaging between our customers (companies) and people who are the customers of those companies. Many electronic media. We uniquely identify each message and so as various events occur on each message we can build a queue called "MessageEvents" which aggregates over time into a Mongo collection called "MessageAudit" that contains one document per unique message, appending to that document the events that occur over time. Each event will have a classifier that may be matched by a tariff pre-computed for that customer. First pass finds any applicable cost and updates that event in the document holding that message's audit trail. Further passes may then aggregate some documents into super documents (one message may have produced several messages - show bill for one with an aggregate total). Eventually a billing sweep will occur - pushing new known costs into a message=cost tuple - such a sweep may occur at any time, usually one a once and hour schedule perhaps. A further sweep might then aggregate new typles into daily then monthly aggregates showing total number of messages and total cost. This final tuple needs to be pushed into our existing billing system which operates at this high level to invoice the customer. Internally we are finishing up code to create the events themselves. What we do with the events appears reasonably clear but much more work than we'd like take on without convincing ourselves that such systems don't already exist... Hope that gives a little more detail to the problem. James On 26 April 2013 04:22, Hadrian Zbarcea <hzbar...@gmail.com> wrote: > Hi James, > > Very good and interesting use case. You could use BPM, indeed, but I think > it's way to heavy weight for what you need to do. CEP would be another > idea, but I digress. > > Wire-tap won't help you, I believe Willem misunderstood your question. > > So here's what I think: you have your system, it emits some events > (messages) when things take place on behalf of a user. (I think in Camel 3 > we'll have even more goodies to handle your scenario... and I digress > again). So I assume your events are send via some protocol that Camel > understand, something from tcp/ip (camel-mina or camel-netty would help), > to maybe http, or a web service call (rest maybe?) or even a database. So > you'd need a Camel endpoint to send your events to and start a Camel route. > Using the dsl it's trivial, you probably saw tons of examples. If your > events are actually api calls, it is a bit less straightforward, you may > need to write a component (which is easy) and there are other options. I > won't get into details unless you confirm that this is the case. > > Once you have your endpoint, you need to setup a route to do your > aggregation. The route and processing is easy to define using the dsl. > Based on what you described your route would probably contain some > aggregation [1], I assume you have some correlation ids there identifying > both the user/account and the activity you're charing for. You may use > other things depending on your logic, such as a filter, content based > router, who knows. If you are not familiar with the EIPs [2] it's worth > taking a quick look. At the end of the route, once you're done with the > processing you'll need to send your result to the billing service (also > available at some url). > > What's nice about your scenario is that it sounds non intrusive to your > business logic, it's async, one way (in-only) so it would scale really well > and should not create any headaches. > > I hope this helps and I hope I did not misunderstand your request. > Obviously more details you provide about your scenario, more concrete > answers you're gonna get. > > Cheers, > Hadrian > > > [1] > http://camel.apache.org/**aggregator2.html<http://camel.apache.org/aggregator2.html> > [2] http://camel.apache.org/eip.**html <http://camel.apache.org/eip.html> > > > > > On 04/25/2013 09:15 PM, Willem jiang wrote: > >> I'm not sure if the billing service needs the data from the business >> analysis. >> If it doesn't need those data, it could be more easy by using wire-tap[1] >> >> [1]http://camel.apache.org/**wire-tap.html<http://camel.apache.org/wire-tap.html> >> >> -- >> Willem Jiang >> >> Red Hat, Inc. >> FuseSource is now part of Red Hat >> Web: http://www.fusesource.com | http://www.redhat.com >> Blog: http://willemjiang.blogspot.**com <http://willemjiang.blogspot.com>( >> http://willemjiang.blogspot.**com/ <http://willemjiang.blogspot.com/>) >> (English) >> http://jnn.iteye.com (http://jnn.javaeye.com/) (Chinese) >> Twitter: willemjiang >> Weibo: 姜宁willem >> >> >> >> >> >> On Thursday, April 25, 2013 at 9:29 PM, James Green wrote: >> >> This isn't to do with Camel per se but I'm hopeful I'm asking this >>> question >>> of the the right audience! >>> >>> Basically we're going to be routing event messages into some storage >>> component where some of those messages may be billable to customers. >>> >>> We have an existing service that charges customers based on a high level >>> aggregate summary of activity. We are missing a component that takes >>> these >>> new event messages, performs business analysis on them, then sends them >>> onwards into our existing billing service. >>> >>> BPM sounds almost like the thing we want, but it's rather broad given our >>> somewhat narrow scope. >>> >>> Can any Camel users advise on what they use to aggregate events for >>> billing >>> purposes? >>> >>> Thanks, >>> >>> James >>> >> >> >>
