Re: Design considerations for batch and speed layers

2016-09-30 Thread Rodrick Brown
We do processing millions of records using Kafka, Elastic Search, Accumulo, Mesos, Spark & Vertica. Their a pattern for this type of pipeline today called SMACK more about here -- http://www.slideshare.net/akirillov/data-processing-platforms-architectures-with-spark-mesos-akka-cassandra-and-kafka

Re: Design considerations for batch and speed layers

2016-09-30 Thread Ashok Kumar
Can one design a fast pipeline with Kafka, Spark streaming and Hbase  or something similar? On Friday, 30 September 2016, 17:17, Mich Talebzadeh wrote: I have designed this prototype for a risk business. Here I would like to discuss issues with batch layer. Apologies about being lo

Design considerations for batch and speed layers

2016-09-30 Thread Mich Talebzadeh
I have designed this prototype for a risk business. Here I would like to discuss issues with batch layer. *Apologies about being long winded.* *Business objective* Reduce risk in the credit business while making better credit and trading decisions. Specifically, to identify risk trends within cer