You can design a receiver to receive data every 5 sec (batch size) & pull data of last 5 sec from http API, you can shard data by time further within those 5 sec to distribute it further. You can also bind TSDB nodes to each receiver to translate HBase data to improve performance.
Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi <https://twitter.com/mayur_rustagi> On Fri, Aug 1, 2014 at 5:21 PM, bumble123 <tc1...@att.com> wrote: > So is there no way to do this through SparkStreaming? Won't I have to do > batch processing if I use the http api rather than getting it directly into > Spark? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-read-from-OpenTSDB-using-PySpark-or-Scala-Spark-tp11211p11234.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >
