Spark structured streaming time series forecasting

2018-01-08 Thread Bogdan Cojocar
Hello, Is there a method to do time series forecasting in spark structured streaming? Is there any integration going on with spark-ts or a similar library? Many thanks, Bogdan Cojocar

Different behaviour when querying a spark DataFrame from dynamodb

2017-12-13 Thread Bogdan Cojocar
I am reading some data in a dataframe from a dynamo db table: val data = spark.read.dynamodb("table") data.filter($"field1".like("%hello%")).createOrReplaceTempView("temp") spark.sql("select * from temp").show() When I do the last statement I get results. If however I try to do:

Spark Structured Streaming how to read data from AWS SQS

2017-12-11 Thread Bogdan Cojocar
nector? Or is there a way to create a custom connector similar to the way one can be created in a spark streaming <http://spark.apache.org/docs/latest/streaming-custom-receivers.html> application? Many thanks, Bogdan Cojocar