Re: Structured Streaming, Reading and Updating a variable

2018-05-16 Thread Martin Engen
util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Any ideas about how to handle this error? Thanks, Martin Engen From: Lalwani, Jayesh Sent: Tuesday, May 15, 2018 9:59 PM To: Marti

Re: Structured Streaming, Reading and Updating a variable

2018-05-15 Thread Lalwani, Jayesh
Do you have a code sample, and detailed error message/exception to show? From: Martin Engen Date: Tuesday, May 15, 2018 at 9:24 AM To: "user@spark.apache.org" Subject: Structured Streaming, Reading and Updating a variable Hello, I'm working with Structured Streaming, and I need a method of kee

Re: Structured Streaming, Reading and Updating a variable

2018-05-15 Thread Koert Kuipers
You use a windowed aggregation for this On Tue, May 15, 2018, 09:23 Martin Engen wrote: > Hello, > > > > I'm working with Structured Streaming, and I need a method of keeping a > running average based on last 24hours of data. > > To help with this, I can use Exponential Smoothing, which means I