Ok but how about something similar to val countByValueAndWindow = price.filter(_ > 95.0).countByValueAndWindow(Seconds(windowLength), Seconds(slidingInterval))
Using a new count => c*ountDistinctByValueAndWindow ?* val countDistinctByValueAndWindow = price.filter(_ > 95.0).countDistinctByValueAndWindow(Seconds(windowLength), Seconds(slidingInterval)) HTH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com On 17 May 2016 at 20:02, Michael Armbrust <mich...@databricks.com> wrote: > In 2.0 you won't be able to do this. The long term vision would be to > make this possible, but a window will be required (like the 24 hours you > suggest). > > On Tue, May 17, 2016 at 1:36 AM, Todd <bit1...@163.com> wrote: > >> Hi, >> We have a requirement to do count(distinct) in a processing batch against >> all the streaming data(eg, last 24 hours' data),that is,when we do >> count(distinct),we actually want to compute distinct against last 24 hours' >> data. >> Does structured streaming support this scenario?Thanks! >> > >
