Re: [SQL][SPARK-14160] Maximum interval for o.a.s.sql.functions.window

2017-01-18 Thread Michael Armbrust
+1, we should just fix the error to explain why months aren't allowed and suggest that you manually specify some number of days. On Wed, Jan 18, 2017 at 9:52 AM, Maciej Szymkiewicz wrote: > Thanks for the response Burak, > > As any sane person I try to steer away from

Re: [SQL][SPARK-14160] Maximum interval for o.a.s.sql.functions.window

2017-01-18 Thread Maciej Szymkiewicz
Thanks for the response Burak, As any sane person I try to steer away from the objects which have both calendar and unsafe in their fully qualified names but if there is no bigger picture I missed here I would go with 1 as well. And of course fix the error message. I understand this has been

Re: [SQL][SPARK-14160] Maximum interval for o.a.s.sql.functions.window

2017-01-18 Thread Burak Yavuz
Hi Maciej, I believe it would be useful to either fix the documentation or fix the implementation. I'll leave it to the community to comment on. The code right now disallows intervals provided in months and years, because they are not a "consistently" fixed amount of time. A month can be 28, 29,

[SQL][SPARK-14160] Maximum interval for o.a.s.sql.functions.window

2017-01-18 Thread Maciej Szymkiewicz
Hi, Can I ask for some clarifications regarding intended behavior of window / TimeWindow? PySpark documentation states that "Windows in the order of months are not supported". This is further confirmed by the checks in TimeWindow.getIntervalInMicroseconds (https://git.io/vMP5l). Surprisingly