Re: [SQL] Windowing and triggering changes proposal
I'm late to the party as usual, but also added some comments. Overall supportive of this work. Thanks for the clear analysis, Anton. On Tue, Jan 16, 2018 at 10:58 AM Mingmin Xu wrote: > Thanks @Anton for the proposal. Window(w/ trigger) support in SQL is > limited now, you're very welcome to join the improvement. > > There's a balance between injected DSL mode and CLI mode when we were > implementing BealmSQL overall, not only widowing. Many default behaviors > are introduced to make it workable in pure SQL CLI scenario. If it limits > the potential with DSL mode, we should adjust it absolutely. > > Mingmin > > On Tue, Jan 16, 2018 at 9:56 AM, Kenneth Knowles wrote: > >> I've commented on the doc. This is a really nice analysis and I think the >> proposal is good for making SQL work with Beam windowing and triggering in >> a way that will make sense to users. >> >> Kenn >> >> On Thu, Jan 11, 2018 at 4:05 PM, Anton Kedin wrote: >> >>> Hi, >>> >>> Wanted to gather feedback on changes I propose to the behavior of some >>> aspects of windowing and triggering in Beam SQL. >>> >>> In short: >>> >>> Beam SQL currently overrides input PCollections' windowing/triggering >>> configuration in few cases. For example if a query has a simple GROUP BY >>> clause, we would apply GlobalWindows. And it's not configurable by the >>> user, it happens under the hood of SQL. >>> >>> Proposal is to update the Beam SQL implementation in these cases to >>> avoid changing the input PCollections' configuration as much as possible. >>> >>> More details here: >>> https://docs.google.com/document/d/1RmyV9e1Qab-axsLI1WWpw5oGAJDv0X7y9OSnPnrZWJk >>> >>> Regards, >>> Anton >>> >> >> > > > -- > > Mingmin >
Re: [SQL] Windowing and triggering changes proposal
Thanks @Anton for the proposal. Window(w/ trigger) support in SQL is limited now, you're very welcome to join the improvement. There's a balance between injected DSL mode and CLI mode when we were implementing BealmSQL overall, not only widowing. Many default behaviors are introduced to make it workable in pure SQL CLI scenario. If it limits the potential with DSL mode, we should adjust it absolutely. Mingmin On Tue, Jan 16, 2018 at 9:56 AM, Kenneth Knowles wrote: > I've commented on the doc. This is a really nice analysis and I think the > proposal is good for making SQL work with Beam windowing and triggering in > a way that will make sense to users. > > Kenn > > On Thu, Jan 11, 2018 at 4:05 PM, Anton Kedin wrote: > >> Hi, >> >> Wanted to gather feedback on changes I propose to the behavior of some >> aspects of windowing and triggering in Beam SQL. >> >> In short: >> >> Beam SQL currently overrides input PCollections' windowing/triggering >> configuration in few cases. For example if a query has a simple GROUP BY >> clause, we would apply GlobalWindows. And it's not configurable by the >> user, it happens under the hood of SQL. >> >> Proposal is to update the Beam SQL implementation in these cases to avoid >> changing the input PCollections' configuration as much as possible. >> >> More details here: https://docs.google.com/docume >> nt/d/1RmyV9e1Qab-axsLI1WWpw5oGAJDv0X7y9OSnPnrZWJk >> >> Regards, >> Anton >> > > -- Mingmin
Re: [SQL] Windowing and triggering changes proposal
I've commented on the doc. This is a really nice analysis and I think the proposal is good for making SQL work with Beam windowing and triggering in a way that will make sense to users. Kenn On Thu, Jan 11, 2018 at 4:05 PM, Anton Kedin wrote: > Hi, > > Wanted to gather feedback on changes I propose to the behavior of some > aspects of windowing and triggering in Beam SQL. > > In short: > > Beam SQL currently overrides input PCollections' windowing/triggering > configuration in few cases. For example if a query has a simple GROUP BY > clause, we would apply GlobalWindows. And it's not configurable by the > user, it happens under the hood of SQL. > > Proposal is to update the Beam SQL implementation in these cases to avoid > changing the input PCollections' configuration as much as possible. > > More details here: https://docs.google.com/document/d/1RmyV9e1Qab- > axsLI1WWpw5oGAJDv0X7y9OSnPnrZWJk > > Regards, > Anton >
[SQL] Windowing and triggering changes proposal
Hi, Wanted to gather feedback on changes I propose to the behavior of some aspects of windowing and triggering in Beam SQL. In short: Beam SQL currently overrides input PCollections' windowing/triggering configuration in few cases. For example if a query has a simple GROUP BY clause, we would apply GlobalWindows. And it's not configurable by the user, it happens under the hood of SQL. Proposal is to update the Beam SQL implementation in these cases to avoid changing the input PCollections' configuration as much as possible. More details here: https://docs.google.com/document/d/1RmyV9e1Qab-axsLI1WWpw5oGAJDv0X7y9OSnPnrZWJk Regards, Anton
