nyingping commented on code in PR #36737: URL: https://github.com/apache/spark/pull/36737#discussion_r891847061
########## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala: ########## @@ -3963,8 +3966,10 @@ object TimeWindowing extends Rule[LogicalPlan] { def getWindow(i: Int, dataType: DataType): Expression = { val timestamp = PreciseTimestampConversion(window.timeColumn, dataType, LongType) - val lastStart = timestamp - (timestamp - window.startTime - + window.slideDuration) % window.slideDuration + val remainder = (timestamp - window.startTime) % window.slideDuration + val lastStart = CaseWhen( + Seq((LessThan(remainder, 0), timestamp - remainder - window.slideDuration)) Review Comment: Thank you for reviewing. For any event timestamp, the start time of the last window (whether tumbling or sliding) obtained by it should always be less than the current timestamp. When `(timestamp - window.starttime)% window.slideduration <0`, the obtained laststart will be greater than timestamp. At this time, it is necessary to shift the window to the right, which is the correct laststart value. For example code ``` val timestamp = -13 val offset = 0 val windowSize = 7 // old code val lastStartOld = timestamp - (timestamp - offset - windowSize) % windowSize // new code val remainder = (timestamp - offset) % windowSize val lastStartNew = if (remainder < 0) { timestamp - remainder - windowSize } else { timestamp - remainder - windowSize } println(s"lastStartOld = $lastStartOld lastStartNew = $lastStartNew") ``` result ``` lastStartOld = -7 lastStartNew = -14 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org