Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR closed pull request #45051: [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator URL: https://github.com/apache/spark/pull/45051 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on PR #45051: URL: https://github.com/apache/spark/pull/45051#issuecomment-1995685935 Thanks! Merging to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1522327291 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1522188039 ## sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateSuite.scala: ## @@ -53,8 +57,134 @@ class RunningCountStatefulProcessor extends

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1522187745 ## sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateSuite.scala: ## @@ -195,6 +327,115 @@ class TransformWithStateSuite extends

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1520644679 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TransformWithStateExec.scala: ## @@ -163,6 +249,16 @@ case class TransformWithStateExec(

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1520644342 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulProcessorHandleImpl.scala: ## @@ -121,6 +123,46 @@ class StatefulProcessorHandleImpl(

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1520520070 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TransformWithStateExec.scala: ## @@ -103,8 +116,12 @@ case class TransformWithStateExec(

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1520519676 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulProcessorHandleImpl.scala: ## @@ -121,6 +123,46 @@ class StatefulProcessorHandleImpl(

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1520472228 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulProcessorHandleImpl.scala: ## @@ -121,6 +123,46 @@ class StatefulProcessorHandleImpl(

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

sahnib commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1519943584 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TransformWithStateExec.scala: ## @@ -103,8 +116,12 @@ case class TransformWithStateExec( val

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1519251919 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1519192303 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1519192303 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1519187563 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1519185494 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1519178347 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1519006106 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1519009915 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1519006106 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1519006106 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1519006106 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

neilramaswamy commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1518995350 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1518795524 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1518795524 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1518795524 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1518795524 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1518795524 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1518795524 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

neilramaswamy commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1518738187 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1518723182 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1518698125 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

neilramaswamy commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1518669207 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

neilramaswamy commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1518669207 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,224 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1517217945 ## sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StatefulProcessorHandleSuite.scala: ## @@ -0,0 +1,299 @@ +/* + * Licensed to the Apache

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

HeartSaVioR commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1517119556 ## sql/api/src/main/scala/org/apache/spark/sql/streaming/ExpiredTimerInfo.scala: ## @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1507971789 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulProcessorHandleImpl.scala: ## @@ -121,6 +123,42 @@ class StatefulProcessorHandleImpl(

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

sahnib commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1507925323 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulProcessorHandleImpl.scala: ## @@ -121,6 +123,42 @@ class StatefulProcessorHandleImpl(

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1505248645 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulProcessorHandleImpl.scala: ## @@ -121,6 +123,42 @@ class StatefulProcessorHandleImpl(

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1505126889 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulProcessorHandleImpl.scala: ## @@ -121,6 +123,42 @@ class StatefulProcessorHandleImpl(

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1505126284 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1505125882 ## sql/api/src/main/scala/org/apache/spark/sql/streaming/ExpiredTimerInfo.scala: ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

sahnib commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1504503668 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala: ## @@ -252,16 +252,30 @@ class RocksDB( } } + /** + * Check

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

sahnib commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1504459373 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TimerStateImpl.scala: ## @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1503733977 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TransformWithStateExec.scala: ## @@ -69,8 +70,20 @@ case class TransformWithStateExec(

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1503733123 ## sql/api/src/main/scala/org/apache/spark/sql/streaming/StatefulProcessor.scala: ## @@ -48,16 +48,19 @@ private[sql] trait StatefulProcessor[K, I, O] extends

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1503732422 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulProcessorHandleImpl.scala: ## @@ -121,6 +123,42 @@ class StatefulProcessorHandleImpl(

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1503731606 ## sql/api/src/main/scala/org/apache/spark/sql/streaming/ExpiredTimerInfo.scala: ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

sahnib commented on code in PR #45051: URL: https://github.com/apache/spark/pull/45051#discussion_r1503581091 ## sql/api/src/main/scala/org/apache/spark/sql/streaming/ExpiredTimerInfo.scala: ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] [SPARK-46913][SS] Add support for processing/event time based timers with transformWithState operator [spark]

anishshri-db commented on PR #45051: URL: https://github.com/apache/spark/pull/45051#issuecomment-1960408558 @HeartSaVioR - PTAL, thx ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the