[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=387700=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387700
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 14/Feb/20 21:44
Start Date: 14/Feb/20 21:44
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10871: [BEAM-6857]
Classify unbounded dynamic timers tests in the UsesUnboundedPCollections
category
URL: https://github.com/apache/beam/pull/10871
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 387700)
Time Spent: 31h 10m (was: 31h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Fix For: 2.20.0
>
> Time Spent: 31h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=387671=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387671
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 14/Feb/20 20:56
Start Date: 14/Feb/20 20:56
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10871: [BEAM-6857]
Classify unbounded dynamic timers tests in the UsesUnboundedPCollections
category
URL: https://github.com/apache/beam/pull/10871#issuecomment-586469227
Thanks for the fix!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 387671)
Time Spent: 31h (was: 30h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Fix For: 2.20.0
>
> Time Spent: 31h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=387670=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387670
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 14/Feb/20 20:56
Start Date: 14/Feb/20 20:56
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10871: [BEAM-6857]
Classify unbounded dynamic timers tests in the UsesUnboundedPCollections
category
URL: https://github.com/apache/beam/pull/10871#issuecomment-586469189
LGTM
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 387670)
Time Spent: 30h 50m (was: 30h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Fix For: 2.20.0
>
> Time Spent: 30h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=387669=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387669
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 14/Feb/20 20:55
Start Date: 14/Feb/20 20:55
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10871: [BEAM-6857] Classify
unbounded dynamic timers tests in the UsesUnboundedPCollections category
URL: https://github.com/apache/beam/pull/10871#issuecomment-586468863
Run Flink ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 387669)
Time Spent: 30h 40m (was: 30.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Fix For: 2.20.0
>
> Time Spent: 30h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=387668=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387668
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 14/Feb/20 20:54
Start Date: 14/Feb/20 20:54
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10871: [BEAM-6857]
Classify unbounded dynamic timers tests in the UsesUnboundedPCollections
category
URL: https://github.com/apache/beam/pull/10871
This fixes the classic Flink runner breaking ValidatesRunner Tests.
R: @reuvenlax
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 387668)
Time Spent: 30.5h (was: 30h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Fix For: 2.20.0
>
> Time Spent: 30.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=381773=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381773
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 04/Feb/20 17:58
Start Date: 04/Feb/20 17:58
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-582036586
@rehmanmuradali it appears that the TimerMap PR did not work for Dataflow.
Unfortunately due to a bug in the test description (one I didn't catch in code
review), the test never ran on Dataflow.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 381773)
Time Spent: 30h 20m (was: 30h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 30h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=381440=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381440
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 04/Feb/20 08:19
Start Date: 04/Feb/20 08:19
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 381440)
Time Spent: 30h 10m (was: 30h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 30h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=381439=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381439
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 04/Feb/20 08:17
Start Date: 04/Feb/20 08:17
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#discussion_r374526099
##
File path: runners/google-cloud-dataflow-java/build.gradle
##
@@ -137,6 +137,7 @@ def commonExcludeCategories = [
'org.apache.beam.sdk.testing.UsesGaugeMetrics',
'org.apache.beam.sdk.testing.UsesSetState',
'org.apache.beam.sdk.testing.UsesMapState',
+ 'org.apache.beam.sdk.testing.UsesTimerMap',
Review comment:
Yes, My first approach was to enable for Dataflow but then the build fail on
the `UsesTimerMap` tests:
https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_PR/210/
https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_PR/76/
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 381439)
Time Spent: 30h (was: 29h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 30h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=381396=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381396
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 04/Feb/20 05:07
Start Date: 04/Feb/20 05:07
Worklog Time Spent: 10m
Work Description: reuvenlax commented on pull request #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#discussion_r374476134
##
File path: runners/google-cloud-dataflow-java/build.gradle
##
@@ -137,6 +137,7 @@ def commonExcludeCategories = [
'org.apache.beam.sdk.testing.UsesGaugeMetrics',
'org.apache.beam.sdk.testing.UsesSetState',
'org.apache.beam.sdk.testing.UsesMapState',
+ 'org.apache.beam.sdk.testing.UsesTimerMap',
Review comment:
If it was broken, we need to fix. Support for this was added for Dataflow,
though it now appears that the test was never being run :(
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 381396)
Time Spent: 29h 50m (was: 29h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 29h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=381383=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381383
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 04/Feb/20 04:35
Start Date: 04/Feb/20 04:35
Worklog Time Spent: 10m
Work Description: reuvenlax commented on pull request #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#discussion_r374470396
##
File path: runners/google-cloud-dataflow-java/build.gradle
##
@@ -137,6 +137,7 @@ def commonExcludeCategories = [
'org.apache.beam.sdk.testing.UsesGaugeMetrics',
'org.apache.beam.sdk.testing.UsesSetState',
'org.apache.beam.sdk.testing.UsesMapState',
+ 'org.apache.beam.sdk.testing.UsesTimerMap',
Review comment:
do we need to exclude for Dataflow?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 381383)
Time Spent: 29.5h (was: 29h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 29.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=381384=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381384
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 04/Feb/20 04:35
Start Date: 04/Feb/20 04:35
Worklog Time Spent: 10m
Work Description: reuvenlax commented on pull request #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#discussion_r374470501
##
File path:
sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ParDoTest.java
##
@@ -4292,7 +4292,7 @@ public void onTimer(OutputReceiver r,
@TimerId("timer") Timer timer) {
public static class TimerFamilyTests extends SharedTestBase implements
Serializable {
@Test
-@Category({NeedsRunner.class, UsesTimersInParDo.class, UsesTimerMap.class})
+@Category({ValidatesRunner.class, UsesTimersInParDo.class,
UsesTimerMap.class})
Review comment:
Good catch - I missed this in the code review.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 381384)
Time Spent: 29h 40m (was: 29.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 29h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380537=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380537
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 14:19
Start Date: 02/Feb/20 14:19
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581140061
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380537)
Time Spent: 29h 10m (was: 29h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 29h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380538=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380538
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 14:19
Start Date: 02/Feb/20 14:19
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581140061
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380538)
Time Spent: 29h 20m (was: 29h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 29h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380526=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380526
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:30
Start Date: 02/Feb/20 13:30
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581135836
Run Spark StructuredStreaming ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380526)
Time Spent: 28h 20m (was: 28h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 28h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380528=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380528
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:30
Start Date: 02/Feb/20 13:30
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581135871
Run Java Flink PortableValidatesRunner Batch
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380528)
Time Spent: 28h 40m (was: 28.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 28h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380523=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380523
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:30
Start Date: 02/Feb/20 13:30
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581135813
Run Gearpump ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380523)
Time Spent: 27h 50m (was: 27h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 27h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380521=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380521
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:30
Start Date: 02/Feb/20 13:30
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581135794
Run Direct ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380521)
Time Spent: 27.5h (was: 27h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 27.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380529=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380529
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:30
Start Date: 02/Feb/20 13:30
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581135880
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380529)
Time Spent: 28h 50m (was: 28h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 28h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380525=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380525
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:30
Start Date: 02/Feb/20 13:30
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581135829
Run Spark ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380525)
Time Spent: 28h 10m (was: 28h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 28h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380527=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380527
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:30
Start Date: 02/Feb/20 13:30
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581135846
Run Dataflow PortabilityApi ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380527)
Time Spent: 28.5h (was: 28h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 28.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380524=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380524
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:30
Start Date: 02/Feb/20 13:30
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581135822
Run Samza ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380524)
Time Spent: 28h (was: 27h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 28h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380530=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380530
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:30
Start Date: 02/Feb/20 13:30
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581135888
Run Java Spark PortableValidatesRunner Batch
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380530)
Time Spent: 29h (was: 28h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 29h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380522=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380522
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:30
Start Date: 02/Feb/20 13:30
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581135802
Run Flink ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380522)
Time Spent: 27h 40m (was: 27.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 27h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380520=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380520
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:30
Start Date: 02/Feb/20 13:30
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581135781
Run Dataflow ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380520)
Time Spent: 27h 20m (was: 27h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 27h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380514=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380514
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:29
Start Date: 02/Feb/20 13:29
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117426
Run Spark StructuredStreaming ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380514)
Time Spent: 26h 20m (was: 26h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 26h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380516=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380516
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:29
Start Date: 02/Feb/20 13:29
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117489
'Run Java Flink PortableValidatesRunner Batch
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380516)
Time Spent: 26h 40m (was: 26.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 26h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380519=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380519
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:29
Start Date: 02/Feb/20 13:29
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581135777
Run Apex ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380519)
Time Spent: 27h 10m (was: 27h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 27h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380509=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380509
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:29
Start Date: 02/Feb/20 13:29
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117308
Run Direct ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380509)
Time Spent: 25.5h (was: 25h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 25.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380511=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380511
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:29
Start Date: 02/Feb/20 13:29
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117357
Run Gearpump ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380511)
Time Spent: 25h 50m (was: 25h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 25h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380518=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380518
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:29
Start Date: 02/Feb/20 13:29
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117527
Run Java Spark PortableValidatesRunner Batch
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380518)
Time Spent: 27h (was: 26h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 27h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380512=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380512
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:29
Start Date: 02/Feb/20 13:29
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117383
Run Samza ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380512)
Time Spent: 26h (was: 25h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 26h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380508=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380508
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:29
Start Date: 02/Feb/20 13:29
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117279
Run Dataflow ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380508)
Time Spent: 25h 20m (was: 25h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 25h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380515=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380515
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:29
Start Date: 02/Feb/20 13:29
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117451
Run Dataflow PortabilityApi ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380515)
Time Spent: 26.5h (was: 26h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 26.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380517=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380517
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:29
Start Date: 02/Feb/20 13:29
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117510
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380517)
Time Spent: 26h 50m (was: 26h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 26h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380513=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380513
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:29
Start Date: 02/Feb/20 13:29
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117403
Run Spark ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380513)
Time Spent: 26h 10m (was: 26h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 26h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380510=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380510
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:29
Start Date: 02/Feb/20 13:29
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117328
Run Flink ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380510)
Time Spent: 25h 40m (was: 25.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 25h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380507=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380507
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 13:28
Start Date: 02/Feb/20 13:28
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117249
Run Apex ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380507)
Time Spent: 25h 10m (was: 25h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 25h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380481=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380481
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 09:38
Start Date: 02/Feb/20 09:38
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117510
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380481)
Time Spent: 24h 50m (was: 24h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 24h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380482=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380482
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 09:38
Start Date: 02/Feb/20 09:38
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117527
Run Java Spark PortableValidatesRunner Batch
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380482)
Time Spent: 25h (was: 24h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 25h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380480=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380480
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 09:38
Start Date: 02/Feb/20 09:38
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117489
'Run Java Flink PortableValidatesRunner Batch
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380480)
Time Spent: 24h 40m (was: 24.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 24h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380478=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380478
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 09:38
Start Date: 02/Feb/20 09:38
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117451
Run Dataflow PortabilityApi ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380478)
Time Spent: 24.5h (was: 24h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 24.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380477=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380477
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 09:37
Start Date: 02/Feb/20 09:37
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117426
Run Spark StructuredStreaming ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380477)
Time Spent: 24h 20m (was: 24h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 24h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380476=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380476
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 09:37
Start Date: 02/Feb/20 09:37
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117403
Run Spark ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380476)
Time Spent: 24h 10m (was: 24h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 24h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380475=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380475
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 09:37
Start Date: 02/Feb/20 09:37
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117383
Run Samza ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380475)
Time Spent: 24h (was: 23h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 24h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380473=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380473
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 09:37
Start Date: 02/Feb/20 09:37
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117328
Run Flink ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380473)
Time Spent: 23h 40m (was: 23.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 23h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380474=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380474
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 09:37
Start Date: 02/Feb/20 09:37
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117357
Run Gearpump ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380474)
Time Spent: 23h 50m (was: 23h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 23h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380471=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380471
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 09:36
Start Date: 02/Feb/20 09:36
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117279
Run Dataflow ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380471)
Time Spent: 23h 20m (was: 23h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 23h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380470=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380470
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 09:36
Start Date: 02/Feb/20 09:36
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117249
Run Apex ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380470)
Time Spent: 23h 10m (was: 23h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 23h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380472=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380472
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 09:36
Start Date: 02/Feb/20 09:36
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747#issuecomment-581117308
Run Direct ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380472)
Time Spent: 23.5h (was: 23h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 23.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380469=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380469
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 02/Feb/20 09:34
Start Date: 02/Feb/20 09:34
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10747: [BEAM-6857]
Recategorize UsesTimerMap tests to ValidatesRunner
URL: https://github.com/apache/beam/pull/10747
NeedsRunner tests are not run against all runners so we need to
recategorize those to validate runners implementation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 380469)
Time Spent: 23h (was: 22h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 23h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=373244=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373244
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 20:23
Start Date: 16/Jan/20 20:23
Worklog Time Spent: 10m
Work Description: reuvenlax commented on pull request #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 373244)
Time Spent: 22h 50m (was: 22h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 22h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=373221=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373221
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 19:22
Start Date: 16/Jan/20 19:22
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575304287
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 373221)
Time Spent: 22h 40m (was: 22.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 22h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=373208=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373208
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 18:49
Start Date: 16/Jan/20 18:49
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575291325
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 373208)
Time Spent: 22.5h (was: 22h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 22.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=373205=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373205
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 18:42
Start Date: 16/Jan/20 18:42
Worklog Time Spent: 10m
Work Description: pabloem commented on issue #10316: [BEAM-6857] Support
Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575288342
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 373205)
Time Spent: 22h 20m (was: 22h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 22h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=373158=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373158
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 17:22
Start Date: 16/Jan/20 17:22
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575256384
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 373158)
Time Spent: 22h 10m (was: 22h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 22h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=373025=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373025
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 14:16
Start Date: 16/Jan/20 14:16
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #10316: [BEAM-6857] Support
Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575170802
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 373025)
Time Spent: 22h (was: 21h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 22h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372878=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372878
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 09:04
Start Date: 16/Jan/20 09:04
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575007437
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372878)
Time Spent: 20h 50m (was: 20h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 20h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372881=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372881
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 09:04
Start Date: 16/Jan/20 09:04
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575053107
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372881)
Time Spent: 21h 20m (was: 21h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 21h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372882=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372882
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 09:04
Start Date: 16/Jan/20 09:04
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575053052
Retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372882)
Time Spent: 21.5h (was: 21h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 21.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372884=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372884
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 09:04
Start Date: 16/Jan/20 09:04
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575052995
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372884)
Time Spent: 21h 40m (was: 21.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 21h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372880=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372880
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 09:04
Start Date: 16/Jan/20 09:04
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575053052
Retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372880)
Time Spent: 21h 10m (was: 21h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 21h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372885=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372885
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 09:04
Start Date: 16/Jan/20 09:04
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575053107
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372885)
Time Spent: 21h 50m (was: 21h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 21h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372879=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372879
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 09:04
Start Date: 16/Jan/20 09:04
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575052995
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372879)
Time Spent: 21h (was: 20h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 21h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372828=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372828
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 07:13
Start Date: 16/Jan/20 07:13
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575016496
run flink validatesrunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372828)
Time Spent: 20.5h (was: 20h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 20.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372829=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372829
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 07:13
Start Date: 16/Jan/20 07:13
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575016535
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372829)
Time Spent: 20h 40m (was: 20.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 20h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372827=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372827
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 07:13
Start Date: 16/Jan/20 07:13
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575016416
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372827)
Time Spent: 20h 20m (was: 20h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 20h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372825=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372825
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 07:03
Start Date: 16/Jan/20 07:03
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575013627
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372825)
Time Spent: 20h 10m (was: 20h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 20h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372824=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372824
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 07:03
Start Date: 16/Jan/20 07:03
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575013627
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372824)
Time Spent: 20h (was: 19h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 20h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372819=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372819
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 06:40
Start Date: 16/Jan/20 06:40
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575007437
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372819)
Time Spent: 19.5h (was: 19h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 19.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372821=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372821
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 06:40
Start Date: 16/Jan/20 06:40
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575007454
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372821)
Time Spent: 19h 50m (was: 19h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 19h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372820=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372820
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 06:40
Start Date: 16/Jan/20 06:40
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575007454
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372820)
Time Spent: 19h 40m (was: 19.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 19h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372815=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372815
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 06:29
Start Date: 16/Jan/20 06:29
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575004478
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372815)
Time Spent: 19h 20m (was: 19h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 19h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372808=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372808
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 06:16
Start Date: 16/Jan/20 06:16
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-575001304
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372808)
Time Spent: 19h 10m (was: 19h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 19h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372803=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372803
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 06:02
Start Date: 16/Jan/20 06:02
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on pull request #10316:
[BEAM-6857] Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#discussion_r367243852
##
File path:
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WindmillTimerInternals.java
##
@@ -285,13 +308,27 @@ public static TimerData windmillTimerToTimerData(
* This is necessary because Windmill will deduplicate based only on this
tag.
*/
public static ByteString timerTag(WindmillNamespacePrefix prefix, TimerData
timerData) {
-String tagString =
-new StringBuilder()
-.append(prefix.byteString().toStringUtf8()) // this never ends
with a slash
-.append(timerData.getNamespace().stringKey()) // this must begin
and end with a slash
-.append('+')
-.append(timerData.getTimerId()) // this is arbitrary; currently
unescaped
-.toString();
+String tagString;
+// Timers without timerFamily would have timerFamily would be an empty
string
+if ("".equals(timerData.getTimerFamilyId())) {
+ tagString =
+ new StringBuilder()
+ .append(prefix.byteString().toStringUtf8()) // this never ends
with a slash
+ .append(timerData.getNamespace().stringKey()) // this must begin
and end with a slash
+ .append('+')
+ .append(timerData.getTimerId()) // this is arbitrary; currently
unescaped
+ .toString();
+} else {
+ tagString =
+ new StringBuilder()
+ .append(prefix.byteString().toStringUtf8()) // this never ends
with a slash
+ .append(timerData.getNamespace().stringKey()) // this must begin
and end with a slash
+ .append('+')
+ .append(timerData.getTimerId()) // this is arbitrary; currently
unescaped
+ .append('+')
+ .append(timerData.getTimerFamilyId())
+ .toString();
+}
Review comment:
> actually maybe we don't need a new tag after all, as these tests are
surprisingly passing
@reuvenlax , timerHoldTag updated
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372803)
Time Spent: 19h (was: 18h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 19h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372797=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372797
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 05:40
Start Date: 16/Jan/20 05:40
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on pull request #10316:
[BEAM-6857] Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#discussion_r367239320
##
File path:
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WindmillTimerInternals.java
##
@@ -285,13 +308,27 @@ public static TimerData windmillTimerToTimerData(
* This is necessary because Windmill will deduplicate based only on this
tag.
*/
public static ByteString timerTag(WindmillNamespacePrefix prefix, TimerData
timerData) {
-String tagString =
-new StringBuilder()
-.append(prefix.byteString().toStringUtf8()) // this never ends
with a slash
-.append(timerData.getNamespace().stringKey()) // this must begin
and end with a slash
-.append('+')
-.append(timerData.getTimerId()) // this is arbitrary; currently
unescaped
-.toString();
+String tagString;
+// Timers without timerFamily would have timerFamily would be an empty
string
+if ("".equals(timerData.getTimerFamilyId())) {
+ tagString =
+ new StringBuilder()
+ .append(prefix.byteString().toStringUtf8()) // this never ends
with a slash
+ .append(timerData.getNamespace().stringKey()) // this must begin
and end with a slash
+ .append('+')
+ .append(timerData.getTimerId()) // this is arbitrary; currently
unescaped
+ .toString();
+} else {
+ tagString =
+ new StringBuilder()
+ .append(prefix.byteString().toStringUtf8()) // this never ends
with a slash
+ .append(timerData.getNamespace().stringKey()) // this must begin
and end with a slash
+ .append('+')
+ .append(timerData.getTimerId()) // this is arbitrary; currently
unescaped
+ .append('+')
+ .append(timerData.getTimerFamilyId())
+ .toString();
+}
Review comment:
> Overall this looks good now! Good job debugging the overlapping timers.
>
> I think the only thing that remaining will be to add a UsesTimerMap test
category so that we can exclude the runners that don't implement this. I'll
trigger ValidatesRunner for those runners, though I expect them to break.
Thank you @reuvenlax for sparing your time to guide me whenever I get
stuck. I am adding a new category for TImerMap. Seems like I only need to
exclude Flink runner for this category.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372797)
Time Spent: 18h 50m (was: 18h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 18h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
>
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372791=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372791
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 05:13
Start Date: 16/Jan/20 05:13
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574986765
actually maybe we don't need a new tag after all, as these tests are
surprisingly passing
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372791)
Time Spent: 18h 40m (was: 18.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 18h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372743=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372743
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 03:04
Start Date: 16/Jan/20 03:04
Worklog Time Spent: 10m
Work Description: reuvenlax commented on pull request #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#discussion_r367210910
##
File path:
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WindmillTimerInternals.java
##
@@ -285,13 +308,27 @@ public static TimerData windmillTimerToTimerData(
* This is necessary because Windmill will deduplicate based only on this
tag.
*/
public static ByteString timerTag(WindmillNamespacePrefix prefix, TimerData
timerData) {
-String tagString =
-new StringBuilder()
-.append(prefix.byteString().toStringUtf8()) // this never ends
with a slash
-.append(timerData.getNamespace().stringKey()) // this must begin
and end with a slash
-.append('+')
-.append(timerData.getTimerId()) // this is arbitrary; currently
unescaped
-.toString();
+String tagString;
+// Timers without timerFamily would have timerFamily would be an empty
string
+if ("".equals(timerData.getTimerFamilyId())) {
+ tagString =
+ new StringBuilder()
+ .append(prefix.byteString().toStringUtf8()) // this never ends
with a slash
+ .append(timerData.getNamespace().stringKey()) // this must begin
and end with a slash
+ .append('+')
+ .append(timerData.getTimerId()) // this is arbitrary; currently
unescaped
+ .toString();
+} else {
+ tagString =
+ new StringBuilder()
+ .append(prefix.byteString().toStringUtf8()) // this never ends
with a slash
+ .append(timerData.getNamespace().stringKey()) // this must begin
and end with a slash
+ .append('+')
+ .append(timerData.getTimerId()) // this is arbitrary; currently
unescaped
+ .append('+')
+ .append(timerData.getTimerFamilyId())
+ .toString();
+}
Review comment:
I just realized that timerHoldTag was not updated, which will cause this
feature to be incompatible with the previous outputTimestamp feature. This
logic needs to be executed there as well (also backwards compatible)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372743)
Time Spent: 18.5h (was: 18h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 18.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372706=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372706
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 02:01
Start Date: 16/Jan/20 02:01
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574945433
Run Spark ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372706)
Time Spent: 18h 20m (was: 18h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 18h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372703=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372703
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 02:00
Start Date: 16/Jan/20 02:00
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574945334
Run Dataflow PortabilityApi ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372703)
Time Spent: 17h 50m (was: 17h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 17h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372705=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372705
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 02:00
Start Date: 16/Jan/20 02:00
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574945393
Run Samza ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372705)
Time Spent: 18h 10m (was: 18h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 18h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372702=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372702
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 02:00
Start Date: 16/Jan/20 02:00
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574945236
Run Java Flink PortableValidatesRunner Streaming
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372702)
Time Spent: 17h 40m (was: 17.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 17h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372704=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372704
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 02:00
Start Date: 16/Jan/20 02:00
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574945364
Run Apex ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372704)
Time Spent: 18h (was: 17h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 18h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372699=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372699
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 01:59
Start Date: 16/Jan/20 01:59
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574945160
run Dataflow PortableValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372699)
Time Spent: 17h 20m (was: 17h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 17h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372697=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372697
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 01:59
Start Date: 16/Jan/20 01:59
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574945027
Overall this looks good now! Good job debugging the overlapping timers.
I think the only thing that remaining will be to add a UsesTimerMap test
category so that we can exclude the runners that don't implement this. I'll
trigger ValidatesRunner for those runners, though I expect them to break.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372697)
Time Spent: 17h (was: 16h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 17h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372700=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372700
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 02:00
Start Date: 16/Jan/20 02:00
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574945202
Run Flink ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372700)
Time Spent: 17.5h (was: 17h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 17.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372698=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372698
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 01:59
Start Date: 16/Jan/20 01:59
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574945064
run Dataflow ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372698)
Time Spent: 17h 10m (was: 17h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 17h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372692=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372692
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 01:42
Start Date: 16/Jan/20 01:42
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574941034
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372692)
Time Spent: 16h 40m (was: 16.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 16h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372693=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372693
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 16/Jan/20 01:42
Start Date: 16/Jan/20 01:42
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574941080
Run Python PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372693)
Time Spent: 16h 50m (was: 16h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 16h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372417=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372417
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 15/Jan/20 17:34
Start Date: 15/Jan/20 17:34
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574769708
@reuvenlax test case with multiple timerFamily and same timerId is passing
now. Kindly review this PR.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372417)
Time Spent: 16.5h (was: 16h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 16.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372397=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372397
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 15/Jan/20 16:39
Start Date: 15/Jan/20 16:39
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574745063
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372397)
Time Spent: 16h 20m (was: 16h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 16h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372342=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372342
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 15/Jan/20 14:18
Start Date: 15/Jan/20 14:18
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574654628
Run Dataflow ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372342)
Time Spent: 16h 10m (was: 16h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 16h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372339=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372339
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 15/Jan/20 14:17
Start Date: 15/Jan/20 14:17
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574678637
Retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372339)
Time Spent: 15h 50m (was: 15h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 15h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372341=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372341
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 15/Jan/20 14:17
Start Date: 15/Jan/20 14:17
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574675558
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372341)
Time Spent: 16h (was: 15h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 16h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372332=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372332
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 15/Jan/20 14:09
Start Date: 15/Jan/20 14:09
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574675558
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372332)
Time Spent: 15h 40m (was: 15.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 15h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372330=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372330
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 15/Jan/20 14:09
Start Date: 15/Jan/20 14:09
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574675331
Retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372330)
Time Spent: 15h 20m (was: 15h 10m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 15h 20m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372331=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372331
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 15/Jan/20 14:09
Start Date: 15/Jan/20 14:09
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574674712
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372331)
Time Spent: 15.5h (was: 15h 20m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 15.5h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372329=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372329
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 15/Jan/20 14:09
Start Date: 15/Jan/20 14:09
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574675331
Retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372329)
Time Spent: 15h 10m (was: 15h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 15h 10m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372325=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372325
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 15/Jan/20 14:07
Start Date: 15/Jan/20 14:07
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574674712
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372325)
Time Spent: 14h 40m (was: 14.5h)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 14h 40m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372327=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372327
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 15/Jan/20 14:07
Start Date: 15/Jan/20 14:07
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574669289
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372327)
Time Spent: 15h (was: 14h 50m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 15h
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[
https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372326=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372326
]
ASF GitHub Bot logged work on BEAM-6857:
Author: ASF GitHub Bot
Created on: 15/Jan/20 14:07
Start Date: 15/Jan/20 14:07
Worklog Time Spent: 10m
Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857]
Support Dynamic Timers
URL: https://github.com/apache/beam/pull/10316#issuecomment-574670691
run java precommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372326)
Time Spent: 14h 50m (was: 14h 40m)
> Support dynamic timers
> --
>
> Key: BEAM-6857
> URL: https://issues.apache.org/jira/browse/BEAM-6857
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 14h 50m
> Remaining Estimate: 0h
>
> The Beam timers API currently requires each timer to be statically specified
> in the DoFn. The user must provide a separate callback method per timer. For
> example:
>
> {code:java}
> DoFn()
> {
> @TimerId("timer1")
> private final TimerSpec timer1 = TimerSpecs.timer(...);
> @TimerId("timer2")
> private final TimerSpec timer2 = TimerSpecs.timer(...);
> .. set timers in processElement
> @OnTimer("timer1")
> public void onTimer1() { .}
> @OnTimer("timer2")
> public void onTimer2() {}
> }
> {code}
>
> However there are many cases where the user does not know the set of timers
> statically when writing their code. This happens when the timer tag should be
> based on the data. It also happens when writing a DSL on top of Beam, where
> the DSL author has to create DoFns but does not know statically which timers
> their users will want to set (e.g. Scio).
>
> The goal is to support dynamic timers. Something as follows;
>
> {code:java}
> DoFn()
> {
> @TimerId("timer")
> private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
> @ProcessElement process(@TimerId("timer") DynamicTimer timer)
> {
> timer.set("tag1'", ts);
>timer.set("tag2", ts);
> }
> @OnTimer("timer")
> public void onTimer1(@TimerTag String tag) { .}
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
