[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=387700&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387700 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 14/Feb/20 21:44 Start Date: 14/Feb/20 21:44 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #10871: [BEAM-6857] Classify unbounded dynamic timers tests in the UsesUnboundedPCollections category URL: https://github.com/apache/beam/pull/10871 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 387700) Time Spent: 31h 10m (was: 31h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: New Feature > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Fix For: 2.20.0 > > Time Spent: 31h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=387671&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387671 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 14/Feb/20 20:56 Start Date: 14/Feb/20 20:56 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10871: [BEAM-6857] Classify unbounded dynamic timers tests in the UsesUnboundedPCollections category URL: https://github.com/apache/beam/pull/10871#issuecomment-586469227 Thanks for the fix! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 387671) Time Spent: 31h (was: 30h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: New Feature > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Fix For: 2.20.0 > > Time Spent: 31h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=387670&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387670 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 14/Feb/20 20:56 Start Date: 14/Feb/20 20:56 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10871: [BEAM-6857] Classify unbounded dynamic timers tests in the UsesUnboundedPCollections category URL: https://github.com/apache/beam/pull/10871#issuecomment-586469189 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 387670) Time Spent: 30h 50m (was: 30h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: New Feature > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Fix For: 2.20.0 > > Time Spent: 30h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=387669&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387669 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 14/Feb/20 20:55 Start Date: 14/Feb/20 20:55 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10871: [BEAM-6857] Classify unbounded dynamic timers tests in the UsesUnboundedPCollections category URL: https://github.com/apache/beam/pull/10871#issuecomment-586468863 Run Flink ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 387669) Time Spent: 30h 40m (was: 30.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: New Feature > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Fix For: 2.20.0 > > Time Spent: 30h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=387668&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387668 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 14/Feb/20 20:54 Start Date: 14/Feb/20 20:54 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #10871: [BEAM-6857] Classify unbounded dynamic timers tests in the UsesUnboundedPCollections category URL: https://github.com/apache/beam/pull/10871 This fixes the classic Flink runner breaking ValidatesRunner Tests. R: @reuvenlax This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 387668) Time Spent: 30.5h (was: 30h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: New Feature > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Fix For: 2.20.0 > > Time Spent: 30.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=381773&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381773 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 04/Feb/20 17:58 Start Date: 04/Feb/20 17:58 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-582036586 @rehmanmuradali it appears that the TimerMap PR did not work for Dataflow. Unfortunately due to a bug in the test description (one I didn't catch in code review), the test never ran on Dataflow. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 381773) Time Spent: 30h 20m (was: 30h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 30h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=381440&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381440 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 04/Feb/20 08:19 Start Date: 04/Feb/20 08:19 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 381440) Time Spent: 30h 10m (was: 30h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 30h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=381439&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381439 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 04/Feb/20 08:17 Start Date: 04/Feb/20 08:17 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#discussion_r374526099 ## File path: runners/google-cloud-dataflow-java/build.gradle ## @@ -137,6 +137,7 @@ def commonExcludeCategories = [ 'org.apache.beam.sdk.testing.UsesGaugeMetrics', 'org.apache.beam.sdk.testing.UsesSetState', 'org.apache.beam.sdk.testing.UsesMapState', + 'org.apache.beam.sdk.testing.UsesTimerMap', Review comment: Yes, My first approach was to enable for Dataflow but then the build fail on the `UsesTimerMap` tests: https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_PR/210/ https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_PR/76/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 381439) Time Spent: 30h (was: 29h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 30h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=381396&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381396 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 04/Feb/20 05:07 Start Date: 04/Feb/20 05:07 Worklog Time Spent: 10m Work Description: reuvenlax commented on pull request #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#discussion_r374476134 ## File path: runners/google-cloud-dataflow-java/build.gradle ## @@ -137,6 +137,7 @@ def commonExcludeCategories = [ 'org.apache.beam.sdk.testing.UsesGaugeMetrics', 'org.apache.beam.sdk.testing.UsesSetState', 'org.apache.beam.sdk.testing.UsesMapState', + 'org.apache.beam.sdk.testing.UsesTimerMap', Review comment: If it was broken, we need to fix. Support for this was added for Dataflow, though it now appears that the test was never being run :( This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 381396) Time Spent: 29h 50m (was: 29h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 29h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=381383&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381383 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 04/Feb/20 04:35 Start Date: 04/Feb/20 04:35 Worklog Time Spent: 10m Work Description: reuvenlax commented on pull request #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#discussion_r374470396 ## File path: runners/google-cloud-dataflow-java/build.gradle ## @@ -137,6 +137,7 @@ def commonExcludeCategories = [ 'org.apache.beam.sdk.testing.UsesGaugeMetrics', 'org.apache.beam.sdk.testing.UsesSetState', 'org.apache.beam.sdk.testing.UsesMapState', + 'org.apache.beam.sdk.testing.UsesTimerMap', Review comment: do we need to exclude for Dataflow? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 381383) Time Spent: 29.5h (was: 29h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 29.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=381384&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-381384 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 04/Feb/20 04:35 Start Date: 04/Feb/20 04:35 Worklog Time Spent: 10m Work Description: reuvenlax commented on pull request #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#discussion_r374470501 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ParDoTest.java ## @@ -4292,7 +4292,7 @@ public void onTimer(OutputReceiver r, @TimerId("timer") Timer timer) { public static class TimerFamilyTests extends SharedTestBase implements Serializable { @Test -@Category({NeedsRunner.class, UsesTimersInParDo.class, UsesTimerMap.class}) +@Category({ValidatesRunner.class, UsesTimersInParDo.class, UsesTimerMap.class}) Review comment: Good catch - I missed this in the code review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 381384) Time Spent: 29h 40m (was: 29.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 29h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380537&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380537 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 14:19 Start Date: 02/Feb/20 14:19 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581140061 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380537) Time Spent: 29h 10m (was: 29h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 29h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380538&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380538 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 14:19 Start Date: 02/Feb/20 14:19 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581140061 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380538) Time Spent: 29h 20m (was: 29h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 29h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380528&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380528 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:30 Start Date: 02/Feb/20 13:30 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581135871 Run Java Flink PortableValidatesRunner Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380528) Time Spent: 28h 40m (was: 28.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 28h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380523&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380523 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:30 Start Date: 02/Feb/20 13:30 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581135813 Run Gearpump ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380523) Time Spent: 27h 50m (was: 27h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 27h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380521&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380521 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:30 Start Date: 02/Feb/20 13:30 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581135794 Run Direct ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380521) Time Spent: 27.5h (was: 27h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 27.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380526&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380526 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:30 Start Date: 02/Feb/20 13:30 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581135836 Run Spark StructuredStreaming ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380526) Time Spent: 28h 20m (was: 28h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 28h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380529&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380529 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:30 Start Date: 02/Feb/20 13:30 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581135880 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380529) Time Spent: 28h 50m (was: 28h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 28h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380525&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380525 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:30 Start Date: 02/Feb/20 13:30 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581135829 Run Spark ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380525) Time Spent: 28h 10m (was: 28h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 28h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380527&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380527 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:30 Start Date: 02/Feb/20 13:30 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581135846 Run Dataflow PortabilityApi ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380527) Time Spent: 28.5h (was: 28h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 28.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380524&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380524 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:30 Start Date: 02/Feb/20 13:30 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581135822 Run Samza ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380524) Time Spent: 28h (was: 27h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 28h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380530&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380530 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:30 Start Date: 02/Feb/20 13:30 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581135888 Run Java Spark PortableValidatesRunner Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380530) Time Spent: 29h (was: 28h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 29h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380522&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380522 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:30 Start Date: 02/Feb/20 13:30 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581135802 Run Flink ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380522) Time Spent: 27h 40m (was: 27.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 27h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380520&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380520 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:30 Start Date: 02/Feb/20 13:30 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581135781 Run Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380520) Time Spent: 27h 20m (was: 27h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 27h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380516&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380516 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:29 Start Date: 02/Feb/20 13:29 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117489 'Run Java Flink PortableValidatesRunner Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380516) Time Spent: 26h 40m (was: 26.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 26h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380519&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380519 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:29 Start Date: 02/Feb/20 13:29 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581135777 Run Apex ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380519) Time Spent: 27h 10m (was: 27h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 27h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380509&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380509 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:29 Start Date: 02/Feb/20 13:29 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117308 Run Direct ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380509) Time Spent: 25.5h (was: 25h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 25.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380514&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380514 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:29 Start Date: 02/Feb/20 13:29 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117426 Run Spark StructuredStreaming ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380514) Time Spent: 26h 20m (was: 26h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 26h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380511&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380511 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:29 Start Date: 02/Feb/20 13:29 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117357 Run Gearpump ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380511) Time Spent: 25h 50m (was: 25h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 25h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380518&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380518 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:29 Start Date: 02/Feb/20 13:29 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117527 Run Java Spark PortableValidatesRunner Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380518) Time Spent: 27h (was: 26h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 27h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380515&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380515 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:29 Start Date: 02/Feb/20 13:29 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117451 Run Dataflow PortabilityApi ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380515) Time Spent: 26.5h (was: 26h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 26.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380517&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380517 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:29 Start Date: 02/Feb/20 13:29 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117510 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380517) Time Spent: 26h 50m (was: 26h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 26h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380513&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380513 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:29 Start Date: 02/Feb/20 13:29 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117403 Run Spark ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380513) Time Spent: 26h 10m (was: 26h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 26h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380510&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380510 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:29 Start Date: 02/Feb/20 13:29 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117328 Run Flink ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380510) Time Spent: 25h 40m (was: 25.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 25h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380512&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380512 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:29 Start Date: 02/Feb/20 13:29 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117383 Run Samza ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380512) Time Spent: 26h (was: 25h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 26h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380508&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380508 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:29 Start Date: 02/Feb/20 13:29 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117279 Run Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380508) Time Spent: 25h 20m (was: 25h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 25h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380507&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380507 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 13:28 Start Date: 02/Feb/20 13:28 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117249 Run Apex ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380507) Time Spent: 25h 10m (was: 25h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 25h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380481&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380481 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 09:38 Start Date: 02/Feb/20 09:38 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117510 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380481) Time Spent: 24h 50m (was: 24h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 24h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380482&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380482 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 09:38 Start Date: 02/Feb/20 09:38 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117527 Run Java Spark PortableValidatesRunner Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380482) Time Spent: 25h (was: 24h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 25h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380480&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380480 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 09:38 Start Date: 02/Feb/20 09:38 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117489 'Run Java Flink PortableValidatesRunner Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380480) Time Spent: 24h 40m (was: 24.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 24h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380478&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380478 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 09:38 Start Date: 02/Feb/20 09:38 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117451 Run Dataflow PortabilityApi ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380478) Time Spent: 24.5h (was: 24h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 24.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380477&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380477 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 09:37 Start Date: 02/Feb/20 09:37 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117426 Run Spark StructuredStreaming ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380477) Time Spent: 24h 20m (was: 24h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 24h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380473&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380473 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 09:37 Start Date: 02/Feb/20 09:37 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117328 Run Flink ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380473) Time Spent: 23h 40m (was: 23.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 23h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380474&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380474 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 09:37 Start Date: 02/Feb/20 09:37 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117357 Run Gearpump ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380474) Time Spent: 23h 50m (was: 23h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 23h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380476&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380476 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 09:37 Start Date: 02/Feb/20 09:37 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117403 Run Spark ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380476) Time Spent: 24h 10m (was: 24h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 24h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380475&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380475 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 09:37 Start Date: 02/Feb/20 09:37 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117383 Run Samza ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380475) Time Spent: 24h (was: 23h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 24h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380471&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380471 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 09:36 Start Date: 02/Feb/20 09:36 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117279 Run Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380471) Time Spent: 23h 20m (was: 23h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 23h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380470&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380470 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 09:36 Start Date: 02/Feb/20 09:36 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117249 Run Apex ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380470) Time Spent: 23h 10m (was: 23h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 23h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380472&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380472 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 09:36 Start Date: 02/Feb/20 09:36 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747#issuecomment-581117308 Run Direct ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380472) Time Spent: 23.5h (was: 23h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 23.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=380469&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380469 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 02/Feb/20 09:34 Start Date: 02/Feb/20 09:34 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #10747: [BEAM-6857] Recategorize UsesTimerMap tests to ValidatesRunner URL: https://github.com/apache/beam/pull/10747 NeedsRunner tests are not run against all runners so we need to recategorize those to validate runners implementation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 380469) Time Spent: 23h (was: 22h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 23h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=373244&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373244 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 20:23 Start Date: 16/Jan/20 20:23 Worklog Time Spent: 10m Work Description: reuvenlax commented on pull request #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373244) Time Spent: 22h 50m (was: 22h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 22h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=373221&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373221 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 19:22 Start Date: 16/Jan/20 19:22 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575304287 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373221) Time Spent: 22h 40m (was: 22.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 22h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=373208&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373208 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 18:49 Start Date: 16/Jan/20 18:49 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575291325 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373208) Time Spent: 22.5h (was: 22h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 22.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=373205&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373205 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 18:42 Start Date: 16/Jan/20 18:42 Worklog Time Spent: 10m Work Description: pabloem commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575288342 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373205) Time Spent: 22h 20m (was: 22h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 22h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=373158&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373158 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 17:22 Start Date: 16/Jan/20 17:22 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575256384 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373158) Time Spent: 22h 10m (was: 22h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 22h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=373025&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373025 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 14:16 Start Date: 16/Jan/20 14:16 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575170802 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373025) Time Spent: 22h (was: 21h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 22h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372878&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372878 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 09:04 Start Date: 16/Jan/20 09:04 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575007437 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372878) Time Spent: 20h 50m (was: 20h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 20h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372881&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372881 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 09:04 Start Date: 16/Jan/20 09:04 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575053107 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372881) Time Spent: 21h 20m (was: 21h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 21h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372882&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372882 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 09:04 Start Date: 16/Jan/20 09:04 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575053052 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372882) Time Spent: 21.5h (was: 21h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 21.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372884&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372884 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 09:04 Start Date: 16/Jan/20 09:04 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575052995 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372884) Time Spent: 21h 40m (was: 21.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 21h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372880&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372880 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 09:04 Start Date: 16/Jan/20 09:04 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575053052 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372880) Time Spent: 21h 10m (was: 21h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 21h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372885&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372885 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 09:04 Start Date: 16/Jan/20 09:04 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575053107 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372885) Time Spent: 21h 50m (was: 21h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 21h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372879&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372879 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 09:04 Start Date: 16/Jan/20 09:04 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575052995 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372879) Time Spent: 21h (was: 20h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 21h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372828&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372828 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 07:13 Start Date: 16/Jan/20 07:13 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575016496 run flink validatesrunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372828) Time Spent: 20.5h (was: 20h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 20.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372829&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372829 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 07:13 Start Date: 16/Jan/20 07:13 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575016535 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372829) Time Spent: 20h 40m (was: 20.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 20h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372827&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372827 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 07:13 Start Date: 16/Jan/20 07:13 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575016416 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372827) Time Spent: 20h 20m (was: 20h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 20h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372825&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372825 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 07:03 Start Date: 16/Jan/20 07:03 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575013627 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372825) Time Spent: 20h 10m (was: 20h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 20h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372824&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372824 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 07:03 Start Date: 16/Jan/20 07:03 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575013627 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372824) Time Spent: 20h (was: 19h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 20h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372820&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372820 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 06:40 Start Date: 16/Jan/20 06:40 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575007454 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372820) Time Spent: 19h 40m (was: 19.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 19h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372819&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372819 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 06:40 Start Date: 16/Jan/20 06:40 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575007437 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372819) Time Spent: 19.5h (was: 19h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 19.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372821&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372821 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 06:40 Start Date: 16/Jan/20 06:40 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575007454 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372821) Time Spent: 19h 50m (was: 19h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 19h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372815&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372815 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 06:29 Start Date: 16/Jan/20 06:29 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575004478 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372815) Time Spent: 19h 20m (was: 19h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 19h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372808&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372808 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 06:16 Start Date: 16/Jan/20 06:16 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-575001304 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372808) Time Spent: 19h 10m (was: 19h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 19h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372803&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372803 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 06:02 Start Date: 16/Jan/20 06:02 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on pull request #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#discussion_r367243852 ## File path: runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WindmillTimerInternals.java ## @@ -285,13 +308,27 @@ public static TimerData windmillTimerToTimerData( * This is necessary because Windmill will deduplicate based only on this tag. */ public static ByteString timerTag(WindmillNamespacePrefix prefix, TimerData timerData) { -String tagString = -new StringBuilder() -.append(prefix.byteString().toStringUtf8()) // this never ends with a slash -.append(timerData.getNamespace().stringKey()) // this must begin and end with a slash -.append('+') -.append(timerData.getTimerId()) // this is arbitrary; currently unescaped -.toString(); +String tagString; +// Timers without timerFamily would have timerFamily would be an empty string +if ("".equals(timerData.getTimerFamilyId())) { + tagString = + new StringBuilder() + .append(prefix.byteString().toStringUtf8()) // this never ends with a slash + .append(timerData.getNamespace().stringKey()) // this must begin and end with a slash + .append('+') + .append(timerData.getTimerId()) // this is arbitrary; currently unescaped + .toString(); +} else { + tagString = + new StringBuilder() + .append(prefix.byteString().toStringUtf8()) // this never ends with a slash + .append(timerData.getNamespace().stringKey()) // this must begin and end with a slash + .append('+') + .append(timerData.getTimerId()) // this is arbitrary; currently unescaped + .append('+') + .append(timerData.getTimerFamilyId()) + .toString(); +} Review comment: > actually maybe we don't need a new tag after all, as these tests are surprisingly passing @reuvenlax , timerHoldTag updated This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372803) Time Spent: 19h (was: 18h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 19h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372797&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372797 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 05:40 Start Date: 16/Jan/20 05:40 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on pull request #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#discussion_r367239320 ## File path: runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WindmillTimerInternals.java ## @@ -285,13 +308,27 @@ public static TimerData windmillTimerToTimerData( * This is necessary because Windmill will deduplicate based only on this tag. */ public static ByteString timerTag(WindmillNamespacePrefix prefix, TimerData timerData) { -String tagString = -new StringBuilder() -.append(prefix.byteString().toStringUtf8()) // this never ends with a slash -.append(timerData.getNamespace().stringKey()) // this must begin and end with a slash -.append('+') -.append(timerData.getTimerId()) // this is arbitrary; currently unescaped -.toString(); +String tagString; +// Timers without timerFamily would have timerFamily would be an empty string +if ("".equals(timerData.getTimerFamilyId())) { + tagString = + new StringBuilder() + .append(prefix.byteString().toStringUtf8()) // this never ends with a slash + .append(timerData.getNamespace().stringKey()) // this must begin and end with a slash + .append('+') + .append(timerData.getTimerId()) // this is arbitrary; currently unescaped + .toString(); +} else { + tagString = + new StringBuilder() + .append(prefix.byteString().toStringUtf8()) // this never ends with a slash + .append(timerData.getNamespace().stringKey()) // this must begin and end with a slash + .append('+') + .append(timerData.getTimerId()) // this is arbitrary; currently unescaped + .append('+') + .append(timerData.getTimerFamilyId()) + .toString(); +} Review comment: > Overall this looks good now! Good job debugging the overlapping timers. > > I think the only thing that remaining will be to add a UsesTimerMap test category so that we can exclude the runners that don't implement this. I'll trigger ValidatesRunner for those runners, though I expect them to break. Thank you @reuvenlax for sparing your time to guide me whenever I get stuck. I am adding a new category for TImerMap. Seems like I only need to exclude Flink runner for this category. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372797) Time Spent: 18h 50m (was: 18h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 18h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { >
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372791&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372791 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 05:13 Start Date: 16/Jan/20 05:13 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574986765 actually maybe we don't need a new tag after all, as these tests are surprisingly passing This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372791) Time Spent: 18h 40m (was: 18.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 18h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372743&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372743 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 03:04 Start Date: 16/Jan/20 03:04 Worklog Time Spent: 10m Work Description: reuvenlax commented on pull request #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#discussion_r367210910 ## File path: runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WindmillTimerInternals.java ## @@ -285,13 +308,27 @@ public static TimerData windmillTimerToTimerData( * This is necessary because Windmill will deduplicate based only on this tag. */ public static ByteString timerTag(WindmillNamespacePrefix prefix, TimerData timerData) { -String tagString = -new StringBuilder() -.append(prefix.byteString().toStringUtf8()) // this never ends with a slash -.append(timerData.getNamespace().stringKey()) // this must begin and end with a slash -.append('+') -.append(timerData.getTimerId()) // this is arbitrary; currently unescaped -.toString(); +String tagString; +// Timers without timerFamily would have timerFamily would be an empty string +if ("".equals(timerData.getTimerFamilyId())) { + tagString = + new StringBuilder() + .append(prefix.byteString().toStringUtf8()) // this never ends with a slash + .append(timerData.getNamespace().stringKey()) // this must begin and end with a slash + .append('+') + .append(timerData.getTimerId()) // this is arbitrary; currently unescaped + .toString(); +} else { + tagString = + new StringBuilder() + .append(prefix.byteString().toStringUtf8()) // this never ends with a slash + .append(timerData.getNamespace().stringKey()) // this must begin and end with a slash + .append('+') + .append(timerData.getTimerId()) // this is arbitrary; currently unescaped + .append('+') + .append(timerData.getTimerFamilyId()) + .toString(); +} Review comment: I just realized that timerHoldTag was not updated, which will cause this feature to be incompatible with the previous outputTimestamp feature. This logic needs to be executed there as well (also backwards compatible) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372743) Time Spent: 18.5h (was: 18h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 18.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372706&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372706 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 02:01 Start Date: 16/Jan/20 02:01 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574945433 Run Spark ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372706) Time Spent: 18h 20m (was: 18h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 18h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372703&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372703 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 02:00 Start Date: 16/Jan/20 02:00 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574945334 Run Dataflow PortabilityApi ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372703) Time Spent: 17h 50m (was: 17h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 17h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372705&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372705 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 02:00 Start Date: 16/Jan/20 02:00 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574945393 Run Samza ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372705) Time Spent: 18h 10m (was: 18h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 18h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372702&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372702 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 02:00 Start Date: 16/Jan/20 02:00 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574945236 Run Java Flink PortableValidatesRunner Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372702) Time Spent: 17h 40m (was: 17.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 17h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372704&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372704 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 02:00 Start Date: 16/Jan/20 02:00 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574945364 Run Apex ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372704) Time Spent: 18h (was: 17h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 18h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372699&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372699 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 01:59 Start Date: 16/Jan/20 01:59 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574945160 run Dataflow PortableValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372699) Time Spent: 17h 20m (was: 17h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 17h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372697&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372697 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 01:59 Start Date: 16/Jan/20 01:59 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574945027 Overall this looks good now! Good job debugging the overlapping timers. I think the only thing that remaining will be to add a UsesTimerMap test category so that we can exclude the runners that don't implement this. I'll trigger ValidatesRunner for those runners, though I expect them to break. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372697) Time Spent: 17h (was: 16h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 17h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372700&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372700 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 02:00 Start Date: 16/Jan/20 02:00 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574945202 Run Flink ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372700) Time Spent: 17.5h (was: 17h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 17.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372698&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372698 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 01:59 Start Date: 16/Jan/20 01:59 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574945064 run Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372698) Time Spent: 17h 10m (was: 17h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 17h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372692&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372692 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 01:42 Start Date: 16/Jan/20 01:42 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574941034 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372692) Time Spent: 16h 40m (was: 16.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 16h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372693&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372693 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 16/Jan/20 01:42 Start Date: 16/Jan/20 01:42 Worklog Time Spent: 10m Work Description: reuvenlax commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574941080 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372693) Time Spent: 16h 50m (was: 16h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 16h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372417&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372417 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 15/Jan/20 17:34 Start Date: 15/Jan/20 17:34 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574769708 @reuvenlax test case with multiple timerFamily and same timerId is passing now. Kindly review this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372417) Time Spent: 16.5h (was: 16h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 16.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372397&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372397 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 15/Jan/20 16:39 Start Date: 15/Jan/20 16:39 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574745063 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372397) Time Spent: 16h 20m (was: 16h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 16h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372342&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372342 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 15/Jan/20 14:18 Start Date: 15/Jan/20 14:18 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574654628 Run Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372342) Time Spent: 16h 10m (was: 16h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 16h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372339&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372339 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 15/Jan/20 14:17 Start Date: 15/Jan/20 14:17 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574678637 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372339) Time Spent: 15h 50m (was: 15h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 15h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372341&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372341 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 15/Jan/20 14:17 Start Date: 15/Jan/20 14:17 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574675558 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372341) Time Spent: 16h (was: 15h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 16h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372332&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372332 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 15/Jan/20 14:09 Start Date: 15/Jan/20 14:09 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574675558 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372332) Time Spent: 15h 40m (was: 15.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 15h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372330&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372330 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 15/Jan/20 14:09 Start Date: 15/Jan/20 14:09 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574675331 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372330) Time Spent: 15h 20m (was: 15h 10m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 15h 20m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372331&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372331 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 15/Jan/20 14:09 Start Date: 15/Jan/20 14:09 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574674712 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372331) Time Spent: 15.5h (was: 15h 20m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 15.5h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372329&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372329 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 15/Jan/20 14:09 Start Date: 15/Jan/20 14:09 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574675331 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372329) Time Spent: 15h 10m (was: 15h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 15h 10m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372325&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372325 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 15/Jan/20 14:07 Start Date: 15/Jan/20 14:07 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574674712 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372325) Time Spent: 14h 40m (was: 14.5h) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 14h 40m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372327&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372327 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 15/Jan/20 14:07 Start Date: 15/Jan/20 14:07 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574669289 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372327) Time Spent: 15h (was: 14h 50m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 15h > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-6857) Support dynamic timers
[ https://issues.apache.org/jira/browse/BEAM-6857?focusedWorklogId=372326&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372326 ] ASF GitHub Bot logged work on BEAM-6857: Author: ASF GitHub Bot Created on: 15/Jan/20 14:07 Start Date: 15/Jan/20 14:07 Worklog Time Spent: 10m Work Description: rehmanmuradali commented on issue #10316: [BEAM-6857] Support Dynamic Timers URL: https://github.com/apache/beam/pull/10316#issuecomment-574670691 run java precommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372326) Time Spent: 14h 50m (was: 14h 40m) > Support dynamic timers > -- > > Key: BEAM-6857 > URL: https://issues.apache.org/jira/browse/BEAM-6857 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Reuven Lax >Assignee: Shehzaad Nakhoda >Priority: Major > Time Spent: 14h 50m > Remaining Estimate: 0h > > The Beam timers API currently requires each timer to be statically specified > in the DoFn. The user must provide a separate callback method per timer. For > example: > > {code:java} > DoFn() > { > @TimerId("timer1") > private final TimerSpec timer1 = TimerSpecs.timer(...); > @TimerId("timer2") > private final TimerSpec timer2 = TimerSpecs.timer(...); > .. set timers in processElement > @OnTimer("timer1") > public void onTimer1() { .} > @OnTimer("timer2") > public void onTimer2() {} > } > {code} > > However there are many cases where the user does not know the set of timers > statically when writing their code. This happens when the timer tag should be > based on the data. It also happens when writing a DSL on top of Beam, where > the DSL author has to create DoFns but does not know statically which timers > their users will want to set (e.g. Scio). > > The goal is to support dynamic timers. Something as follows; > > {code:java} > DoFn() > { > @TimerId("timer") > private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...); > @ProcessElement process(@TimerId("timer") DynamicTimer timer) > { > timer.set("tag1'", ts); >timer.set("tag2", ts); > } > @OnTimer("timer") > public void onTimer1(@TimerTag String tag) { .} > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)