RE: Debug job execution from savepoint
Great ! Thanks for your help. I think I will have all the functionalities I need for my use case with these test classes. Regards, Manuel De : Chesnay Schepler Envoyé : vendredi 22 juin 2018 15:03:01 À : Haddadi Manuel Cc : user@flink.apache.org Objet : Re: Debug job execution from savepoint Similarly, the AbstractOperatorRestoreTestBase shows how to start en embedded cluster and submit a job from a savepoint. see * #restoreJob * #createJobGraph * MiniClusterResource On 22.06.2018 13:53, Tzu-Li (Gordon) Tai wrote: Hi, The tests in Flink uses a `AbstractStreamOperatorTestHarness` that allows wrapping an operator, input elements into the operator, getting the outputs, and also snapshotting / restoring operator state. I’m not sure of your specific case, but in general that test harness utility can be used to test the operator’s output and state update logic for a given set of input records. Cheers, Gordon On 22 June 2018 at 11:23:28 AM, Haddadi Manuel (manuel.hadd...@gfi.fr<mailto:manuel.hadd...@gfi.fr>) wrote: Hi Fabian, thanks for your response. Here is the Jira issue : https://issues.apache.org/jira/browse/FLINK-9644 Would you have any devlopper tips on how testing my intializeState methods to validate that my savepoints are corrects ? Thank you again. Regards, Manuel De : Fabian Hueske <mailto:fhue...@gmail.com> Envoyé : jeudi 21 juin 2018 10:33:10 À : Haddadi Manuel Cc : user@flink.apache.org<mailto:user@flink.apache.org> Objet : Re: Debug job execution from savepoint Hi Manuel, I had a look and couldn't find a way to do it. However, this sounds like a very useful feature to me. Would you mind creating a Jira issue [1] for that? Thanks, Fabian [1] https://issues.apache.org/jira/projects/FLINK 2018-06-18 16:23 GMT+02:00 Haddadi Manuel mailto:manuel.hadd...@gfi.fr>>: Hi all, I would like to test my checkpointing implementation doing a step-by-step debugging under an IDE. Is there a way to restore a job from a local savepoint in a local stream environnement, like a command "flink run -s :savepontpath" would do ? Thanks, Manuel
Re: Debug job execution from savepoint
Similarly, the AbstractOperatorRestoreTestBase shows how to start en embedded cluster and submit a job from a savepoint. see * #restoreJob * #createJobGraph * MiniClusterResource On 22.06.2018 13:53, Tzu-Li (Gordon) Tai wrote: Hi, The tests in Flink uses a `AbstractStreamOperatorTestHarness` that allows wrapping an operator, input elements into the operator, getting the outputs, and also snapshotting / restoring operator state. I’m not sure of your specific case, but in general that test harness utility can be used to test the operator’s output and state update logic for a given set of input records. Cheers, Gordon On 22 June 2018 at 11:23:28 AM, Haddadi Manuel (manuel.hadd...@gfi.fr <mailto:manuel.hadd...@gfi.fr>) wrote: Hi Fabian, thanks for your response. Here is the Jira issue : https://issues.apache.org/jira/browse/FLINK-9644 Would you have any devlopper tips on how testing my intializeState methods to validate that my savepoints are corrects ? Thank you again. Regards, Manuel *De :* Fabian Hueske *Envoyé :* jeudi 21 juin 2018 10:33:10 *À :* Haddadi Manuel *Cc :* user@flink.apache.org *Objet :* Re: Debug job execution from savepoint Hi Manuel, I had a look and couldn't find a way to do it. However, this sounds like a very useful feature to me. Would you mind creating a Jira issue [1] for that? Thanks, Fabian [1] https://issues.apache.org/jira/projects/FLINK 2018-06-18 16:23 GMT+02:00 Haddadi Manuel <mailto:manuel.hadd...@gfi.fr>>: Hi all, I would like to test my checkpointing implementation doing a step-by-step debugging under an IDE. Is there a way to restore a job from a local savepoint in a local stream environnement, like a command "flink run -s :savepontpath" would do ? Thanks, Manuel
RE: Debug job execution from savepoint
Hi, The tests in Flink uses a `AbstractStreamOperatorTestHarness` that allows wrapping an operator, input elements into the operator, getting the outputs, and also snapshotting / restoring operator state. I’m not sure of your specific case, but in general that test harness utility can be used to test the operator’s output and state update logic for a given set of input records. Cheers, Gordon On 22 June 2018 at 11:23:28 AM, Haddadi Manuel (manuel.hadd...@gfi.fr) wrote: Hi Fabian, thanks for your response. Here is the Jira issue : https://issues.apache.org/jira/browse/FLINK-9644 Would you have any devlopper tips on how testing my intializeState methods to validate that my savepoints are corrects ? Thank you again. Regards, Manuel De : Fabian Hueske Envoyé : jeudi 21 juin 2018 10:33:10 À : Haddadi Manuel Cc : user@flink.apache.org Objet : Re: Debug job execution from savepoint Hi Manuel, I had a look and couldn't find a way to do it. However, this sounds like a very useful feature to me. Would you mind creating a Jira issue [1] for that? Thanks, Fabian [1] https://issues.apache.org/jira/projects/FLINK 2018-06-18 16:23 GMT+02:00 Haddadi Manuel : Hi all, I would like to test my checkpointing implementation doing a step-by-step debugging under an IDE. Is there a way to restore a job from a local savepoint in a local stream environnement, like a command "flink run -s :savepontpath" would do ? Thanks, Manuel
RE: Debug job execution from savepoint
Hi Fabian, thanks for your response. Here is the Jira issue : https://issues.apache.org/jira/browse/FLINK-9644 Would you have any devlopper tips on how testing my intializeState methods to validate that my savepoints are corrects ? Thank you again. Regards, Manuel De : Fabian Hueske Envoyé : jeudi 21 juin 2018 10:33:10 À : Haddadi Manuel Cc : user@flink.apache.org Objet : Re: Debug job execution from savepoint Hi Manuel, I had a look and couldn't find a way to do it. However, this sounds like a very useful feature to me. Would you mind creating a Jira issue [1] for that? Thanks, Fabian [1] https://issues.apache.org/jira/projects/FLINK 2018-06-18 16:23 GMT+02:00 Haddadi Manuel mailto:manuel.hadd...@gfi.fr>>: Hi all, I would like to test my checkpointing implementation doing a step-by-step debugging under an IDE. Is there a way to restore a job from a local savepoint in a local stream environnement, like a command "flink run -s :savepontpath" would do ? Thanks, Manuel
Re: Debug job execution from savepoint
Hi Manuel, I had a look and couldn't find a way to do it. However, this sounds like a very useful feature to me. Would you mind creating a Jira issue [1] for that? Thanks, Fabian [1] https://issues.apache.org/jira/projects/FLINK 2018-06-18 16:23 GMT+02:00 Haddadi Manuel : > Hi all, > > > I would like to test my checkpointing implementation doing a step-by-step > debugging under an IDE. > > > Is there a way to restore a job from a local savepoint in a local stream > environnement, like a command "flink run -s :savepontpath" would do ? > > > Thanks, > > > Manuel >
Debug job execution from savepoint
Hi all, I would like to test my checkpointing implementation doing a step-by-step debugging under an IDE. Is there a way to restore a job from a local savepoint in a local stream environnement, like a command "flink run -s :savepontpath" would do ? Thanks, Manuel
