It is one way to go, to have an operator implement the scheduling. You can signal the operator using the change of a property. Look at the set-operator-property command in the apex cli, it can be used to change a property which will result in the corresponding setter method being called in the operator and you can use that as a signal to do further operations.
On Tue, Jun 13, 2017 at 3:30 PM, Guilherme Hott <[email protected]> wrote: > I was thinking about to use a cron sending a kafka message and in my DAG > I'll have a kafka input operator to consume this message and start the > process. I think this work but I would like to know if have something more > appropriate. > > On Tue, Jun 13, 2017 at 3:25 PM, Pramod Immaneni <[email protected]> > wrote: > >> There is no built scheduler to schedule the DAGs at a prescribed time, >> you would need to use some external mechanisms. Because it is a daily >> one-time activity, would something like cron work for you? >> >> On Tue, Jun 13, 2017 at 3:22 PM, Guilherme Hott <[email protected]> >> wrote: >> >>> Because I am syncing my data from a table in a database to HDFS and I >>> want to do this just once a day to save processing use. >>> >>> On Tue, Jun 13, 2017 at 2:45 PM, Ganelin, Ilya < >>> [email protected]> wrote: >>> >>>> Why don’t you want your dag to continue running? Are there resources >>>> you wish to release? >>>> >>>> >>>> >>>> - Ilya Ganelin >>>> >>>> [image: id:[email protected]] >>>> >>>> >>>> >>>> *From: *Guilherme Hott <[email protected]> >>>> *Reply-To: *"[email protected]" <[email protected]> >>>> *Date: *Tuesday, June 13, 2017 at 2:29 PM >>>> *To: *"[email protected]" <[email protected]> >>>> *Subject: *Is there a way to schedule an operator? >>>> >>>> >>>> >>>> Hi guys, >>>> >>>> >>>> >>>> Is there a way to schedule an operator? I need an operator start the >>>> DAG once a day at 00am. >>>> >>>> >>>> >>>> Best >>>> >>>> >>>> >>>> -- >>>> >>>> *Guilherme Hott* >>>> >>>> *Software Engineer* >>>> >>>> Skype: guilhermehott >>>> >>>> @guilhermehott >>>> >>>> https://www.linkedin.com/in/guilhermehott >>>> >>>> >>>> >>>> ------------------------------ >>>> >>>> The information contained in this e-mail is confidential and/or >>>> proprietary to Capital One and/or its affiliates and may only be used >>>> solely in performance of work or services for Capital One. The information >>>> transmitted herewith is intended only for use by the individual or entity >>>> to which it is addressed. If the reader of this message is not the intended >>>> recipient, you are hereby notified that any review, retransmission, >>>> dissemination, distribution, copying or other use of, or taking of any >>>> action in reliance upon this information is strictly prohibited. If you >>>> have received this communication in error, please contact the sender and >>>> delete the material from your computer. >>>> >>> >>> >>> >>> -- >>> *Guilherme Hott* >>> *Software Engineer* >>> Skype: guilhermehott >>> @guilhermehott >>> https://www.linkedin.com/in/guilhermehott >>> >>> >> > > > -- > *Guilherme Hott* > *Software Engineer* > Skype: guilhermehott > @guilhermehott > https://www.linkedin.com/in/guilhermehott > >
