I think I originally dismissed that idea because I was thinking in terms in prev_ds. If your do 2-6 then on 2, you’re prev_ds in the TI context is 6, but if you just use yesterday_ds that is just crazy enough to work. I can’t believe it took me this long to realize that.
James Coder > On May 10, 2019, at 4:05 AM, Bas Harenslak <basharens...@godatadriven.com> > wrote: > > Yea that’s why I also suggested 2-6. > > If that does not work for you, could you explain the use case? > > Bas > >> On 9 May 2019, at 22:00, James Coder <jcode...@gmail.com> wrote: >> >> Yes, but doing 1-5 would not run 5 until 1. >> >> James Coder >> >>> On May 9, 2019, at 2:40 PM, Bas Harenslak <basharens...@godatadriven.com> >>> wrote: >>> >>> Perhaps needless to say, but you can do this with a cron expression, e.g. >>> “0 0 * * 1-5” to schedule on weekdays only. Or “0 0 * * 2-6” to ensure you >>> start running from Tuesday 00:00 to Saturday 00:00 so you also process >>> Fridays data. >>> >>> Does that help? >>> >>> Bas >>> >>> On 9 May 2019, at 18:53, James Coder >>> <jcode...@gmail.com<mailto:jcode...@gmail.com>> wrote: >>> >>> Hi All, >>> I feel like this has probably been discussed more times than necessary, but >>> I wanted to get the community opinion on running dags for business days. In >>> my case I want to run M-F but I don't want to wait for Monday to run >>> Fridays data. As far as I can tell the only way to do this right now is to >>> schedule it to run everyday and short circuit on Saturday and Sunday. >>> I would like to explore options other than adding an additional operator to >>> all dags. >>> I have seen it suggested to add a "schedule type" argument to dag that >>> would allow for running at "the right side or left side" of the interval. >>> Other options I can think of to solve for this would be specify add an >>> additional dag argument that allows you to explicitly set the interval >>> length rather than relying on cronitor to provide the following schedule. >>> The last option I can think of would be to add support for passing a >>> function as schedule_interval (a function that returns a time delta). >>> >>> To summarize possible changes: >>> 1. add "schedule_type" kwarg (start or end) >>> 2. add "interval_length" kwarg (time delta that is used in conjunction with >>> cronitor) >>> 3. support functions as schedule_interval. >>> >>> Thoughts? >>> >>> Thanks, >>> James >>> >
