Salutations Gubbi, It would the same idea. Create a handler to execute your pipeline <https://cloud.google.com/dataflow/pipelines/executing-your-pipeline> then have your cron hit it.
On Friday, July 3, 2015 at 9:37:34 AM UTC-4, Gubbi wrote: > > Hi Ryan, > > Thanks for the pointers. But I was asking about scheduling a Dataflow > pipeline not Mapreduce pipeline. The above FAQ section is from Dataflow > documentation. Can you please help? > > Thanks. > > > > On Wed, Jun 24, 2015 at 9:38 PM, Ryan (Cloud Platform Support) < > rbruy...@google.com> wrote: > >> You would create a cron job >> <https://cloud.google.com/appengine/docs/python/config/cron> to hit a >> URL in your app. That URL would then start the mapreduce >> <https://github.com/GoogleCloudPlatform/appengine-mapreduce/wiki/3.1-Creating-and-Running-a-Job> >> >> job programmably. >> >> >> On Wednesday, June 24, 2015 at 4:00:42 AM UTC-4, Gubbi wrote: >>> >>> Thanks Ryan. >>> >>> We were looking at the Mapreduce code that does the backup and also the >>> Cloud Dataflow code for DatastoreIO. We can do our own custom backups today >>> using a query via dataflow. Only restriction seems to be that the query >>> can't have inequality operators. >>> >>> However, we are not able to find a way to schedule Dataflow pipelines. >>> Questions to this effect on StackOverflow have gone unanswered. The >>> Dataflow FAQ however has this: >>> >>> Is there a built-in scheduling mechanism to execute pipelines at given >>> time or interval? >>> >>> You can automate pipeline execution through Google App Engine or custom >>> (CRON) job processes on GCE. Future releases of the SDK will support >>> command line options for finer grained control over job management. >>> >>> >>> But there is no documentation on how to automate execution using >>> Appengine or Cron. If you could please help me with this, I'll have a >>> solution. >>> >>> >>> Thanks, >>> Vinuth. >>> >>> >>> >>> On Thu, Jun 18, 2015 at 7:10 PM, Ryan (Cloud Platform Support) < >>> rbruy...@google.com> wrote: >>> >>>> I will process your PIT later today. >>>> >>>> Unfortunately there is no way to do incremental backups at this time. >>>> >>>> >>>> On Thursday, June 18, 2015 at 1:02:25 AM UTC-4, Gubbi wrote: >>>>> >>>>> I have created a new feature request here: >>>>> https://code.google.com/p/googleappengine/issues/detail?id=12071 >>>>> >>>>> starred the old feature request as well. >>>>> >>>>> Are there any strategies I can use now for incremental backups? >>>>> >>>>> >>>>> On Wed, Jun 17, 2015 at 12:56 AM, Jason Collins < >>>>> jason.a.coll...@gmail.com> wrote: >>>>> >>>>>> I have an old feature request here if you want to star it: >>>>>> >>>>>> https://code.google.com/p/googleappengine/issues/detail?id=7428 >>>>>> >>>>>> >>>>>> On Tuesday, 16 June 2015 08:37:46 UTC-7, Ryan (Cloud Platform >>>>>> Support) wrote: >>>>>>> >>>>>>> Salutations Gubbi, >>>>>>> >>>>>>> This Feature Request would best be posted in the Public Issue >>>>>>> Tracker >>>>>>> <https://code.google.com/p/googleappengine/issues/entry?template=Feature%20request>. >>>>>>> >>>>>>> When you fill out the report let me know on this thread and I will >>>>>>> process >>>>>>> it for you. >>>>>>> >>>>>>> On Tuesday, June 16, 2015 at 10:53:47 AM UTC-4, Gubbi wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> The official backup method only supports full backup as of now. I >>>>>>>> don't know when, if at all, it'll support incremental backup. It'll >>>>>>>> be >>>>>>>> wasteful to take full backups every time. >>>>>>>> >>>>>>>> What strategies are you using for taking incremental backups of >>>>>>>> your datastore? >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Vinuth. >>>>>>>> >>>>>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Google App Engine" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to google-appengine+unsubscr...@googlegroups.com. >>>> To post to this group, send email to google-appengine@googlegroups.com. >>>> Visit this group at http://groups.google.com/group/google-appengine. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/google-appengine/06f79b85-be73-4dc4-a774-07dfc6cf9cb0%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/google-appengine/06f79b85-be73-4dc4-a774-07dfc6cf9cb0%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to google-appengine+unsubscr...@googlegroups.com. >> To post to this group, send email to google-appengine@googlegroups.com. >> Visit this group at http://groups.google.com/group/google-appengine. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/google-appengine/143a8860-8148-48a0-941c-aeb27c66063b%40googlegroups.com >> >> <https://groups.google.com/d/msgid/google-appengine/143a8860-8148-48a0-941c-aeb27c66063b%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/90af1c98-5aff-4bf6-9192-462c6e4c1b1a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
