Re: [google-appengine] Re: How export BiqQuery data to google Storage bucket either using with API or Java SDK automatically ?
There are currently no predefined templates from BigQuery to Google Cloud Storage. However, Dataflow does allow you to create your own templates to fit your use case. You can find information pertaining to creating your own template at the following documentation[1]. Information regarding the required classes and functions to read from BigQuery and write to Google Cloud Storage for Java are available on the Apache Beam API reference page[2]. Additionally, as your use case is to automate the export, you can do so using App Engine Cron services[3]. [1] https://cloud.google.com/dataflow/docs/templates/creating-templates [2] https://beam.apache.org/documentation/sdks/javadoc/2.6.0/ [3] https://cloud.google.com/blog/products/gcp/scheduling-dataflow-pipelines-using-app-engine-cron-service-or-cloud-functions -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/2f115eb5-be50-476e-8212-3c5b7c341d83%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: How export BiqQuery data to google Storage bucket either using with API or Java SDK automatically ?
It’s possible to do so. This would be like any other application that does some manipulation on data from BigQuery . Your application can be done using the client libraries[1] or through the rest API using extract property[2]. Once the application setup, you can use Cron jobs[3] to schedule your app to run every “n” time interval. Moreover, as outlined on the official exporting data from BigQuery documentation[4], this can also be done through Dataflow jobs which can also be scheduled to run automatically at a certain time interval. [1] https://cloud.google.com/bigquery/docs/exporting-data#exporting_table_data [2] https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/insert [3] https://cloud.google.com/appengine/docs/standard/python/config/cron [4] https://cloud.google.com/bigquery/docs/exporting-data -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/fd888e79-8687-4a82-9757-604d535a6919%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
