Thanks Tom.
The next part of my story is that we use the backup files with a BigQuery
ingestion job - that is, the BigQuery ingestion job uses the native output
from the Datastore Admin Scheduled Backups stored on Cloud Storage.
('sourceFormat': 'DATASTORE_BACKUP')
Presumably, I'd also have to replicate the format if I were to roll my own
GCS/MapReduce hybrid and continue to use the same BigQuery ingestion
approach.
Any suggestions on that front? Or maybe just an approximate ETA and save me
a bunch of work? ;)
j
On Monday, 8 July 2013 18:23:38 UTC-6, Tom Kaitchuck wrote:
>
> This is something we are working hard on. We're updating many code paths
> to fix a lot of issues and migrate over to the GCS client. Changes won't
> roll out in one big release, rather updates will be released as they are
> completed. If you don't want to wait, it is absolutely supported to use the
> GCS client to write out data from within your own MapReduce.
>
>
> On Thu, Jul 4, 2013 at 9:01 AM, Jason Collins
> <jason.a...@gmail.com<javascript:>
> > wrote:
>
>> Our backups are really flakey and take a long time, apparently because of
>> the sketchy Files API link.
>>
>> Will the GAE 1.8.2 release of Datastore Admin Scheduled Backup tool
>> support the new Cloud Storage Client Library?
>>
>> This is all wrapped up in MapReduce/Pipelines, so there are a lot of
>> moving parts. My goal is to simply use stock, Google-supplied tools and
>> have this stuff be resilient and predictable.
>>
>> Any info is appreciated. If something in this area is not imminent, we
>> will have to start looking for alternate ways to achieve reliable backups.
>>
>> Thanks,
>> j
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to google-appengi...@googlegroups.com <javascript:>.
>> To post to this group, send email to
>> google-a...@googlegroups.com<javascript:>
>> .
>> Visit this group at http://groups.google.com/group/google-appengine.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.
