[google-appengine] Re: How to download data from my datastore to the local machine

2017-03-06 Thread 'Alex (Cloud Platform Support)' via Google App Engine 


Glad to hear that you’ve made it work, as the Remote API may in fact 
require additional setup according to your environment. Nevertheless, I 
would recommend another alternative potentially easier to configure for 
accessing the Datastore in the future. The alternative consists in using 
the Python NDB Client Library 
 as part of 
an App Engine instance written in Python to access the Cloud Datastore. 
More information on how to customize queries to retrieve Datastore’s 
content can be retrieved from this NDB Query Class reference 
.

Also, here is the referenced Issue Tracker link 
 that was missing in my previous response. 
Feel free to open a Feature Request regarding the re-implementation of the 
"appcfg.py” [download_data] feature there.

Hope it helps!

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/db381823-20d4-4a9f-991c-db070d7a95f4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How to download data from my datastore to the local machine

2017-03-03 Thread Delyan Spasov 
Thank you Alex,

I have read the docs for the remote_api. For some reason it isn't working 
and the error isn't very useful to find our why 
- NotSupportedOnThisPlatform.
I have found bug reports about this, but nobody is giving a working 
solution. 

Fortunately I can connect the remote_api from the console and I wrote a 
simple script to download the data to a file (Good that my database was a 
small one).

Regards

On Friday, March 3, 2017 at 9:43:59 PM UTC+2, Alex (Cloud Platform Support) 
wrote:
>
> In fact, current Cloud Datastore’s NoSQL implementation of relationships 
> between data objects differs from traditional databases. And the observed 
> feature differences in this thread can be related to what is described on 
> the Google Cloud Datastore Overview documentation 
> .
>
> Nevertheless, in case you would want to migrate your data to another 
> Google Cloud application, you can use the Cloud Storage backing up 
> functionality to export a datastore backup into a Storage bucket (See this 
> Backing up data section 
> 
>  
> of the Managing Datastore guide). Note that using this method, also allows 
> to export your data to other Google Cloud Platform services, such as Google 
> BigQuery (See Backup and restore considerations 
> 
> ).
>
> Lastly, here is a guide 
>  
> about Accessing App Engine with Remote API, that may get you further 
> insights on how to write a client to download the data according to your 
> specifications. See step 6 of Using the Remote API in a local client 
> section 
> 
>  
> for an example on how to format and retrieve data from a Datastore instance.
>
> Hope you’ll find this information insightful and feel free to look at 
> different Cloud products as Cloud SQL  
> that may more accurately suit your needs. In case you would like to see the 
> previously available "appcfg.py” [download_data] available in this Cloud 
> Datastore implementation, it is possible to open a Features request on this 
> Issue Tracker and making sure to specify “Feature Requests” under Component.
>
> Regards,
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3bc69663-dca2-4c83-bf18-eb34299a5436%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How to download data from my datastore to the local machine

2017-03-03 Thread 'Alex (Cloud Platform Support)' via Google App Engine 


In fact, current Cloud Datastore’s NoSQL implementation of relationships 
between data objects differs from traditional databases. And the observed 
feature differences in this thread can be related to what is described on 
the Google Cloud Datastore Overview documentation 
.

Nevertheless, in case you would want to migrate your data to another Google 
Cloud application, you can use the Cloud Storage backing up functionality 
to export a datastore backup into a Storage bucket (See this Backing up 
data section 

 
of the Managing Datastore guide). Note that using this method, also allows 
to export your data to other Google Cloud Platform services, such as Google 
BigQuery (See Backup and restore considerations 

).

Lastly, here is a guide 
 
about Accessing App Engine with Remote API, that may get you further 
insights on how to write a client to download the data according to your 
specifications. See step 6 of Using the Remote API in a local client section 

 
for an example on how to format and retrieve data from a Datastore instance.

Hope you’ll find this information insightful and feel free to look at 
different Cloud products as Cloud SQL  
that may more accurately suit your needs. In case you would like to see the 
previously available "appcfg.py” [download_data] available in this Cloud 
Datastore implementation, it is possible to open a Features request on this 
Issue Tracker and making sure to specify “Feature Requests” under Component.

Regards,

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/69a02536-6441-4251-afd1-782eb1c74525%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How to download data from my datastore to the local machine

2017-03-03 Thread Delyan Spasov 
Thanks Nicola,

>From the interactive shell I can access the remote_api. But not from my 
application. It is raising NotSupportedOnThisPlatform(). Since this is in 
Python you probably don't know about this error. I found it as an issue but 
I didn't find a solution of it. 

I don't know how to write a client to download the data. I don't know how 
to format it in a way that the local server will accept it and I also don't 
know how to import it in the local server. There is not any documentation 
about it. 

But I think this should be included. It was but they have changed it 
because of the Oauth. OK, then include it again using Oauth. It isn't 
normal to not be able to export your db. This is very low quality service. 


On Tuesday, February 28, 2017 at 9:19:18 AM UTC+2, Nicola Spreafico wrote:
>
> The appcfg client cannot be used anymore because it use a 
> username/password authentication, which has been deprecated in favor of 
> oauth2 (and not supported by the client itself).
> About 1 year ago I had a very very long ticket with google support (issue 
> number #08922843) where we analyzed all this situation, here a couple of 
> comments:
>
> We ended up writing a very simple remote API handler (both server and 
> client) in order to reproduce the same need as you.
> You need to deploy online a version with remote_api enabled and the write 
> a simple client which will download the data (for all the kinds you need) 
> from online and load it inside the local dev-server
>
> In my case the client was written in Java, but because the remote_api 
> exists in python I'm very confident that you can build a client of your own.
> The main thing to take care is (this example is for java language, i think 
> a python counterpart exists as well)
> if ( ... remote api connected to development server ...) {
> options.useDevelopmentServerCredential();
> } else {
> options.useApplicationDefaultCredential();
> }
>
> When you need to connect with the online environment, the 
> application-default is used which need to be configured using GCloud SDK
>
>
>> https://cloud.google.com/sdk/gcloud/reference/beta/auth/application-default/login
>
>
>
>
> Il giorno lunedì 27 febbraio 2017 21:56:47 UTC+1, Delyan Spasov ha scritto:
>>
>> I've wasted lot of time trying to download the datatstore to my local 
>> machine. Nothing works!
>>
>> The appcfg download_data doesn't seem to work any more. There is nothing 
>> about it in the new docs. It's not even mentioned even as deprecation 
>> notice.
>>
>> It is still there in the appcfg. But it doesn't work. 
>> I've tried to upload some old data. It doesn't work.
>> I've tried the remote_api. Tones of errors.
>>
>> They all used to work. What happened?
>> How to work with my database on the local machine?  
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/af71a864-4b98-4687-a704-67de033d1b88%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How to download data from my datastore to the local machine

2017-03-03 Thread Delyan Spasov 
I read it but I don't understand how to download my database. 

The lack of any normal way of exporting my database is making me nervous 
about using GAE in the future. What if I decide to migrate because of some 
reason? 

I will find some workaround for the current projects (I will have to). But 
I don't see myself starting a new project on GAE. 

Thanks!

On Tuesday, February 28, 2017 at 10:15:39 PM UTC+2, Alex (Cloud Platform 
Support) wrote:
>
> To add up to Nicola’s reply, you may be able to download your data to your 
> local machine by opening the local Datastore viewer as proposed on this 
> Issue Tracker thread 
> . To do 
> so, you will have to access the viewer locally at “
> http://localhost:8000/_ah/admin/datastore” and get the whole data flushed 
> to you local disk.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/51f0d4fa-1e68-4897-b8a9-f482522f1cd5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How to download data from my datastore to the local machine

2017-02-28 Thread 'Alex (Cloud Platform Support)' via Google App Engine 
To add up to Nicola’s reply, you may be able to download your data to your 
local machine by opening the local Datastore viewer as proposed on this 
Issue Tracker thread 
. To do 
so, you will have to access the viewer locally at “
http://localhost:8000/_ah/admin/datastore” and get the whole data flushed 
to you local disk.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/bd4708ad-feca-47e8-93bc-2008fd5a8a16%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How to download data from my datastore to the local machine

2017-02-27 Thread 'Nicola Spreafico' via Google App Engine 
The appcfg client cannot be used anymore because it use a username/password 
authentication, which has been deprecated in favor of oauth2 (and not 
supported by the client itself).
About 1 year ago I had a very very long ticket with google support (issue 
number #08922843) where we analyzed all this situation, here a couple of 
comments:

We ended up writing a very simple remote API handler (both server and 
client) in order to reproduce the same need as you.
You need to deploy online a version with remote_api enabled and the write a 
simple client which will download the data (for all the kinds you need) 
from online and load it inside the local dev-server

In my case the client was written in Java, but because the remote_api 
exists in python I'm very confident that you can build a client of your own.
The main thing to take care is (this example is for java language, i think 
a python counterpart exists as well)
if ( ... remote api connected to development server ...) {
options.useDevelopmentServerCredential();
} else {
options.useApplicationDefaultCredential();
}

When you need to connect with the online environment, the 
application-default is used which need to be configured using GCloud SDK

https://cloud.google.com/sdk/gcloud/reference/beta/auth/application-default/login




Il giorno lunedì 27 febbraio 2017 21:56:47 UTC+1, Delyan Spasov ha scritto:
>
> I've wasted lot of time trying to download the datatstore to my local 
> machine. Nothing works!
>
> The appcfg download_data doesn't seem to work any more. There is nothing 
> about it in the new docs. It's not even mentioned even as deprecation 
> notice.
>
> It is still there in the appcfg. But it doesn't work. 
> I've tried to upload some old data. It doesn't work.
> I've tried the remote_api. Tones of errors.
>
> They all used to work. What happened?
> How to work with my database on the local machine?  
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/eb89bebe-305b-4f04-a600-69d59e234a7b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.