[google-appengine] upload data via appcfg
Hey, I'm trying to upload data to local Datastore using the file I've downloaded from out production using appcfg download_data. I get an error almost every time about ProtocolBufferDecodeError: corrupted. Sometimes (on the same file) it is just working. This is really disturbing and I'd like to know if there's another way to use my local datastore with my local file and having a solution for this problem please. Everytime I update an entity in production and want to update my local Datastore it takes me for 10-20 tries. Stack trace: File "/home/rotem/Desktop/google_appengine/google/appengine/tools/adaptive_thread_pool.py", line 172, in WorkOnItems status, instruction = item.PerformWork(self.__thread_pool) File "/home/rotem/Desktop/google_appengine/google/appengine/tools/bulkloader.py", line 750, in PerformWork transfer_time = self._TransferItem(thread_pool) File "/home/rotem/Desktop/google_appengine/google/appengine/tools/bulkloader.py", line 921, in _TransferItem self.request_manager.PostEntities(self.content) File "/home/rotem/Desktop/google_appengine/google/appengine/tools/bulkloader.py", line 1396, in PostEntities datastore.Put(entities) File "/home/rotem/Desktop/google_appengine/google/appengine/api/datastore.py", line 606, in Put return PutAsync(entities, **kwargs).get_result() File "/home/rotem/Desktop/google_appengine/google/appengine/datastore/datastore_rpc.py", line 923, in get_result results = self.__rpcs[0].get_result() File "/home/rotem/Desktop/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result return self.__get_result_hook(self) File "/home/rotem/Desktop/google_appengine/google/appengine/datastore/datastore_rpc.py", line 1881, in __put_hook self.check_rpc_success(rpc) File "/home/rotem/Desktop/google_appengine/google/appengine/datastore/datastore_rpc.py", line 1371, in check_rpc_success rpc.check_success() File "/home/rotem/Desktop/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 579, in check_success self.__rpc.CheckSuccess() File "/home/rotem/Desktop/google_appengine/google/appengine/api/apiproxy_rpc.py", line 157, in _WaitImpl self.request, self.response) File "/home/rotem/Desktop/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 286, in MakeSyncCall handler(request, response) File "/home/rotem/Desktop/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 460, in _Dynamic_Put 'datastore_v3', 'Put', put_request, put_response) File "/home/rotem/Desktop/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 201, in MakeSyncCall self._MakeRealSyncCall(service, call, request, response) File "/home/rotem/Desktop/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 228, in _MakeRealSyncCall response_pb.ParseFromString(encoded_response) File "/home/rotem/Desktop/google_appengine/google/net/proto/ProtocolBuffer.py", line 140, in ParseFromString self.MergeFromString(s) File "/home/rotem/Desktop/google_appengine/google/net/proto/ProtocolBuffer.py", line 152, in MergeFromString self.MergePartialFromString(s) File "/home/rotem/Desktop/google_appengine/google/net/proto/ProtocolBuffer.py", line 168, in MergePartialFromString self.TryMerge(d) File "/home/rotem/Desktop/google_appengine/google/appengine/ext/remote_api/remote_api_pb.py", line 770, in TryMerge d.skipData(tt) File "/home/rotem/Desktop/google_appengine/google/net/proto/ProtocolBuffer.py", line 673, in skipData raise ProtocolBufferDecodeError, "corrupted" ProtocolBufferDecodeError: corrupted Thanks! -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/43b0721d-0848-49fa-9db3-99010adf5c76%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Upload data from local SQL Server instance?
I have some local data that I would like to upload to the app engine. I´ve read about the bulk uploader, but since I don´t know python I´d like to know if there is any way of importing data using Java? Or do I have to learn python? :-) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Upload data to development server
Hi, I am new to the Google app engine. I have developed a simple app which has a datastore component. I want to upload some data to the development server. I am using the 'appcfg.py upload_data' but I am getting a login/password prompt and I am not sure what to put there. I tried using my gmail a/c password but that fails. Could you please let me know what I am missing here? Thanks rc Uploading data records. [INFO] Logging to bulkloader-log-20090921.085252 [INFO] Throttling transfers: [INFO] Bandwidth: 25 bytes/second [INFO] HTTP connections: 8/second [INFO] Entities inserted/fetched/modified: 20/second [INFO] Opening database: bulkloader-progress-20090921.085252.sql3 Please enter login credentials for localhost Email: r**...@gmail.com Password for r**...@gmail.com: [INFO] Connecting to localhost:8080/remote_api [ERROR ] Exception during authentication Traceback (most recent call last): File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/tools/bulkloader.py, line 2985, in Run request_manager.Authenticate() File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/tools/bulkloader.py, line 1152, in Authenticate remote_api_stub.MaybeInvokeAuthentication() File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/ext/remote_api/remote_api_stub.py, line 494, in MaybeInvokeAuthentication datastore_stub._server.Send(datastore_stub._path, payload=None) File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/tools/appengine_rpc.py, line 344, in Send f = self.opener.open(req) File /System/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/urllib2.py, line 374, in open response = self._open(req, data) File /System/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/urllib2.py, line 392, in _open '_open', req) File /System/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/urllib2.py, line 353, in _call_chain result = func(*args) File /System/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/urllib2.py, line 1100, in http_open return self.do_open(httplib.HTTPConnection, req) File /System/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/urllib2.py, line 1075, in do_open raise URLError(err) URLError: urlopen error (61, 'Connection refused') [INFO] Authentication Failed --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Upload data
I cant get 'Uploading Data' going. Got the impression the Google description http://code.google.com/appengine/docs/python/tools/uploadingdata.html contains only part of the necessary code. Is there a downloadable and working demo or any other further advice on uploading data with remote_api and bulkloader? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Upload data to datastore
Hi there.
I'm trying to upload some data to datastore, I can't use CSV bulk
uploader because my data is not just text. Part of the data is an
image. So I decide to make a python script to call a POST method on
the server and go saving each row of my data.
I test on local environment and works well, but after upload the
application, my script receive an error 405 Method POST not allowed.
I define my request class on server:
class BulkUpload(webapp.RequestHandler):
def post(self):
model.put()
On the client i'm using a script to call a url
content_type, body = encode_multipart_formdata(fields, files)
#here i put all my data
h = httplib.HTTP(host)
h.putrequest('POST', selector)
h.putheader('content-type', content_type)
h.putheader('content-length', str(len(body)))
h.endheaders()
h.send(body)
errcode, errmsg, headers = h.getreply()
Thanks for your help.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
