Hi all, I've been struggling to get the Python bulk uploader working reliably with my Java GAE app.
My bulkuploader.yaml is correct (I think) yet sometimes when I'm partway through uploading a CSV file the uploader barfs with the error shown at the bottom of this post. The log viewer in GAE shows the warning "com.google.apphosting.api.DeadlineExceededException: This request (a5a74b909963ecc7) started at 2010/11/15 16:28:41.971 UTC and was still executing at 2010/11/15 16:29:11.422 UTC." - I'm not sure this is relevant though. My CSV file is 500,000 lines. I split it up into 15,000-line files but it's hit and miss as to whether those smaller files upload. I reduced the number of worker threads, bandwidth limit, batch size, etc, but that didn't seem to make the uploader more reliable. I guess I could change my app to use app-assigned IDs for keys so I don't get duplicate entities when I re-upload a file that's failed partway through, but it'd be great to know why the bulk uploader doesn't work out of the box. Is anyone else having this problem and is there a simple solution or workaround? Thanks in advance Andrew. [ERROR ] [WorkerThread-1] WorkerThread: Traceback (most recent call last): File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/tools/adaptive_thread_pool.py", line 150, in WorkOnItems status, instruction = item.PerformWork(self.__thread_pool) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/tools/bulkloader.py", line 691, in PerformWork transfer_time = self._TransferItem(thread_pool) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/tools/bulkloader.py", line 848, in _TransferItem self.request_manager.PostEntities(self.content) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/tools/bulkloader.py", line 1300, in PostEntities datastore.Put(entities) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/api/datastore.py", line 291, in Put 'datastore_v3', 'Put', req, datastore_pb.PutResponse(), rpc) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/api/datastore.py", line 195, in _MakeSyncCall rpc.check_success() File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/api/apiproxy_stub_map.py", line 499, in check_success self.__rpc.CheckSuccess() File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/api/apiproxy_rpc.py", line 149, in _WaitImpl self.request, self.response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/ext/remote_api/remote_api_stub.py", line 224, in MakeSyncCall handler(request, response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/ext/remote_api/remote_api_stub.py", line 350, in _Dynamic_Put 'datastore_v3', 'Put', put_request, put_response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/ext/remote_api/remote_api_stub.py", line 156, in MakeSyncCall self._MakeRealSyncCall(service, call, request, response) File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/ GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/ google/appengine/ext/remote_api/remote_api_stub.py", line 178, in _MakeRealSyncCall raise UnknownJavaServerError("An unknown error has occured in the " UnknownJavaServerError: An unknown error has occured in the Java remote_api handler for this call. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.