[appengine-java] Using the Python bulk exporter tool with a Java application
Ikai has posted nice recipe on subject: http://appengine-cookbook.appspot.com/recipe/using-the-python-bulk-exporter-tool-with-a-java-application/ -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=.
[appengine-java] Using the Python bulk exporter tool with a Java application
i had created webappliccation project using java,I want to bulk load the data in google app engine datastore,But it shows the following error how to solve this appcfg.py create_bulkloader_config --filename=bulkloader.yaml -- url=http://logchecking1.appspot.com/remote_api appcfg.py:41: DeprecationWarning: the sha module is deprecated; use the hashlib module instead os.path.join(DIR_PATH, 'lib', 'antlr3'), /home/crimson/Desktop/google_appengine/google/appengine/tools/ dev_appserver_login.py:33: DeprecationWarning: the md5 module is deprecated; use hashlib instead import md5 Creating bulkloader configuration. [INFO] Logging to bulkloader-log-20100917.105157 [INFO] Throttling transfers: [INFO] Bandwidth: 25 bytes/second [INFO] HTTP connections: 8/second [INFO] Entities inserted/fetched/modified: 20/second [INFO] Batch Size: 10 [INFO] Opening database: bulkloader-progress-20100917.105157.sql3 [INFO] Opening database: bulkloader-results-20100917.105157.sql3 [INFO] Connecting to logchecking1.appspot.com/remote_api [ERROR ] Exception during authentication Traceback (most recent call last): File "/home/crimson/Desktop/google_appengine/google/appengine/tools/ bulkloader.py", line 3167, in Run self.request_manager.Authenticate() File "/home/crimson/Desktop/google_appengine/google/appengine/tools/ bulkloader.py", line 1176, in Authenticate remote_api_stub.MaybeInvokeAuthentication() File "/home/crimson/Desktop/google_appengine/google/appengine/ext/ remote_api/remote_api_stub.py", line 533, in MaybeInvokeAuthentication datastore_stub._server.Send(datastore_stub._path, payload=None) File "/home/crimson/Desktop/google_appengine/google/appengine/tools/ appengine_rpc.py", line 336, in Send f = self.opener.open(req) File "/usr/lib/python2.6/urllib2.py", line 389, in open response = self._open(req, data) File "/usr/lib/python2.6/urllib2.py", line 407, in _open '_open', req) File "/usr/lib/python2.6/urllib2.py", line 367, in _call_chain result = func(*args) File "/usr/lib/python2.6/urllib2.py", line 1146, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib/python2.6/urllib2.py", line 1121, in do_open raise URLError(err) URLError: [INFO] Authentication Failed How to solve this? -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Using the Python bulk exporter tool with a Java application
Looks like something went wrong with some escaping somewhere: Let's suppose your Java application's appengine-web.xml is defined as > follows: > > appid > java > > On Fri, Nov 20, 2009 at 5:26 PM, Peter Ondruska wrote: > Ikai has posted nice recipe on subject: > > http://appengine-cookbook.appspot.com/recipe/using-the-python-bulk-exporter-tool-with-a-java-application/ > > -- > > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=. > > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=.
Re: [appengine-java] Using the Python bulk exporter tool with a Java application
Bah! Well, we'll need to give the cookbook app some love. I'll post an addendum in the comments. Looks like all my <>s got stripped out instead of being escaped to < >. Hopefully that hasn't prevented anyone from understanding the article. Here's a pastie of what should have gone in there: http://pastie.org/708339 On Fri, Nov 20, 2009 at 2:43 PM, Toby Reyelts wrote: > Looks like something went wrong with some escaping somewhere: > > Let's suppose your Java application's appengine-web.xml is defined as >> follows: >> >> appid >> java >> >> > > On Fri, Nov 20, 2009 at 5:26 PM, Peter Ondruska > wrote: > >> Ikai has posted nice recipe on subject: >> >> http://appengine-cookbook.appspot.com/recipe/using-the-python-bulk-exporter-tool-with-a-java-application/ >> >> -- >> >> You received this message because you are subscribed to the Google Groups >> "Google App Engine for Java" group. >> To post to this group, send email to >> google-appengine-j...@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine-java+unsubscr...@googlegroups.com >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine-java?hl=. >> >> >> > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=. > -- Ikai Lan Developer Programs Engineer, Google App Engine -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=.
Re: [appengine-java] Using the Python bulk exporter tool with a Java application
If you feel more comfortable working with Java, I use this code to bulk load and backup data: http://code.google.com/p/remote-datastore/ It intercepts the raw bytes sent to the local datastore and sends them to another remote datastore. Because it operates at this very low level it is very efficient. On 17 Sep 2010, at 12:23, megala dhinakaran wrote: i had created webappliccation project using java,I want to bulk load the data in google app engine datastore,But it shows the following error how to solve this appcfg.py create_bulkloader_config --filename=bulkloader.yaml -- url=http://logchecking1.appspot.com/remote_api appcfg.py:41: DeprecationWarning: the sha module is deprecated; use the hashlib module instead os.path.join(DIR_PATH, 'lib', 'antlr3'), /home/crimson/Desktop/google_appengine/google/appengine/tools/ dev_appserver_login.py:33: DeprecationWarning: the md5 module is deprecated; use hashlib instead import md5 Creating bulkloader configuration. [INFO] Logging to bulkloader-log-20100917.105157 [INFO] Throttling transfers: [INFO] Bandwidth: 25 bytes/second [INFO] HTTP connections: 8/second [INFO] Entities inserted/fetched/modified: 20/second [INFO] Batch Size: 10 [INFO] Opening database: bulkloader-progress-20100917.105157.sql3 [INFO] Opening database: bulkloader-results-20100917.105157.sql3 [INFO] Connecting to logchecking1.appspot.com/remote_api [ERROR ] Exception during authentication Traceback (most recent call last): File "/home/crimson/Desktop/google_appengine/google/appengine/tools/ bulkloader.py", line 3167, in Run self.request_manager.Authenticate() File "/home/crimson/Desktop/google_appengine/google/appengine/tools/ bulkloader.py", line 1176, in Authenticate remote_api_stub.MaybeInvokeAuthentication() File "/home/crimson/Desktop/google_appengine/google/appengine/ext/ remote_api/remote_api_stub.py", line 533, in MaybeInvokeAuthentication datastore_stub._server.Send(datastore_stub._path, payload=None) File "/home/crimson/Desktop/google_appengine/google/appengine/tools/ appengine_rpc.py", line 336, in Send f = self.opener.open(req) File "/usr/lib/python2.6/urllib2.py", line 389, in open response = self._open(req, data) File "/usr/lib/python2.6/urllib2.py", line 407, in _open '_open', req) File "/usr/lib/python2.6/urllib2.py", line 367, in _call_chain result = func(*args) File "/usr/lib/python2.6/urllib2.py", line 1146, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib/python2.6/urllib2.py", line 1121, in do_open raise URLError(err) URLError: [INFO] Authentication Failed How to solve this? -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Using the Python bulk exporter tool with a Java application
Seems like the account credentials you are using to login may not have admin privileges (i.e. should be a valid user/developer account in your app.), or there is some other authentication mechanism (e.g. spring security) that is invalidating the login. Arun From: John Patterson To: google-appengine-java@googlegroups.com Sent: Mon, 20 September, 2010 6:50:43 AM Subject: Re: [appengine-java] Using the Python bulk exporter tool with a Java application If you feel more comfortable working with Java, I use this code to bulk load and backup data: http://code.google.com/p/remote-datastore/ It intercepts the raw bytes sent to the local datastore and sends them to another remote datastore. Because it operates at this very low level it is very efficient. On 17 Sep 2010, at 12:23, megala dhinakaran wrote: i had created webappliccation project using java,I want to bulk load >the data in google app engine datastore,But it shows the following >error how to solve this > >appcfg.py create_bulkloader_config --filename=bulkloader.yaml -- >url=http://logchecking1.appspot.com/remote_api >appcfg.py:41: DeprecationWarning: the sha module is deprecated; use >the hashlib module instead > os.path.join(DIR_PATH, 'lib', 'antlr3'), >/home/crimson/Desktop/google_appengine/google/appengine/tools/ >dev_appserver_login.py:33: DeprecationWarning: the md5 module is >deprecated; use hashlib instead > import md5 >Creating bulkloader configuration. >[INFO] Logging to bulkloader-log-20100917.105157 >[INFO] Throttling transfers: >[INFO] Bandwidth: 25 bytes/second >[INFO] HTTP connections: 8/second >[INFO] Entities inserted/fetched/modified: 20/second >[INFO] Batch Size: 10 >[INFO] Opening database: bulkloader-progress-20100917.105157.sql3 >[INFO] Opening database: bulkloader-results-20100917.105157.sql3 >[INFO] Connecting to logchecking1.appspot.com/remote_api >[ERROR ] Exception during authentication >Traceback (most recent call last): > File "/home/crimson/Desktop/google_appengine/google/appengine/tools/ >bulkloader.py", line 3167, in Run > self.request_manager.Authenticate() > File "/home/crimson/Desktop/google_appengine/google/appengine/tools/ >bulkloader.py", line 1176, in Authenticate > remote_api_stub.MaybeInvokeAuthentication() > File "/home/crimson/Desktop/google_appengine/google/appengine/ext/ >remote_api/remote_api_stub.py", line 533, in MaybeInvokeAuthentication > datastore_stub._server.Send(datastore_stub._path, payload=None) > File "/home/crimson/Desktop/google_appengine/google/appengine/tools/ >appengine_rpc.py", line 336, in Send > f = self.opener.open(req) > File "/usr/lib/python2.6/urllib2.py", line 389, in open > response = self._open(req, data) > File "/usr/lib/python2.6/urllib2.py", line 407, in _open > '_open', req) > File "/usr/lib/python2.6/urllib2.py", line 367, in _call_chain > result = func(*args) > File "/usr/lib/python2.6/urllib2.py", line 1146, in http_open > return self.do_open(httplib.HTTPConnection, req) > File "/usr/lib/python2.6/urllib2.py", line 1121, in do_open > raise URLError(err) >URLError: >[INFO] Authentication Failed > >How to solve this? > >-- >You received this message because you are subscribed to the Google Groups >"Google App Engine for Java" group. >To post to this group, send email to google-appengine-j...@googlegroups.com. >To unsubscribe from this group, send email to >google-appengine-java+unsubscr...@googlegroups.com. >For more options, visit this group at >http://groups.google.com/group/google-appengine-java?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
