[appengine-java] Re: Automatically increment version number
Or you could simply use the -V flag in appcfg. (Recently added to Java.) On Oct 4, 10:31 am, Ronoaldo José de Lana Pereira rpere...@beneficiofacil.com.br wrote: Since you are using a shell script, you may want to try something smaller that don't require you to add Ant to your deploy proccess (i.e., if your app is in Python): sed -e s/@version/$(hg id -i)/g appengine-web.xml.in appengine-web.xml This is just a in-mind-one-liner script, not sure about this particular sed syntax ;) Best Regards, -Ronoaldo -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How to export logs details
You'll need to add the 'severity' flag, which will get all of the additional logs, but will suppress log lined without the additional application created logs. You may be interested in http://groups.google.com/group/google-appengine/browse_thread/thread/a8ec10b5f376920f which describes a (python implemented) parser for those logs. I imagine there are other tools which can handle them too. --Matthew On Nov 18, 3:00 am, Featheast Lee featheast@gmail.com wrote: Hi All, I would like to ask a question about how to export logs from App Engine server. Currently, inhttp://code.google.com/appengine/docs/java/tools/uploadinganapp.html there is a way to download the logs. However, after I followed the steps within, it seems the txt file will only record the request name but without more details. An example would be: 0.1.0.1 - - [18/Nov/2010:02:09:19 -0800] GET /cron/*** HTTP/1.1 200 0 - AppEngine-Google; (+http://code.google.com/appengine) 0.1.0.1 - - [18/Nov/2010:02:08:16 -0800] GET /cron/*** HTTP/1.1 200 0 - AppEngine-Google; (+http://code.google.com/appengine) In the server console logs, however, we are able to see all logs in more details including those added by developers. I wonder is there any way to also download those logs. Cheers! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Downloading application logs from app engine
You can download them using the --severity flag http://code.google.com/appengine/docs/java/tools/uploadinganapp.html#Command_Line_Arguments On Oct 2, 5:25 am, Vikas Hazrati vhazr...@gmail.com wrote: I guess copy paste is the only way out ;) On Sep 30, 5:45 pm, Vikas Hazrati vhazr...@gmail.com wrote: I am trying to download the application logs from the app engine but with the command that I am using appcfg.sh request_logs myapp/war 0051.txt I am able to get the app engine logs and not the ones that I get when i expand the logs. So in a nutshell instead of just this as a part of download 09-30 04:58AM 40.954 / 302 24240ms 22236cpu_ms 1080api_cpu_ms 0kb Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.63 Safari/534.3,gzip(gfe) I would like to get the following logs which i see when i expand the above log in the admin console 09-30 04:58AM 40.954 / 302 24240ms 22236cpu_ms 1080api_cpu_ms 0kb Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.63 Safari/534.3,gzip(gfe) [myappid/0-0-5-1.345167134118388057].stdout: 2010-09-30 11:58:45,996 INFO com.myapp.util.admin.LogLoadingRequest - GAE ALERT !! Loading request occuring. I 09-30 04:58AM 47.802 javax.servlet.ServletContext log: Initializing Spring root WebApplicationContext Regards | Vikas -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Python bulk loader tool with local dev server of GAE Java
While the bulkloader may ask for a username and password, any username and password should be accepted by the dev_appserver. You can use the --email and --passin options to script this from the command line. On Aug 1, 10:33 pm, Jan jan.ja...@gmail.com wrote: Hi John, your remote-datastore tool looks great! Thank you very much for pointing me to it! I will have a look. It would still be a great improvement if Google's App Engine team could improve on the bulk loader or maybe even provide a Java version. That would complete an already excellent app hosting / development platform. Thanks a lot, Jan. On Aug 1, 11:06 pm, John Patterson jdpatter...@gmail.com wrote: You need to check out the source code (or even just copy and past the classes) into your own project - no downloads or jars (as stated on the project homepage) http://code.google.com/p/remote-datastore/source/checkout To insert data locally just call: RemoteDatastore.install(myAppId, war/); from a static initializer (or some other place guaranteed to only run once) and then you can use the datastore APIs from any Java code e.g. from a main(String[]) method. There is example code on the homepage using the low-level datastore API to insert data. I have used Twig to insert data too and JDO should work. If you need to read data from a CSV file you will need to parse the file and create Entities from it - there is an Apache CSV file reader you can use for that. On 2 Aug 2010, at 03:33, emmanuel harel wrote: John, could you give more information about how to use your library please ? I am not sure it is obvious for anyone. What are the dependencies of your project : the jar used. Thanks a lot 2010/8/1 John Patterson jdpatter...@gmail.com Hi Jan, I have used the Java RemoteDatastore to upload and download data in bulk. It allows you to use the low-level API, JDO, Twig, Objectify or whatever to store your data remotely or locally. http://code.google.com/p/remote-datastore/ On 1 Aug 2010, at 19:49, Jan wrote: Hi, I still did not find any solution to use the bulk uploader (now version 1.3.5) locally with GAE/Java. Communication with the production server works fine. As I cannot find any information about this issue, I assume that nobody else faces this problem. Can someone please post the steps necessary to make the Python bulk loader tool work locally with a Java app? Are there any alternatives available for the Python bulk uploader? Thank you, Jan. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How to upload primary key as an id instead of name
Sorry, this won't be available until 1.3.6. You should be able to do
something like this:
- property: __key__
external_name: CityId
export_transform: datastore.Key.id
import_transform: lambda value: datastore.Key.from_path('City',
int(value))
--Matthew
On Jul 10, 5:53 pm, Pasha pavel.selivers...@gmail.com wrote:
Could you please post an example. Thank you in advance.
On Jun 30, 1:18 pm, Matthew Blain matthew.bl...@google.com wrote:
The 1.3.5 bulkloader client will allow you to specify a numeric key;
you must use the Key constructor explicitly to do this, integers will
still be converted into strings.
On Jun 30, 12:12 am, MANISH DHIMAN manisd...@gmail.com wrote:
Hi All
When I upload data using CSV file on G A E. Primary key is stored
there as a name instead of id.
Example.
Format of .yaml is Given below
transformers:
- kind: City
connector: csv
connector_options:
encoding: utf-8
columns: from_header
property_map:
- property: __key__
external_name: CityId
export_transform: datastore.Key.id
- property: Name
external_name: Name
Primary key stored there is:
ID/Name Name
name=1 Delhi
name=2 London
Due to uploaded data with name=keyvalue, when I am try to get Key
from any Fetched Entities, then Key contains only name value but id
value is 0 and also name contains long value as a String instance.
Is it possible to store primary key(Using CSV while uploading data) in
a format given below :
ID/Name Name
id=1 Delhi
id=2 London
When data is stored with id=keyvalue, then Key contains id value is
keyvalue as long instance.
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Any ETA for a backup/restore facility?
The bulk loader does indeed provide backup and restore functionality today. While the implementation is in Python it can talk to your Java server and the entities are shared between Java and Python. On Jun 28, 4:46 pm, mscwd01 mscw...@gmail.com wrote: Thanks for your replying guys, it would be great if we could get an official Google response to update us on this as well though! I'll definitely have a look at the Bulk Loader in the meantime... On Jun 28, 1:52 pm, Mike Dillon mikedillo...@gmail.com wrote: Mscwd01, I'm not a google employee, but I have had success backing up our datastore and then restoring our datastore with the bulkloader utility. Currently the bulkloader is in the python sdk, so all you need to do is download that and then youll be able to do backups and restore from those backups. Check out the literature on the app engine, and try a couple searches to get your bearings. Nick Johnson has commented or written every search result that comes back when I was researching this a couple of weeks back, and there is also an I/O video on the bulkloader. Make sure you read some of the fine print, i.e. one caveat is that you must have had your datastore populated for at least 12 hours which is the amount of time it takes for the statistics info to be built. That what the bulkloader works off of to do its datastore dump. - Mike On Jun 28, 7:16 am, mscwd01 mscw...@gmail.com wrote: Hey Google Employees, There has been a backup/restore facility on the roadmap for a while now. Can you let us know if and when we may get an in-built tool that will allow us to easily save the contents of the datastore to our local machines and restore it if needs be? In the meantime, what is the preferred solution to backup/restore our Java based apps? Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: bulk download error 403
The Google Apps control panel does not affect who is a admin on a particular app engine application. That's controlled by the App Engine admin console, e.g. http://appengine.google.com/permissions?app_id=myprogram . For remote_api to work you also need the admin to match the authentication option of the application, e.g. whatever it says on http://appengine.google.com/domain?app_id=myprogram for Authentication Options. If it says anyone with a valid Google Account then you need to use a non-Google Apps account for the admin with remote API. This is often simplest to do with a Gmail account. If it says anyone with a valid *yourdomain* Google Apps domain, then only admins on the Permissions page with Google Apps account in that domain will work. Hopefully this clarifies things--it's confusing because there are so many different places to check. --Matthew On Dec 24 2009, 1:48 am, lembas keremo...@gmail.com wrote: First of all, sorry for writing remote instead of bulkload in my last 2 posts. There is no remote version. The only version is bulkload. I created it with Python like I said in my first post in this thread. I logged out my gmail.com account and mydomain.com account in my browser. Then I visithttp://bulkload.latest.myprogram.appspot.com/remote_api. There comes a login page of mydomain.com. When I login with my admin account on mydomain.com, it says Error: Forbidden Your client does not have permission to get URL /remote_api from this server. I do not use GAE UserService anywhere in my code. I do not have any security-constraint in my web.xml. I looked at the logs as you said. I have the following in my request- only-filtered log: 12-24 01:33AM 02.761 /remote_api 403 36ms 0cpu_ms 0kb Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.6 (KHTML, like Gecko) Chrome/4.0.266.0 Safari/532.6,gzip(gfe) See details 193.xxx.yyy.zzz - myid [24/Dec/2009:01:33:02 -0800] GET /remote_api HTTP/1.1 403 0 https://www.google.com/a/mydomain.com/ServiceLogin? service=ahpassive=truecontinue=http:// bulkload.latest.myprogram.appspot.com/_ah/login%3Fcontinue%3Dhttp:// bulkload.latest.myprogram.appspot.com/ remote_apiltmpl=gaahname=dummysig=a0d1d0bdb0ea9947083be67280bb1e39 Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.6 (KHTML, like Gecko) Chrome/4.0.266.0 Safari/532.6,gzip(gfe) How can I check if my admin account on mydomain.com is really an admin forhttp://bulkload.latest.myprogram.appspot.com/remote_api? I checked it by logging inhttps://www.google.com/a/cpanel/kavancha.com/Users and see that admin is an Administrator as stated in Status column next to Username column. Am I right? Thanks for your help. Now what should I do next? On Dec 24, 4:54 am, Matthew Blain matthew.bl...@google.com wrote: (Note that the log in page is only needed to test interactive login.) On Dec 23, 6:53 pm, Matthew Blain matthew.bl...@google.com wrote: You need to log in to the application, not just the Google control panel--do you have a login page you can visit onhttp://remote.latest.myprogram.appspot.com/remote_api? What do the logs look like in the admin console for your app? On Dec 19, 2:36 am, lembas keremo...@gmail.com wrote: First I visithttps://www.google.com/a/cpanel/mydomain.com then login as ad...@mydomain.com and then I visithttp://remote.latest.myprogram.appspot.com/remote_api, which has remoteapi definition in web.xml as stated inhttp://groups.google.com/group/google-appengine-java/browse_thread/th..., it says Error: Forbidden Your client does not have permission to get URL /remote_api from this server. but if i visithttp://myprogram.appspot.com/remote_api, which is the current version without any remoteapi definition in web.xml, it says Error: Not Found The requested URL /remote_api was not found on this server. what should I do now? On Dec 19, 2:24 am, Matthew Blain matthew.bl...@google.com wrote: A quick test is if you can visit yourapp/remote_api in the web browser--it should work if you log in as an admin, well it should say This request did not contain a necessary header but not any other errors. On Dec 16, 2:52 pm, lembas keremo...@gmail.com wrote: I get exactly the same error when I try to bulkupload with this command: appcfg.py upload_data --auth_domain=mydomain.com --db_filename=skip -- url=http://remote.latest.myprogram.appspot.com/remote_api--no_cookies --has_header --config_file=loader.py --filename=kind.csv --kind=Kind myprogram On Dec 17, 12:37 am, lembas keremo...@gmail.com wrote: Hi, I have a java app on myprogram.appspot.com. I use google eclipse plugin to deploy it.(Eclipse 3.5, App Engine SDK 1.3.0, GWT SDK 2.0.0) I set the version to bulkload and deployed it. Now I have bulkload.latest.myprogram.appspot.com as I can see
[appengine-java] Re: Bulk Upload with Authentication
I've responded in the other thread. On Feb 14, 1:01 pm, lembas keremo...@gmail.com wrote: I have a Java-app-with-authentication. I created it this way. I wish I had not. I need to upload bulk data. I am waiting for months but Java still cannot do it. I tried to do it with Python. I cannot do with it either because I have created my app with authentication and I cannot change my app back to a non-authentication-app. I can create another Java-app-without-authentication. But this time I need to bulk download my data but I cannot because my app is a Java- app-with-authentication. Any help? When will it be possible to bulk-upload to a Java app with authentication via Python or Java or any other way? Any workaround? Neither works for me as stated inhttp://groups.google.com/group/google-appengine-java/browse_thread/th... -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: bulk download error 403
A quick test is if you can visit yourapp/remote_api in the web browser--it should work if you log in as an admin, well it should say This request did not contain a necessary header but not any other errors. On Dec 16, 2:52 pm, lembas keremo...@gmail.com wrote: I get exactly the same error when I try to bulkupload with this command: appcfg.py upload_data --auth_domain=mydomain.com --db_filename=skip -- url=http://remote.latest.myprogram.appspot.com/remote_api--no_cookies --has_header --config_file=loader.py --filename=kind.csv --kind=Kind myprogram On Dec 17, 12:37 am, lembas keremo...@gmail.com wrote: Hi, I have a java app on myprogram.appspot.com. I use google eclipse plugin to deploy it.(Eclipse 3.5, App Engine SDK 1.3.0, GWT SDK 2.0.0) I set the version to bulkload and deployed it. Now I have bulkload.latest.myprogram.appspot.com as I can see onhttps://appengine.google.com/deployment?app_id=myprogram I try to bulkdownload data to my app but cannot succeed. I use: bulkloader.py --dump --batch_size=5 --kind=Kind -- auth_domain=mydomain.com --url=http:// bulkload.latest.myprogram.appspot.com/remote_api --filename=dump.sql3 myprogram when I execute this command it asks my an email and a password. I enter ad...@mydomain.com and its password but the result is: [ERROR ] Exception during authentication Traceback (most recent call last): File C:\Program Files\Google\google_appengine\google\appengine\tools \bulkload er.py, line 2985, in Run request_manager.Authenticate() File C:\Program Files\Google\google_appengine\google\appengine\tools \bulkload er.py, line 1152, in Authenticate remote_api_stub.MaybeInvokeAuthentication() File C:\Program Files\Google\google_appengine\google\appengine\ext \remote_api \remote_api_stub.py, line 494, in MaybeInvokeAuthentication datastore_stub._server.Send(datastore_stub._path, payload=None) File C:\Program Files\Google\google_appengine\google\appengine\tools \appengin e_rpc.py, line 344, in Send f = self.opener.open(req) File C:\Python26\lib\urllib2.py, line 389, in open response = meth(req, response) File C:\Python26\lib\urllib2.py, line 502, in http_response 'http', request, response, code, msg, hdrs) File C:\Python26\lib\urllib2.py, line 427, in error return self._call_chain(*args) File C:\Python26\lib\urllib2.py, line 361, in _call_chain result = func(*args) File C:\Python26\lib\urllib2.py, line 510, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 403: Forbidden [INFO ] Authentication Failed I got exactly the same error above when I tried downloading with this command: appcfg.py download_data --auth_domain=mydomain.com --url=http:// bulkload.latest.myprogram.appspot.com/remote_api -- config_file=exporter.py --filename=kind_data_archive.csv --kind=Kind myprogram If I enter my gmail.com account and password, which I use to deploy the app in google eclipse plugin, the error is: Invalid username or password. Please enter login credentials for bukload.latest.myprogram.appspot.com My app.yaml in myprogram folder is: application: myprogram version: bulkload runtime: python api_version: 1 handlers: - url: /remote_api script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py login: admin Any help? Or should I ask it on google-appengine-python group or google-appengine group? Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Looking for Bulk Loader beta testers
Hi everyone, Thanks for the response so far--we'll be getting back to you over the next few days. I'd like to point out an existing feature which will be helpful to many of you: --dump and --restore. This will download or upload all of the entities for a particular Kind and save them in a local sqlite database. It's useful for general backup (e.g. a weekly backup), and also useful for moving data across applications, such as between your app running on the dev_appserver and on App Engine, or to load a staging instance with a known set of test data. You can find more information here: http://code.google.com/appengine/docs/python/tools/uploadingdata.html#Downloading_and_Uploading_All_Data Also, for Java developers, there is a remote API handler available; adding following to your web xml file should work (disclaimer: I have not yet personally tested this.) servlet servlet-nameremoteapi/servlet-name servlet-classcom.google.apphosting.utils.remoteapi.RemoteApiServlet/servlet-class /servlet servlet-mapping servlet-nameremoteapi/servlet-name url-pattern/remote_api/url-pattern /servlet-mapping security-constraint web-resource-collection web-resource-nameremoteapi/web-resource-name url-pattern/remote_api/url-pattern /web-resource-collection auth-constraint role-nameadmin/role-name /auth-constraint /security-constraint --Matthew On Tue, Nov 17, 2009 at 12:53 PM, Matthew Blain matthew.bl...@google.comwrote: Hi App Engine developers, We're working on some improvements and additions to the bulk loader to make it easier to move data between the App Engine Datastore and other data files you may have. If you're doing this right now, we're looking for some beta testers. Two specific issues we're addressing at this time are: * More language-neutral: Java developers, this will help you! * More format-neutral: If you use some format other than CSV, this will help you! If you're interested, fill out the form at https://spreadsheets.google.com/viewform?formkey=dC15V2hwczhpZ1VVWFhPZGhXR1dydUE6MQ to get started. --Matthew -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=.
[appengine-java] Looking for Bulk Loader beta testers
Hi App Engine developers, We're working on some improvements and additions to the bulk loader to make it easier to move data between the App Engine Datastore and other data files you may have. If you're doing this right now, we're looking for some beta testers. Two specific issues we're addressing at this time are: * More language-neutral: Java developers, this will help you! * More format-neutral: If you use some format other than CSV, this will help you! If you're interested, fill out the form at https://spreadsheets.google.com/viewform?formkey=dC15V2hwczhpZ1VVWFhPZGhXR1dydUE6MQ to get started. --Matthew -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=.
[appengine-java] Re: The API call datastore_v3.Put() is temporarily unavailable.
We are currently in planned maintenance. See http://groups.google.com/group/google-appengine-downtime-notify/browse_thread/thread/af970296d9a0b9c8 http://groups.google.com/group/google-appengine-downtime-notify/browse_thread/thread/78f2590cd347fdd0 and http://googleappengine.blogspot.com/2009/09/migration-to-better-datastore.html Apologies for the inconvenience, On Sep 22, 5:58 pm, Hani Naguib haninag...@gmail.com wrote: I am getting this aswell. Basically my apps are down because of this! On Sep 22, 5:45 pm, David davesmith@gmail.com wrote: Any idea what this means? com.google.apphosting.api.ApiProxy$CapabilityDisabledException: The API call datastore_v3.Put() is temporarily unavailable. Shows up in my production logs... Dave --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
