[google-appengine] Re: Bulk upload datastore and files

2016-02-02 Thread Emanuele Ziglioli
How many rows? within 10m in a task queue, you should be able to upload >100k rows. If you need more, there are many tools, but the simplest one (Java) is probably: https://github.com/icoloma/queue4gae On Tuesday, 2 February 2016 21:47:23 UTC+13, pinesh shah wrote: > > Hi, > > I have a scenario

[google-appengine] Re: Bulk Upload Images

2009-07-06 Thread Nick Johnson (Google)
You can also simply include the filename in the csv, and make the converter function "lambda x:open(x,'rb').read()". -Nick Johnson On Mon, Jul 6, 2009 at 12:57 PM, Tim Hoffman wrote: > > Hi > > You can put images in your csv programtaically. > > base64 encode them (you will need to increase the

[google-appengine] Re: Bulk Upload Images

2009-07-06 Thread Albert
Hi Tim! Wow. Thanks! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to g

[google-appengine] Re: Bulk Upload Images

2009-07-06 Thread John
That is great. Thanks for that idea. On Mon, Jul 6, 2009 at 7:57 PM, Tim Hoffman wrote: > > Hi > > You can put images in your csv programtaically. > > base64 encode them (you will need to increase the max field size in > the csv reader) > > then in your handler base64 decode them and the db.Blob

[google-appengine] Re: Bulk Upload Images

2009-07-06 Thread Tim Hoffman
Hi You can put images in your csv programtaically. base64 encode them (you will need to increase the max field size in the csv reader) then in your handler base64 decode them and the db.BlobProperty the string. is myobj.image = db.BlobProperty(b64decode(val)) I am round tripping polymodels v

[google-appengine] Re: Bulk Upload Images

2009-07-06 Thread John
You cannot do it with CSVs. I do not know of an easier way to do this but for my application, every image is found online so I could use the URLFetch service to fetch each image and then pass the response into db.Blob() and save it with .put() On Mon, Jul 6, 2009 at 7:23 PM, Albert wrote: > > Hi

[google-appengine] Re: Bulk Upload/Download for List and Unicode properties

2009-05-21 Thread Rodrigo Moraes
On Thu, May 21, 2009 at 9:29 AM, 风笑雪 wrote: > Change it to: >> >> for (name, converter), val in zip(self.__properties, values): >> >> if converter is bool and val.lower() in ('0', 'false', 'no'): >> >> val = False >> >> if isinstance(val,str) and not isinstance(val, uni

[google-appengine] Re: Bulk Upload/Download for List and Unicode properties

2009-05-21 Thread 风笑雪
I find a blog gives a solution: http://www.cnblogs.com/step1/archive/2009/03/12/1409899.html Search these code in \google_appengine\google\appengine\tools\bulkloader.py: for (name, converter), val in zip(self.__properties, values): if converter is bool and val.lower() in ('0', 'false',

[google-appengine] Re: Bulk Upload/Download for List and Unicode properties

2009-05-21 Thread Rodrigo Moraes
On Thu, May 21, 2009 at 9:11 AM, Alex Popescu wrote: > For lists, things sound even more complex. I must confess that I > haven't tried it yet, but (I don't know why exactly) I believe that > specifying 'list' as a conversion type will not be enough. > > Has anyone tried any of these scenarios and

[google-appengine] Re: Bulk upload specifying a key name

2009-05-18 Thread ecognium
Thanks, Nick. It works! On May 18, 5:00 am, "Nick Johnson (Google)" wrote: > Hi ecognium, > > You can do this by implementing the generate_key method in your > bulkloader.Loader class. See the code for > details:http://code.google.com/p/googleappengine/source/browse/trunk/python/g... > > -Nick

[google-appengine] Re: Bulk upload specifying a key name

2009-05-18 Thread Nick Johnson (Google)
Hi Dan, Unfortunately, no - as I mentioned in my other responses, currently the easiest way to do this appears to be overloading output_entities and handling the writing of entities to disk yourself. -Nick On Mon, May 18, 2009 at 4:43 PM, Devel63 wrote: > > Fantastic, Nick, thanks for the info

[google-appengine] Re: Bulk upload specifying a key name

2009-05-18 Thread Devel63
Fantastic, Nick, thanks for the info. Is there an equally simple way to output the key_name on bulk download? On May 18, 5:00 am, "Nick Johnson (Google)" wrote: > Hi ecognium, > > You can do this by implementing the generate_key method in your > bulkloader.Loader class. See the code for > deta

[google-appengine] Re: Bulk upload specifying a key name

2009-05-18 Thread Nick Johnson (Google)
Hi ecognium, You can do this by implementing the generate_key method in your bulkloader.Loader class. See the code for details: http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/tools/bulkloader.py#3215 -Nick Johnson On Mon, May 18, 2009 at 4:54 AM, ecognium

[google-appengine] Re: Bulk Upload Referential Integrity

2009-01-14 Thread djidjadji
The keys are probably not the same on dev and production, unlikely that the ids will be the same for the object or the parent, and maybe the dev server uses a different algorithm to construct keys. Make sure you upload first all the object that you want to reference too. They must have some sort

[google-appengine] Re: bulk upload

2008-11-20 Thread Marzia Niccolai
Hi, This first error looks to be caused by this issue: http://code.google.com/p/googleappengine/issues/detail?id=157 For the second, to upload text fields, you can specify the value type as db.Text. -Marzia On Wed, Nov 19, 2008 at 11:02 PM, Gampesh <[EMAIL PROTECTED]> wrote: > > Hi, > > my bul

[google-appengine] Re: bulk upload

2008-11-19 Thread Gampesh
yes thank Marzia in log file i got that error was in my data there are some ascii code which is not compatible with Gql Bulkupload parse. Can you please help me on this : if i want to store data in local server using same script just change is in "--url=http://localhost:8080/load"; then i got fo

[google-appengine] Re: bulk upload

2008-11-19 Thread Marzia Niccolai
Hi, What error is listed in your logs for this message? http://appengine.google.com/logs?&app_id=xx -Marzia On Wed, Nov 19, 2008 at 5:36 AM, Gampesh <[EMAIL PROTECTED]> wrote: > > my bulkupload was working properly but after two or three upload of > data i got following error > > can any b

[google-appengine] Re: bulk upload users

2008-10-24 Thread Luigi Semenzato
So is there any way of retrieving a user object from the user's current email address? If there isn't a way, is this for security reasons (rather, spam- prevention reasons) or some other reason? Thanks! Luigi On Oct 24, 10:48 am, "Marzia Niccolai" <[EMAIL PROTECTED]> wrote: > If you know the u

[google-appengine] Re: bulk upload users

2008-10-24 Thread Marzia Niccolai
If you know the user's email address for which you want to create profiles, you can create the user object fairly easily. new_user = users.User(email='[EMAIL PROTECTED]') Be aware (from: http://code.google.com/appengine/docs/users/userobjects.html ): "If the User constructor is called with an em

[google-appengine] Re: bulk upload "version 1" format?

2008-10-20 Thread Marzia Niccolai
Hi Angel, There is only one version of the bulkload tool, which is the one currently released. I think the comment was written to acknowledge that in the future we will likely continue to improve the upload/download data functionality of App Engine, so this tool is not likely to be the final word