Hi Ryan,
Thanks for the note. You can also do the same thing in your transform
with
transform.none_if_empty(transform.generate_foreign_key(...))
Perhaps this should be the default on a few more of the supplied
transforms.
--Matthew
On May 8, 3:42 pm, Ryan <ryanleeschnei...@gmail.com> wrote:
> Not sure where you want bug reports, but I fixed a little something:
>
> Basically, I couldn't have null/empty reference attributes, I would
> get this exception:
>
> Traceback (most recent call last):
> File "/Users/ryan/Downloads/google_appengine/google/appengine/tools/
> adaptive_thread_pool.py", line 150, in WorkOnItems
> status, instruction = item.PerformWork(self.__thread_pool)
> File
> "/Users/ryan/Downloads/google_appengine/google/appengine/tools/bulkloader.py",
> line 695, in PerformWork
> transfer_time = self._TransferItem(thread_pool)
> File
> "/Users/ryan/Downloads/google_appengine/google/appengine/tools/bulkloader.py",
> line 850, in _TransferItem
> self.content = self.request_manager.EncodeContent(self.rows)
> File
> "/Users/ryan/Downloads/google_appengine/google/appengine/tools/bulkloader.py",
> line 1271, in EncodeContent
> entity = loader.create_entity(values, key_name=key, parent=parent)
> File "/Users/ryan/Downloads/google_appengine/google/appengine/ext/
> bulkload/bulkloader_config.py", line 383, in create_entity
> return self.dict_to_entity(input_dict, self.bulkload_state)
> File "/Users/ryan/Downloads/google_appengine/google/appengine/ext/
> bulkload/bulkloader_config.py", line 129, in dict_to_entity
> self.__run_import_transforms(input_dict, instance, bulkload_state)
> File "/Users/ryan/Downloads/google_appengine/google/appengine/ext/
> bulkload/bulkloader_config.py", line 231, in __run_import_transforms
> value = self.__dict_to_prop(transform, input_dict, bulkload_state)
> File "/Users/ryan/Downloads/google_appengine/google/appengine/ext/
> bulkload/bulkloader_config.py", line 186, in __dict_to_prop
> value = transform.import_transform(value)
> File "/Users/ryan/Downloads/google_appengine/google/appengine/ext/
> bulkload/bulkloader_parser.py", line 90, in __call__
> return self.method(*args, **kwargs)
> File "/Users/ryan/Downloads/google_appengine/google/appengine/ext/
> bulkload/transform.py", line 141, in generate_foreign_key_lambda
> return datastore.Key.from_path(kind, value)
> File "/Users/ryan/Downloads/google_appengine/google/appengine/api/
> datastore_types.py", line 382, in from_path
> ValidateString(id_or_name, 'name')
> File "/Users/ryan/Downloads/google_appengine/google/appengine/api/
> datastore_types.py", line 107, in ValidateString
> raise exception('%s must not be empty.' % name)
> BadValueError: name must not be empty.
>
> The fix is pretty simple (in ext/bulkload/transform.py):
>
> def generate_foreign_key_lambda(value):
> if key_is_id:
> value = int(value)
> if not value: # ADDED
> return None # ADDED
> return datastore.Key.from_path(kind, value)
>
> return generate_foreign_key_lambda
>
> Other than that, working great for me.
>
> Thanks,
> Ryan
>
> On May 4, 3:42 pm, Craig Berry <cdbe...@gmail.com> wrote:
>
>
>
>
>
> > I did that check, and got prompted to log in. But the script dies
> > without ever having prompted me.
>
> > On Tue, May 4, 2010 at 14:34, Matthew Blain <matthew.bl...@google.com>
> > wrote:
> > > You should get prompted for authentication. Make sure remote_api is
> > > installed correctly on your server by visiting whatever you used for
> > > the --url argument in your browser. It should require you to log in
> > > then say "This request did not contain a necessary header"
>
> > > --Matthew
>
> > > On May 4, 10:51 am, Craig Berry <cdbe...@gmail.com> wrote:
> > >> Anybody there? I could really use a pointer on this. To clarify my
> > >> situation:
>
> > >> 1. I'm using the Eclipse java plugin to build and deploy my app.
> > >> 2. I'm trying to use the previewbulkloadertool to transform and
> > >> upload data from an existing db.
> > >> 3. I set up the app-side java remote_api servlet as described.
> > >> 4. I obtained the zip file, extracted it to a new dir on disk, and ran
> > >> the script as described in the doc.
> > >> 5. When I do that, I get an authentication failure.
>
> > >> This is perhaps not surprising, as I never provided my credentials
> > >> anywhere. How and where is that supposed to happen?
>
> > >> On May 3, 7:53 pm, Craig Berry <cdbe...@gmail.com> wrote:
>
> > >> > Perhaps this is documented somewhere, but I can't find it. I'm trying
> > >> > to upload to a Java app, configured as recommended, if that matters.
> > >> > How do I provide authentication information (username and password) to
> > >> >bulkloader.py so that it can connect to my remote_api?
>
> > >> > On Apr 9, 3:26 pm, Matthew Blain <matthew.bl...@google.com> wrote:
>
> > >> > > I’d like to announce a wider prerelease of the new App Engine
> > >> > >Bulkloaderconfiguration format. This new format extends the existing
> > >> > >bulkloaderwith a new format with several advantages:
>
> > >> > > * Semi-Automatic configuration generation: The SDK can generate a
> > >> > >bulkloaderconfiguration based on your existing data.
> > >> > > * Supports more data formats. CSV support has been extended to
> > >> > > files
> > >> > > with headers, basic XML support has been added, and it is simpler to
> > >> > > use alternate text encodings in the files. Additional and custom data
> > >> > > connectors are now easier to create.
> > >> > > * Easier to use -- the new syntax is more declarative and
> > >> > > descriptive than the old one. Developers who use languages other than
> > >> > > Python no longer need to write code in the Python language, although
> > >> > > the Python SDK is still required to run the tool.
>
> > >> > > To try out the newbulkloader, please
> > >> > > visithttp://bulkloadersample.appspot.com/
> > >> > > . Preliminary documentation is available on the README on that site
> > >> > > along with a version of the 1.3.2 Python SDK containing the new
> > >> > >bulkloader.
>
> > >> > > You can send feedback to the group or directly to me.
>
> > >> > > --Matthew Blain
> > >> > > Google App Engine Team
> > >> > > matthew.blain+bulkloa...@google.com
>
> > >> > --
> > >> > You received this message because you are subscribed to the Google
> > >> > Groups "Google App Engine" group.
> > >> > To post to this group, send email to google-appeng...@googlegroups.com.
> > >> > To unsubscribe from this group, send email to
> > >> > google-appengine+unsubscr...@googlegroups.com.
> > >> > For more options, visit this group
> > >> > athttp://groups.google.com/group/google-appengine?hl=en.
>
> > >> --
> > >> You received this message because you are subscribed to the Google
> > >> Groups "Google App Engine" group.
> > >> To post to this group, send email to google-appeng...@googlegroups.com.
> > >> To unsubscribe from this group, send email to
> > >> google-appengine+unsubscr...@googlegroups.com.
> > >> For more options, visit this group
> > >> athttp://groups.google.com/group/google-appengine?hl=en.
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "Google App Engine" group.
> > > To post to this group, send email to google-appeng...@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > google-appengine+unsubscr...@googlegroups.com.
> > > For more options, visit this group
> > > athttp://groups.google.com/group/google-appengine?hl=en.
>
> > --
> > Craig Berry -http://lapidum.org/home.html
> > "Magicians lie to the universe, and the
> > universe believes them." -- Lenore Berry
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com.
> > For more options, visit this group
> > athttp://groups.google.com/group/google-appengine?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group
> athttp://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
