Well, I think I have it solved.
I was using an older version of the mapreduce library, which uses
mapreduce.lib.files to interact with the blobstore.
The newer version of mapreduce uses google.appengine.api.files instead,
which doesn't cause this problem.
These two libraries seem pretty similar -- so I'm not sure what precisely
was causing the issue.
On Saturday, March 2, 2013 4:25:17 AM UTC, Jamie Niemasik wrote:
>
> I've been receiving intermittent errors from MapReduce jobs. I'm running
> Python 2.7.
>
> The specific error is "BadValueError: name must be under 500 bytes" which
> is raised when calling datastore.Key.from_path() within
> blobstore.get_blob_key(); the filename being provided is way too long to
> make a key from.
>
> This all occurs within the code in the mapreduce package… nothing in my
> code seems to affect it.
>
> Some of the filenames are 288 bytes long, while some are 992. The M/R spec
> name and id in each case is nearly the same and is very short; I don't see
> where this variance comes from.
>
> The sequence of events is this:
> mapreduce.output_writers.init_job() creates a reasonable, short filename
> and passes it to files.blobstore.create()
> create() calls files.file._create('blobstore', …, filename)
> _create() sets up an rpc with that filename and calls _make_call('Create',
> ...)
>
> And that call sometimes returns a filename that's 288 bytes, sometimes
> 992. I have no idea why or how to work around this — any help would be
> appreciated.
>
> Thanks,
> Jamie
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
