Stan, you have to serialize the schema into the job conf using the
signature passed to you in setUDFContextSignature.
your code will look something like this:
@Override
public void checkSchema(ResourceSchema s) throws IOException {
this.schema = s;
getUDFProperties().setProperty(contex
Hi All,
I'd like to get the schema of a relation that is used in conjunction
with my custom StorageFunc. I found 'checkSchema' to be useful for
this case, however, it seems to work only in local mode. When run in
distributed mode, 'checkSchema' is not invoked in mappers.
Is there some other mean
Hi, Alejandro,
I understand your concern but creating multiple pig.jar is inevitable. See
my comments below.
Daniel
On Mon, Nov 7, 2011 at 11:40 AM, Alejandro Abdelnur wrote:
> Hi Olga,
>
> Regarding #1, does this means we'd have a build of Pig X for each
> version of Hadoop we support? It seems
Hi Olga,
Regarding #1, does this means we'd have a build of Pig X for each
version of Hadoop we support? It seems to me this would be a bit
complex to maintain.
Regarding #2, If Hadoop does a good job at maintaing public API
backwards compatibility and Pig uses only Hadoop public API we would
be
Hi,
In the past we have for the most part avoided supporting multiple versions of
Hadoop with the same version of Pig. This is about to change with release of
Hadoop 23. We need to come up with a strategy on how to support that. There are
a couple of issues to consider:
(1)Version numberi
