We could also just serialize the script to more than one value and paste it
together.
2012/6/11 Bill Graham
> That's expected. It's a cap on the size of how much of the script can be
> stored. I'm not sure what the exact size limit is though, but if it's
> causing issues I'm sure we could make i
That's expected. It's a cap on the size of how much of the script can be
stored. I'm not sure what the exact size limit is though, but if it's
causing issues I'm sure we could make it a configurable value.
On Mon, Jun 11, 2012 at 2:33 PM, Prashant Kommireddi wrote:
> Bill,
>
> Would you know if
Bill,
Would you know if that is expected or a bug?
On Wed, Jun 6, 2012 at 5:56 PM, Bill Graham wrote:
> One thing to be aware of when accessing the pig.script option is that AFAIK
> there's a limit to how large the script can be, after which the rest would
> be truncated.
>
>
> On Wed, Jun 6
One thing to be aware of when accessing the pig.script option is that AFAIK
there's a limit to how large the script can be, after which the rest would
be truncated.
On Wed, Jun 6, 2012 at 5:44 PM, Prashant Kommireddi wrote:
> I completely agree that's an option. But IMHO being able to do that up
I completely agree that's an option. But IMHO being able to do that upfront
would be a nice feature, adding cron is just an additional process we could
avoid if possible.
On Wed, Jun 6, 2012 at 5:39 PM, Dmitriy Ryaboy wrote:
> You can write a nightly cron that runs the JobHistoryLoader job and
>
You can write a nightly cron that runs the JobHistoryLoader job and
stores parsed scripts to hdfs...
D
On Wed, Jun 6, 2012 at 5:16 PM, Prashant Kommireddi wrote:
> I think that would be more of a post-process vs having Pig write the same
> to a HDFS location. That would avoid having to parse it
I think that would be more of a post-process vs having Pig write the same
to a HDFS location. That would avoid having to parse it from job.xml.
On Wed, Jun 6, 2012 at 4:19 PM, Daniel Dai wrote:
> One existing solution is "pig.script" entry inside job.xml, it is the
> serialized Pig script. JobHi
One existing solution is "pig.script" entry inside job.xml, it is the
serialized Pig script. JobHistoryLoader can load job.xml files and grab
those entries. Does that solve your problem?
Daniel
On Wed, Jun 6, 2012 at 3:52 PM, Prashant Kommireddi wrote:
> Hi All,
>
> What do you guys think about
