Hi Ben,
Ok, Doh moment from me. Thanks for the script, but the hint was enough
to remind me that there's a similar version already in SVN trunk.
I dont think there's no need for a separate issue, as upgrading to work
on Hod4 is already an unresolved issue. Presumably an upgraded Pig
would no longer required the expect script (though notably, I dont think
hod 4 produces all the required output ;-)
Ta muchly
Craig
Benjamin Reed wrote:
Ah yes, sorry about that. We had a problem with HOD not working well with
piped inputs and outputs, so we actually use an expect script to interface to
hod. (We should open an issue on this.)
I'm attaching the script that we use.
ben
On Wednesday 28 November 2007 11:38:09 Craig Macdonald wrote:
Hi all,
I've been trying to setup Pig using Hadoop on Demand. Using some
hackery, my incantation now looks like
PATH=/users/tr.craigm/OF_tools/python/bin/:$PATH ROOT=$PWD
scripts/pig.pl -Dlog4j.level=debug -Dhod.server=local
-Dhod.expect.root=$PWD -Dhod.command=hod/bin/hod
-Dhod.expect.uselatest=hodrc/released -Dyinst.cluster=
-Dhadoop.root.logger=DEBUG,console --cluster hodrc
(the name of my hodrc file is hodrc).
However, the HOD connection code in PigContext mystifies me. Does it
correspond to any released version of HOD?
It seems to connect to HOD, and parse the response.
PIG-18 (https://issues.apache.org/jira/browse/PIG-18) states that Pig
needs to be fixed to work with hod 4.
So I presume that Pig does not worth with the HOD version
hod-open-4.tar.gz attached to
https://issues.apache.org/jira/browse/HADOOP-1301
However, it doesnt look like Pig works with the other version of Hod
attached to the same JIRA issue: hod.0.2.2.tar.gz
PigContent.java looks for output from HOD in the form of lines starting:
hdfsUI:
hdfs:
mapredUI:
mapred:
hadoopConf:
I cant find any source in either versions of HOD that resemble this.
Does anyone know if Pig will currently work with any currently openly
available version of HOD?
Thanks in advance
Craig
