> On Feb. 4, 2013, 8:44 p.m., Mohammad Islam wrote: > > branches/hcat-intre/core/src/main/resources/oozie-default.xml, line 798 > > <https://reviews.apache.org/r/9277/diff/1/?file=254962#file254962line798> > > > > Wandering why there is a set of EL functions specifically for pig? Does > > it mean for MR job, we will provide another set of EL functions? > > > > Is there any difference between pig and MR filter? > > Mona Chitnis wrote: > Most probably yes. These EL functions are named so to be explicit about > the signature of the HCatStorer's and HCatLoader's arguments when used via > Pig scripts. The filter is different for Hive scripts. For MR filters, it > needs to be tried out. If it happens to comply with the same as Pig/Hive > filter, we can document it. > > Mohammad Islam wrote: > Last time I remember, it was same for Pig and MR. I don't know the usage > of Hive. Is there any use case for Hive? any link? > > The reason I brought this: if it is not absolutely required, we should > not explode the list of functions. > > > Mona Chitnis wrote: > Here's an example of 'Loading data via Hive' > https://cwiki.apache.org/Hive/tutorial.html#Tutorial-LoadingData > > The partitions need to be given with single '=' as opposed to double '==' > in Pig. > > Mohammad Islam wrote: > We could control this, by passing appropriate arguments to the function. > Basically giving the name of EL function with a specific product may not > be a good idea. As can be seen, we will need three methods already. > I would say, alternatively, put a generic one covering MR and pig which > are identical. We can provide an extended version of the function with > multiple parameters such as comparator and delimiter. > > > > Rohini Palaniswamy wrote: > It will only be pig and hive. There is no filter for MR. HCatInputFormat > and HCatOutputFormat cannot be used directly through MRAction. Need to use a > java action that launches a job with HCatInputFormat and HCatOutputFormat. > Users can parse the hcat uri's from dataIn and dataOut and use them as they > wish in case of the java action. > > Mohammad Islam wrote: > I think in MR also we need to parametrize this by passing those args. > For example, (from the presentation associated with Jira #OOZIE-561, > slide 9): > > InputJobInfo inputJobInfo = > InputJobInfo.create(dbName,tableName,filter,thriftUri,null); > > The first four parameters could be passed through job configurations. > This way we can easily hide the parsing of our format. The same logic is more > or less true for pig and hive. > > > > > >
Good point Mohammad. I think we should give the map-reduce example a whirl to verify these following API get the required arguments. API exposed by HCatInputFormat - setInput - setOutputSchema - getTableSchema API exposed by HCatOutputFormat: - setOutput - setSchema - getTableSchema Reference doc - http://docs.hortonworks.com/HCatalog/CURRENT/inputoutput.html - Mona ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9277/#review16060 ----------------------------------------------------------- On Feb. 3, 2013, 2:07 a.m., Mona Chitnis wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/9277/ > ----------------------------------------------------------- > > (Updated Feb. 3, 2013, 2:07 a.m.) > > > Review request for oozie. > > > Description > ------- > > https://issues.apache.org/jira/browse/OOZIE-1196 > > Patch also incorporates twiki review comments by Tucu > > > This addresses bug OOZIE-1196. > https://issues.apache.org/jira/browse/OOZIE-1196 > > > Diffs > ----- > > > branches/hcat-intre/core/src/main/java/org/apache/oozie/coord/HCatELFunctions.java > 1441275 > branches/hcat-intre/core/src/main/resources/oozie-default.xml 1441275 > > branches/hcat-intre/core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java > 1441275 > branches/hcat-intre/docs/src/site/twiki/CoordinatorFunctionalSpec.twiki > 1441275 > > Diff: https://reviews.apache.org/r/9277/diff/ > > > Testing > ------- > > unit tests modified > > > Thanks, > > Mona Chitnis > >
