[
https://issues.apache.org/jira/browse/PIG-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353380#comment-15353380
]
Prashant Kommireddi commented on PIG-4939:
------------------------------------------
There are utils that check for the filesystem (s3 vs hdfs vs ... ) but I don't
think there is an interface for non-fs based loaders. I'd be in favor of option
2, a marker interface that defines the behavior for non-direct-fs loaders. That
could evolve with a base class and loaders being able to implement custom
behaviors, but a marker interface might suffice for now.
[~daijy] thoughts?
> QueryParserUtils.setHdfsServers(QueryParserUtils.java:104) should not be
> called for non-dfs methods
> ---------------------------------------------------------------------------------------------------
>
> Key: PIG-4939
> URL: https://issues.apache.org/jira/browse/PIG-4939
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Reporter: Siddhi Mehta
> Priority: Minor
>
> {code}
> A = load 'hbase://query/SELECT ID,NAME,DATE FROM HIRES WHERE DATE >
> TO_DATE('1990-12-21 05:55:00.000');
> STORE A into 'output';
> {code}
> The above script throws an exception because it treats the location as an fs
> path and tries to convert it to a URI after splitting it based on comma.
> The code that tries to the same is
> {code}
> String buildLoadOp(SourceLocation loc, String alias, String filename,
> FuncSpec funcSpec, LogicalSchema schema)
> throws ParserValidationException {
> String absolutePath;
> LoadFunc loFunc;
> try {
> // Load LoadFunc class from default properties if funcSpec is
> null. Fallback on PigStorage if LoadFunc is not specified in properties.
> funcSpec = funcSpec == null ? new
> FuncSpec(pigContext.getProperties().getProperty(PigConfiguration.PIG_DEFAULT_LOAD_FUNC,
> PigStorage.class.getName())) : funcSpec;
> loFunc = (LoadFunc)PigContext.instantiateFuncFromSpec(funcSpec);
> ......
> .......
> if (absolutePath == null) {
> absolutePath = loFunc.relativeToAbsolutePath( filename,
> QueryParserUtils.getCurrentDir( pigContext ) );
> if (absolutePath!=null) {
> QueryParserUtils.setHdfsServers( absolutePath, pigContext
> );
> }
> .....
> }
> {code}
> We should not be calling
> QueryParserUtils.setHdfsServers(QueryParserUtils.java:104) should not be
> called for non-dfs methods
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
