[
https://issues.apache.org/jira/browse/PIG-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15632679#comment-15632679
]
Adam Szita commented on PIG-4939:
---------------------------------
Hi [~daijy], although it would be quicker to hardcode "hbase" scheme to be
handled differently in getRemoteHosts, I think we should rather go with the
marker interface option. From an OO point of view it is a cleaner solution - to
distinguish LoadFunc implementations that aren't file system related. This is
also safer in my opinion, but of course in the case above PhoenixHBaseLoader
has to be modified too in order to make use of this.
I attached patch for the Pig change [^PIG-4939.patch] and also one for Phoenix
just for illustration [^PIG-4939-PHOENIX.patch].
> QueryParserUtils.setHdfsServers(QueryParserUtils.java:104) should not be
> called for non-dfs methods
> ---------------------------------------------------------------------------------------------------
>
> Key: PIG-4939
> URL: https://issues.apache.org/jira/browse/PIG-4939
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Reporter: Siddhi Mehta
> Assignee: Adam Szita
> Priority: Minor
> Attachments: PIG-4939-PHOENIX.patch, PIG-4939.patch
>
>
> {code}
> A = load 'hbase://query/SELECT ID,NAME,DATE FROM HIRES WHERE DATE >
> TO_DATE('1990-12-21 05:55:00.000');
> STORE A into 'output';
> {code}
> The above script throws an exception because it treats the location as an fs
> path and tries to convert it to a URI after splitting it based on comma.
> The code that tries to the same is
> {code}
> String buildLoadOp(SourceLocation loc, String alias, String filename,
> FuncSpec funcSpec, LogicalSchema schema)
> throws ParserValidationException {
> String absolutePath;
> LoadFunc loFunc;
> try {
> // Load LoadFunc class from default properties if funcSpec is
> null. Fallback on PigStorage if LoadFunc is not specified in properties.
> funcSpec = funcSpec == null ? new
> FuncSpec(pigContext.getProperties().getProperty(PigConfiguration.PIG_DEFAULT_LOAD_FUNC,
> PigStorage.class.getName())) : funcSpec;
> loFunc = (LoadFunc)PigContext.instantiateFuncFromSpec(funcSpec);
> ......
> .......
> if (absolutePath == null) {
> absolutePath = loFunc.relativeToAbsolutePath( filename,
> QueryParserUtils.getCurrentDir( pigContext ) );
> if (absolutePath!=null) {
> QueryParserUtils.setHdfsServers( absolutePath, pigContext
> );
> }
> .....
> }
> {code}
> We should not be calling
> QueryParserUtils.setHdfsServers(QueryParserUtils.java:104) should not be
> called for non-dfs methods
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
