[
https://issues.apache.org/jira/browse/PIG-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352143#comment-15352143
]
Siddhi Mehta commented on PIG-4939:
-----------------------------------
[~prkommireddi]
Is already a notion of non-fs loaders.
If not for the approach I was thinking
1. A property that has the comma separated list of non-dfs paths. The property
could be by the loader in the relativeToAbsolutePath mapping phase.
2. An Interface that the load/store func implements to indicate
Thoughts?
> QueryParserUtils.setHdfsServers(QueryParserUtils.java:104) should not be
> called for non-dfs methods
> ---------------------------------------------------------------------------------------------------
>
> Key: PIG-4939
> URL: https://issues.apache.org/jira/browse/PIG-4939
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Reporter: Siddhi Mehta
> Priority: Minor
>
> QueryParserUtils.setHdfsServers(QueryParserUtils.java:104) should not be
> called for non-dfs methods
> {code}
> String buildLoadOp(SourceLocation loc, String alias, String filename,
> FuncSpec funcSpec, LogicalSchema schema)
> throws ParserValidationException {
> String absolutePath;
> LoadFunc loFunc;
> try {
> // Load LoadFunc class from default properties if funcSpec is
> null. Fallback on PigStorage if LoadFunc is not specified in properties.
> funcSpec = funcSpec == null ? new
> FuncSpec(pigContext.getProperties().getProperty(PigConfiguration.PIG_DEFAULT_LOAD_FUNC,
> PigStorage.class.getName())) : funcSpec;
> loFunc = (LoadFunc)PigContext.instantiateFuncFromSpec(funcSpec);
> ......
> .......
> if (absolutePath == null) {
> absolutePath = loFunc.relativeToAbsolutePath( filename,
> QueryParserUtils.getCurrentDir( pigContext ) );
> if (absolutePath!=null) {
> QueryParserUtils.setHdfsServers( absolutePath, pigContext
> );
> }
> .....
> }
> {code}
> Email Conversation
> On Mon, Jun 27, 2016 at 1:04 PM, Prashant Kommireddi <[email protected]>
> wrote:
> Agreed. This method call isn't needed for phoenix loader (or any such
> non-direct-fs loaders). You should allow a config to handle it.
> On Mon, Jun 27, 2016 at 12:14 PM, Siddhi Mehta <[email protected]> wrote:
> > Hello All,
> >
> > I am getting a URISyntaxException when I try to execute my pig script using
> > PHoenixHBaseLoader. Traced attached below.
> > Looking through the code Pig splits multiple paths provided to it based on
> > comma(',') and during the query parsing step
> > QueryParserUtils.setHdfsServers(absolutePath, pigContext) tried to split
> > paths based on comma(',') and create URI's/PATHS for the same.
> >
> > Certain loaders like 'PhoenixHBaseLoader' donot pass hdfs locations and
> > instead work with passing PhoenixQueryStatement in the location.
> > e.g.
> > *A = load 'hbase://query/SELECT ID,NAME,DATE FROM HIRES WHERE DATE >
> > TO_DATE('1990-12-21 05:55:00.000')*
> >
> > This locations needs not be parsed to get hdfsservers path from them.
> > Does it make sense to introduce a config/loader property to annotate if the
> > loader/store is dealing with hdfs locations and based on the property make
> > a function call to QueryParserUtils.setHdfsServers(absolutePath,
> > pigContext).
> >
> > *Thoughts?*
> >
> > ***** Stack trace *****
> >
> > Caused by: Failed to parse: Pig script failed to parse:
> > <line 1, column 23> pig script failed to validate:
> > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> > path in absolute URI: CREATED_DATE FROM HIRES WHERE
> > CREATED_DATE>=TO_DATE('1990-12-21
> > 05:55:00.000')%20AND%20CREATED_DATE%3CTO_DATE('2016-03-08%2008:00:00.000')
> > at
> > org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
> > at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1712)
> > ... 30 more
> > Caused by:
> > <line 1, column 23> pig script failed to validate:
> > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> > path in absolute URI: CREATED_DATE FROM HIRES WHERE
> > CREATED_DATE>=TO_DATE('1990-12-21
> > 05:55:00.000')%20AND%20CREATED_DATE%3CTO_DATE('2016-03-08%2008:00:00.000')
> > at
> >
> > org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:897)
> > at
> >
> > org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3568)
> > at
> >
> > org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
> > at
> >
> > org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
> > at
> >
> > org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
> > at
> >
> > org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
> > at
> > org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
> > ... 31 more
> > Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> > Relative path in absolute URI: CREATED_DATE FROM HIRES WHERE
> > CREATED_DATE>=TO_DATE('1990-12-21
> > 05:55:00.000')%20AND%20CREATED_DATE%3CTO_DATE('2016-03-08%2008:00:00.000')
> > at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> > at org.apache.hadoop.fs.Path.<init>(Path.java:172)
> > at
> >
> > org.apache.pig.parser.QueryParserUtils.getRemoteHosts(QueryParserUtils.java:138)
> > at
> >
> > org.apache.pig.parser.QueryParserUtils.setHdfsServers(QueryParserUtils.java:104)
> > at
> >
> > org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:892)
> > ... 37 more
> > Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> > CREATED_DATE FROM HIRES WHERE CREATED_DATE>=TO_DATE('1990-12-21
> > 05:55:00.000')%20AND%20CREATED_DATE%3CTO_DATE('2016-03-08%2008:00:00.000')
> > at java.net.URI.checkPath(URI.java:1823)
> > at java.net.URI.<init>(URI.java:745)
> > at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> > ... 41 more
> >
> > Thanks,
> > Siddhi
> >
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
