[
https://issues.apache.org/jira/browse/PHOENIX-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14352661#comment-14352661
]
maghamravikiran commented on PHOENIX-1609:
------------------------------------------
[~jamestaylor]
I noticed few tests in the phoenix-pig module are failing due the changes
done in ColumnInfo.java class where we are explicitly escaping each column.
Apparently, I have changed the toString() and fromString() methods of
ColumnInfo class as below to ensure I don't get impacted with the occurrence of
: in the column name . Can you please share few cases where things would
break when functional indexes come into picture despite the changes.
{code}
@Override
public String toString() {
return getPDataType().getSqlTypeName() + STR_SEPARATOR + columnName ;
}
/**
* Instantiate a {@code ColumnInfo} from a string representation created by
the {@link
* #toString()} method.
*
* @param stringRepresentation string representation of a ColumnInfo
* @return the corresponding ColumnInfo
* @throws java.lang.IllegalArgumentException if the given string
representation cannot be
* parsed
*/
public static ColumnInfo fromString(String stringRepresentation) {
List<String> components =
Lists.newArrayList(stringRepresentation.split(":",2));
if (components.size() != 2) {
throw new IllegalArgumentException("Unparseable string: " +
stringRepresentation);
}
return new ColumnInfo(
components.get(1),
PDataType.fromSqlTypeName(components.get(0)).getSqlType());
}
{code}
The exception trace from PhoenixHBaseLoaderIT.testGroupingOfDataForTable is
{code}
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
parsing. Cannot find field SAL in "ID":int,"NAME":chararray,"AGE":int,"SAL":int
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1689)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1623)
at org.apache.pig.PigServer.registerQuery(PigServer.java:575)
at org.apache.pig.PigServer.registerQuery(PigServer.java:588)
at
org.apache.phoenix.pig.PhoenixHBaseLoaderIT.testGroupingOfDataForTable(PhoenixHBaseLoaderIT.java:411)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: Failed to parse: Pig script failed to parse:
<line 1, column 119> pig script failed to validate:
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1128: Cannot find
field SAL in "ID":int,"NAME":chararray,"AGE":int,"SAL":int
at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:196)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1676)
... 31 more
Caused by:
<line 1, column 119> pig script failed to validate:
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1128: Cannot find
field SAL in "ID":int,"NAME":chararray,"AGE":int,"SAL":int
at
org.apache.pig.parser.LogicalPlanBuilder.expandAndResetVisitor(LogicalPlanBuilder.java:410)
at
org.apache.pig.parser.LogicalPlanBuilder.buildForeachOp(LogicalPlanBuilder.java:979)
at
org.apache.pig.parser.LogicalPlanGenerator.foreach_clause(LogicalPlanGenerator.java:15781)
at
org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1844)
at
org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1013)
at
org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:553)
at
org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
... 32 more
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1128:
Cannot find field SAL in "ID":int,"NAME":chararray,"AGE":int,"SAL":int
at
org.apache.pig.newplan.logical.expression.DereferenceExpression.translateAliasToPos(DereferenceExpression.java:215)
at
org.apache.pig.newplan.logical.expression.DereferenceExpression.getFieldSchema(DereferenceExpression.java:149)
at
org.apache.pig.newplan.logical.optimizer.FieldSchemaResetter.execute(SchemaResetter.java:264)
at
org.apache.pig.newplan.logical.expression.AllSameExpressionVisitor.visit(AllSameExpressionVisitor.java:148)
at
org.apache.pig.newplan.logical.expression.DereferenceExpression.accept(DereferenceExpression.java:84)
at
org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70)
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
at
org.apache.pig.newplan.logical.optimizer.SchemaResetter.visitAll(SchemaResetter.java:67)
at
org.apache.pig.newplan.logical.optimizer.SchemaResetter.visit(SchemaResetter.java:122)
at
org.apache.pig.newplan.logical.relational.LOGenerate.accept(LOGenerate.java:246)
at
org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
at
org.apache.pig.newplan.logical.optimizer.SchemaResetter.visit(SchemaResetter.java:114)
at
org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:76)
at
org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
at
org.apache.pig.parser.LogicalPlanBuilder.expandAndResetVisitor(LogicalPlanBuilder.java:408)
... 39 more
{code}
> MR job to populate index tables
> --------------------------------
>
> Key: PHOENIX-1609
> URL: https://issues.apache.org/jira/browse/PHOENIX-1609
> Project: Phoenix
> Issue Type: New Feature
> Reporter: maghamravikiran
> Assignee: maghamravikiran
> Attachments: 0001-PHOENIX-1609-4.0.patch,
> 0001-PHOENIX-1609-4.0.patch, 0001-PHOENIX-1609-wip.patch,
> 0001-PHOENIX_1609.patch
>
>
> Often, we need to create new indexes on master tables way after the data
> exists on the master tables. It would be good to have a simple MR job given
> by the phoenix code that users can call to have indexes in sync with the
> master table.
> Users can invoke the MR job using the following command
> hadoop jar org.apache.phoenix.mapreduce.Index -st MASTER_TABLE -tt
> INDEX_TABLE -columns a,b,c
> Is this ideal?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
