[
https://issues.apache.org/jira/browse/PHOENIX-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889848#comment-16889848
]
Kadir OZDEMIR commented on PHOENIX-5401:
----------------------------------------
The reported issue here was caused by setting BUILD_INDEX_AT_ATTRIB in
PhoenixServerBuildIndexInputFormat.getQueryPlan(), which was not necessary.
After fixing this issue, I have run a very long index build job and noticed
that it failed due to various timeouts (rpc, and mapper task timeout so on).
It turned out that the default timeouts are too small for building indexes.
MetaDataRegionObserver increases these timeout values to let the server side
partial index rebuild operations complete. I followed the same approach for now
although it is not bullet proof. I also needed to increase
mapreduce.task.timeout accordingly. In future, we need to breakup these
operations into smaller ones in order to eliminate the need for increasing
timeout values.
> Error in running
> apache.phoenix.mapreduce.index.automation.PhoenixMRJobSubmitter
> --------------------------------------------------------------------------------
>
> Key: PHOENIX-5401
> URL: https://issues.apache.org/jira/browse/PHOENIX-5401
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.14.1
> Reporter: Ashutosh Parekh
> Assignee: Kadir OZDEMIR
> Priority: Major
> Fix For: 4.15.0, 5.1.0, 4.14.3
>
> Attachments: PHOENIX-5401.master.001.patch, yarnIssueIT.java
>
>
> Initially, the following query is executed:
>
> {code:java}
> CREATE TABLE IF NOT EXISTS us_population (
> state CHAR(2) NOT NULL,
> city VARCHAR NOT NULL,
> population BIGINT,
> CONSTRAINT my_pk PRIMARY KEY (state, city)) COLUMN_ENCODED_BYTES=0;
> UPSERT INTO us_population VALUES('NY','New York',8143197);
> UPSERT INTO us_population VALUES('CA','Los Angeles',3844829);
> UPSERT INTO us_population VALUES('IL','Chicago',2842518);
> UPSERT INTO us_population VALUES('TX','Houston',2016582);
> UPSERT INTO us_population VALUES('PA','Philadelphia',1463281);
> UPSERT INTO us_population VALUES('AZ','Phoenix',1461575);
> UPSERT INTO us_population VALUES('TX','San Antonio',1256509);
> UPSERT INTO us_population VALUES('CA','San Diego',1255540);
> UPSERT INTO us_population VALUES('TX','Dallas',1213825);
> UPSERT INTO us_population VALUES('CA','San Jose',912332);
> CREATE VIEW us_population_global_view (name VARCHAR,
> age BIGINT) AS
> SELECT * FROM us_population
> WHERE state = 'CA';
> CREATE INDEX us_population_gv_gi_1 ON us_population_global_view(age) include
> (city) async;
> {code}
>
> Then,
> {code:java}
> org.apache.phoenix.mapreduce.index.automation.PhoenixMRJobSubmitter{code}
> is run.
> The YARN application logs then show the following error:
> {code:java}
> 2019-07-16 19:40:50,397 ERROR [main]
> org.apache.phoenix.mapreduce.PhoenixServerBuildIndexInputFormat: Failed to
> get the query plan with error [ERROR 534 (42911): If both specified, values
> of CURRENT_SCN and BUILD_INDEX_AT must be equal.]
> 2019-07-16 19:40:50,398 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: java.sql.SQLException:
> ERROR 534 (42911): If both specified, values of CURRENT_SCN and
> BUILD_INDEX_AT must be equal.
> at
> org.apache.phoenix.mapreduce.PhoenixServerBuildIndexInputFormat.getQueryPlan(PhoenixServerBuildIndexInputFormat.java:108)
> at
> org.apache.phoenix.mapreduce.PhoenixInputFormat.createRecordReader(PhoenixInputFormat.java:75)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:515)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:758)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1760)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
> Caused by: java.sql.SQLException: ERROR 534 (42911): If both specified,
> values of CURRENT_SCN and BUILD_INDEX_AT must be equal.
> at
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:494)
> at
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:150)
> at
> org.apache.phoenix.jdbc.PhoenixConnection.checkScnAndBuildIndexAtEquality(PhoenixConnection.java:415)
> at
> org.apache.phoenix.jdbc.PhoenixConnection.<init>(PhoenixConnection.java:285)
> at
> org.apache.phoenix.jdbc.PhoenixConnection.<init>(PhoenixConnection.java:229)
> at
> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2533)
> at
> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2499)
> at
> org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:76)
> at
> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2499)
> at
> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:255)
> at
> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:147)
> at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
> at java.sql.DriverManager.getConnection(DriverManager.java:664)
> at java.sql.DriverManager.getConnection(DriverManager.java:208)
> at
> org.apache.phoenix.mapreduce.util.ConnectionUtil.getConnection(ConnectionUtil.java:113)
> at
> org.apache.phoenix.mapreduce.util.ConnectionUtil.getInputConnection(ConnectionUtil.java:58)
> at
> org.apache.phoenix.mapreduce.PhoenixServerBuildIndexInputFormat.getQueryPlan(PhoenixServerBuildIndexInputFormat.java:82)
> ... 9 more{code}
>
> Will require an appropriate resolution for this.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
