[
https://issues.apache.org/jira/browse/HDFS-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16881197#comment-16881197
]
TanYuxin commented on HDFS-14622:
---------------------------------
Thanks [~xkrogen] and [~hexiaoqiao] very much. I think the patch can not only
fix the issue caused by the presence of Centralized Cache Management
functionality, but also improves compatibility with any other Fsimage XML
Section generated by other feature. Thanks again.
> [Dynamometer] State transition err when CCM( HDFS Centralized Cache
> Management) feature is used
> -----------------------------------------------------------------------------------------------
>
> Key: HDFS-14622
> URL: https://issues.apache.org/jira/browse/HDFS-14622
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: tools
> Reporter: TanYuxin
> Assignee: Erik Krogen
> Priority: Major
> Attachments: HDFS-14622.000.patch
>
>
> When we used dynamometer to test HDFS performance, the test encountered a
> error when generate DataNode Block info, then the generation process failed.
> The error stack is
> {code:java}
> Error: java.io.IOException: State transition not allowed; from DEFAULT to
> FILE_WITH_REPLICATION at
> com.linkedin.dynamometer.blockgenerator.XMLParser.transitionTo(XMLParser.java:107)
> at
> com.linkedin.dynamometer.blockgenerator.XMLParser.parseLine(XMLParser.java:77)
> at
> com.linkedin.dynamometer.blockgenerator.XMLParserMapper.map(XMLParserMapper.java:53)
> at
> com.linkedin.dynamometer.blockgenerator.XMLParserMapper.map(XMLParserMapper.java:26)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:151) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:828) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1690)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> {code}
> After checking Fsimage xml and the source code, I find that *XMLParser* can
> not parse the lines correctly, these lines are like
>
> {code:java}
> <directive><id>8963</id><path>/user/somepath/path1</path><replication>3</replication><pool>cache_other_pool</pool><expiration><millis>1544454142310</millis><relatilve>false</relatilve></expiration>
>
> <directive><id>8964</id><path>/user/somepath/path2</path><replication>3</replication><pool>cache_hadoop-data_pool</pool><expiration><millis>1544497817686</millis><relatilve>false</relatilve></expiration>
>
> <directive><id>8965</id><path>/user/somepath/path3</path><replication>3</replication><pool>cache_hadoop-peisong_pool</pool><expiration><millis>1544451500312</millis><relatilve>false</relatilve></expiration>
>
> <directive><id>8967</id><path>/user/somepath/path4</path><replication>3</replication><pool>cache_other_pool</pool><expiration><millis>1544497602570</millis><relatilve>false</relatilve></expiration>
> {code}
>
> These fsimage xml lines are generated when [HDFS Centralized Cache Management
> (CCM)|https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html]
> feature is used.
> I have a discuss with [~xkrogen]
> [here|[https://github.com/linkedin/dynamometer/pull/77]], and some patches
> provided can fix the issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
