[
https://issues.apache.org/jira/browse/YARN-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14234847#comment-14234847
]
Zhijie Shen commented on YARN-2837:
-----------------------------------
bq. It’ll throw inCompatible exception, even though it should not.
I guess it's way we want to do. When currently CURRENT_VERSION_INFO = 1.0, we
make it compatible with the existing store without version info. When
CURRENT_VERSION_INFO is upgraded to 2.0, the db schema is no longer compatible
to 1.0 and the exception is thrown. Going directly from no version to 2.0 may
not be a valid use case.
Here I simply reuse the same logic that other leveldb impl is using. If you
have some concern, we can figure out it separately.
bq. RMDelegationTokenIdentifierData
I noticed it before, but I don't think it is really necessary to create another
protobuf obj just because we have one additional long integer to ser/des.
bq. it’s always adding new token, the old token still remain, we should remove
the old token
{{db.put(k, v);}} will update the value if key already exists.
bq. the following looks ok , a simpler way might be to just concatenate two
strings.
That's different. See KeyBuilder and KeyParser for more detail. It's need to
be broken into two section for the convenience of parsing it.
bq. Or we could force user to configure the path properly and throw exception
otherwise.
I prefer to keep state store and data store under the same dir by default. For
advanced deployment, the user is free to config them separately.
bq. we need to keep track of the latest sequenceNumber so that it can be
recovered.
It is updated here.
{code}
public void recover(TimelineServiceState state) throws IOException {
LOG.info("Recovering " + getClass().getSimpleName());
for (DelegationKey key : state.getTokenMasterKeyState()) {
addKey(key);
}
for (Entry<TimelineDelegationTokenIdentifier, Long> entry :
state.getTokenState().entrySet()) {
addPersistedDelegationToken(entry.getKey(), entry.getValue());
}
}
{code}
Otherwise, I updated the patch accordingly. BTW, I've removed the cache config,
as it is not so important to the state store.
> Timeline server needs to recover the timeline DT when restarting
> ----------------------------------------------------------------
>
> Key: YARN-2837
> URL: https://issues.apache.org/jira/browse/YARN-2837
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: timelineserver
> Reporter: Zhijie Shen
> Assignee: Zhijie Shen
> Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: YARN-2837.1.patch, YARN-2837.2.patch, YARN-2837.3.patch
>
>
> Timeline server needs to recover the stateful information when restarting as
> RM/NM/JHS does now. So far the stateful information only includes the
> timeline DT. Without recovery, the timeline DT of the existing YARN apps is
> not long valid, and cannot be renewed any more after the timeline server is
> restarted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
