[
https://issues.apache.org/jira/browse/FLINK-4154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15368130#comment-15368130
]
ASF GitHub Bot commented on FLINK-4154:
---------------------------------------
GitHub user greghogan opened a pull request:
https://github.com/apache/flink/pull/2223
[FLINK-4154] [core] Correction of murmur hash breaks backwards compatibility
Revert "[FLINK-3623] [runtime] Adjust MurmurHash Algorithm"
This reverts commit 641a0d436c9b7a34ff33ceb370cf29962cac4dee.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/greghogan/flink
4154_correction_of_murmurhash_breaks_backwards_compatibility
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2223.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2223
----
commit 99696c2fc40db7b5767c9c1b20bcc1fa5edfc890
Author: Greg Hogan <c...@greghogan.com>
Date: 2016-07-08T14:57:42Z
[FLINK-4154] [core] Correction of murmur hash breaks backwards compatibility
Revert "[FLINK-3623] [runtime] Adjust MurmurHash Algorithm"
This reverts commit 641a0d436c9b7a34ff33ceb370cf29962cac4dee.
----
> Correction of murmur hash breaks backwards compatibility
> --------------------------------------------------------
>
> Key: FLINK-4154
> URL: https://issues.apache.org/jira/browse/FLINK-4154
> Project: Flink
> Issue Type: Bug
> Components: Distributed Coordination
> Affects Versions: 1.1.0
> Reporter: Till Rohrmann
> Assignee: Greg Hogan
> Priority: Blocker
> Fix For: 1.1.0
>
>
> The correction of Flink's murmur hash with commit [1], breaks Flink's
> backwards compatibility with respect to savepoints. The reason is that the
> changed murmur hash which is used to partition elements in a {{KeyedStream}}
> changes the mapping from keys to sub tasks. This changes the assigned key
> spaces for a sub task. Consequently, an old savepoint (version 1.0) assigns
> states with a different key space to the sub tasks.
> I think that this must be fixed for the upcoming 1.1 release. I see two
> options to solve the problem:
> - revert the changes, but then we don't know how the flawed murmur hash
> performs
> - develop tooling to repartition state of old savepoints. This is probably
> not trivial since a keyed stream can also contain non-partitioned state which
> is not partitionable in all cases. And even if only partitioned state is
> used, we would need some kind of special operator which can repartition the
> state wrt the key.
> I think that the latter option requires some more thoughts and is thus
> unlikely to be done before the release 1.1. Therefore, as a workaround, I
> think that we should revert the murmur hash changes.
> [1]
> https://github.com/apache/flink/commit/641a0d436c9b7a34ff33ceb370cf29962cac4dee
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
