[
https://issues.apache.org/jira/browse/FLINK-5480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823919#comment-15823919
]
ASF GitHub Bot commented on FLINK-5480:
---------------------------------------
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/3117#discussion_r96222946
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamGraphUserHashHasher.java
---
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.api.graph;
+
+import org.apache.flink.streaming.api.operators.ChainingStrategy;
+import org.apache.flink.streaming.api.operators.StreamOperator;
+import org.apache.flink.streaming.runtime.partitioner.ForwardPartitioner;
+import org.apache.flink.util.StringUtils;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * StreamGraphHasher that works with user provided hashes.
+ */
+public class StreamGraphUserHashHasher implements StreamGraphHasher {
+ @Override
+ public Map<Integer, byte[]>
traverseStreamGraphAndGenerateHashes(StreamGraph streamGraph) {
+ HashMap<Integer, byte[]> hashResult = new HashMap<>();
+ for (StreamNode streamNode : streamGraph.getStreamNodes()) {
+ String userHash = streamNode.getUserHash();
+ if (null != userHash) {
+
+ for (StreamEdge inEdge :
streamNode.getInEdges()) {
+ if (isChainable(inEdge,
streamGraph.isChainingEnabled())) {
+ throw new
UnsupportedOperationException("Cannot assign user-specified hash "
+ + "to
intermediate node in chain. This will be supported in future "
+ + "versions of
Flink. As a work around start new chain at task "
+ +
streamNode.getOperatorName() + ".");
+ }
+ }
+
+ hashResult.put(streamNode.getId(),
StringUtils.hexStringToByte(userHash));
+ }
+ }
+
+ return hashResult;
+ }
+
+ private boolean isChainable(StreamEdge edge, boolean isChainingEnabled)
{
--- End diff --
Can't we reuse the ```isChainable``` method in the
```StreamJobGraphGenerator``` instead? Otherwise we risk these conditions going
out of sync.
> User-provided hashes for operators
> ----------------------------------
>
> Key: FLINK-5480
> URL: https://issues.apache.org/jira/browse/FLINK-5480
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
> Affects Versions: 1.2.0
> Reporter: Stefan Richter
> Assignee: Stefan Richter
>
> We could allow users to provided (alternative) hashes for operators in a
> StreamGraph. This can make migration between Flink versions easier, in case
> the automatically produced hashes between versions are incompatible. For
> example, users could just copy the old hashes from the web ui to their job.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
