[GitHub] flink pull request: [FLINK-951] Reworking of Iteration Synchroniza...
GitHub user markus-h opened a pull request: https://github.com/apache/flink/pull/570 [FLINK-951] Reworking of Iteration Synchronization, Accumulators and Aggregators Iteration synchronization through JobManager Unification of Accumulators and Aggregators (removal of former Aggregators) Adjusted testcases accordingly I redid the work of my very old pull request https://github.com/apache/flink/pull/36 A more detailed description can be found in jira https://issues.apache.org/jira/browse/FLINK-951 I came across some unexpected behaviour with akka that made a small hack neccessary. Perhaps somebody with more experience in akka can find a better solution. See IterationHeadPactTask line 392. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markus-h/incubator-flink iterationsAndAccumulatorsRework2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/570.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #570 commit 5492487892ff99f10fccdb075404dedaa3371ff7 Author: Markus Holzemer Date: 2015-04-02T15:56:19Z Iteration synchronization through JobManager Unification of Accumulators and Aggregators (removal of former Aggregators) Adjusted testcases accordingly --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: Reworking of Iteration Synchronization, Accumu...
Github user markus-h commented on the pull request: https://github.com/apache/flink/pull/36#issuecomment-89529411 This change is continued in https://github.com/apache/flink/pull/570 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-951) Reworking of Iteration Synchronization, Accumulators and Aggregators
[ https://issues.apache.org/jira/browse/FLINK-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395628#comment-14395628 ] ASF GitHub Bot commented on FLINK-951: -- GitHub user markus-h opened a pull request: https://github.com/apache/flink/pull/570 [FLINK-951] Reworking of Iteration Synchronization, Accumulators and Aggregators Iteration synchronization through JobManager Unification of Accumulators and Aggregators (removal of former Aggregators) Adjusted testcases accordingly I redid the work of my very old pull request https://github.com/apache/flink/pull/36 A more detailed description can be found in jira https://issues.apache.org/jira/browse/FLINK-951 I came across some unexpected behaviour with akka that made a small hack neccessary. Perhaps somebody with more experience in akka can find a better solution. See IterationHeadPactTask line 392. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markus-h/incubator-flink iterationsAndAccumulatorsRework2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/570.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #570 commit 5492487892ff99f10fccdb075404dedaa3371ff7 Author: Markus Holzemer Date: 2015-04-02T15:56:19Z Iteration synchronization through JobManager Unification of Accumulators and Aggregators (removal of former Aggregators) Adjusted testcases accordingly > Reworking of Iteration Synchronization, Accumulators and Aggregators > > > Key: FLINK-951 > URL: https://issues.apache.org/jira/browse/FLINK-951 > Project: Flink > Issue Type: Improvement > Components: Iterations, Optimizer >Affects Versions: 0.6-incubating >Reporter: Markus Holzemer >Assignee: Markus Holzemer > Labels: refactoring > Original Estimate: 168h > Remaining Estimate: 168h > > I just realized that there is no real Jira issue for the task I am currently > working on. > I am currently reworking a few things regarding Iteration Synchronization, > Accumulators and Aggregators. Currently the synchronization at the end of one > superstep is done through channel events. That makes it hard to track the > current status of iterations. That is why I am changing this synchronization > to use RPC calls with the JobManager, so that the JobManager manages the > current status of all iterations. > Currently we use Accumulators outside of iterations and Aggregators inside of > iterations. Both have a similiar function, but a bit different interfaces and > handling. I want to unify these two concepts. I propose that we stick in the > future to Accumulators only. Aggregators therefore are removed and > Accumulators are extended to cover the usecases Aggregators were used fore > before. The switch to RPC for iterations makes it possible to also send the > current Accumulator values at the end of each superstep, so that the > JobManager (and thereby the webinterface) will be able to print intermediate > accumulation results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: Reworking of Iteration Synchronization, Accumu...
Github user markus-h closed the pull request at: https://github.com/apache/flink/pull/36 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-951) Reworking of Iteration Synchronization, Accumulators and Aggregators
[ https://issues.apache.org/jira/browse/FLINK-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395630#comment-14395630 ] ASF GitHub Bot commented on FLINK-951: -- Github user markus-h closed the pull request at: https://github.com/apache/flink/pull/36 > Reworking of Iteration Synchronization, Accumulators and Aggregators > > > Key: FLINK-951 > URL: https://issues.apache.org/jira/browse/FLINK-951 > Project: Flink > Issue Type: Improvement > Components: Iterations, Optimizer >Affects Versions: 0.6-incubating >Reporter: Markus Holzemer >Assignee: Markus Holzemer > Labels: refactoring > Original Estimate: 168h > Remaining Estimate: 168h > > I just realized that there is no real Jira issue for the task I am currently > working on. > I am currently reworking a few things regarding Iteration Synchronization, > Accumulators and Aggregators. Currently the synchronization at the end of one > superstep is done through channel events. That makes it hard to track the > current status of iterations. That is why I am changing this synchronization > to use RPC calls with the JobManager, so that the JobManager manages the > current status of all iterations. > Currently we use Accumulators outside of iterations and Aggregators inside of > iterations. Both have a similiar function, but a bit different interfaces and > handling. I want to unify these two concepts. I propose that we stick in the > future to Accumulators only. Aggregators therefore are removed and > Accumulators are extended to cover the usecases Aggregators were used fore > before. The switch to RPC for iterations makes it possible to also send the > current Accumulator values at the end of each superstep, so that the > JobManager (and thereby the webinterface) will be able to print intermediate > accumulation results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLINK-951) Reworking of Iteration Synchronization, Accumulators and Aggregators
[ https://issues.apache.org/jira/browse/FLINK-951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Markus Holzemer updated FLINK-951: -- Affects Version/s: (was: 0.6-incubating) 0.9 > Reworking of Iteration Synchronization, Accumulators and Aggregators > > > Key: FLINK-951 > URL: https://issues.apache.org/jira/browse/FLINK-951 > Project: Flink > Issue Type: Improvement > Components: Iterations, Optimizer >Affects Versions: 0.9 >Reporter: Markus Holzemer >Assignee: Markus Holzemer > Labels: refactoring > Original Estimate: 168h > Remaining Estimate: 168h > > I just realized that there is no real Jira issue for the task I am currently > working on. > I am currently reworking a few things regarding Iteration Synchronization, > Accumulators and Aggregators. Currently the synchronization at the end of one > superstep is done through channel events. That makes it hard to track the > current status of iterations. That is why I am changing this synchronization > to use RPC calls with the JobManager, so that the JobManager manages the > current status of all iterations. > Currently we use Accumulators outside of iterations and Aggregators inside of > iterations. Both have a similiar function, but a bit different interfaces and > handling. I want to unify these two concepts. I propose that we stick in the > future to Accumulators only. Aggregators therefore are removed and > Accumulators are extended to cover the usecases Aggregators were used fore > before. The switch to RPC for iterations makes it possible to also send the > current Accumulator values at the end of each superstep, so that the > JobManager (and thereby the webinterface) will be able to print intermediate > accumulation results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-1827) Move test classes in test folders and fix scope of test dependencies
Flavio Pompermaier created FLINK-1827:
-
Summary: Move test classes in test folders and fix scope of test
dependencies
Key: FLINK-1827
URL: https://issues.apache.org/jira/browse/FLINK-1827
Project: Flink
Issue Type: Improvement
Components: Build System
Affects Versions: 0.9
Reporter: Flavio Pompermaier
Priority: Minor
Right now it is not possible to avoid compilation of test classes
(-Dmaven.test.skip=true) because some project (e.g. flink-test-utils) requires
test classes in non-test sources (e.g. scalatest_${scala.binary.version})
Test classes should be moved to src/main/test (if Java) and src/test/scala (if
scala) and use scope=test for test dependencies
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink pull request: Fix issue where Windows paths were not recogni...
Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/491#issuecomment-89548376 I checked what are [valid absolute paths on windows](https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.aspx#paths). Summary: A path is relative if it does not begin with any of the following: - A disk designator **with a backslash**, for example `C:\` or `d:\`. - A single backslash, for example, `\directory` or `\file.txt`. This is also referred to as an absolute path. - A UNC name of any format, which always start with two backslash characters (`\\`). **Note:** A path with disk designator but without backslash (such as `c:test.tmp`, or `D:tmp/test.tmp`) is relative. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (FLINK-1828) Impossible to output data to an HBase table
Flavio Pompermaier created FLINK-1828: - Summary: Impossible to output data to an HBase table Key: FLINK-1828 URL: https://issues.apache.org/jira/browse/FLINK-1828 Project: Flink Issue Type: Bug Components: Hadoop Compatibility Affects Versions: 0.9 Reporter: Flavio Pompermaier Fix For: 0.9 Right now it is not possible to use HBase TableOutputFormat as output format because Configurable.setConf is not called in the configure() method of the HadoopOutputFormatBase -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: Fixed Configurable HadoopOutputFormat (FLINK-1...
GitHub user fpompermaier opened a pull request: https://github.com/apache/flink/pull/571 Fixed Configurable HadoopOutputFormat (FLINK-1828) See https://issues.apache.org/jira/browse/FLINK-1828 You can merge this pull request into a Git repository by running: $ git pull https://github.com/fpompermaier/flink master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/571.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #571 commit 83655bf2773a871c0fb88481be51c6d61ee98881 Author: fpompermaier Date: 2015-04-04T10:57:36Z Fixed Configurable Hadoop output format initialization, added a simple HBase sink test and upgraded HBase dependencies (from 0.98.6 to 0.98.11) commit 85dbacf46c6f97f6033a4247cdd60ded87b93641 Author: fpompermaier Date: 2015-04-04T10:57:36Z Fixed Configurable Hadoop output format initialization, added a simple HBase sink test and upgraded HBase dependencies (from 0.98.6 to 0.98.11) commit da39bd2da2ab6ae03ff90b4434e167b8278d2df2 Author: fpompermaier Date: 2015-04-04T11:11:55Z Merge branch 'master' of https://github.com/fpompermaier/flink.git Conflicts: flink-java/src/main/java/org/apache/flink/api/java/hadoop/mapreduce/HadoopOutputFormatBase.java --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1828) Impossible to output data to an HBase table
[ https://issues.apache.org/jira/browse/FLINK-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395694#comment-14395694 ] ASF GitHub Bot commented on FLINK-1828: --- GitHub user fpompermaier opened a pull request: https://github.com/apache/flink/pull/571 Fixed Configurable HadoopOutputFormat (FLINK-1828) See https://issues.apache.org/jira/browse/FLINK-1828 You can merge this pull request into a Git repository by running: $ git pull https://github.com/fpompermaier/flink master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/571.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #571 commit 83655bf2773a871c0fb88481be51c6d61ee98881 Author: fpompermaier Date: 2015-04-04T10:57:36Z Fixed Configurable Hadoop output format initialization, added a simple HBase sink test and upgraded HBase dependencies (from 0.98.6 to 0.98.11) commit 85dbacf46c6f97f6033a4247cdd60ded87b93641 Author: fpompermaier Date: 2015-04-04T10:57:36Z Fixed Configurable Hadoop output format initialization, added a simple HBase sink test and upgraded HBase dependencies (from 0.98.6 to 0.98.11) commit da39bd2da2ab6ae03ff90b4434e167b8278d2df2 Author: fpompermaier Date: 2015-04-04T11:11:55Z Merge branch 'master' of https://github.com/fpompermaier/flink.git Conflicts: flink-java/src/main/java/org/apache/flink/api/java/hadoop/mapreduce/HadoopOutputFormatBase.java > Impossible to output data to an HBase table > --- > > Key: FLINK-1828 > URL: https://issues.apache.org/jira/browse/FLINK-1828 > Project: Flink > Issue Type: Bug > Components: Hadoop Compatibility >Affects Versions: 0.9 >Reporter: Flavio Pompermaier > Labels: hadoop, hbase > Fix For: 0.9 > > > Right now it is not possible to use HBase TableOutputFormat as output format > because Configurable.setConf is not called in the configure() method of the > HadoopOutputFormatBase -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: Fixed Configurable HadoopOutputFormat (FLINK-1...
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/571#discussion_r27768571
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/hadoop/mapreduce/HadoopOutputFormatBase.java
---
@@ -66,6 +67,8 @@ public
HadoopOutputFormatBase(org.apache.hadoop.mapreduce.OutputFormat map
@Override
public void configure(Configuration parameters) {
// nothing to do
--- End diff --
The comment should be removed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (FLINK-1828) Impossible to output data to an HBase table
[
https://issues.apache.org/jira/browse/FLINK-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395707#comment-14395707
]
ASF GitHub Bot commented on FLINK-1828:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/571#discussion_r27768571
--- Diff:
flink-java/src/main/java/org/apache/flink/api/java/hadoop/mapreduce/HadoopOutputFormatBase.java
---
@@ -66,6 +67,8 @@ public
HadoopOutputFormatBase(org.apache.hadoop.mapreduce.OutputFormat map
@Override
public void configure(Configuration parameters) {
// nothing to do
--- End diff --
The comment should be removed.
> Impossible to output data to an HBase table
> ---
>
> Key: FLINK-1828
> URL: https://issues.apache.org/jira/browse/FLINK-1828
> Project: Flink
> Issue Type: Bug
> Components: Hadoop Compatibility
>Affects Versions: 0.9
>Reporter: Flavio Pompermaier
> Labels: hadoop, hbase
> Fix For: 0.9
>
>
> Right now it is not possible to use HBase TableOutputFormat as output format
> because Configurable.setConf is not called in the configure() method of the
> HadoopOutputFormatBase
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1828) Impossible to output data to an HBase table
[
https://issues.apache.org/jira/browse/FLINK-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395709#comment-14395709
]
ASF GitHub Bot commented on FLINK-1828:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/571#discussion_r27768581
--- Diff:
flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.addons.hbase.example;
+
+import org.apache.flink.api.common.functions.FlatMapFunction;
+import org.apache.flink.api.common.functions.RichMapFunction;
+import org.apache.flink.api.java.DataSet;
+import org.apache.flink.api.java.ExecutionEnvironment;
+import org.apache.flink.api.java.hadoop.mapreduce.HadoopOutputFormat;
+import org.apache.flink.api.java.tuple.Tuple2;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.util.Collector;
+import org.apache.hadoop.hbase.client.Mutation;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.mapreduce.TableOutputFormat;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.mapreduce.Job;
+
+@SuppressWarnings("serial")
+public class HBaseWriteExample {
+
+ //
*
+ // PROGRAM
+ //
*
+
+ public static void main(String[] args) throws Exception {
+
+ if(!parseParameters(args)) {
+ return;
+ }
+
+ // set up the execution environment
+ final ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
+
+ // get input data
+ DataSet text = getTextDataSet(env);
+
+ DataSet> counts =
+ // split up the lines in pairs (2-tuples)
containing: (word,1)
+ text.flatMap(new Tokenizer())
+ // group by the tuple field "0" and sum up
tuple field "1"
+ .groupBy(0)
+ .sum(1);
+
+ // emit result
+// if(fileOutput) {
--- End diff --
the `if` statement should be completely removed.
> Impossible to output data to an HBase table
> ---
>
> Key: FLINK-1828
> URL: https://issues.apache.org/jira/browse/FLINK-1828
> Project: Flink
> Issue Type: Bug
> Components: Hadoop Compatibility
>Affects Versions: 0.9
>Reporter: Flavio Pompermaier
> Labels: hadoop, hbase
> Fix For: 0.9
>
>
> Right now it is not possible to use HBase TableOutputFormat as output format
> because Configurable.setConf is not called in the configure() method of the
> HadoopOutputFormatBase
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink pull request: Fixed Configurable HadoopOutputFormat (FLINK-1...
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/571#discussion_r27768581
--- Diff:
flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.addons.hbase.example;
+
+import org.apache.flink.api.common.functions.FlatMapFunction;
+import org.apache.flink.api.common.functions.RichMapFunction;
+import org.apache.flink.api.java.DataSet;
+import org.apache.flink.api.java.ExecutionEnvironment;
+import org.apache.flink.api.java.hadoop.mapreduce.HadoopOutputFormat;
+import org.apache.flink.api.java.tuple.Tuple2;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.util.Collector;
+import org.apache.hadoop.hbase.client.Mutation;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.mapreduce.TableOutputFormat;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.mapreduce.Job;
+
+@SuppressWarnings("serial")
+public class HBaseWriteExample {
+
+ //
*
+ // PROGRAM
+ //
*
+
+ public static void main(String[] args) throws Exception {
+
+ if(!parseParameters(args)) {
+ return;
+ }
+
+ // set up the execution environment
+ final ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
+
+ // get input data
+ DataSet text = getTextDataSet(env);
+
+ DataSet> counts =
+ // split up the lines in pairs (2-tuples)
containing: (word,1)
+ text.flatMap(new Tokenizer())
+ // group by the tuple field "0" and sum up
tuple field "1"
+ .groupBy(0)
+ .sum(1);
+
+ // emit result
+// if(fileOutput) {
--- End diff --
the `if` statement should be completely removed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: Fixed Configurable HadoopOutputFormat (FLINK-1...
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/571#discussion_r27768585
--- Diff:
flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.addons.hbase.example;
+
+import org.apache.flink.api.common.functions.FlatMapFunction;
+import org.apache.flink.api.common.functions.RichMapFunction;
+import org.apache.flink.api.java.DataSet;
+import org.apache.flink.api.java.ExecutionEnvironment;
+import org.apache.flink.api.java.hadoop.mapreduce.HadoopOutputFormat;
+import org.apache.flink.api.java.tuple.Tuple2;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.util.Collector;
+import org.apache.hadoop.hbase.client.Mutation;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.mapreduce.TableOutputFormat;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.mapreduce.Job;
+
+@SuppressWarnings("serial")
+public class HBaseWriteExample {
+
+ //
*
+ // PROGRAM
+ //
*
+
+ public static void main(String[] args) throws Exception {
+
+ if(!parseParameters(args)) {
+ return;
+ }
+
+ // set up the execution environment
+ final ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
+
+ // get input data
+ DataSet text = getTextDataSet(env);
+
+ DataSet> counts =
+ // split up the lines in pairs (2-tuples)
containing: (word,1)
+ text.flatMap(new Tokenizer())
+ // group by the tuple field "0" and sum up
tuple field "1"
+ .groupBy(0)
+ .sum(1);
+
+ // emit result
+// if(fileOutput) {
+ Job job = Job.getInstance();
+
job.getConfiguration().set(TableOutputFormat.OUTPUT_TABLE, outputTableName);
+ // TODO is "mapred.output.dir" really useful?
+
job.getConfiguration().set("mapred.output.dir","/tmp/test");
+ counts.map(new RichMapFunction ,
Tuple2>() {
+ private final byte[] CF_SOME =
Bytes.toBytes("test-column");
+ private final byte[] Q_SOME =
Bytes.toBytes("value");
+ private transient Tuple2 reuse;
+
+ @Override
+ public void open(Configuration parameters)
throws Exception {
+ super.open(parameters);
+ reuse = new Tuple2();
+ }
+
+ @Override
+ public Tuple2
map(Tuple2 t) throws Exception {
+ reuse.f0 = new Text(t.f0);
+ Put put = new Put(t.f0.getBytes());
+ put.add(CF_SOME, Q_SOME,
Bytes.toBytes(t.f1));
+ reuse.f1 = put;
+ return reuse;
+ }
+ }).output(new HadoopOutputFormat(new
TableOutputFormat(), job));
+// } else {
--- End diff --
`else` branch not necessary
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but
[jira] [Commented] (FLINK-1828) Impossible to output data to an HBase table
[
https://issues.apache.org/jira/browse/FLINK-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395710#comment-14395710
]
ASF GitHub Bot commented on FLINK-1828:
---
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/571#discussion_r27768585
--- Diff:
flink-staging/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.addons.hbase.example;
+
+import org.apache.flink.api.common.functions.FlatMapFunction;
+import org.apache.flink.api.common.functions.RichMapFunction;
+import org.apache.flink.api.java.DataSet;
+import org.apache.flink.api.java.ExecutionEnvironment;
+import org.apache.flink.api.java.hadoop.mapreduce.HadoopOutputFormat;
+import org.apache.flink.api.java.tuple.Tuple2;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.util.Collector;
+import org.apache.hadoop.hbase.client.Mutation;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.mapreduce.TableOutputFormat;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.mapreduce.Job;
+
+@SuppressWarnings("serial")
+public class HBaseWriteExample {
+
+ //
*
+ // PROGRAM
+ //
*
+
+ public static void main(String[] args) throws Exception {
+
+ if(!parseParameters(args)) {
+ return;
+ }
+
+ // set up the execution environment
+ final ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
+
+ // get input data
+ DataSet text = getTextDataSet(env);
+
+ DataSet> counts =
+ // split up the lines in pairs (2-tuples)
containing: (word,1)
+ text.flatMap(new Tokenizer())
+ // group by the tuple field "0" and sum up
tuple field "1"
+ .groupBy(0)
+ .sum(1);
+
+ // emit result
+// if(fileOutput) {
+ Job job = Job.getInstance();
+
job.getConfiguration().set(TableOutputFormat.OUTPUT_TABLE, outputTableName);
+ // TODO is "mapred.output.dir" really useful?
+
job.getConfiguration().set("mapred.output.dir","/tmp/test");
+ counts.map(new RichMapFunction ,
Tuple2>() {
+ private final byte[] CF_SOME =
Bytes.toBytes("test-column");
+ private final byte[] Q_SOME =
Bytes.toBytes("value");
+ private transient Tuple2 reuse;
+
+ @Override
+ public void open(Configuration parameters)
throws Exception {
+ super.open(parameters);
+ reuse = new Tuple2();
+ }
+
+ @Override
+ public Tuple2
map(Tuple2 t) throws Exception {
+ reuse.f0 = new Text(t.f0);
+ Put put = new Put(t.f0.getBytes());
+ put.add(CF_SOME, Q_SOME,
Bytes.toBytes(t.f1));
+ reuse.f1 = put;
+ return reuse;
+ }
+ }).output(new HadoopOutputFormat(new
TableOutputFormat(), job));
+// } else {
--- End di
[jira] [Commented] (FLINK-1828) Impossible to output data to an HBase table
[ https://issues.apache.org/jira/browse/FLINK-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395711#comment-14395711 ] ASF GitHub Bot commented on FLINK-1828: --- Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/571#issuecomment-89559472 Looks good to me, except for some comments that should be removed. Regarding the `mapred.output.dir` parameter I am not sure whether this is generally expected for all Hadoop OutputFormats or only required for file-based OutputFormats. I would keep it for now and open a JIRA to investigate the issue and fix it if necessary. > Impossible to output data to an HBase table > --- > > Key: FLINK-1828 > URL: https://issues.apache.org/jira/browse/FLINK-1828 > Project: Flink > Issue Type: Bug > Components: Hadoop Compatibility >Affects Versions: 0.9 >Reporter: Flavio Pompermaier > Labels: hadoop, hbase > Fix For: 0.9 > > > Right now it is not possible to use HBase TableOutputFormat as output format > because Configurable.setConf is not called in the configure() method of the > HadoopOutputFormatBase -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: Fixed Configurable HadoopOutputFormat (FLINK-1...
Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/571#issuecomment-89559472 Looks good to me, except for some comments that should be removed. Regarding the `mapred.output.dir` parameter I am not sure whether this is generally expected for all Hadoop OutputFormats or only required for file-based OutputFormats. I would keep it for now and open a JIRA to investigate the issue and fix it if necessary. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: Fixed Configurable HadoopOutputFormat (FLINK-1...
Github user fpompermaier commented on the pull request: https://github.com/apache/flink/pull/571#issuecomment-89570344 Removed comments and commented code as suggested by Fabian. Do I have also to create a JIRA ticket about mapred.output.dir parameter? I think that it can be defaulted to the Flink temp directory or flinkTempDir/hadoop/job-id --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1828) Impossible to output data to an HBase table
[ https://issues.apache.org/jira/browse/FLINK-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395714#comment-14395714 ] ASF GitHub Bot commented on FLINK-1828: --- Github user fpompermaier commented on the pull request: https://github.com/apache/flink/pull/571#issuecomment-89570344 Removed comments and commented code as suggested by Fabian. Do I have also to create a JIRA ticket about mapred.output.dir parameter? I think that it can be defaulted to the Flink temp directory or flinkTempDir/hadoop/job-id > Impossible to output data to an HBase table > --- > > Key: FLINK-1828 > URL: https://issues.apache.org/jira/browse/FLINK-1828 > Project: Flink > Issue Type: Bug > Components: Hadoop Compatibility >Affects Versions: 0.9 >Reporter: Flavio Pompermaier > Labels: hadoop, hbase > Fix For: 0.9 > > > Right now it is not possible to use HBase TableOutputFormat as output format > because Configurable.setConf is not called in the configure() method of the > HadoopOutputFormatBase -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-1829) Conflicting Jackson version in the Flink POMs
Alexander Alexandrov created FLINK-1829:
---
Summary: Conflicting Jackson version in the Flink POMs
Key: FLINK-1829
URL: https://issues.apache.org/jira/browse/FLINK-1829
Project: Flink
Issue Type: Bug
Components: Build System
Affects Versions: 0.9
Reporter: Alexander Alexandrov
Fix For: 0.9
The current POM setup transitively includes multiple conflicting versions of
the Jackson library over
* {{com.amazonaws:aws-java-sdk}} (v. 2.1.1)
* {{org.apache.avro:avro}} (v. 1.9.13)
* {{org.apache.hbase:hbase-client}} (v. 1.8.8)
When running jobs against a Flink local runtime embedded with:
{code:xml}
org.apache.flink
flink-scala
${flink.version}
provided
org.apache.flink
flink-java
${flink.version}
provided
org.apache.flink
flink-clients
${flink.version}
provided
{code}
I get the following error:
{noformat}
15-04-04 15:52:04 ERROR exception during creation
akka.actor.ActorInitializationException: exception during creation
at akka.actor.ActorInitializationException$.apply(Actor.scala:164)
at akka.actor.ActorCell.create(ActorCell.scala:596)
at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:279)
at akka.dispatch.Mailbox.run(Mailbox.scala:220)
at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at akka.util.Reflect$.instantiate(Reflect.scala:66)
at akka.actor.ArgsReflectConstructor.produce(Props.scala:352)
at akka.actor.Props.newActor(Props.scala:252)
at akka.actor.ActorCell.newActor(ActorCell.scala:552)
at akka.actor.ActorCell.create(ActorCell.scala:578)
... 9 more
Caused by: java.lang.NoSuchMethodError:
com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
at
com.fasterxml.jackson.databind.ObjectMapper.(ObjectMapper.java:445)
at
com.fasterxml.jackson.databind.ObjectMapper.(ObjectMapper.java:366)
at
org.apache.flink.runtime.taskmanager.TaskManager.(TaskManager.scala:134)
... 18 more
{noformat}
Fixing the Jackson version on the client side, e.g, with the following snippet
{code:xml}
com.fasterxml.jackson.core
jackson-core
2.2.1
provided
com.fasterxml.jackson.core
jackson-databind
2.2.1
provided
com.fasterxml.jackson.core
jackson-annotations
2.2.1
provided
{code}
solves the problem, but I guess it will be better if we can stick with one
version in the build artifacts.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (FLINK-1670) Collect method for streaming
[ https://issues.apache.org/jira/browse/FLINK-1670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Gevay reassigned FLINK-1670: -- Assignee: Gabor Gevay > Collect method for streaming > > > Key: FLINK-1670 > URL: https://issues.apache.org/jira/browse/FLINK-1670 > Project: Flink > Issue Type: New Feature > Components: Streaming >Affects Versions: 0.9 >Reporter: Márton Balassi >Assignee: Gabor Gevay >Priority: Minor > > A convenience method for streaming back the results of a job to the client. > As the client itself is a bottleneck anyway an easy solution would be to > provide a socket sink with degree of parallelism 1, from which a client > utility can read. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1826]Remove the redundant codes never b...
Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/569#issuecomment-89617988 Hey, I personally like that the check is performed there even though it is currently redundant. It is true that the `DefaultMemoryManager` is for internal usage, but I think it is generally better be safe than sorry with these checks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1826) Remove the code never be executed in function getNumPages
[ https://issues.apache.org/jira/browse/FLINK-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395825#comment-14395825 ] ASF GitHub Bot commented on FLINK-1826: --- Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/569#issuecomment-89617988 Hey, I personally like that the check is performed there even though it is currently redundant. It is true that the `DefaultMemoryManager` is for internal usage, but I think it is generally better be safe than sorry with these checks. > Remove the code never be executed in function getNumPages > - > > Key: FLINK-1826 > URL: https://issues.apache.org/jira/browse/FLINK-1826 > Project: Flink > Issue Type: Improvement > Components: Local Runtime >Affects Versions: master >Reporter: Sibao Hong >Assignee: Sibao Hong >Priority: Minor > Fix For: master > > > Remove the code never be executed in function getNumPages, because the input > of numBytes has be validataion by its caller of DefaultMemoryManager(long > memorySize, int numberOfSlots, int pageSize). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1825]Fix the constructor comments with ...
Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/568#issuecomment-89618942 LGTM, merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1825) Fix the constructor comments with correct parameter in DefaultMemoryManager
[ https://issues.apache.org/jira/browse/FLINK-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395827#comment-14395827 ] ASF GitHub Bot commented on FLINK-1825: --- Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/568#issuecomment-89618942 LGTM, merging. > Fix the constructor comments with correct parameter in DefaultMemoryManager > --- > > Key: FLINK-1825 > URL: https://issues.apache.org/jira/browse/FLINK-1825 > Project: Flink > Issue Type: Improvement > Components: Local Runtime >Affects Versions: master >Reporter: Sibao Hong >Assignee: Sibao Hong >Priority: Minor > Fix For: master > > > The comment in DefaultMemoryManager's constructor of parameters missing > 'numberOfSlots' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1824] [streaming] Support added for mis...
Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/567#issuecomment-89619837 LGTM, merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1824) Allow missing types for DataStreams
[ https://issues.apache.org/jira/browse/FLINK-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395828#comment-14395828 ] ASF GitHub Bot commented on FLINK-1824: --- Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/567#issuecomment-89619837 LGTM, merging. > Allow missing types for DataStreams > --- > > Key: FLINK-1824 > URL: https://issues.apache.org/jira/browse/FLINK-1824 > Project: Flink > Issue Type: Improvement > Components: Streaming >Reporter: Gyula Fora >Assignee: Gyula Fora > > The streaming api currently only uses the methods of the type extractor which > don't allow missing types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1800][docs] Add beta badge to component...
Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/555#issuecomment-89620103 +1 for Ufuk's suggestion. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1800) Add a "Beta" badge in the documentation to components in flink-staging
[ https://issues.apache.org/jira/browse/FLINK-1800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395829#comment-14395829 ] ASF GitHub Bot commented on FLINK-1800: --- Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/555#issuecomment-89620103 +1 for Ufuk's suggestion. > Add a "Beta" badge in the documentation to components in flink-staging > -- > > Key: FLINK-1800 > URL: https://issues.apache.org/jira/browse/FLINK-1800 > Project: Flink > Issue Type: Task > Components: Documentation >Reporter: Robert Metzger >Assignee: Robert Metzger >Priority: Minor > > As per mailing list discussion: > http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Add-a-quot-Beta-quot-badge-in-the-documentation-to-components-in-flink-staging-td4801.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1808] Send barrier requests only when t...
Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/551#issuecomment-89620475 Thanks for the fix, merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1808) Omit sending checkpoint barriers when the execution graph is not running
[ https://issues.apache.org/jira/browse/FLINK-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395830#comment-14395830 ] ASF GitHub Bot commented on FLINK-1808: --- Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/551#issuecomment-89620475 Thanks for the fix, merging. > Omit sending checkpoint barriers when the execution graph is not running > > > Key: FLINK-1808 > URL: https://issues.apache.org/jira/browse/FLINK-1808 > Project: Flink > Issue Type: Improvement > Components: Streaming >Reporter: Paris Carbone >Assignee: Paris Carbone > > Currently the StreamCheckpointCoordinator sends barrier requests even when > the executionGraph is in FAILING or RESTARTING status which results in > unneeded potential communication and space overhead until the job restarts > again. It should therefore simply omit sending barriers requests when the > execution graph is not in a RUNNING state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1826) Remove the code never be executed in function getNumPages
[ https://issues.apache.org/jira/browse/FLINK-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14396034#comment-14396034 ] ASF GitHub Bot commented on FLINK-1826: --- Github user matadorhong closed the pull request at: https://github.com/apache/flink/pull/569 > Remove the code never be executed in function getNumPages > - > > Key: FLINK-1826 > URL: https://issues.apache.org/jira/browse/FLINK-1826 > Project: Flink > Issue Type: Improvement > Components: Local Runtime >Affects Versions: master >Reporter: Sibao Hong >Assignee: Sibao Hong >Priority: Minor > Fix For: master > > > Remove the code never be executed in function getNumPages, because the input > of numBytes has be validataion by its caller of DefaultMemoryManager(long > memorySize, int numberOfSlots, int pageSize). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1826]Remove the redundant codes never b...
Github user matadorhong closed the pull request at: https://github.com/apache/flink/pull/569 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1826) Remove the code never be executed in function getNumPages
[ https://issues.apache.org/jira/browse/FLINK-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14396033#comment-14396033 ] ASF GitHub Bot commented on FLINK-1826: --- Github user matadorhong commented on the pull request: https://github.com/apache/flink/pull/569#issuecomment-89706960 OK. I accept it. I will close the PR. Thanks for your answer. > Remove the code never be executed in function getNumPages > - > > Key: FLINK-1826 > URL: https://issues.apache.org/jira/browse/FLINK-1826 > Project: Flink > Issue Type: Improvement > Components: Local Runtime >Affects Versions: master >Reporter: Sibao Hong >Assignee: Sibao Hong >Priority: Minor > Fix For: master > > > Remove the code never be executed in function getNumPages, because the input > of numBytes has be validataion by its caller of DefaultMemoryManager(long > memorySize, int numberOfSlots, int pageSize). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1826]Remove the redundant codes never b...
Github user matadorhong commented on the pull request: https://github.com/apache/flink/pull/569#issuecomment-89706960 OK. I accept it. I will close the PR. Thanks for your answer. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (FLINK-1826) Remove the code never be executed in function getNumPages
[ https://issues.apache.org/jira/browse/FLINK-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Márton Balassi closed FLINK-1826. - Resolution: Not a Problem > Remove the code never be executed in function getNumPages > - > > Key: FLINK-1826 > URL: https://issues.apache.org/jira/browse/FLINK-1826 > Project: Flink > Issue Type: Improvement > Components: Local Runtime >Affects Versions: master >Reporter: Sibao Hong >Assignee: Sibao Hong >Priority: Minor > Fix For: master > > > Remove the code never be executed in function getNumPages, because the input > of numBytes has be validataion by its caller of DefaultMemoryManager(long > memorySize, int numberOfSlots, int pageSize). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
