[GitHub] flink pull request: [FLINK-1330] [build] Build creates a link in t...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/333#issuecomment-71365114 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1330) Restructure directory layout
[ https://issues.apache.org/jira/browse/FLINK-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291031#comment-14291031 ] ASF GitHub Bot commented on FLINK-1330: --- Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/333#issuecomment-71365114 LGTM > Restructure directory layout > > > Key: FLINK-1330 > URL: https://issues.apache.org/jira/browse/FLINK-1330 > Project: Flink > Issue Type: Improvement > Components: Build System, Documentation >Reporter: Max Michels >Priority: Minor > Labels: usability > > When building Flink, the build results can currently be found under > "flink-root/flink-dist/target/flink-$FLINKVERSION-incubating-SNAPSHOT-bin/flink-$YARNVERSION-$FLINKVERSION-incubating-SNAPSHOT/". > I think we could improve the directory layout with the following: > - provide the bin folder in the root by default > - let the start up and submissions scripts in bin assemble the class path > - in case the project hasn't been build yet, inform the user > The changes would make it easier to work with Flink from source. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1444) Add data properties for data sources
[ https://issues.apache.org/jira/browse/FLINK-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291033#comment-14291033 ] Robert Metzger commented on FLINK-1444: --- I've prototyped something like this here: https://github.com/rmetzger/flink/compare/local_joins Maybe its helpful for you. > Add data properties for data sources > > > Key: FLINK-1444 > URL: https://issues.apache.org/jira/browse/FLINK-1444 > Project: Flink > Issue Type: New Feature > Components: Java API, JobManager, Optimizer >Affects Versions: 0.9 >Reporter: Fabian Hueske >Priority: Minor > > This issue proposes to add support for attaching data properties to data > sources. These data properties are defined with respect to input splits. > Possible properties are: > - partitioning across splits: all elements of the same key (combination) are > contained in one split > - sorting / grouping with splits: elements are sorted or grouped on certain > keys within a split > - key uniqueness: a certain key (combination) is unique for all elements of > the data source. This property is not defined wrt. input splits. > The optimizer can leverage this information to generate more efficient > execution plans. > The InputFormat will be responsible to generate input splits such that the > promised data properties are actually in place. Otherwise, the program will > produce invalid results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1389) Allow setting custom file extensions for files created by the FileOutputFormat
[ https://issues.apache.org/jira/browse/FLINK-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291038#comment-14291038 ] ASF GitHub Bot commented on FLINK-1389: --- Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/301#issuecomment-71366256 The only reason why I added the two variants is that I want to give users freedom to choose between these two variants. Its one more line of code that might (someday) make a user happy. I hope you agree with me that its easier and more efficient for me now to just merge it as it is rather than removing the feature again? > Allow setting custom file extensions for files created by the FileOutputFormat > -- > > Key: FLINK-1389 > URL: https://issues.apache.org/jira/browse/FLINK-1389 > Project: Flink > Issue Type: New Feature >Reporter: Robert Metzger >Assignee: Robert Metzger >Priority: Minor > > A user requested the ability to name avro files with the "avro" extension. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1389] Allow changing the filenames of t...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/301#issuecomment-71366256 The only reason why I added the two variants is that I want to give users freedom to choose between these two variants. Its one more line of code that might (someday) make a user happy. I hope you agree with me that its easier and more efficient for me now to just merge it as it is rather than removing the feature again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1389) Allow setting custom file extensions for files created by the FileOutputFormat
[ https://issues.apache.org/jira/browse/FLINK-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291039#comment-14291039 ] ASF GitHub Bot commented on FLINK-1389: --- Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/301#issuecomment-71366397 To be honest, I find this additional option more confusing than helpful. > Allow setting custom file extensions for files created by the FileOutputFormat > -- > > Key: FLINK-1389 > URL: https://issues.apache.org/jira/browse/FLINK-1389 > Project: Flink > Issue Type: New Feature >Reporter: Robert Metzger >Assignee: Robert Metzger >Priority: Minor > > A user requested the ability to name avro files with the "avro" extension. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1389] Allow changing the filenames of t...
Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/301#issuecomment-71366397 To be honest, I find this additional option more confusing than helpful. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1389] Allow changing the filenames of t...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/301#issuecomment-71366480 Okay. Let me implement a different approach that allows users to overwrite a method to put anything they want for the file name. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1389) Allow setting custom file extensions for files created by the FileOutputFormat
[ https://issues.apache.org/jira/browse/FLINK-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291040#comment-14291040 ] ASF GitHub Bot commented on FLINK-1389: --- Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/301#issuecomment-71366480 Okay. Let me implement a different approach that allows users to overwrite a method to put anything they want for the file name. > Allow setting custom file extensions for files created by the FileOutputFormat > -- > > Key: FLINK-1389 > URL: https://issues.apache.org/jira/browse/FLINK-1389 > Project: Flink > Issue Type: New Feature >Reporter: Robert Metzger >Assignee: Robert Metzger >Priority: Minor > > A user requested the ability to name avro files with the "avro" extension. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1434] [FLINK-1401] Streaming support ad...
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/334#discussion_r23503538
--- Diff: flink-addons/flink-streaming/flink-streaming-core/pom.xml ---
@@ -48,6 +48,12 @@ under the License.
${project.version}
test
+
+
+org.apache.sling
+org.apache.sling.commons.json
+2.0.6
+
--- End diff --
how many new / and which transitive dependencies are added by this
dependency?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (FLINK-1434) Web interface cannot be used to run streaming programs
[
https://issues.apache.org/jira/browse/FLINK-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291041#comment-14291041
]
ASF GitHub Bot commented on FLINK-1434:
---
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/334#discussion_r23503538
--- Diff: flink-addons/flink-streaming/flink-streaming-core/pom.xml ---
@@ -48,6 +48,12 @@ under the License.
${project.version}
test
+
+
+org.apache.sling
+org.apache.sling.commons.json
+2.0.6
+
--- End diff --
how many new / and which transitive dependencies are added by this
dependency?
> Web interface cannot be used to run streaming programs
> --
>
> Key: FLINK-1434
> URL: https://issues.apache.org/jira/browse/FLINK-1434
> Project: Flink
> Issue Type: Bug
> Components: Streaming, Webfrontend
>Affects Versions: 0.9
>Reporter: Gyula Fora
>Assignee: Gyula Fora
>
> Flink streaming programs currently cannot be submitted through the web
> client. When you try run the jar you get a ProgramInvocationException.
> The reason for this might be that streaming programs completely bypass the
> use of Plans for job execution and the streaming execution environment
> directly submits the jobgraph to the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1434] [FLINK-1401] Streaming support ad...
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/334#discussion_r23503540
--- Diff:
flink-addons/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/StreamGraph.java
---
@@ -536,4 +549,79 @@ public long getIterationTimeout(String vertexName) {
return iterationTimeouts.get(vertexName);
}
+ public String getOperatorName(String vertexName) {
+ return operatorNames.get(vertexName);
+ }
+
+ @Override
+ public String getStreamingPlanAsJSON() {
+
+ try {
+ JSONObject json = new JSONObject();
+ JSONArray nodes = new JSONArray();
+
+ json.put("nodes", nodes);
+
+ for (String id : operatorNames.keySet()) {
+ JSONObject node = new JSONObject();
+ nodes.put(node);
+
+ node.put("id", Integer.valueOf(id));
+ node.put("type", getOperatorName(id));
+
+ if (sources.contains(id)) {
+ node.put("pact", "Data Source");
+ } else {
+ node.put("pact", "Data Stream");
+ }
+
+ node.put("contents", getOperatorName(id) + " at
"
+ +
getInvokable(id).getUserFunction().getClass().getSimpleName());
+ node.put("parallelism", getParallelism(id));
+
+ int numIn = getInEdges(id).size();
+ if (numIn > 0) {
+
+ JSONArray inputs = new JSONArray();
+ node.put("predecessors", inputs);
+
+ for (int i = 0; i < numIn; i++) {
+
+ String inID =
getInEdges(id).get(i);
+
+ JSONObject input = new
JSONObject();
+ inputs.put(input);
+
+ input.put("id",
Integer.valueOf(inID));
+ input.put("ship_strategy",
getOutPartitioner(inID, id).getStrategy());
+ if (i == 0) {
+ input.put("side",
"first");
+ } else if (i == 1) {
+ input.put("side",
"second");
+ }
+ }
+ }
+
+ }
+ return json.toString();
+ } catch (Exception e) {
--- End diff --
maybe we should at least LOG.debug() the exception?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (FLINK-1434) Web interface cannot be used to run streaming programs
[
https://issues.apache.org/jira/browse/FLINK-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291042#comment-14291042
]
ASF GitHub Bot commented on FLINK-1434:
---
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/334#discussion_r23503540
--- Diff:
flink-addons/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/StreamGraph.java
---
@@ -536,4 +549,79 @@ public long getIterationTimeout(String vertexName) {
return iterationTimeouts.get(vertexName);
}
+ public String getOperatorName(String vertexName) {
+ return operatorNames.get(vertexName);
+ }
+
+ @Override
+ public String getStreamingPlanAsJSON() {
+
+ try {
+ JSONObject json = new JSONObject();
+ JSONArray nodes = new JSONArray();
+
+ json.put("nodes", nodes);
+
+ for (String id : operatorNames.keySet()) {
+ JSONObject node = new JSONObject();
+ nodes.put(node);
+
+ node.put("id", Integer.valueOf(id));
+ node.put("type", getOperatorName(id));
+
+ if (sources.contains(id)) {
+ node.put("pact", "Data Source");
+ } else {
+ node.put("pact", "Data Stream");
+ }
+
+ node.put("contents", getOperatorName(id) + " at
"
+ +
getInvokable(id).getUserFunction().getClass().getSimpleName());
+ node.put("parallelism", getParallelism(id));
+
+ int numIn = getInEdges(id).size();
+ if (numIn > 0) {
+
+ JSONArray inputs = new JSONArray();
+ node.put("predecessors", inputs);
+
+ for (int i = 0; i < numIn; i++) {
+
+ String inID =
getInEdges(id).get(i);
+
+ JSONObject input = new
JSONObject();
+ inputs.put(input);
+
+ input.put("id",
Integer.valueOf(inID));
+ input.put("ship_strategy",
getOutPartitioner(inID, id).getStrategy());
+ if (i == 0) {
+ input.put("side",
"first");
+ } else if (i == 1) {
+ input.put("side",
"second");
+ }
+ }
+ }
+
+ }
+ return json.toString();
+ } catch (Exception e) {
--- End diff --
maybe we should at least LOG.debug() the exception?
> Web interface cannot be used to run streaming programs
> --
>
> Key: FLINK-1434
> URL: https://issues.apache.org/jira/browse/FLINK-1434
> Project: Flink
> Issue Type: Bug
> Components: Streaming, Webfrontend
>Affects Versions: 0.9
>Reporter: Gyula Fora
>Assignee: Gyula Fora
>
> Flink streaming programs currently cannot be submitted through the web
> client. When you try run the jar you get a ProgramInvocationException.
> The reason for this might be that streaming programs completely bypass the
> use of Plans for job execution and the streaming execution environment
> directly submits the jobgraph to the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1330] [build] Build creates a link in t...
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/333#issuecomment-71366786 This looks good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1330) Restructure directory layout
[ https://issues.apache.org/jira/browse/FLINK-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291043#comment-14291043 ] ASF GitHub Bot commented on FLINK-1330: --- Github user mxm commented on the pull request: https://github.com/apache/flink/pull/333#issuecomment-71366786 This looks good. > Restructure directory layout > > > Key: FLINK-1330 > URL: https://issues.apache.org/jira/browse/FLINK-1330 > Project: Flink > Issue Type: Improvement > Components: Build System, Documentation >Reporter: Max Michels >Priority: Minor > Labels: usability > > When building Flink, the build results can currently be found under > "flink-root/flink-dist/target/flink-$FLINKVERSION-incubating-SNAPSHOT-bin/flink-$YARNVERSION-$FLINKVERSION-incubating-SNAPSHOT/". > I think we could improve the directory layout with the following: > - provide the bin folder in the root by default > - let the start up and submissions scripts in bin assemble the class path > - in case the project hasn't been build yet, inform the user > The changes would make it easier to work with Flink from source. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1441) Documentation SVN checkout link is wrong
[ https://issues.apache.org/jira/browse/FLINK-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291044#comment-14291044 ] Max Michels commented on FLINK-1441: Thank you, we will merge this as soon as possible. > Documentation SVN checkout link is wrong > > > Key: FLINK-1441 > URL: https://issues.apache.org/jira/browse/FLINK-1441 > Project: Flink > Issue Type: Bug > Components: Documentation >Reporter: Rahul Mahindrakar >Priority: Trivial > Attachments: improvement.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > In the documentation over here > http://flink.apache.org/how-to-contribute.html it states > -- > To make changes to the website, you have to checkout the source code of it > first: > svn checkout https://svn.apache.org/repos/asf/incubator/flink/ > cd flink > -- > when I try this it states. > hduser@ubuntu:~/Desktop/Flink/website$ svn checkout > https://svn.apache.org/repos/asf/incubator/flink/ > svn: E175011: Repository moved permanently to > 'https://svn.apache.org/repos/asf/flink'; please relocate > Though there is clear message on what to do I would like to have a first try > to change documentation and get to understand the process of contribution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1434) Web interface cannot be used to run streaming programs
[
https://issues.apache.org/jira/browse/FLINK-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291047#comment-14291047
]
ASF GitHub Bot commented on FLINK-1434:
---
Github user gyfora commented on a diff in the pull request:
https://github.com/apache/flink/pull/334#discussion_r23503611
--- Diff:
flink-addons/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/StreamGraph.java
---
@@ -536,4 +549,79 @@ public long getIterationTimeout(String vertexName) {
return iterationTimeouts.get(vertexName);
}
+ public String getOperatorName(String vertexName) {
+ return operatorNames.get(vertexName);
+ }
+
+ @Override
+ public String getStreamingPlanAsJSON() {
+
+ try {
+ JSONObject json = new JSONObject();
+ JSONArray nodes = new JSONArray();
+
+ json.put("nodes", nodes);
+
+ for (String id : operatorNames.keySet()) {
+ JSONObject node = new JSONObject();
+ nodes.put(node);
+
+ node.put("id", Integer.valueOf(id));
+ node.put("type", getOperatorName(id));
+
+ if (sources.contains(id)) {
+ node.put("pact", "Data Source");
+ } else {
+ node.put("pact", "Data Stream");
+ }
+
+ node.put("contents", getOperatorName(id) + " at
"
+ +
getInvokable(id).getUserFunction().getClass().getSimpleName());
+ node.put("parallelism", getParallelism(id));
+
+ int numIn = getInEdges(id).size();
+ if (numIn > 0) {
+
+ JSONArray inputs = new JSONArray();
+ node.put("predecessors", inputs);
+
+ for (int i = 0; i < numIn; i++) {
+
+ String inID =
getInEdges(id).get(i);
+
+ JSONObject input = new
JSONObject();
+ inputs.put(input);
+
+ input.put("id",
Integer.valueOf(inID));
+ input.put("ship_strategy",
getOutPartitioner(inID, id).getStrategy());
+ if (i == 0) {
+ input.put("side",
"first");
+ } else if (i == 1) {
+ input.put("side",
"second");
+ }
+ }
+ }
+
+ }
+ return json.toString();
+ } catch (Exception e) {
--- End diff --
Fixed
> Web interface cannot be used to run streaming programs
> --
>
> Key: FLINK-1434
> URL: https://issues.apache.org/jira/browse/FLINK-1434
> Project: Flink
> Issue Type: Bug
> Components: Streaming, Webfrontend
>Affects Versions: 0.9
>Reporter: Gyula Fora
>Assignee: Gyula Fora
>
> Flink streaming programs currently cannot be submitted through the web
> client. When you try run the jar you get a ProgramInvocationException.
> The reason for this might be that streaming programs completely bypass the
> use of Plans for job execution and the streaming execution environment
> directly submits the jobgraph to the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1434] [FLINK-1401] Streaming support ad...
Github user gyfora commented on a diff in the pull request:
https://github.com/apache/flink/pull/334#discussion_r23503611
--- Diff:
flink-addons/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/StreamGraph.java
---
@@ -536,4 +549,79 @@ public long getIterationTimeout(String vertexName) {
return iterationTimeouts.get(vertexName);
}
+ public String getOperatorName(String vertexName) {
+ return operatorNames.get(vertexName);
+ }
+
+ @Override
+ public String getStreamingPlanAsJSON() {
+
+ try {
+ JSONObject json = new JSONObject();
+ JSONArray nodes = new JSONArray();
+
+ json.put("nodes", nodes);
+
+ for (String id : operatorNames.keySet()) {
+ JSONObject node = new JSONObject();
+ nodes.put(node);
+
+ node.put("id", Integer.valueOf(id));
+ node.put("type", getOperatorName(id));
+
+ if (sources.contains(id)) {
+ node.put("pact", "Data Source");
+ } else {
+ node.put("pact", "Data Stream");
+ }
+
+ node.put("contents", getOperatorName(id) + " at
"
+ +
getInvokable(id).getUserFunction().getClass().getSimpleName());
+ node.put("parallelism", getParallelism(id));
+
+ int numIn = getInEdges(id).size();
+ if (numIn > 0) {
+
+ JSONArray inputs = new JSONArray();
+ node.put("predecessors", inputs);
+
+ for (int i = 0; i < numIn; i++) {
+
+ String inID =
getInEdges(id).get(i);
+
+ JSONObject input = new
JSONObject();
+ inputs.put(input);
+
+ input.put("id",
Integer.valueOf(inID));
+ input.put("ship_strategy",
getOutPartitioner(inID, id).getStrategy());
+ if (i == 0) {
+ input.put("side",
"first");
+ } else if (i == 1) {
+ input.put("side",
"second");
+ }
+ }
+ }
+
+ }
+ return json.toString();
+ } catch (Exception e) {
--- End diff --
Fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1434] [FLINK-1401] Streaming support ad...
Github user gyfora commented on a diff in the pull request:
https://github.com/apache/flink/pull/334#discussion_r23503616
--- Diff: flink-addons/flink-streaming/flink-streaming-core/pom.xml ---
@@ -48,6 +48,12 @@ under the License.
${project.version}
test
+
+
+org.apache.sling
+org.apache.sling.commons.json
+2.0.6
+
--- End diff --
This is already a dependency of the flink-connectors.
It might be good at some point to refactor the batch JSON plan generator to
use some actual JSON parser to make it look less messy
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (FLINK-1434) Web interface cannot be used to run streaming programs
[
https://issues.apache.org/jira/browse/FLINK-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291048#comment-14291048
]
ASF GitHub Bot commented on FLINK-1434:
---
Github user gyfora commented on a diff in the pull request:
https://github.com/apache/flink/pull/334#discussion_r23503616
--- Diff: flink-addons/flink-streaming/flink-streaming-core/pom.xml ---
@@ -48,6 +48,12 @@ under the License.
${project.version}
test
+
+
+org.apache.sling
+org.apache.sling.commons.json
+2.0.6
+
--- End diff --
This is already a dependency of the flink-connectors.
It might be good at some point to refactor the batch JSON plan generator to
use some actual JSON parser to make it look less messy
> Web interface cannot be used to run streaming programs
> --
>
> Key: FLINK-1434
> URL: https://issues.apache.org/jira/browse/FLINK-1434
> Project: Flink
> Issue Type: Bug
> Components: Streaming, Webfrontend
>Affects Versions: 0.9
>Reporter: Gyula Fora
>Assignee: Gyula Fora
>
> Flink streaming programs currently cannot be submitted through the web
> client. When you try run the jar you get a ProgramInvocationException.
> The reason for this might be that streaming programs completely bypass the
> use of Plans for job execution and the streaming execution environment
> directly submits the jobgraph to the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-629) Add support for null values to the java api
[ https://issues.apache.org/jira/browse/FLINK-629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291060#comment-14291060 ] Robert Metzger commented on FLINK-629: -- Cool. I'm glad its working. I'll push the fix to master and release-0.8 branch. Thank you for verifying the fix. > Add support for null values to the java api > --- > > Key: FLINK-629 > URL: https://issues.apache.org/jira/browse/FLINK-629 > Project: Flink > Issue Type: Improvement > Components: Java API >Reporter: Stephan Ewen >Assignee: Gyula Fora >Priority: Critical > Labels: github-import > Fix For: pre-apache > > Attachments: Selection_006.png, SimpleTweetInputFormat.java, > Tweet.java, model.tar.gz > > > Currently, many runtime operations fail when encountering a null value. Tuple > serialization should allow null fields. > I suggest to add a method to the tuples called `getFieldNotNull()` which > throws a meaningful exception when the accessed field is null. That way, we > simplify the logic of operators that should not dead with null fields, like > key grouping or aggregations. > Even though SQL allows grouping and aggregating of null values, I suggest to > exclude this from the java api, because the SQL semantics of aggregating null > fields are messy. > Imported from GitHub > Url: https://github.com/stratosphere/stratosphere/issues/629 > Created by: [StephanEwen|https://github.com/StephanEwen] > Labels: enhancement, java api, > Milestone: Release 0.5.1 > Created at: Wed Mar 26 00:27:49 CET 2014 > State: open -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-1447) Flink Web Submission Client does not work on Ubuntu 14.04 firefox browser
Rahul Mahindrakar created FLINK-1447: Summary: Flink Web Submission Client does not work on Ubuntu 14.04 firefox browser Key: FLINK-1447 URL: https://issues.apache.org/jira/browse/FLINK-1447 Project: Flink Issue Type: Bug Affects Versions: 0.8 Reporter: Rahul Mahindrakar Priority: Minor Hi I have the Hadoop 2 version of Flink and I tried to use the Flink Web Submission Client to submit a job. This is on Ubuntu 14.04 with Fireforx 35. "Mozilla Firefox for Ubuntu" When I upload a jar I dont see the jar in the options to select as a program to run. Since I am running Ubuntu on a virtual Machine using VMware Player, I tried to access the console from my windows host. This clearly shows me the Program. I have tried to attach this screenshot. Rahul Mahindrakar -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-629) Add support for null values to the java api
[ https://issues.apache.org/jira/browse/FLINK-629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291061#comment-14291061 ] Robert Metzger commented on FLINK-629: -- I'll track the fix here: https://issues.apache.org/jira/browse/FLINK-1446 > Add support for null values to the java api > --- > > Key: FLINK-629 > URL: https://issues.apache.org/jira/browse/FLINK-629 > Project: Flink > Issue Type: Improvement > Components: Java API >Reporter: Stephan Ewen >Assignee: Gyula Fora >Priority: Critical > Labels: github-import > Fix For: pre-apache > > Attachments: Selection_006.png, SimpleTweetInputFormat.java, > Tweet.java, model.tar.gz > > > Currently, many runtime operations fail when encountering a null value. Tuple > serialization should allow null fields. > I suggest to add a method to the tuples called `getFieldNotNull()` which > throws a meaningful exception when the accessed field is null. That way, we > simplify the logic of operators that should not dead with null fields, like > key grouping or aggregations. > Even though SQL allows grouping and aggregating of null values, I suggest to > exclude this from the java api, because the SQL semantics of aggregating null > fields are messy. > Imported from GitHub > Url: https://github.com/stratosphere/stratosphere/issues/629 > Created by: [StephanEwen|https://github.com/StephanEwen] > Labels: enhancement, java api, > Milestone: Release 0.5.1 > Created at: Wed Mar 26 00:27:49 CET 2014 > State: open -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-1446) Make KryoSerializer.createInstance() return new instances instead of null
Robert Metzger created FLINK-1446: - Summary: Make KryoSerializer.createInstance() return new instances instead of null Key: FLINK-1446 URL: https://issues.apache.org/jira/browse/FLINK-1446 Project: Flink Issue Type: Bug Reporter: Robert Metzger Fix For: 0.9, 0.8.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLINK-1447) Flink Web Submission Client does not work on Ubuntu 14.04 firefox browser
[ https://issues.apache.org/jira/browse/FLINK-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rahul Mahindrakar updated FLINK-1447: - Attachment: Capture.PNG I cannot see this with my Ubuntu 14.04 firefox browser. This is visible from outside my vm in my Windows Mozilla 35 browser. > Flink Web Submission Client does not work on Ubuntu 14.04 firefox browser > - > > Key: FLINK-1447 > URL: https://issues.apache.org/jira/browse/FLINK-1447 > Project: Flink > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Rahul Mahindrakar >Priority: Minor > Attachments: Capture.PNG > > > Hi > I have the Hadoop 2 version of Flink and I tried to use the Flink Web > Submission Client to submit a job. > This is on Ubuntu 14.04 with Fireforx 35. "Mozilla Firefox for Ubuntu" > When I upload a jar I dont see the jar in the options to select as a program > to run. > Since I am running Ubuntu on a virtual Machine using VMware Player, I tried > to access the console from my windows host. This clearly shows me the > Program. I have tried to attach this screenshot. > Rahul Mahindrakar -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1447) Flink Web Submission Client does not work on Ubuntu 14.04 firefox browser
[ https://issues.apache.org/jira/browse/FLINK-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291065#comment-14291065 ] Robert Metzger commented on FLINK-1447: --- On the Ubuntu Firefox browser, can you activate the JavaScript console and see if there are any errors reported when loading the page? > Flink Web Submission Client does not work on Ubuntu 14.04 firefox browser > - > > Key: FLINK-1447 > URL: https://issues.apache.org/jira/browse/FLINK-1447 > Project: Flink > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Rahul Mahindrakar >Priority: Minor > Attachments: Capture.PNG > > > Hi > I have the Hadoop 2 version of Flink and I tried to use the Flink Web > Submission Client to submit a job. > This is on Ubuntu 14.04 with Fireforx 35. "Mozilla Firefox for Ubuntu" > When I upload a jar I dont see the jar in the options to select as a program > to run. > Since I am running Ubuntu on a virtual Machine using VMware Player, I tried > to access the console from my windows host. This clearly shows me the > Program. I have tried to attach this screenshot. > Rahul Mahindrakar -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-1448) Documentation Clone your repository Link is wrong
Rahul Mahindrakar created FLINK-1448: Summary: Documentation Clone your repository Link is wrong Key: FLINK-1448 URL: https://issues.apache.org/jira/browse/FLINK-1448 Project: Flink Issue Type: Improvement Components: Documentation Reporter: Rahul Mahindrakar Priority: Trivial Hi http://flink.apache.org/how-to-contribute.html States --- Next, clone your repository fork to your local machine. git clone https://github.com//incubator-flink.git -- However due to change in status of the project from incubation this is wrong now It should point to git clone https://github.com//flink.git Rahul Mahindrakar -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1447) Flink Web Submission Client does not work on Ubuntu 14.04 firefox browser
[ https://issues.apache.org/jira/browse/FLINK-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291075#comment-14291075 ] Rahul Mahindrakar commented on FLINK-1447: -- See these errors. Need to investigate further and isolate the problem. This is definetly a browser compatibility issue since it works within Windows and not in Ubuntu --- The page was reloaded, because the character encoding declaration of the HTML document was not found when prescanning the first 1024 bytes of the file. The encoding declaration needs to be moved to be within the first 1024 bytes of the file. launch.html:26 NS_ERROR_XPC_SECURITY_MANAGER_VETO: unity-global-property-initializer.js:238 mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create d3.js:490 "navigated; resolution of _defaultNode aborted" Promise-backend.js:868 GET http://localhost:8080/css/bootstrap.css.map [HTTP/1.1 404 Not Found 5ms] A promise chain failed to handle a rejection. Did you forget to '.catch', or did you forget to 'return'? See https://developer.mozilla.org/Mozilla/JavaScript_code_modules/Promise.jsm/Promise Date: Sun Jan 25 2015 04:18:32 GMT-0800 (PST) Full Message: SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data SourceMap.jsm:67 --- > Flink Web Submission Client does not work on Ubuntu 14.04 firefox browser > - > > Key: FLINK-1447 > URL: https://issues.apache.org/jira/browse/FLINK-1447 > Project: Flink > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Rahul Mahindrakar >Priority: Minor > Attachments: Capture.PNG > > > Hi > I have the Hadoop 2 version of Flink and I tried to use the Flink Web > Submission Client to submit a job. > This is on Ubuntu 14.04 with Fireforx 35. "Mozilla Firefox for Ubuntu" > When I upload a jar I dont see the jar in the options to select as a program > to run. > Since I am running Ubuntu on a virtual Machine using VMware Player, I tried > to access the console from my windows host. This clearly shows me the > Program. I have tried to attach this screenshot. > Rahul Mahindrakar -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (FLINK-1409) Connected datastream functionality broken since the introduction of intermediate results
[ https://issues.apache.org/jira/browse/FLINK-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyula Fora closed FLINK-1409. - Resolution: Fixed > Connected datastream functionality broken since the introduction of > intermediate results > > > Key: FLINK-1409 > URL: https://issues.apache.org/jira/browse/FLINK-1409 > Project: Flink > Issue Type: Bug > Components: Streaming >Affects Versions: 0.9 >Reporter: Gyula Fora > > The connected data stream functionality which allows joint transformations on > two data streams of arbitrary type is broken since Ufuk's commit which > introduces the intermediate results. > The problem is most likely in the CoRecordReader which should allow > nonblocking read from inputs with different types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-1449) Extend Streaming guide to include the Scala API
Gyula Fora created FLINK-1449: - Summary: Extend Streaming guide to include the Scala API Key: FLINK-1449 URL: https://issues.apache.org/jira/browse/FLINK-1449 Project: Flink Issue Type: Task Components: Documentation, Streaming Affects Versions: 0.9, 0.8.1 Reporter: Gyula Fora The streaming guide currently only contains the java api features, we need to extend this for the scala api as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: Add flink-gelly to flink-addons (with commit h...
GitHub user vasia opened a pull request: https://github.com/apache/flink/pull/335 Add flink-gelly to flink-addons (with commit history) Hi everyone, here's the new PR for the graph api. I have addressed the comments of #326 and looking forward to additional feedback! -V. You can merge this pull request into a Git repository by running: $ git pull https://github.com/vasia/flink gelly-with-history Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/335.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #335 commit f6d3949f203343ae389b5650042f9dfed8aed2b1 Author: Stephan Ewen Date: 2014-08-30T17:19:21Z [FLINK-1201] [gelly] First mockup of initial API functions commit a01c2d7b2730f0fcd39439e2147ad0e582fa6949 Author: Stephan Ewen Date: 2014-10-08T10:18:04Z [FLINK-1201] [gelly] Initial package commit 07049741e3e502cbcd6be77f8da89c389caec63c Author: Stephan Ewen Date: 2014-10-08T11:50:11Z [FLINK-1201] [gelly] Stubs for basic classes commit 0f3130cd6fb8e48ee0fb22d1afa6ad0763407f67 Author: Stephan Ewen Date: 2014-10-08T12:29:46Z [FLINK-1201] [gelly] fix project setup commit ee8fadecb6300a91da39e0a3818fd66f77447786 Author: vasia Date: 2014-10-08T13:27:40Z [FLINK-1201] [gelly] get undirected graph commit f1a238970ebf2413fd96e425a8688611a29ac062 Author: vasia Date: 2014-10-09T11:22:24Z [FLINK-1201] [gelly] edge and value extend tuple types commit 3064a403acc8e2e45a66e43404456ea0aeb9164f Author: vasia Date: 2014-10-09T12:49:34Z [FLINK-1201] [gelly] reverse and graph input methods commit 4620ffbf2d6fda865fcdf1f1a2cf1e63e6b86280 Author: Theodore Vasiloudis Date: 2014-10-09T14:11:03Z [FLINK-1201] [gelly] Added Push-Gather-Apply, mapVertices, subgraph, outDegrees Added: mapVertices + Test subgraph (Unfinished) outDegrees Push-Gather-Apply Neighborhood Model Junit Dependency Everything is untested as we are blocked on graph creation commit 91ec75a7899616d1f9e6a7edb5157eff0add0264 Author: Theodore Vasiloudis Date: 2014-10-10T14:09:36Z [FLINK-1201] [gelly] Reverted to using Tuples, added tests. commit 10dc89d764fd99b51d27beb1783f7bd53dd001ad Author: Kostas Tzoumas Date: 2014-10-21T06:11:02Z [FLINK-1201] [gelly] Added Kostas' functions from the hackathon; added ExecutionEnvironment member to Graph commit 2cbc1d63719927459d41042cc0118eedd440b680 Author: vasia Date: 2014-10-25T14:14:42Z [FLINK-1201] [gelly] type information in Graph and in getUndirected method commit a3a28bc5915057a0c3305dc64816f1c544a0fc95 Author: vasia Date: 2014-10-25T15:08:53Z [FLINK-1201] [gelly] typeinfo in reverse and getOutdegrees commit 995fd38e582d2f8bf50e2b47042b5d67537cca77 Author: vasia Date: 2014-10-25T16:27:15Z [FLINK-1201] [gelly] test for undirected, reverse and outDegrees remove unused imports commit 73f8570ac6a1862fe7e1c560e7bfd4200a7a9797 Author: vasia Date: 2014-10-26T19:40:32Z [FLINK-1201] [gelly] made functions for getUndirected, outDegrees and reverse static inner commit 21f00aa4978a04cb7e830c973aec4ac7c10e692f Author: vasia Date: 2014-10-28T21:39:42Z [FLINK-1201] [gelly] test mapVertices commit e844c0154365992328b76adcdcdd3f56285f0873 Author: vasia Date: 2014-10-28T21:51:26Z [FLINK-1201] [gelly] remove unecessary type from mapVertices commit 79e8f2a0b0232fce26f879e51005b2483f18b3a6 Author: vasia Date: 2014-10-28T22:47:55Z [FLINK-1201] [gelly] fixed subGraph() and added test commit 3044b8ef8ba5d33251d1281ffd81229d7dd08745 Author: vasia Date: 2014-11-05T00:03:42Z [FLINK-1201] [gelly] mapVertices allows changing the value type commit dd9ba00c3415c5e0f89d5c409232166d334ea0c2 Author: vasia Date: 2014-11-11T14:05:10Z [FLINK-1201] [gelly] remove old hackathon test file commit ef8da751884a048b2a35d1a9850e8d3e5f577359 Author: vasia Date: 2014-11-11T16:18:44Z [FLINK-1201] [gelly] added execution environment to create calls in test commit 52861969eb63f5449ed8cb1868b03d1b5a3296e2 Author: vasia Date: 2014-11-12T09:25:48Z [FLINK-1201] [gelly] separate test for mapVertices commit cbc2fe2a64050e6bfa3d136073402e454d705666 Author: balidani Date: 2014-11-12T14:21:32Z [FLINK-1201] [gelly] Added mapEdges commit 0680f2d27aaf0603e9ce8aedc1ef6a4ad2f4356a Author: Daniel Bali Date: 2014-11-12T14:31:32Z [FLINK-1201] [gelly] Added mapEdges tests commit 0a1f7f429820c230ce0a81c0fc9d92b0eaa1f99b Author: balidani Date: 2014-11-12T18:09:45Z [FLINK-1201] [gelly] Removed unnecessary edgeKeyType field commit 4bf9c7697b97a93bf8363cefd294668f1b3f92b7 Author: vasia Date: 2014-11-13T10:19:26Z [FLINK-1201] [gelly] empty test todos for implemented operations commit 9ada02f638d196dd4de5c8
[jira] [Created] (FLINK-1450) Add Fold operator to the Streaming api
Gyula Fora created FLINK-1450: - Summary: Add Fold operator to the Streaming api Key: FLINK-1450 URL: https://issues.apache.org/jira/browse/FLINK-1450 Project: Flink Issue Type: New Feature Components: Streaming Affects Versions: 0.9 Reporter: Gyula Fora Priority: Minor The streaming API currently doesn't support a fold operator. This operator would work as the foldLeft method in Scala. This would allow effective implementations in a lot of cases where a the simple reduce is inappropriate due to different return types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (FLINK-1449) Extend Streaming guide to include the Scala API
[ https://issues.apache.org/jira/browse/FLINK-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Márton Balassi closed FLINK-1449. - Resolution: Duplicate This is a duplicate of FlINK-1429. > Extend Streaming guide to include the Scala API > --- > > Key: FLINK-1449 > URL: https://issues.apache.org/jira/browse/FLINK-1449 > Project: Flink > Issue Type: Task > Components: Documentation, Streaming >Affects Versions: 0.9, 0.8.1 >Reporter: Gyula Fora > > The streaming guide currently only contains the java api features, we need to > extend this for the scala api as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1434] [FLINK-1401] Streaming support ad...
Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/334#issuecomment-71374393 Now it is +1 for merge on my side. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1434) Web interface cannot be used to run streaming programs
[ https://issues.apache.org/jira/browse/FLINK-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291102#comment-14291102 ] ASF GitHub Bot commented on FLINK-1434: --- Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/334#issuecomment-71374393 Now it is +1 for merge on my side. > Web interface cannot be used to run streaming programs > -- > > Key: FLINK-1434 > URL: https://issues.apache.org/jira/browse/FLINK-1434 > Project: Flink > Issue Type: Bug > Components: Streaming, Webfrontend >Affects Versions: 0.9 >Reporter: Gyula Fora >Assignee: Gyula Fora > > Flink streaming programs currently cannot be submitted through the web > client. When you try run the jar you get a ProgramInvocationException. > The reason for this might be that streaming programs completely bypass the > use of Plans for job execution and the streaming execution environment > directly submits the jobgraph to the client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1446] Fix Kryo createInstance() method
GitHub user rmetzger opened a pull request: https://github.com/apache/flink/pull/336 [FLINK-1446] Fix Kryo createInstance() method You can merge this pull request into a Git repository by running: $ git pull https://github.com/rmetzger/flink FLINK-1446 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/336.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #336 commit b1aedaef11ccdd6740d0cce466f7e6be13cd492d Author: Robert Metzger Date: 2015-01-25T12:18:25Z [FLINK-1446] Fix Kryo createInstance() method --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1446) Make KryoSerializer.createInstance() return new instances instead of null
[ https://issues.apache.org/jira/browse/FLINK-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291109#comment-14291109 ] ASF GitHub Bot commented on FLINK-1446: --- GitHub user rmetzger opened a pull request: https://github.com/apache/flink/pull/336 [FLINK-1446] Fix Kryo createInstance() method You can merge this pull request into a Git repository by running: $ git pull https://github.com/rmetzger/flink FLINK-1446 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/336.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #336 commit b1aedaef11ccdd6740d0cce466f7e6be13cd492d Author: Robert Metzger Date: 2015-01-25T12:18:25Z [FLINK-1446] Fix Kryo createInstance() method > Make KryoSerializer.createInstance() return new instances instead of null > - > > Key: FLINK-1446 > URL: https://issues.apache.org/jira/browse/FLINK-1446 > Project: Flink > Issue Type: Bug >Reporter: Robert Metzger > Fix For: 0.9, 0.8.1 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-532) (Chained Map) Tasks simply keep in state Running
[ https://issues.apache.org/jira/browse/FLINK-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291114#comment-14291114 ] Mathias Peters commented on FLINK-532: -- I did not encounter this issue anymore in "recent" versions (from 0.7 upwards). It could be closed, I guess. > (Chained Map) Tasks simply keep in state Running > > > Key: FLINK-532 > URL: https://issues.apache.org/jira/browse/FLINK-532 > Project: Flink > Issue Type: Bug >Reporter: GitHub Import > Labels: github-import > Fix For: pre-apache > > > Sometimes tasks simply stay in state Running. This was experienced on a small > data set (1Gb) and dop 4. Of the 4 Maps chained to a data source 2 just > don't finish. This is not reproducable deterministically and the hanging > instances do not have error messages in the task manager logs. > > Imported from GitHub > Url: https://github.com/stratosphere/stratosphere/issues/532 > Created by: [mathiaspet|https://github.com/mathiaspet] > Labels: bug, core, > Milestone: Release 0.5.1 > Created at: Wed Mar 05 15:02:42 CET 2014 > State: open -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (FLINK-532) (Chained Map) Tasks simply keep in state Running
[ https://issues.apache.org/jira/browse/FLINK-532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Metzger closed FLINK-532. Resolution: Won't Fix Thank you. I'm closing the issue. > (Chained Map) Tasks simply keep in state Running > > > Key: FLINK-532 > URL: https://issues.apache.org/jira/browse/FLINK-532 > Project: Flink > Issue Type: Bug >Reporter: GitHub Import > Labels: github-import > Fix For: pre-apache > > > Sometimes tasks simply stay in state Running. This was experienced on a small > data set (1Gb) and dop 4. Of the 4 Maps chained to a data source 2 just > don't finish. This is not reproducable deterministically and the hanging > instances do not have error messages in the task manager logs. > > Imported from GitHub > Url: https://github.com/stratosphere/stratosphere/issues/532 > Created by: [mathiaspet|https://github.com/mathiaspet] > Labels: bug, core, > Milestone: Release 0.5.1 > Created at: Wed Mar 05 15:02:42 CET 2014 > State: open -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLINK-1441) Documentation SVN checkout link is wrong
[ https://issues.apache.org/jira/browse/FLINK-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Metzger updated FLINK-1441: -- Assignee: Rahul Mahindrakar > Documentation SVN checkout link is wrong > > > Key: FLINK-1441 > URL: https://issues.apache.org/jira/browse/FLINK-1441 > Project: Flink > Issue Type: Bug > Components: Documentation >Reporter: Rahul Mahindrakar >Assignee: Rahul Mahindrakar >Priority: Trivial > Attachments: improvement.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > In the documentation over here > http://flink.apache.org/how-to-contribute.html it states > -- > To make changes to the website, you have to checkout the source code of it > first: > svn checkout https://svn.apache.org/repos/asf/incubator/flink/ > cd flink > -- > when I try this it states. > hduser@ubuntu:~/Desktop/Flink/website$ svn checkout > https://svn.apache.org/repos/asf/incubator/flink/ > svn: E175011: Repository moved permanently to > 'https://svn.apache.org/repos/asf/flink'; please relocate > Though there is clear message on what to do I would like to have a first try > to change documentation and get to understand the process of contribution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1441) Documentation SVN checkout link is wrong
[ https://issues.apache.org/jira/browse/FLINK-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291117#comment-14291117 ] Robert Metzger commented on FLINK-1441: --- Hi Rahul, thank you for the patch. I'll merge it now. I gave you Contributor permissions in JIRA, so we can assign issues to you now. > Documentation SVN checkout link is wrong > > > Key: FLINK-1441 > URL: https://issues.apache.org/jira/browse/FLINK-1441 > Project: Flink > Issue Type: Bug > Components: Documentation >Reporter: Rahul Mahindrakar >Assignee: Rahul Mahindrakar >Priority: Trivial > Attachments: improvement.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > In the documentation over here > http://flink.apache.org/how-to-contribute.html it states > -- > To make changes to the website, you have to checkout the source code of it > first: > svn checkout https://svn.apache.org/repos/asf/incubator/flink/ > cd flink > -- > when I try this it states. > hduser@ubuntu:~/Desktop/Flink/website$ svn checkout > https://svn.apache.org/repos/asf/incubator/flink/ > svn: E175011: Repository moved permanently to > 'https://svn.apache.org/repos/asf/flink'; please relocate > Though there is clear message on what to do I would like to have a first try > to change documentation and get to understand the process of contribution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1441) Documentation SVN checkout link is wrong
[ https://issues.apache.org/jira/browse/FLINK-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291118#comment-14291118 ] Robert Metzger commented on FLINK-1441: --- Merged in http://svn.apache.org/r1654660 > Documentation SVN checkout link is wrong > > > Key: FLINK-1441 > URL: https://issues.apache.org/jira/browse/FLINK-1441 > Project: Flink > Issue Type: Bug > Components: Documentation >Reporter: Rahul Mahindrakar >Assignee: Rahul Mahindrakar >Priority: Trivial > Attachments: improvement.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > In the documentation over here > http://flink.apache.org/how-to-contribute.html it states > -- > To make changes to the website, you have to checkout the source code of it > first: > svn checkout https://svn.apache.org/repos/asf/incubator/flink/ > cd flink > -- > when I try this it states. > hduser@ubuntu:~/Desktop/Flink/website$ svn checkout > https://svn.apache.org/repos/asf/incubator/flink/ > svn: E175011: Repository moved permanently to > 'https://svn.apache.org/repos/asf/flink'; please relocate > Though there is clear message on what to do I would like to have a first try > to change documentation and get to understand the process of contribution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1433) Add HADOOP_CLASSPATH to start scripts
[ https://issues.apache.org/jira/browse/FLINK-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291127#comment-14291127 ] ASF GitHub Bot commented on FLINK-1433: --- GitHub user rmetzger opened a pull request: https://github.com/apache/flink/pull/337 [FLINK-1433] Add HADOOP_CLASSPATH to start scripts You can merge this pull request into a Git repository by running: $ git pull https://github.com/rmetzger/flink FLINK-1433 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/337.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #337 commit b9d1140df9b3232be53c105636f370f1d11aca37 Author: Robert Metzger Date: 2015-01-25T15:05:20Z [FLINK-1433] Add HADOOP_CLASSPATH to start scripts > Add HADOOP_CLASSPATH to start scripts > - > > Key: FLINK-1433 > URL: https://issues.apache.org/jira/browse/FLINK-1433 > Project: Flink > Issue Type: Improvement >Reporter: Robert Metzger > Fix For: 0.8.1 > > > With the Hadoop file system wrapper, its important to have access to the > hadoop filesystem classes. > The HADOOP_CLASSPATH seems to be a standard environment variable used by > Hadoop for such libraries. > Deployments like Google Compute Cloud set this variable containing the > "Google Cloud Storage Hadoop Wrapper". So if users want to use the Cloud > Storage in an non-yarn environment, we need to address this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1433] Add HADOOP_CLASSPATH to start scr...
GitHub user rmetzger opened a pull request: https://github.com/apache/flink/pull/337 [FLINK-1433] Add HADOOP_CLASSPATH to start scripts You can merge this pull request into a Git repository by running: $ git pull https://github.com/rmetzger/flink FLINK-1433 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/337.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #337 commit b9d1140df9b3232be53c105636f370f1d11aca37 Author: Robert Metzger Date: 2015-01-25T15:05:20Z [FLINK-1433] Add HADOOP_CLASSPATH to start scripts --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1201) Graph API for Flink
[ https://issues.apache.org/jira/browse/FLINK-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291129#comment-14291129 ] ASF GitHub Bot commented on FLINK-1201: --- Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-71376640 Thank you. Looks good. History is preserved and you addressed my comments. +1 for merging it. > Graph API for Flink > > > Key: FLINK-1201 > URL: https://issues.apache.org/jira/browse/FLINK-1201 > Project: Flink > Issue Type: New Feature >Reporter: Kostas Tzoumas >Assignee: Vasia Kalavri > > This issue tracks the development of a Graph API/DSL for Flink. > Until the code is pushed to the Flink repository, collaboration is happening > here: https://github.com/project-flink/flink-graph -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-71376640 Thank you. Looks good. History is preserved and you addressed my comments. +1 for merging it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (FLINK-1431) Add news mailing list to website
[ https://issues.apache.org/jira/browse/FLINK-1431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Metzger reassigned FLINK-1431: - Assignee: Robert Metzger > Add news mailing list to website > > > Key: FLINK-1431 > URL: https://issues.apache.org/jira/browse/FLINK-1431 > Project: Flink > Issue Type: Task > Components: Project Website >Reporter: Fabian Hueske >Assignee: Robert Metzger > > Flink got a new mailing list for monthly news and other announcements. > That list should be added to the project website. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLINK-1431) Add news mailing list to website
[ https://issues.apache.org/jira/browse/FLINK-1431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Metzger resolved FLINK-1431. --- Resolution: Fixed Resolved in http://svn.apache.org/r1654662 > Add news mailing list to website > > > Key: FLINK-1431 > URL: https://issues.apache.org/jira/browse/FLINK-1431 > Project: Flink > Issue Type: Task > Components: Project Website >Reporter: Fabian Hueske >Assignee: Robert Metzger > > Flink got a new mailing list for monthly news and other announcements. > That list should be added to the project website. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1428) Typos in Java code example for RichGroupReduceFunction
[ https://issues.apache.org/jira/browse/FLINK-1428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291132#comment-14291132 ] Robert Metzger commented on FLINK-1428: --- Would be cool if you could provide a patch for that. (We can apply it to all branches affected by this issue) > Typos in Java code example for RichGroupReduceFunction > -- > > Key: FLINK-1428 > URL: https://issues.apache.org/jira/browse/FLINK-1428 > Project: Flink > Issue Type: Bug > Components: Project Website >Reporter: Felix Neutatz >Priority: Minor > > http://flink.apache.org/docs/0.7-incubating/dataset_transformations.html > String key = null //missing ';' > public void combine(Iterable> in, > Collector> out)) > --> one ')' too much -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [Flink-1436] refactor CLiFrontend to provide m...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/331#issuecomment-71376859 @mxm: Since you have some experience with the CliFrontend now, would you mind looking into https://issues.apache.org/jira/browse/FLINK-1424 as well? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1424) bin/flink run does not recognize -c parameter anymore
[
https://issues.apache.org/jira/browse/FLINK-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291133#comment-14291133
]
ASF GitHub Bot commented on FLINK-1424:
---
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/331#issuecomment-71376859
@mxm: Since you have some experience with the CliFrontend now, would you
mind looking into https://issues.apache.org/jira/browse/FLINK-1424 as well?
> bin/flink run does not recognize -c parameter anymore
> -
>
> Key: FLINK-1424
> URL: https://issues.apache.org/jira/browse/FLINK-1424
> Project: Flink
> Issue Type: Bug
> Components: TaskManager
>Affects Versions: master
>Reporter: Carsten Brandt
>
> bin/flink binary does not recognize `-c` parameter anymore which specifies
> the class to run:
> {noformat}
> $ ./flink run "/path/to/target/impro3-ws14-flink-1.0-SNAPSHOT.jar" -c
> de.tu_berlin.impro3.flink.etl.FollowerGraphGenerator /tmp/flink/testgraph.txt
> 1
> usage: emma-experiments-impro3-ss14-flink
>[-?]
> emma-experiments-impro3-ss14-flink: error: unrecognized arguments: '-c'
> {noformat}
> before this command worked fine and executed the job.
> I tracked it down to the following commit using `git bisect`:
> {noformat}
> 93eadca782ee8c77f89609f6d924d73021dcdda9 is the first bad commit
> commit 93eadca782ee8c77f89609f6d924d73021dcdda9
> Author: Alexander Alexandrov
> Date: Wed Dec 24 13:49:56 2014 +0200
> [FLINK-1027] [cli] Added support for '--' and '-' prefixed tokens in CLI
> program arguments.
>
> This closes #278
> :04 04 a1358e6f7fe308b4d51a47069f190a29f87fdeda
> d6f11bbc9444227d5c6297ec908e44b9644289a9 Mflink-clients
> {noformat}
> https://github.com/apache/flink/commit/93eadca782ee8c77f89609f6d924d73021dcdda9
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (FLINK-1448) Documentation Clone your repository Link is wrong
[ https://issues.apache.org/jira/browse/FLINK-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Metzger resolved FLINK-1448. --- Resolution: Fixed Assignee: Robert Metzger Resovled in http://svn.apache.org/r1654661 > Documentation Clone your repository Link is wrong > - > > Key: FLINK-1448 > URL: https://issues.apache.org/jira/browse/FLINK-1448 > Project: Flink > Issue Type: Improvement > Components: Documentation >Reporter: Rahul Mahindrakar >Assignee: Robert Metzger >Priority: Trivial > > Hi > http://flink.apache.org/how-to-contribute.html > States > --- > Next, clone your repository fork to your local machine. > git clone https://github.com//incubator-flink.git > -- > However due to change in status of the project from incubation this is wrong > now > It should point to > git clone https://github.com//flink.git > Rahul Mahindrakar -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1448) Documentation Clone your repository Link is wrong
[ https://issues.apache.org/jira/browse/FLINK-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291142#comment-14291142 ] Rahul Mahindrakar commented on FLINK-1448: -- Wow you fixed all of them! Great. Rahul Mahindrakar > Documentation Clone your repository Link is wrong > - > > Key: FLINK-1448 > URL: https://issues.apache.org/jira/browse/FLINK-1448 > Project: Flink > Issue Type: Improvement > Components: Documentation >Reporter: Rahul Mahindrakar >Assignee: Robert Metzger >Priority: Trivial > > Hi > http://flink.apache.org/how-to-contribute.html > States > --- > Next, clone your repository fork to your local machine. > git clone https://github.com//incubator-flink.git > -- > However due to change in status of the project from incubation this is wrong > now > It should point to > git clone https://github.com//flink.git > Rahul Mahindrakar -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1433] Add HADOOP_CLASSPATH to start scr...
Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/337#issuecomment-71378388 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1433) Add HADOOP_CLASSPATH to start scripts
[ https://issues.apache.org/jira/browse/FLINK-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291154#comment-14291154 ] ASF GitHub Bot commented on FLINK-1433: --- Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/337#issuecomment-71378388 LGTM > Add HADOOP_CLASSPATH to start scripts > - > > Key: FLINK-1433 > URL: https://issues.apache.org/jira/browse/FLINK-1433 > Project: Flink > Issue Type: Improvement >Reporter: Robert Metzger > Fix For: 0.8.1 > > > With the Hadoop file system wrapper, its important to have access to the > hadoop filesystem classes. > The HADOOP_CLASSPATH seems to be a standard environment variable used by > Hadoop for such libraries. > Deployments like Google Compute Cloud set this variable containing the > "Google Cloud Storage Hadoop Wrapper". So if users want to use the Cloud > Storage in an non-yarn environment, we need to address this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1381] Allow multiple output splitters f...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/332 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1381) Only one output splitter supported per operator
[ https://issues.apache.org/jira/browse/FLINK-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291166#comment-14291166 ] ASF GitHub Bot commented on FLINK-1381: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/332 > Only one output splitter supported per operator > --- > > Key: FLINK-1381 > URL: https://issues.apache.org/jira/browse/FLINK-1381 > Project: Flink > Issue Type: Bug > Components: Streaming >Affects Versions: 0.9 >Reporter: Gyula Fora >Assignee: Mingliang Qi >Priority: Minor > > Currently the streaming api only supports output splitting once per operator. > The splitting logic should be reworked to allow any number of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (FLINK-1446) Make KryoSerializer.createInstance() return new instances instead of null
[ https://issues.apache.org/jira/browse/FLINK-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Metzger reassigned FLINK-1446: - Assignee: Robert Metzger > Make KryoSerializer.createInstance() return new instances instead of null > - > > Key: FLINK-1446 > URL: https://issues.apache.org/jira/browse/FLINK-1446 > Project: Flink > Issue Type: Bug >Reporter: Robert Metzger >Assignee: Robert Metzger > Fix For: 0.9, 0.8.1 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1447) Flink Web Submission Client does not work on Ubuntu 14.04 firefox browser
[ https://issues.apache.org/jira/browse/FLINK-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291217#comment-14291217 ] Robert Metzger commented on FLINK-1447: --- Cool, thank you for taking a closer look into the issue. > Flink Web Submission Client does not work on Ubuntu 14.04 firefox browser > - > > Key: FLINK-1447 > URL: https://issues.apache.org/jira/browse/FLINK-1447 > Project: Flink > Issue Type: Bug >Affects Versions: 0.8 >Reporter: Rahul Mahindrakar >Priority: Minor > Attachments: Capture.PNG > > > Hi > I have the Hadoop 2 version of Flink and I tried to use the Flink Web > Submission Client to submit a job. > This is on Ubuntu 14.04 with Fireforx 35. "Mozilla Firefox for Ubuntu" > When I upload a jar I dont see the jar in the options to select as a program > to run. > Since I am running Ubuntu on a virtual Machine using VMware Player, I tried > to access the console from my windows host. This clearly shows me the > Program. I have tried to attach this screenshot. > Rahul Mahindrakar -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLINK-1381) Only one output splitter supported per operator
[ https://issues.apache.org/jira/browse/FLINK-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyula Fora resolved FLINK-1381. --- Resolution: Fixed > Only one output splitter supported per operator > --- > > Key: FLINK-1381 > URL: https://issues.apache.org/jira/browse/FLINK-1381 > Project: Flink > Issue Type: Bug > Components: Streaming >Affects Versions: 0.9 >Reporter: Gyula Fora >Assignee: Mingliang Qi >Priority: Minor > > Currently the streaming api only supports output splitting once per operator. > The splitting logic should be reworked to allow any number of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1424) bin/flink run does not recognize -c parameter anymore
[
https://issues.apache.org/jira/browse/FLINK-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291265#comment-14291265
]
Max Michels commented on FLINK-1424:
[~cebe] The commit you mention makes it possible to have - or -- prefixed
arguments for the jobs. As of this change, everything behind the job jar file
is treated as arguments for the jar.
{{./flink run -c classpath job.jar}} sets the class of the job.jar to start
execution.
{{./flink run job.jar -c classpath}} sets the arguments for job.jar to "-c" and
"classpath"
Currently, the parser stops if it finds a non-option (argument not prefixed
with - or --) after "run". That way, you cannot accidentally set options of the
flink command line tool with your job's arguments. Imagine, your program had
the argument -m. If the above two examples were equivalent, you would also
change the -m flag of the flink command line tool which would overwrite the job
manager url and lead to a failed submission of the job.
Thus, I suggest to adapt the help notice to make it clear where the flink
command line tool options should go.
> bin/flink run does not recognize -c parameter anymore
> -
>
> Key: FLINK-1424
> URL: https://issues.apache.org/jira/browse/FLINK-1424
> Project: Flink
> Issue Type: Bug
> Components: TaskManager
>Affects Versions: master
>Reporter: Carsten Brandt
>
> bin/flink binary does not recognize `-c` parameter anymore which specifies
> the class to run:
> {noformat}
> $ ./flink run "/path/to/target/impro3-ws14-flink-1.0-SNAPSHOT.jar" -c
> de.tu_berlin.impro3.flink.etl.FollowerGraphGenerator /tmp/flink/testgraph.txt
> 1
> usage: emma-experiments-impro3-ss14-flink
>[-?]
> emma-experiments-impro3-ss14-flink: error: unrecognized arguments: '-c'
> {noformat}
> before this command worked fine and executed the job.
> I tracked it down to the following commit using `git bisect`:
> {noformat}
> 93eadca782ee8c77f89609f6d924d73021dcdda9 is the first bad commit
> commit 93eadca782ee8c77f89609f6d924d73021dcdda9
> Author: Alexander Alexandrov
> Date: Wed Dec 24 13:49:56 2014 +0200
> [FLINK-1027] [cli] Added support for '--' and '-' prefixed tokens in CLI
> program arguments.
>
> This closes #278
> :04 04 a1358e6f7fe308b4d51a47069f190a29f87fdeda
> d6f11bbc9444227d5c6297ec908e44b9644289a9 Mflink-clients
> {noformat}
> https://github.com/apache/flink/commit/93eadca782ee8c77f89609f6d924d73021dcdda9
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (FLINK-1424) bin/flink run does not recognize -c parameter anymore
[
https://issues.apache.org/jira/browse/FLINK-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291265#comment-14291265
]
Max Michels edited comment on FLINK-1424 at 1/25/15 8:52 PM:
-
[~cebe] The commit you mention makes it possible to have - or -- prefixed
arguments for the jobs. As of this change, everything behind the job jar file
is treated as arguments for the jar.
{{./flink run -c classpath job.jar}} sets the class of the job.jar to start
execution.
{{./flink run job.jar -c classpath}} sets the arguments for job.jar to "-c" and
"classpath"
Currently, the parser stops if it finds a non-option (argument not prefixed
with - or --) after "run". That way, you cannot accidentally set options of the
flink command line tool with your job's arguments. In the above example, if the
two commands were identical and if your program had the argument -c, you would
also change the -c option of the flink command line tool which would overwrite
your job's class entry point and lead to a failed submission of the job.
Thus, I suggest to adapt the help notice to make it clear where the flink
command line tool options should go.
was (Author: mxm):
[~cebe] The commit you mention makes it possible to have - or -- prefixed
arguments for the jobs. As of this change, everything behind the job jar file
is treated as arguments for the jar.
{{./flink run -c classpath job.jar}} sets the class of the job.jar to start
execution.
{{./flink run job.jar -c classpath}} sets the arguments for job.jar to "-c" and
"classpath"
Currently, the parser stops if it finds a non-option (argument not prefixed
with - or --) after "run". That way, you cannot accidentally set options of the
flink command line tool with your job's arguments. Imagine, your program had
the argument -m. If the above two examples were equivalent, you would also
change the -m flag of the flink command line tool which would overwrite the job
manager url and lead to a failed submission of the job.
Thus, I suggest to adapt the help notice to make it clear where the flink
command line tool options should go.
> bin/flink run does not recognize -c parameter anymore
> -
>
> Key: FLINK-1424
> URL: https://issues.apache.org/jira/browse/FLINK-1424
> Project: Flink
> Issue Type: Bug
> Components: TaskManager
>Affects Versions: master
>Reporter: Carsten Brandt
>
> bin/flink binary does not recognize `-c` parameter anymore which specifies
> the class to run:
> {noformat}
> $ ./flink run "/path/to/target/impro3-ws14-flink-1.0-SNAPSHOT.jar" -c
> de.tu_berlin.impro3.flink.etl.FollowerGraphGenerator /tmp/flink/testgraph.txt
> 1
> usage: emma-experiments-impro3-ss14-flink
>[-?]
> emma-experiments-impro3-ss14-flink: error: unrecognized arguments: '-c'
> {noformat}
> before this command worked fine and executed the job.
> I tracked it down to the following commit using `git bisect`:
> {noformat}
> 93eadca782ee8c77f89609f6d924d73021dcdda9 is the first bad commit
> commit 93eadca782ee8c77f89609f6d924d73021dcdda9
> Author: Alexander Alexandrov
> Date: Wed Dec 24 13:49:56 2014 +0200
> [FLINK-1027] [cli] Added support for '--' and '-' prefixed tokens in CLI
> program arguments.
>
> This closes #278
> :04 04 a1358e6f7fe308b4d51a47069f190a29f87fdeda
> d6f11bbc9444227d5c6297ec908e44b9644289a9 Mflink-clients
> {noformat}
> https://github.com/apache/flink/commit/93eadca782ee8c77f89609f6d924d73021dcdda9
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1081] Add HDFS file-stream source for s...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/226 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1081) Add HDFS file-stream source for streaming
[ https://issues.apache.org/jira/browse/FLINK-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291272#comment-14291272 ] ASF GitHub Bot commented on FLINK-1081: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/226 > Add HDFS file-stream source for streaming > - > > Key: FLINK-1081 > URL: https://issues.apache.org/jira/browse/FLINK-1081 > Project: Flink > Issue Type: Improvement > Components: Streaming >Affects Versions: 0.7.0-incubating >Reporter: Gyula Fora >Assignee: Chiwan Park > Labels: starter > > Add data stream source that will monitor a slected directory on HDFS (or > other filesystems as well) and will process all new files created. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLINK-1081) Add HDFS file-stream source for streaming
[ https://issues.apache.org/jira/browse/FLINK-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyula Fora resolved FLINK-1081. --- Resolution: Fixed > Add HDFS file-stream source for streaming > - > > Key: FLINK-1081 > URL: https://issues.apache.org/jira/browse/FLINK-1081 > Project: Flink > Issue Type: Improvement > Components: Streaming >Affects Versions: 0.7.0-incubating >Reporter: Gyula Fora >Assignee: Chiwan Park > Labels: starter > > Add data stream source that will monitor a slected directory on HDFS (or > other filesystems as well) and will process all new files created. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-1451) Enable parallel execution of streaming file sources
Márton Balassi created FLINK-1451: - Summary: Enable parallel execution of streaming file sources Key: FLINK-1451 URL: https://issues.apache.org/jira/browse/FLINK-1451 Project: Flink Issue Type: Task Components: Streaming Affects Versions: 0.9 Reporter: Márton Balassi Assignee: Márton Balassi Currently the F -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLINK-1451) Enable parallel execution of streaming file sources
[ https://issues.apache.org/jira/browse/FLINK-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Márton Balassi updated FLINK-1451: -- Description: Currently the streaming FileSourceFunction does not support parallelism greater than 1 as it does not implement the ParallelSourceFunction interface. Usage with distributed filesystems should be checked. In relation with this issue possible parallel solution for the FileMonitoringFunction should be considered. (was: Currently the F) > Enable parallel execution of streaming file sources > --- > > Key: FLINK-1451 > URL: https://issues.apache.org/jira/browse/FLINK-1451 > Project: Flink > Issue Type: Task > Components: Streaming >Affects Versions: 0.9 >Reporter: Márton Balassi >Assignee: Márton Balassi > > Currently the streaming FileSourceFunction does not support parallelism > greater than 1 as it does not implement the ParallelSourceFunction interface. > Usage with distributed filesystems should be checked. In relation with this > issue possible parallel solution for the FileMonitoringFunction should be > considered. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1433) Add HADOOP_CLASSPATH to start scripts
[ https://issues.apache.org/jira/browse/FLINK-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291435#comment-14291435 ] ASF GitHub Bot commented on FLINK-1433: --- Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/337#issuecomment-71412124 Does this mean that we now have multiple hadoop dependencies in the class path? The ones that are part of the Flink lib directory, plus the ones in the HADOOP_CLASSPATH variable. > Add HADOOP_CLASSPATH to start scripts > - > > Key: FLINK-1433 > URL: https://issues.apache.org/jira/browse/FLINK-1433 > Project: Flink > Issue Type: Improvement >Reporter: Robert Metzger > Fix For: 0.8.1 > > > With the Hadoop file system wrapper, its important to have access to the > hadoop filesystem classes. > The HADOOP_CLASSPATH seems to be a standard environment variable used by > Hadoop for such libraries. > Deployments like Google Compute Cloud set this variable containing the > "Google Cloud Storage Hadoop Wrapper". So if users want to use the Cloud > Storage in an non-yarn environment, we need to address this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1433] Add HADOOP_CLASSPATH to start scr...
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/337#issuecomment-71412124 Does this mean that we now have multiple hadoop dependencies in the class path? The ones that are part of the Flink lib directory, plus the ones in the HADOOP_CLASSPATH variable. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1446] Fix Kryo createInstance() method
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/336#issuecomment-71416590 +1 to merge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1446) Make KryoSerializer.createInstance() return new instances instead of null
[ https://issues.apache.org/jira/browse/FLINK-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291477#comment-14291477 ] ASF GitHub Bot commented on FLINK-1446: --- Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/336#issuecomment-71416590 +1 to merge > Make KryoSerializer.createInstance() return new instances instead of null > - > > Key: FLINK-1446 > URL: https://issues.apache.org/jira/browse/FLINK-1446 > Project: Flink > Issue Type: Bug >Reporter: Robert Metzger >Assignee: Robert Metzger > Fix For: 0.9, 0.8.1 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1376] [runtime] Add proper shared slot ...
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/317#issuecomment-71423347 I am looking into this right now and testing it locally. Seems to work so far. Will try to give a vote by tomorrow... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1376) SubSlots are not properly released in case that a TaskManager fatally fails, leaving the system in a corrupted state
[ https://issues.apache.org/jira/browse/FLINK-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291540#comment-14291540 ] ASF GitHub Bot commented on FLINK-1376: --- Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/317#issuecomment-71423347 I am looking into this right now and testing it locally. Seems to work so far. Will try to give a vote by tomorrow... > SubSlots are not properly released in case that a TaskManager fatally fails, > leaving the system in a corrupted state > > > Key: FLINK-1376 > URL: https://issues.apache.org/jira/browse/FLINK-1376 > Project: Flink > Issue Type: Bug >Reporter: Till Rohrmann > > In case that the TaskManager fatally fails and some of the failing node's > slots are SharedSlots, then the slots are not properly released by the > JobManager. This causes that the corresponding job will not be properly > failed, leaving the system in a corrupted state. > The reason for that is that the AllocatedSlot is not aware of being treated > as a SharedSlot and thus he cannot release the associated SubSlots. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
