[GitHub] flink pull request: [FLINK-3050] [runtime] Add UnrecoverableExcept...

2015-12-17 Thread gyfora
Github user gyfora commented on the pull request: https://github.com/apache/flink/pull/1461#issuecomment-165382644 I think this is very useful, also for instance when there are not enough task slots etc. --- If your project is set up for it, you can reply to this email and have your

[jira] [Commented] (FLINK-3050) Add custom Exception type to suppress job restarts

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061717#comment-15061717 ] ASF GitHub Bot commented on FLINK-3050: --- Github user gyfora commented on the pull re

[GitHub] flink pull request: [FLINK-3050] [runtime] Add UnrecoverableExcept...

2015-12-17 Thread gyfora
Github user gyfora commented on the pull request: https://github.com/apache/flink/pull/1461#issuecomment-165382891 Looks good +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[jira] [Commented] (FLINK-3050) Add custom Exception type to suppress job restarts

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061721#comment-15061721 ] ASF GitHub Bot commented on FLINK-3050: --- Github user gyfora commented on the pull re

[GitHub] flink pull request: [FLINK-3050] [runtime] Add UnrecoverableExcept...

2015-12-17 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/1461#issuecomment-165398018 Thanks for the review. With task slots we have to be careful though, because it is possible that slots become available after some time. --- If your project is set up for i

[jira] [Commented] (FLINK-3050) Add custom Exception type to suppress job restarts

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061794#comment-15061794 ] ASF GitHub Bot commented on FLINK-3050: --- Github user uce commented on the pull reque

[jira] [Commented] (FLINK-3050) Add custom Exception type to suppress job restarts

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061800#comment-15061800 ] ASF GitHub Bot commented on FLINK-3050: --- Github user tillrohrmann commented on the p

[GitHub] flink pull request: [FLINK-3050] [runtime] Add UnrecoverableExcept...

2015-12-17 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/1461#issuecomment-165399077 And in the foreseeable future it might be possible to scale the job according to the available slots. --- If your project is set up for it, you can reply to this e

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-12-17 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/1255#issuecomment-165400271 Sorry for not responding earlier. I've missed the notification somehow. Batch exchange (the blocking intermediate results) do not work within iterations. Therefore t

[jira] [Commented] (FLINK-7) [GitHub] Enable Range Partitioner

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061803#comment-15061803 ] ASF GitHub Bot commented on FLINK-7: Github user uce commented on the pull request:

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-12-17 Thread fhueske
Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/1255#issuecomment-165404528 Thanks @uce! I extended the `RangePartitionRewriter` to cover that case and will include the changes when I merge the PR. I also found that no combiners are inse

[jira] [Commented] (FLINK-7) [GitHub] Enable Range Partitioner

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061816#comment-15061816 ] ASF GitHub Bot commented on FLINK-7: Github user fhueske commented on the pull request:

[GitHub] flink pull request: Trivial typo in docker-flink/README.md

2015-12-17 Thread fhueske
Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/1463#issuecomment-165406300 Hi @petervandenabeele, thanks for the patch! +1 to merge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] flink pull request: [FLINK-3166] [runtime] The first program in Ob...

2015-12-17 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/1464#issuecomment-165406987 A few comments: * the change to the TestEnvironment is unnecessary; ObjectReuse is enabled/disabled in the JavaProgramTestBase. it doesn't fix anything, removing the

[jira] [Commented] (FLINK-3166) The first program in ObjectReuseITCase has the wrong expected result, and it succeeds

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061837#comment-15061837 ] ASF GitHub Bot commented on FLINK-3166: --- Github user zentol commented on the pull re

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread twalthr
GitHub user twalthr opened a pull request: https://github.com/apache/flink/pull/1465 [FLINK-3140] NULL value data layout in Row Serializer/Comparator This PR implements FLINK-3140 and enables full NULL field support for grouping and sorting rows. You can merge this pull request int

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061848#comment-15061848 ] ASF GitHub Bot commented on FLINK-3140: --- GitHub user twalthr opened a pull request:

[GitHub] flink pull request: Trivial typo in docker-flink/README.md

2015-12-17 Thread petervandenabeele
Github user petervandenabeele commented on the pull request: https://github.com/apache/flink/pull/1463#issuecomment-165415675 Hmmm, one of the Travis tests failed * I presume this is a flickering test from the context of other PR tests passing around that time * my patch is onl

[GitHub] flink pull request: Trivial typo in docker-flink/README.md

2015-12-17 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/1463#issuecomment-165424481 yes it's not related to your change (there are a bunch of unstable tests), it will be merged and you don't have to do anything. --- If your project is set up for it, you

[GitHub] flink pull request: [FLINK-3131] Expose checkpoint metrics

2015-12-17 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/1453#discussion_r47895579 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/StateForTask.java --- @@ -48,18 +48,38 @@ /** The state of the parallel operato

[jira] [Commented] (FLINK-3131) Expose checkpoint metrics

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061929#comment-15061929 ] ASF GitHub Bot commented on FLINK-3131: --- Github user mxm commented on a diff in the

[GitHub] flink pull request: [FLINK-3131] Expose checkpoint metrics

2015-12-17 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/1453#discussion_r47895592 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/StateHandle.java --- @@ -43,4 +43,15 @@ * used any more. */ void di

[jira] [Commented] (FLINK-3131) Expose checkpoint metrics

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061930#comment-15061930 ] ASF GitHub Bot commented on FLINK-3131: --- Github user mxm commented on a diff in the

[GitHub] flink pull request: [FLINK-3131] Expose checkpoint metrics

2015-12-17 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1453#issuecomment-165430143 Thanks @uce for this nice pull request. Looks very good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If y

[jira] [Commented] (FLINK-3131) Expose checkpoint metrics

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061934#comment-15061934 ] ASF GitHub Bot commented on FLINK-3131: --- Github user mxm commented on the pull reque

[GitHub] flink pull request: Trivial typo in docker-flink/README.md

2015-12-17 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1463#issuecomment-165432127 Thanks for the pull request! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] flink pull request: Trivial typo in docker-flink/README.md

2015-12-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/1463 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Created] (FLINK-3180) MemoryLogger does not log direct memory

2015-12-17 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-3180: -- Summary: MemoryLogger does not log direct memory Key: FLINK-3180 URL: https://issues.apache.org/jira/browse/FLINK-3180 Project: Flink Issue Type: Bug C

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/1465#discussion_r47902384 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/typeinfo/NullMaskUtils.scala --- @@ -0,0 +1,92 @@ +/* + * Licensed

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062006#comment-15062006 ] ASF GitHub Bot commented on FLINK-3140: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/1465#discussion_r47903103 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/typeinfo/NullMaskUtils.scala --- @@ -0,0 +1,92 @@ +/* + * Licensed

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062010#comment-15062010 ] ASF GitHub Bot commented on FLINK-3140: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/1465#discussion_r47903321 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/typeinfo/RowComparator.scala --- @@ -0,0 +1,468 @@ +/* + * License

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062017#comment-15062017 ] ASF GitHub Bot commented on FLINK-3140: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/1465#discussion_r47903444 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/typeinfo/RowComparator.scala --- @@ -0,0 +1,468 @@ +/* + * License

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062018#comment-15062018 ] ASF GitHub Bot commented on FLINK-3140: --- Github user tillrohrmann commented on a dif

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062021#comment-15062021 ] ASF GitHub Bot commented on FLINK-3140: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/1465#discussion_r47903615 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/typeinfo/RowComparator.scala --- @@ -0,0 +1,468 @@ +/* + * License

[GitHub] flink pull request: [FLINK-3180] [runtime] Log direct memory usage...

2015-12-17 Thread uce
GitHub user uce opened a pull request: https://github.com/apache/flink/pull/1466 [FLINK-3180] [runtime] Log direct memory usage in MemoryLogger The off-heap stats reported by the memory logger didn't include direct memory. You can merge this pull request into a Git repository by ru

[jira] [Commented] (FLINK-3180) MemoryLogger does not log direct memory

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062031#comment-15062031 ] ASF GitHub Bot commented on FLINK-3180: --- GitHub user uce opened a pull request:

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-12-17 Thread fhueske
Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/1255#issuecomment-165453966 I executed the Flink WordCount example on 4 nodes with 8 parallel tasks and roughly 17GB of input data once with hash partitioning and once with range partitioning. Both

[jira] [Commented] (FLINK-7) [GitHub] Enable Range Partitioner

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062032#comment-15062032 ] ASF GitHub Bot commented on FLINK-7: Github user fhueske commented on the pull request:

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/1465#discussion_r47907286 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/typeinfo/RowComparator.scala --- @@ -0,0 +1,468 @@ +/* + * License

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062058#comment-15062058 ] ASF GitHub Bot commented on FLINK-3140: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread twalthr
Github user twalthr commented on the pull request: https://github.com/apache/flink/pull/1465#issuecomment-165462700 Thanks for your feedback @tillrohrmann. Many parts could be expressed in more Scala style, but for performance reasons and to prevent bugs in this crucial component I

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062088#comment-15062088 ] ASF GitHub Bot commented on FLINK-3140: --- Github user twalthr commented on the pull r

[GitHub] flink pull request: [FLINK-3134][yarn] asynchronous YarnJobManager...

2015-12-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/1450 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (FLINK-3134) Make YarnJobManager's allocate call asynchronous

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062094#comment-15062094 ] ASF GitHub Bot commented on FLINK-3134: --- Github user asfgit closed the pull request

[jira] [Closed] (FLINK-3134) Make YarnJobManager's allocate call asynchronous

2015-12-17 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels closed FLINK-3134. - Resolution: Implemented 4e52fe4304566e5239996b3d48290e0c1f0772e8 > Make YarnJobManager's

[jira] [Assigned] (FLINK-3181) The vertex-centric SSSP example and library method send unnecessary messages during the first superstep

2015-12-17 Thread Vasia Kalavri (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vasia Kalavri reassigned FLINK-3181: Assignee: Vasia Kalavri > The vertex-centric SSSP example and library method send unnecessa

[jira] [Created] (FLINK-3181) The vertex-centric SSSP example and library method send unnecessary messages during the first superstep

2015-12-17 Thread Vasia Kalavri (JIRA)
Vasia Kalavri created FLINK-3181: Summary: The vertex-centric SSSP example and library method send unnecessary messages during the first superstep Key: FLINK-3181 URL: https://issues.apache.org/jira/browse/FLINK-3

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread twalthr
Github user twalthr commented on a diff in the pull request: https://github.com/apache/flink/pull/1465#discussion_r47911141 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/typeinfo/RowComparator.scala --- @@ -0,0 +1,468 @@ +/* + * Licensed to

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062109#comment-15062109 ] ASF GitHub Bot commented on FLINK-3140: --- Github user twalthr commented on a diff in

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/1465#discussion_r47911612 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/typeinfo/RowComparator.scala --- @@ -0,0 +1,468 @@ +/* + * License

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062112#comment-15062112 ] ASF GitHub Bot commented on FLINK-3140: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread twalthr
Github user twalthr commented on a diff in the pull request: https://github.com/apache/flink/pull/1465#discussion_r47912714 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/typeinfo/RowComparator.scala --- @@ -0,0 +1,468 @@ +/* + * Licensed to

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062126#comment-15062126 ] ASF GitHub Bot commented on FLINK-3140: --- Github user twalthr commented on a diff in

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062134#comment-15062134 ] ASF GitHub Bot commented on FLINK-3140: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/1465#discussion_r47913064 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/typeinfo/RowComparator.scala --- @@ -0,0 +1,468 @@ +/* + * License

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062141#comment-15062141 ] ASF GitHub Bot commented on FLINK-3140: --- Github user tillrohrmann commented on the p

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/1465#discussion_r47913887 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/typeinfo/RowComparator.scala --- @@ -0,0 +1,468 @@ +/* + * License

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062144#comment-15062144 ] ASF GitHub Bot commented on FLINK-3140: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/1465#issuecomment-165474222 @aljoscha is absolutely right about the performance aspect. However, the key difference is that he was referring in his comment to the implementation of the `serial

[jira] [Commented] (FLINK-3160) Aggregate operator statistics by TaskManager

2015-12-17 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062178#comment-15062178 ] Stephan Ewen commented on FLINK-3160: - You mean like there is currently an aggregate v

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/1465#discussion_r47918892 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/typeinfo/NullMaskUtils.scala --- @@ -0,0 +1,92 @@ +/* + * Licensed

[GitHub] flink pull request: Set current class loader to user-class loader ...

2015-12-17 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1457#issuecomment-165484795 Good catch! The context class loaded needs to set where ever we reflectively invoke user code. We simply forgot it at that place... +1 to merge --

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062190#comment-15062190 ] ASF GitHub Bot commented on FLINK-3140: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: Set current class loader to user-class loader ...

2015-12-17 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/1457#discussion_r47918946 --- Diff: flink-clients/src/main/java/org/apache/flink/client/program/PackagedProgram.java --- @@ -574,8 +574,11 @@ private static String getEntryPointC

[GitHub] flink pull request: [FLINK-3140] NULL value data layout in Row Ser...

2015-12-17 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/1465#discussion_r47919405 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/typeinfo/NullMaskUtils.scala --- @@ -0,0 +1,92 @@ +/* + * Licensed

[jira] [Commented] (FLINK-3140) NULL value data layout in Row Serializer/Comparator

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062198#comment-15062198 ] ASF GitHub Bot commented on FLINK-3140: --- Github user tillrohrmann commented on a dif

[GitHub] flink pull request: Set current class loader to user-class loader ...

2015-12-17 Thread mjsax
Github user mjsax commented on a diff in the pull request: https://github.com/apache/flink/pull/1457#discussion_r47919597 --- Diff: flink-clients/src/main/java/org/apache/flink/client/program/PackagedProgram.java --- @@ -574,8 +574,11 @@ private static String getEntryPointClassNa

[jira] [Commented] (FLINK-3180) MemoryLogger does not log direct memory

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062201#comment-15062201 ] ASF GitHub Bot commented on FLINK-3180: --- Github user mxm commented on the pull reque

[GitHub] flink pull request: [FLINK-3180] [runtime] Log direct memory usage...

2015-12-17 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1466#issuecomment-165486967 Thanks for the pull request. Could we also include the direct memory in the `getMemoryUsageStatsAsString()` which outputs memory information on task manager startups?

[GitHub] flink pull request: [FLINK-2716] [gelly java scala] New checksum m...

2015-12-17 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/1462#discussion_r47920383 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/DataSet.java --- @@ -394,6 +396,21 @@ public long count() throws Exception {

[jira] [Commented] (FLINK-2716) Checksum method for DataSet and Graph

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062207#comment-15062207 ] ASF GitHub Bot commented on FLINK-2716: --- Github user StephanEwen commented on a diff

[GitHub] flink pull request: [FLINK-2716] [gelly java scala] New checksum m...

2015-12-17 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1462#issuecomment-165489647 I think this is a nice addition. Unsure, though, if `checksum()` should be added to the `DataSet` directly. It is a tradeoff between convenience and API overload.

[jira] [Commented] (FLINK-2716) Checksum method for DataSet and Graph

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062214#comment-15062214 ] ASF GitHub Bot commented on FLINK-2716: --- Github user StephanEwen commented on the pu

[GitHub] flink pull request: [FLINK-2716] [gelly java scala] New checksum m...

2015-12-17 Thread StephanEwen
Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/1462#discussion_r47921105 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/DataSet.java --- @@ -394,6 +396,21 @@ public long count() throws Exception {

[jira] [Commented] (FLINK-2716) Checksum method for DataSet and Graph

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062215#comment-15062215 ] ASF GitHub Bot commented on FLINK-2716: --- Github user StephanEwen commented on a diff

[jira] [Created] (FLINK-3182) TaskManagersHandler can block web frontend

2015-12-17 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-3182: -- Summary: TaskManagersHandler can block web frontend Key: FLINK-3182 URL: https://issues.apache.org/jira/browse/FLINK-3182 Project: Flink Issue Type: Bug

[jira] [Commented] (FLINK-3160) Aggregate operator statistics by TaskManager

2015-12-17 Thread Greg Hogan (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062208#comment-15062208 ] Greg Hogan commented on FLINK-3160: --- Currently when clicking a JobVertex the table expan

[GitHub] flink pull request: [FLINK-3166] [runtime] The first program in Ob...

2015-12-17 Thread greghogan
Github user greghogan commented on the pull request: https://github.com/apache/flink/pull/1464#issuecomment-165499126 I haven't found a prohibition on returning the second input to a user-defined reduce function. I think fixing this in code is preferable to documenting the problem or

[jira] [Commented] (FLINK-3166) The first program in ObjectReuseITCase has the wrong expected result, and it succeeds

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062259#comment-15062259 ] ASF GitHub Bot commented on FLINK-3166: --- Github user greghogan commented on the pull

[jira] [Commented] (FLINK-3166) The first program in ObjectReuseITCase has the wrong expected result, and it succeeds

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062276#comment-15062276 ] ASF GitHub Bot commented on FLINK-3166: --- Github user zentol commented on the pull re

[GitHub] flink pull request: [FLINK-3166] [runtime] The first program in Ob...

2015-12-17 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/1464#issuecomment-165501540 "Object reuse is configured in JavaProgramTestBase but must be set in the ExecutionEnvironmentFactory by TestEnvironment. I don't see that any of the many tests subclassi

[jira] [Created] (FLINK-3183) Display direct memory stats in web frontend

2015-12-17 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-3183: -- Summary: Display direct memory stats in web frontend Key: FLINK-3183 URL: https://issues.apache.org/jira/browse/FLINK-3183 Project: Flink Issue Type: Improvement

[jira] [Commented] (FLINK-3166) The first program in ObjectReuseITCase has the wrong expected result, and it succeeds

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062309#comment-15062309 ] ASF GitHub Bot commented on FLINK-3166: --- Github user greghogan commented on the pull

[GitHub] flink pull request: [FLINK-3166] [runtime] The first program in Ob...

2015-12-17 Thread greghogan
Github user greghogan commented on the pull request: https://github.com/apache/flink/pull/1464#issuecomment-165506005 TestEnvironment was only setting the parallelism when creating and establishing the ExecutionEnvironmentFactory: ``` public void setAsContext() {

[GitHub] flink pull request: [FLINK-3166] [runtime] The first program in Ob...

2015-12-17 Thread zentol
Github user zentol commented on the pull request: https://github.com/apache/flink/pull/1464#issuecomment-165516598 ooohhh...that's a problem. alrighty, your solution for that looks good. actually, would you mind moving that into a separate PR so we can get it in faster? this should be

[jira] [Commented] (FLINK-3166) The first program in ObjectReuseITCase has the wrong expected result, and it succeeds

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062347#comment-15062347 ] ASF GitHub Bot commented on FLINK-3166: --- Github user zentol commented on the pull re

[jira] [Commented] (FLINK-2716) Checksum method for DataSet and Graph

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062337#comment-15062337 ] ASF GitHub Bot commented on FLINK-2716: --- Github user greghogan commented on a diff i

[GitHub] flink pull request: [FLINK-2716] [gelly java scala] New checksum m...

2015-12-17 Thread greghogan
Github user greghogan commented on a diff in the pull request: https://github.com/apache/flink/pull/1462#discussion_r47931079 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/DataSet.java --- @@ -394,6 +396,21 @@ public long count() throws Exception { re

[GitHub] flink pull request: [FLINK-2716] [gelly java scala] New checksum m...

2015-12-17 Thread greghogan
Github user greghogan commented on the pull request: https://github.com/apache/flink/pull/1462#issuecomment-165524009 Were you thinking of `DataSetUtils`? That looks like a nice home for this and already includes zip-with and sampling functions. --- If your project is set up for it,

[jira] [Commented] (FLINK-2716) Checksum method for DataSet and Graph

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062397#comment-15062397 ] ASF GitHub Bot commented on FLINK-2716: --- Github user greghogan commented on the pull

[GitHub] flink pull request: [FLINK-3180] [runtime] Log direct memory usage...

2015-12-17 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/1466#issuecomment-165524534 I've added some related changes to this PR to display the direct memory usage in the web frontend as well. The current structure didn't make a lot of sense to me: #

[jira] [Commented] (FLINK-3180) MemoryLogger does not log direct memory

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062407#comment-15062407 ] ASF GitHub Bot commented on FLINK-3180: --- Github user uce commented on the pull reque

[GitHub] flink pull request: [FLINK-3181] [gelly] avoid unnecessary message...

2015-12-17 Thread vasia
GitHub user vasia opened a pull request: https://github.com/apache/flink/pull/1467 [FLINK-3181] [gelly] avoid unnecessary messages in SSSP examples and library method I added a simple check in the messaging function to avoid sending unnecessary messages during the first superstep o

[jira] [Commented] (FLINK-3181) The vertex-centric SSSP example and library method send unnecessary messages during the first superstep

2015-12-17 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062462#comment-15062462 ] ASF GitHub Bot commented on FLINK-3181: --- GitHub user vasia opened a pull request:

[jira] [Created] (FLINK-3184) Decrease Akka timeouts on cluster side to make system more responsive

2015-12-17 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-3184: Summary: Decrease Akka timeouts on cluster side to make system more responsive Key: FLINK-3184 URL: https://issues.apache.org/jira/browse/FLINK-3184 Project: Flink

[GitHub] flink pull request: [FLINK-3184] [timeouts] Decrease timeouts

2015-12-17 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/1468 [FLINK-3184] [timeouts] Decrease timeouts This PR introduces a client side timeout of 60 s and a cluster side timeout of 10 s. Both timeouts can be configured via `akka.client.timeout` and `ak

  1   2   >