[GitHub] flink pull request #5677: [hotfix] update doc of InternalTimerService.regist...

2018-03-09 Thread bowenli86
GitHub user bowenli86 opened a pull request: https://github.com/apache/flink/pull/5677 [hotfix] update doc of InternalTimerService.registerEventTimeTimer() ## What is the purpose of the change update doc of InternalTimerService.registerEventTimeTimer() ## Brief chan

[jira] [Comment Edited] (FLINK-8690) Update logical rule set to generate FlinkLogicalAggregate explicitly allow distinct agg on DataStream

2018-03-09 Thread Hequn Cheng (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394069#comment-16394069 ] Hequn Cheng edited comment on FLINK-8690 at 3/10/18 6:53 AM: -

[jira] [Commented] (FLINK-8690) Update logical rule set to generate FlinkLogicalAggregate explicitly allow distinct agg on DataStream

2018-03-09 Thread Hequn Cheng (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394069#comment-16394069 ] Hequn Cheng commented on FLINK-8690: Rename current FlinkLogicalAggregateConverter to

[jira] [Comment Edited] (FLINK-8690) Update logical rule set to generate FlinkLogicalAggregate explicitly allow distinct agg on DataStream

2018-03-09 Thread Hequn Cheng (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394069#comment-16394069 ] Hequn Cheng edited comment on FLINK-8690 at 3/10/18 6:53 AM: -

[jira] [Commented] (FLINK-8690) Update logical rule set to generate FlinkLogicalAggregate explicitly allow distinct agg on DataStream

2018-03-09 Thread Rong Rong (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394068#comment-16394068 ] Rong Rong commented on FLINK-8690: -- You are right. Haven't thought about that. I will ver

[jira] [Commented] (FLINK-8690) Update logical rule set to generate FlinkLogicalAggregate explicitly allow distinct agg on DataStream

2018-03-09 Thread Hequn Cheng (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394064#comment-16394064 ] Hequn Cheng commented on FLINK-8690: No need to worry about the {{AggregateExpandDisti

[jira] [Comment Edited] (FLINK-8690) Update logical rule set to generate FlinkLogicalAggregate explicitly allow distinct agg on DataStream

2018-03-09 Thread Rong Rong (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394046#comment-16394046 ] Rong Rong edited comment on FLINK-8690 at 3/10/18 5:54 AM: --- That

[jira] [Comment Edited] (FLINK-8690) Update logical rule set to generate FlinkLogicalAggregate explicitly allow distinct agg on DataStream

2018-03-09 Thread Rong Rong (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394046#comment-16394046 ] Rong Rong edited comment on FLINK-8690 at 3/10/18 5:52 AM: --- That

[jira] [Commented] (FLINK-8690) Update logical rule set to generate FlinkLogicalAggregate explicitly allow distinct agg on DataStream

2018-03-09 Thread Rong Rong (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394046#comment-16394046 ] Rong Rong commented on FLINK-8690: -- That should resolve our problem partially. The real r

[jira] [Commented] (FLINK-8863) Add user-defined function support in SQL Client

2018-03-09 Thread Rong Rong (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394042#comment-16394042 ] Rong Rong commented on FLINK-8863: -- Sounds good. We can deal with that in future. Thanks

[jira] [Commented] (FLINK-8655) Add a default keyspace to CassandraSink

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394012#comment-16394012 ] ASF GitHub Bot commented on FLINK-8655: --- Github user Bekreth commented on the issue:

[GitHub] flink issue #5538: [FLINK-8655] [DataSink] Added default keyspace to Cassand...

2018-03-09 Thread Bekreth
Github user Bekreth commented on the issue: https://github.com/apache/flink/pull/5538 Previously, I was trying to leverage the current annotation parsers. It would be possible to pass in a keyspace more directly into AnotationParser in the shaded Datastax component, but I'm a

[jira] [Commented] (FLINK-8690) Update logical rule set to generate FlinkLogicalAggregate explicitly allow distinct agg on DataStream

2018-03-09 Thread Hequn Cheng (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394002#comment-16394002 ] Hequn Cheng commented on FLINK-8690: Hi, we don't have to create a new logical RelNode

[jira] [Commented] (FLINK-8863) Add user-defined function support in SQL Client

2018-03-09 Thread Xingcan Cui (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393990#comment-16393990 ] Xingcan Cui commented on FLINK-8863: Yes, I agree that there may be conflicts among di

[jira] [Commented] (FLINK-8854) Mapping of SchemaValidator.deriveFieldMapping() is incorrect.

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393985#comment-16393985 ] ASF GitHub Bot commented on FLINK-8854: --- Github user xccui commented on the issue:

[GitHub] flink issue #5662: [FLINK-8854] [table] Fix schema mapping with time attribu...

2018-03-09 Thread xccui
Github user xccui commented on the issue: https://github.com/apache/flink/pull/5662 Thanks for the explanation, @twalthr! I'll update the PR and resolve the conflicts caused. ---

[jira] [Commented] (FLINK-8895) Job failed when one kafka broker shutdown

2018-03-09 Thread godfrey johnson (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393973#comment-16393973 ] godfrey johnson commented on FLINK-8895: [~StephanEwen] After the partition leader

[jira] [Commented] (FLINK-8863) Add user-defined function support in SQL Client

2018-03-09 Thread Rong Rong (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393966#comment-16393966 ] Rong Rong commented on FLINK-8863: -- Thanks [~xccui] for the quick reply. I am assuming us

[jira] [Comment Edited] (FLINK-5486) Lack of synchronization in BucketingSink#handleRestoredBucketState()

2018-03-09 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-5486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351986#comment-16351986 ] Ted Yu edited comment on FLINK-5486 at 3/10/18 1:57 AM: Can this g

[jira] [Commented] (FLINK-6924) ADD LOG(X) supported in TableAPI

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393960#comment-16393960 ] ASF GitHub Bot commented on FLINK-6924: --- Github user walterddr commented on a diff i

[GitHub] flink pull request #5638: [FLINK-6924][table]ADD LOG(X) supported in TableAP...

2018-03-09 Thread walterddr
Github user walterddr commented on a diff in the pull request: https://github.com/apache/flink/pull/5638#discussion_r173607312 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/api/scala/expressionDsl.scala --- @@ -1130,4 +1130,13 @@ object concat_ws {

[jira] [Comment Edited] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

2018-03-09 Thread yanxiaobin (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393956#comment-16393956 ] yanxiaobin edited comment on FLINK-8794 at 3/10/18 1:53 AM: Ab

[jira] [Commented] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

2018-03-09 Thread yanxiaobin (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393956#comment-16393956 ] yanxiaobin commented on FLINK-8794: --- About : 1.What I described above is that there will

[jira] [Commented] (FLINK-8863) Add user-defined function support in SQL Client

2018-03-09 Thread Xingcan Cui (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393957#comment-16393957 ] Xingcan Cui commented on FLINK-8863: Hi [~walterddr], commonly, all the required class

[jira] [Updated] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

2018-03-09 Thread yanxiaobin (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanxiaobin updated FLINK-8794: -- Issue Type: Improvement (was: Bug) > When using BucketingSink, it happens that one of the files is alwa

[jira] [Commented] (FLINK-6924) ADD LOG(X) supported in TableAPI

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393949#comment-16393949 ] ASF GitHub Bot commented on FLINK-6924: --- Github user walterddr commented on a diff i

[GitHub] flink pull request #5638: [FLINK-6924][table]ADD LOG(X) supported in TableAP...

2018-03-09 Thread walterddr
Github user walterddr commented on a diff in the pull request: https://github.com/apache/flink/pull/5638#discussion_r173606447 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/api/scala/expressionDsl.scala --- @@ -1130,4 +1130,13 @@ object concat_ws {

[jira] [Commented] (FLINK-8863) Add user-defined function support in SQL Client

2018-03-09 Thread Rong Rong (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393942#comment-16393942 ] Rong Rong commented on FLINK-8863: -- Hi [~twalthr], in the task description, there's no sp

[jira] [Commented] (FLINK-6206) Log task state transitions as warn/error for FAILURE scenarios

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393756#comment-16393756 ] ASF GitHub Bot commented on FLINK-6206: --- Github user casidiablo commented on the iss

[GitHub] flink issue #5399: [FLINK-6206] [runtime] Use LOG.error() when logging failu...

2018-03-09 Thread casidiablo
Github user casidiablo commented on the issue: https://github.com/apache/flink/pull/5399 Just to be clear, we would revert all the changes, except the ones for `flink-runtime/src/main/java/org/apache/flink/runtime/taskmanager/Task.java`, right? ---

[jira] [Commented] (FLINK-8888) Upgrade AWS SDK in flink-connector-kinesis

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393725#comment-16393725 ] ASF GitHub Bot commented on FLINK-: --- Github user kailashhd commented on the issu

[GitHub] flink issue #5663: [FLINK-8888] [Kinesis Connectors] Update the AWS SDK for ...

2018-03-09 Thread kailashhd
Github user kailashhd commented on the issue: https://github.com/apache/flink/pull/5663 Currently in flink connector we are depending only on aws-sdk-kinesis and not on aws-java-sdk-bundle and also don't depend on kinesisvideo. So by default the dependency on kinesisvideo is not inclu

[jira] [Updated] (FLINK-8913) RocksDB state backend crashes in alpine image

2018-03-09 Thread Joshua Griffith (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua Griffith updated FLINK-8913: --- Summary: RocksDB state backend crashes in alpine image (was: RocksDB state backend crashes in

[jira] [Created] (FLINK-8913) RocksDB state backend crashes in hadoop28-scala_2.11-alpine image

2018-03-09 Thread Joshua Griffith (JIRA)
Joshua Griffith created FLINK-8913: -- Summary: RocksDB state backend crashes in hadoop28-scala_2.11-alpine image Key: FLINK-8913 URL: https://issues.apache.org/jira/browse/FLINK-8913 Project: Flink

[jira] [Commented] (FLINK-6206) Log task state transitions as warn/error for FAILURE scenarios

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393392#comment-16393392 ] ASF GitHub Bot commented on FLINK-6206: --- Github user casidiablo commented on the iss

[GitHub] flink issue #5399: [FLINK-6206] [runtime] Use LOG.error() when logging failu...

2018-03-09 Thread casidiablo
Github user casidiablo commented on the issue: https://github.com/apache/flink/pull/5399 Sure, that makes sense! ---

[jira] [Closed] (FLINK-8786) SpillableSubpartitionView#getNextBuffer always sets isMoreAvailable to false when switching from spillable to spilled

2018-03-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-8786. --- > SpillableSubpartitionView#getNextBuffer always sets isMoreAvailable to false > when switching from spil

[jira] [Resolved] (FLINK-8786) SpillableSubpartitionView#getNextBuffer always sets isMoreAvailable to false when switching from spillable to spilled

2018-03-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-8786. - Resolution: Fixed Fix Version/s: 1.6.0 Fixed in - 1.5.0 via 835adcc373ce169f202055e9b4f9d

[jira] [Commented] (FLINK-8755) SpilledSubpartitionView wrongly relys on the backlog for determining whether more data is available

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393374#comment-16393374 ] ASF GitHub Bot commented on FLINK-8755: --- Github user asfgit closed the pull request

[jira] [Closed] (FLINK-8755) SpilledSubpartitionView wrongly relys on the backlog for determining whether more data is available

2018-03-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-8755. --- > SpilledSubpartitionView wrongly relys on the backlog for determining whether > more data is available >

[jira] [Resolved] (FLINK-8755) SpilledSubpartitionView wrongly relys on the backlog for determining whether more data is available

2018-03-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-8755. - Resolution: Fixed Fix Version/s: 1.6.0 Fixed in - 1.5.0 via 5c7457aa2aece89b77e9a9402cc51

[GitHub] flink pull request #5581: [FLINK-8755][FLINK-8786][network] fix two bugs in ...

2018-03-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/5581 ---

[jira] [Commented] (FLINK-6206) Log task state transitions as warn/error for FAILURE scenarios

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393357#comment-16393357 ] ASF GitHub Bot commented on FLINK-6206: --- Github user StephanEwen commented on the is

[GitHub] flink issue #5399: [FLINK-6206] [runtime] Use LOG.error() when logging failu...

2018-03-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/5399 Do you think it would be a good approach to promote that to ERROR logging on the TaskManager (who actually encountered the exception) and leave the JobManager at INFO (who only handles a regular

[jira] [Commented] (FLINK-8888) Upgrade AWS SDK in flink-connector-kinesis

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393327#comment-16393327 ] ASF GitHub Bot commented on FLINK-: --- Github user bowenli86 commented on the issu

[GitHub] flink issue #5663: [FLINK-8888] [Kinesis Connectors] Update the AWS SDK for ...

2018-03-09 Thread bowenli86
Github user bowenli86 commented on the issue: https://github.com/apache/flink/pull/5663 Is your testing Flink job both reading from and writing to Kinesis, aka both KCL and KPL are tested? If so, +1 ---

[jira] [Commented] (FLINK-8364) Add iterator() to ListState which returns empty iterator when it has no value

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393314#comment-16393314 ] ASF GitHub Bot commented on FLINK-8364: --- Github user bowenli86 commented on the issu

[GitHub] flink issue #5356: [FLINK-8364][state backend] Add iterator() to ListState w...

2018-03-09 Thread bowenli86
Github user bowenli86 commented on the issue: https://github.com/apache/flink/pull/5356 hmmm I think you are right, this actually might be a non-issue in the first place ---

[jira] [Commented] (FLINK-8515) update RocksDBMapState to replace deprecated remove() with delete()

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393312#comment-16393312 ] ASF GitHub Bot commented on FLINK-8515: --- Github user bowenli86 closed the pull reque

[GitHub] flink pull request #5365: [FLINK-8515] update RocksDBMapState to replace dep...

2018-03-09 Thread bowenli86
Github user bowenli86 closed the pull request at: https://github.com/apache/flink/pull/5365 ---

[jira] [Assigned] (FLINK-8897) Rowtime materialization causes "mismatched type" AssertionError

2018-03-09 Thread Timo Walther (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timo Walther reassigned FLINK-8897: --- Assignee: Timo Walther > Rowtime materialization causes "mismatched type" AssertionError > --

[jira] [Closed] (FLINK-8903) Error calculation based on rolling window in table API and SQL API

2018-03-09 Thread Fabian Hueske (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fabian Hueske closed FLINK-8903. Resolution: Not A Problem Hi [~lilizhao], processing time is inherently non-deterministic, i.e., di

[jira] [Commented] (FLINK-6206) Log task state transitions as warn/error for FAILURE scenarios

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393157#comment-16393157 ] ASF GitHub Bot commented on FLINK-6206: --- Github user casidiablo commented on the iss

[GitHub] flink issue #5399: [FLINK-6206] [runtime] Use LOG.error() when logging failu...

2018-03-09 Thread casidiablo
Github user casidiablo commented on the issue: https://github.com/apache/flink/pull/5399 The reason I proposed the change is that any unexpected behavior, even when you can recover from it, can be indicative of a potential bug/misconfiguration. INFO logs are by far noisier than errors

[jira] [Commented] (FLINK-8910) Introduce automated end-to-end test for local recovery (including sticky scheduling)

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393140#comment-16393140 ] ASF GitHub Bot commented on FLINK-8910: --- Github user StefanRRichter commented on the

[jira] [Commented] (FLINK-8910) Introduce automated end-to-end test for local recovery (including sticky scheduling)

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393139#comment-16393139 ] ASF GitHub Bot commented on FLINK-8910: --- GitHub user StefanRRichter opened a pull re

[GitHub] flink issue #5676: [FLINK-8910][tests] Automated end-to-end test for local r...

2018-03-09 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/5676 CC @aljoscha ---

[GitHub] flink pull request #5676: [FLINK-8910][tests] Automated end-to-end test for ...

2018-03-09 Thread StefanRRichter
GitHub user StefanRRichter opened a pull request: https://github.com/apache/flink/pull/5676 [FLINK-8910][tests] Automated end-to-end test for local recovery and sticky scheduling This PR adds an automated end-to-end test for the local recovery feature, which also includes sticky al

[jira] [Updated] (FLINK-8740) Job-level metrics lost during job re-submission in HA mode

2018-03-09 Thread Aljoscha Krettek (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aljoscha Krettek updated FLINK-8740: Priority: Critical (was: Blocker) > Job-level metrics lost during job re-submission in HA m

[jira] [Updated] (FLINK-8740) Job-level metrics lost during job re-submission in HA mode

2018-03-09 Thread Aljoscha Krettek (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aljoscha Krettek updated FLINK-8740: Priority: Blocker (was: Critical) > Job-level metrics lost during job re-submission in HA m

[jira] [Commented] (FLINK-8357) enable rolling in default log settings

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393051#comment-16393051 ] ASF GitHub Bot commented on FLINK-8357: --- Github user StephanEwen commented on the is

[GitHub] flink issue #5371: [FLINK-8357] [conf] Enable rolling in default log setting...

2018-03-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/5371 Two more questions about this: 1. There is code in the shell scripts that rotates log file each time you start / stop the cluster, with .0 /.1/.2/etc suffixes to the log files. Have yo

[jira] [Commented] (FLINK-8755) SpilledSubpartitionView wrongly relys on the backlog for determining whether more data is available

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393028#comment-16393028 ] ASF GitHub Bot commented on FLINK-8755: --- Github user StephanEwen commented on the is

[GitHub] flink issue #5581: [FLINK-8755][FLINK-8786][network] fix two bugs in spilled...

2018-03-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/5581 Thanks for the patch. Looks good, well tested and reviewed (thanks, @pnowojski). Merging this... ---

[jira] [Commented] (FLINK-8599) Improve the failure behavior of the FileInputFormat for bad files

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393002#comment-16393002 ] ASF GitHub Bot commented on FLINK-8599: --- Github user ChengzhiZhao commented on the i

[GitHub] flink issue #5521: [FLINK-8599] Improve the failure behavior of the FileInpu...

2018-03-09 Thread ChengzhiZhao
Github user ChengzhiZhao commented on the issue: https://github.com/apache/flink/pull/5521 @StephanEwen @kl0u Thanks for you feedback, I will put an option for user to choose ---

[jira] [Commented] (FLINK-6206) Log task state transitions as warn/error for FAILURE scenarios

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392999#comment-16392999 ] ASF GitHub Bot commented on FLINK-6206: --- Github user StephanEwen commented on the is

[GitHub] flink issue #5399: [FLINK-6206] [runtime] Use LOG.error() when logging failu...

2018-03-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/5399 I am unsure about this change, so let's discuss the pros and cons a bit. So far, this purposefully logs on *INFO* so far, because from the JobManager's perspective, a task failing and rec

[jira] [Commented] (FLINK-8515) update RocksDBMapState to replace deprecated remove() with delete()

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392978#comment-16392978 ] ASF GitHub Bot commented on FLINK-8515: --- Github user StephanEwen commented on the is

[GitHub] flink issue #5365: [FLINK-8515] update RocksDBMapState to replace deprecated...

2018-03-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/5365 This change has already gotten in via another commit. @bowenli86 can you close this PR? Thank you for the contribution! ---

[jira] [Closed] (FLINK-8898) Materialize time indicators in conditions of LogicalFilter

2018-03-09 Thread Hequn Cheng (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hequn Cheng closed FLINK-8898. -- Resolution: Duplicate > Materialize time indicators in conditions of LogicalFilter > ---

[jira] [Commented] (FLINK-8898) Materialize time indicators in conditions of LogicalFilter

2018-03-09 Thread Hequn Cheng (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392975#comment-16392975 ] Hequn Cheng commented on FLINK-8898: duplicated with [FLINK-8897|https://issues.apache

[GitHub] flink issue #5185: [FLINK-8297] [flink-rocksdb] Optionally store elements of...

2018-03-09 Thread je-ik
Github user je-ik commented on the issue: https://github.com/apache/flink/pull/5185 @StephanEwen I think it should be configurable. As Aljoscha pointed out, it is needed to ensure that these two representations have the same serialized form in checkpoints, because that way users can s

[jira] [Commented] (FLINK-8297) RocksDBListState stores whole list in single byte[]

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392976#comment-16392976 ] ASF GitHub Bot commented on FLINK-8297: --- Github user je-ik commented on the issue:

[jira] [Commented] (FLINK-6214) WindowAssigners do not allow negative offsets

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392966#comment-16392966 ] ASF GitHub Bot commented on FLINK-6214: --- Github user StephanEwen commented on the is

[GitHub] flink issue #5376: [FLINK-6214] WindowAssigners do not allow negative offset...

2018-03-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/5376 Ah, one problem: What happens if the offset is larger (by absolute value) than the window length? Then the offset would still be negative with this change. ---

[jira] [Commented] (FLINK-6214) WindowAssigners do not allow negative offsets

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392962#comment-16392962 ] ASF GitHub Bot commented on FLINK-6214: --- Github user StephanEwen commented on the is

[GitHub] flink issue #5376: [FLINK-6214] WindowAssigners do not allow negative offset...

2018-03-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/5376 If we decide that we want to support negative offsets, this looks like a good implementation. @aljoscha what is your take, should we support negative offsets? ---

[jira] [Commented] (FLINK-8364) Add iterator() to ListState which returns empty iterator when it has no value

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392961#comment-16392961 ] ASF GitHub Bot commented on FLINK-8364: --- Github user StephanEwen commented on the is

[jira] [Commented] (FLINK-8297) RocksDBListState stores whole list in single byte[]

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392957#comment-16392957 ] ASF GitHub Bot commented on FLINK-8297: --- Github user StephanEwen commented on the is

[GitHub] flink issue #5356: [FLINK-8364][state backend] Add iterator() to ListState w...

2018-03-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/5356 I am wondering whether this discussion is a bit confused. All state facing the user in the APIs already has the behavior that there is no `null`, but only empty iterators. That's because

[GitHub] flink issue #5185: [FLINK-8297] [flink-rocksdb] Optionally store elements of...

2018-03-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/5185 Is the general thought to always have list state as a map, or only fallback to that way if the lists cross a certain threshold? Since list state backs many common operations (non aggregat

[jira] [Commented] (FLINK-8822) RotateLogFile may not work well when sed version is below 4.2

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392953#comment-16392953 ] ASF GitHub Bot commented on FLINK-8822: --- Github user zentol commented on the issue:

[GitHub] flink issue #5609: [FLINK-8822] RotateLogFile may not work well when sed ver...

2018-03-09 Thread zentol
Github user zentol commented on the issue: https://github.com/apache/flink/pull/5609 ehh...probably. The PR has a point as `sed --help` (v4.2.2) does not list the `-E` option. If we want to be super safe we could check that `sed -r` is supported and use `-E` as a backup. -

[jira] [Updated] (FLINK-8897) Rowtime materialization causes "mismatched type" AssertionError

2018-03-09 Thread Timo Walther (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timo Walther updated FLINK-8897: Fix Version/s: 1.5.0 > Rowtime materialization causes "mismatched type" AssertionError > ---

[jira] [Updated] (FLINK-8897) Rowtime materialization causes "mismatched type" AssertionError

2018-03-09 Thread Timo Walther (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timo Walther updated FLINK-8897: Priority: Blocker (was: Major) > Rowtime materialization causes "mismatched type" AssertionError >

[jira] [Commented] (FLINK-8854) Mapping of SchemaValidator.deriveFieldMapping() is incorrect.

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392937#comment-16392937 ] ASF GitHub Bot commented on FLINK-8854: --- Github user twalthr commented on the issue:

[GitHub] flink issue #5662: [FLINK-8854] [table] Fix schema mapping with time attribu...

2018-03-09 Thread twalthr
Github user twalthr commented on the issue: https://github.com/apache/flink/pull/5662 Thanks for the comments @xccui. It's never to late for feedback. Sorry, maybe I merged this too quickly. We still need to call `builder.forJsonSchema()` if the schema contains a `proctime` attribute.

[jira] [Updated] (FLINK-8912) Web UI does not render error messages correctly in FLIP-6 mode

2018-03-09 Thread Gary Yao (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Yao updated FLINK-8912: Labels: flip6 (was: ) > Web UI does not render error messages correctly in FLIP-6 mode > --

[jira] [Updated] (FLINK-8909) pyflink.sh not working with yarn

2018-03-09 Thread Gary Yao (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Yao updated FLINK-8909: Affects Version/s: 1.4.2 > pyflink.sh not working with yarn > > >

[jira] [Commented] (FLINK-8854) Mapping of SchemaValidator.deriveFieldMapping() is incorrect.

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392933#comment-16392933 ] ASF GitHub Bot commented on FLINK-8854: --- Github user twalthr commented on a diff in

[GitHub] flink pull request #5662: [FLINK-8854] [table] Fix schema mapping with time ...

2018-03-09 Thread twalthr
Github user twalthr commented on a diff in the pull request: https://github.com/apache/flink/pull/5662#discussion_r173465435 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/descriptors/SchemaValidator.scala --- @@ -198,14 +205,20 @@ object SchemaValidat

[jira] [Created] (FLINK-8912) Web UI does not render error messages correctly in FLIP-6 mode

2018-03-09 Thread Gary Yao (JIRA)
Gary Yao created FLINK-8912: --- Summary: Web UI does not render error messages correctly in FLIP-6 mode Key: FLINK-8912 URL: https://issues.apache.org/jira/browse/FLINK-8912 Project: Flink Issue Typ

[jira] [Commented] (FLINK-8822) RotateLogFile may not work well when sed version is below 4.2

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392919#comment-16392919 ] ASF GitHub Bot commented on FLINK-8822: --- Github user StephanEwen commented on the is

[GitHub] flink issue #5609: [FLINK-8822] RotateLogFile may not work well when sed ver...

2018-03-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/5609 @zentol What do you think? Is this safe to merge? ---

[jira] [Commented] (FLINK-8655) Add a default keyspace to CassandraSink

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392915#comment-16392915 ] ASF GitHub Bot commented on FLINK-8655: --- Github user StephanEwen commented on the is

[GitHub] flink issue #5538: [FLINK-8655] [DataSink] Added default keyspace to Cassand...

2018-03-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/5538 I see the need for the feature, but I am skeptical concerning the implementation. This PR reflectively modifies the contents of a String. This is prone to cause problems, for multiple re

[jira] [Commented] (FLINK-8854) Mapping of SchemaValidator.deriveFieldMapping() is incorrect.

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392913#comment-16392913 ] ASF GitHub Bot commented on FLINK-8854: --- Github user xccui commented on a diff in th

[jira] [Commented] (FLINK-8714) Suggest new users to use env.readTextFile method with 2 arguments (using the charset), not to rely on system charset (which varies across environments)

2018-03-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-8714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392912#comment-16392912 ] ASF GitHub Bot commented on FLINK-8714: --- Github user StephanEwen commented on the is

[GitHub] flink pull request #5662: [FLINK-8854] [table] Fix schema mapping with time ...

2018-03-09 Thread xccui
Github user xccui commented on a diff in the pull request: https://github.com/apache/flink/pull/5662#discussion_r173455747 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/descriptors/SchemaValidator.scala --- @@ -198,14 +205,20 @@ object SchemaValidator

  1   2   3   >