[jira] [Commented] (FLINK-4256) Fine-grained recovery

2016-08-09 Thread wenlong.lyu (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413090#comment-15413090 ] wenlong.lyu commented on FLINK-4256: thanks for explaining, you are right about pre-co

[GitHub] flink issue #2330: FLINK-4311 Fixed several problems in TableInputFormat

2016-08-09 Thread nielsbasjes
Github user nielsbasjes commented on the issue: https://github.com/apache/flink/pull/2330 Question: Is this change good? Or do you have more things that I need to change before it can be committed? --- If your project is set up for it, you can reply to this email and have your re

[jira] [Commented] (FLINK-4311) TableInputFormat fails when reused on next split

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413108#comment-15413108 ] ASF GitHub Bot commented on FLINK-4311: --- Github user nielsbasjes commented on the is

[jira] [Resolved] (FLINK-4332) Savepoint Serializer mixed read()/readFully()

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-4332. - Resolution: Fixed Fixed in - 1.2.0 via 3f3bab10b9ca68eb31a7ef5a31e49145b51006fd - 1.1.1 via 19

[jira] [Closed] (FLINK-4320) Fix misleading ScheduleMode names

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-4320. --- > Fix misleading ScheduleMode names > - > > Key: FLINK-432

[jira] [Resolved] (FLINK-4320) Fix misleading ScheduleMode names

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-4320. - Resolution: Fixed Fixed via cd98e85ddd3c35e5900713266fc38916b53f172d (commit has wrong JIRA issue

[jira] [Closed] (FLINK-4332) Savepoint Serializer mixed read()/readFully()

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-4332. --- > Savepoint Serializer mixed read()/readFully() > - > >

[jira] [Closed] (FLINK-4333) Name mixup in Savepoint versions

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-4333. --- > Name mixup in Savepoint versions > > > Key: FLINK-4333

[jira] [Resolved] (FLINK-4333) Name mixup in Savepoint versions

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-4333. - Resolution: Fixed Fixed via 9a84b04f076f9cdc2fd0037fcc89f31edc596bdd > Name mixup in Savepoint ve

[jira] [Commented] (FLINK-4334) Shaded Hadoop1 jar not fully excluded in Quickstart

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413199#comment-15413199 ] ASF GitHub Bot commented on FLINK-4334: --- Github user StephanEwen commented on the is

[GitHub] flink issue #2341: [FLINK-4334] [quickstarts] Correctly exclude hadoop1 in q...

2016-08-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2341 Thanks, this makes a lot of sense. We should merge this for Flink 1.1.1 For Flink 1.2.0, we should actually fix the root issue, which is that the hadoop1 artifact has a scala suffix e

[jira] [Assigned] (FLINK-4329) Streaming File Source Must Correctly Handle Timestamps/Watermarks

2016-08-09 Thread Kostas Kloudas (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kostas Kloudas reassigned FLINK-4329: - Assignee: Kostas Kloudas > Streaming File Source Must Correctly Handle Timestamps/Waterma

[jira] [Updated] (FLINK-4334) Shaded Hadoop1 jar not fully excluded in Quickstart

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen updated FLINK-4334: Fix Version/s: 1.1.1 > Shaded Hadoop1 jar not fully excluded in Quickstart > ---

[jira] [Created] (FLINK-4337) Remove unnecessary Scala suffix from Hadoop1 artifact

2016-08-09 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-4337: --- Summary: Remove unnecessary Scala suffix from Hadoop1 artifact Key: FLINK-4337 URL: https://issues.apache.org/jira/browse/FLINK-4337 Project: Flink Issue Type:

[jira] [Commented] (FLINK-4326) Flink start-up scripts should optionally start services on the foreground

2016-08-09 Thread JIRA
[ https://issues.apache.org/jira/browse/FLINK-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413252#comment-15413252 ] Ismaël Mejía commented on FLINK-4326: - Well it is good to know that there is interest

[GitHub] flink pull request #2342: FLINK-4253 - Rename "recovery.mode" config key to ...

2016-08-09 Thread ramkrish86
GitHub user ramkrish86 opened a pull request: https://github.com/apache/flink/pull/2342 FLINK-4253 - Rename "recovery.mode" config key to "high-availability" (Ram) Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into c

[GitHub] flink issue #2338: [FLINK-4316] [core] [hadoop compatibility] Make flink-cor...

2016-08-09 Thread rmetzger
Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/2338 +1 to merge, once the failing tests are fixed. I think the exception to the API stability check is okay in this case. The class is still in the same java package. This are the test

[jira] [Commented] (FLINK-4253) Rename "recovery.mode" config key to "high-availability"

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413258#comment-15413258 ] ASF GitHub Bot commented on FLINK-4253: --- GitHub user ramkrish86 opened a pull reques

[jira] [Commented] (FLINK-4316) Make flink-core independent of Hadoop

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413259#comment-15413259 ] ASF GitHub Bot commented on FLINK-4316: --- Github user rmetzger commented on the issue

[jira] [Commented] (FLINK-3298) Streaming connector for ActiveMQ

2016-08-09 Thread Robert Metzger (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413270#comment-15413270 ] Robert Metzger commented on FLINK-3298: --- I've started a discussion on this issue on

[GitHub] flink issue #2338: [FLINK-4316] [core] [hadoop compatibility] Make flink-cor...

2016-08-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2338 Good point. I'll fix those tests... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[jira] [Commented] (FLINK-4316) Make flink-core independent of Hadoop

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413275#comment-15413275 ] ASF GitHub Bot commented on FLINK-4316: --- Github user StephanEwen commented on the is

[jira] [Created] (FLINK-4338) Implement Slot Pool

2016-08-09 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-4338: --- Summary: Implement Slot Pool Key: FLINK-4338 URL: https://issues.apache.org/jira/browse/FLINK-4338 Project: Flink Issue Type: Sub-task Components: Cl

[jira] [Updated] (FLINK-4338) Implement Slot Pool

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen updated FLINK-4338: Issue Type: Bug (was: Sub-task) Parent: (was: FLINK-4319) > Implement Slot Pool > -

[jira] [Updated] (FLINK-4338) Implement Slot Pool

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen updated FLINK-4338: Issue Type: New Feature (was: Bug) > Implement Slot Pool > --- > >

[jira] [Created] (FLINK-4339) Implement Slot Pool Core

2016-08-09 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-4339: --- Summary: Implement Slot Pool Core Key: FLINK-4339 URL: https://issues.apache.org/jira/browse/FLINK-4339 Project: Flink Issue Type: Sub-task Component

[jira] [Commented] (FLINK-4072) EventTimeWindowCheckpointingITCase.testSlidingTimeWindow fails on Travis

2016-08-09 Thread Robert Metzger (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1541#comment-1541 ] Robert Metzger commented on FLINK-4072: --- Once again: https://s3.amazonaws.com/archi

[jira] [Comment Edited] (FLINK-4072) EventTimeWindowCheckpointingITCase.testSlidingTimeWindow fails on Travis

2016-08-09 Thread Robert Metzger (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1541#comment-1541 ] Robert Metzger edited comment on FLINK-4072 at 8/9/16 10:19 AM:

[GitHub] flink issue #2323: [FLINK-2090] toString of CollectionInputFormat takes long...

2016-08-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2323 Looks good, merging this... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[jira] [Commented] (FLINK-2090) toString of CollectionInputFormat takes long time when the collection is huge

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413355#comment-15413355 ] ASF GitHub Bot commented on FLINK-2090: --- Github user StephanEwen commented on the is

[jira] [Commented] (FLINK-2090) toString of CollectionInputFormat takes long time when the collection is huge

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413358#comment-15413358 ] ASF GitHub Bot commented on FLINK-2090: --- Github user mushketyk commented on the issu

[GitHub] flink issue #2323: [FLINK-2090] toString of CollectionInputFormat takes long...

2016-08-09 Thread mushketyk
Github user mushketyk commented on the issue: https://github.com/apache/flink/pull/2323 Awesome! Thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[jira] [Resolved] (FLINK-4334) Shaded Hadoop1 jar not fully excluded in Quickstart

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-4334. - Resolution: Fixed Fix Version/s: (was: 1.1.1) 1.1.2 Fixed for 1.1.2

[jira] [Closed] (FLINK-4334) Shaded Hadoop1 jar not fully excluded in Quickstart

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-4334. --- > Shaded Hadoop1 jar not fully excluded in Quickstart > --

[jira] [Assigned] (FLINK-4337) Remove unnecessary Scala suffix from Hadoop1 artifact

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen reassigned FLINK-4337: --- Assignee: Stephan Ewen > Remove unnecessary Scala suffix from Hadoop1 artifact >

[GitHub] flink pull request #2343: [FLINK-4337] [build] Remove unnecessary Scala Suff...

2016-08-09 Thread StephanEwen
GitHub user StephanEwen opened a pull request: https://github.com/apache/flink/pull/2343 [FLINK-4337] [build] Remove unnecessary Scala Suffix from Hadoop 1 shaded artifact The shaded Hadoop 1 dependency was versioned with a Scala Version Suffix, even though it does not depend on Sc

[GitHub] flink issue #2341: [FLINK-4334] [quickstarts] Correctly exclude hadoop1 in q...

2016-08-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2341 I merged this into the 1.1 release branch. For the 1.2 releases, we should use this fix: #2343 --- If your project is set up for it, you can reply to this email and have your reply appear on

[jira] [Commented] (FLINK-4337) Remove unnecessary Scala suffix from Hadoop1 artifact

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413426#comment-15413426 ] ASF GitHub Bot commented on FLINK-4337: --- GitHub user StephanEwen opened a pull reque

[GitHub] flink issue #2341: [FLINK-4334] [quickstarts] Correctly exclude hadoop1 in q...

2016-08-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2341 @rehevkor5 Since the Apache bot does not automatically close this pull request, could you close it manually? --- If your project is set up for it, you can reply to this email and have your reply

[jira] [Commented] (FLINK-4334) Shaded Hadoop1 jar not fully excluded in Quickstart

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413427#comment-15413427 ] ASF GitHub Bot commented on FLINK-4334: --- Github user StephanEwen commented on the is

[jira] [Commented] (FLINK-4334) Shaded Hadoop1 jar not fully excluded in Quickstart

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413429#comment-15413429 ] ASF GitHub Bot commented on FLINK-4334: --- Github user StephanEwen commented on the is

[GitHub] flink pull request #2344: [hotfix][build] Remove Scala suffix from Hadoop1 s...

2016-08-09 Thread rmetzger
GitHub user rmetzger opened a pull request: https://github.com/apache/flink/pull/2344 [hotfix][build] Remove Scala suffix from Hadoop1 shading project It seems that the hadoop1 shaded artifact has a scala suffix, which is not needed anymore. You can merge this pull request into a G

[jira] [Commented] (FLINK-4282) Add Offset Parameter to WindowAssigners

2016-08-09 Thread Aljoscha Krettek (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413471#comment-15413471 ] Aljoscha Krettek commented on FLINK-4282: - In my opinion the offset parameter shou

[GitHub] flink issue #2251: [FLINK-4212] [scripts] Lock PID file when starting daemon...

2016-08-09 Thread greghogan
Github user greghogan commented on the issue: https://github.com/apache/flink/pull/2251 Merging this ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[jira] [Commented] (FLINK-4212) Lock PID file when starting daemons

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413473#comment-15413473 ] ASF GitHub Bot commented on FLINK-4212: --- Github user greghogan commented on the issu

[jira] [Created] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-4340: --- Summary: Remove RocksDB Semi-Async Checkpoint Mode Key: FLINK-4340 URL: https://issues.apache.org/jira/browse/FLINK-4340 Project: Flink Issue Type: Imp

[GitHub] flink issue #2002: Support for bz2 compression in flink-core

2016-08-09 Thread mtanski
Github user mtanski commented on the issue: https://github.com/apache/flink/pull/2002 Now that 1.1 one is out, is it possible to get this in? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] flink pull request #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint ...

2016-08-09 Thread aljoscha
GitHub user aljoscha opened a pull request: https://github.com/apache/flink/pull/2345 [FLINK-4340] Remove RocksDB Semi-Async Checkpoint Mode This was causing to many problems and is also incompatible with the upcoming key-group/sharding changes. R: @StephanEwen for revie

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413493#comment-15413493 ] ASF GitHub Bot commented on FLINK-4340: --- GitHub user aljoscha opened a pull request:

[GitHub] flink issue #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/2345 Technically, `HDFSCopyFromLocal` and `HDFSCopyToLocal` are now unused. Should we remove them? They might be useful for some stuff in the future. --- If your project is set up for it, you can reply

[GitHub] flink pull request #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint ...

2016-08-09 Thread aljoscha
GitHub user aljoscha reopened a pull request: https://github.com/apache/flink/pull/2345 [FLINK-4340] Remove RocksDB Semi-Async Checkpoint Mode This was causing to many problems and is also incompatible with the upcoming key-group/sharding changes. R: @StephanEwen for rev

[GitHub] flink pull request #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint ...

2016-08-09 Thread aljoscha
Github user aljoscha closed the pull request at: https://github.com/apache/flink/pull/2345 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413497#comment-15413497 ] ASF GitHub Bot commented on FLINK-4340: --- Github user aljoscha closed the pull reques

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413496#comment-15413496 ] ASF GitHub Bot commented on FLINK-4340: --- Github user aljoscha commented on the issue

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413498#comment-15413498 ] ASF GitHub Bot commented on FLINK-4340: --- GitHub user aljoscha reopened a pull reques

[GitHub] flink pull request #2251: [FLINK-4212] [scripts] Lock PID file when starting...

2016-08-09 Thread greghogan
Github user greghogan closed the pull request at: https://github.com/apache/flink/pull/2251 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[jira] [Commented] (FLINK-4212) Lock PID file when starting daemons

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413516#comment-15413516 ] ASF GitHub Bot commented on FLINK-4212: --- Github user greghogan closed the pull reque

[jira] [Closed] (FLINK-4212) Lock PID file when starting daemons

2016-08-09 Thread Greg Hogan (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Hogan closed FLINK-4212. - Resolution: Implemented Implemented in 46b427fac9cfceca7839fc93f06ba758101f4fee > Lock PID file when star

[jira] [Resolved] (FLINK-2090) toString of CollectionInputFormat takes long time when the collection is huge

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-2090. - Resolution: Fixed Fix Version/s: 1.2.0 Fixed via b5d58934d7124e0076e588e74485a60e7c1f484b

[GitHub] flink pull request #2323: [FLINK-2090] toString of CollectionInputFormat tak...

2016-08-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/2323 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] flink issue #2231: [FLINK-4035] Bump Kafka producer in Kafka sink to Kafka 0...

2016-08-09 Thread rmetzger
Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/2231 Sorry for joining this discussion late. I've been on vacation. I also stumbled across the code duplicates. I'll check out the code from this pull request and see if there's a good way of re-using

[jira] [Commented] (FLINK-4035) Bump Kafka producer in Kafka sink to Kafka 0.10.0.0

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413518#comment-15413518 ] ASF GitHub Bot commented on FLINK-4035: --- Github user rmetzger commented on the issue

[jira] [Closed] (FLINK-2090) toString of CollectionInputFormat takes long time when the collection is huge

2016-08-09 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-2090. --- > toString of CollectionInputFormat takes long time when the collection is huge >

[jira] [Commented] (FLINK-4282) Add Offset Parameter to WindowAssigners

2016-08-09 Thread Aditi Viswanathan (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413519#comment-15413519 ] Aditi Viswanathan commented on FLINK-4282: -- Also when we want to assign time zone

[jira] [Commented] (FLINK-2090) toString of CollectionInputFormat takes long time when the collection is huge

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413521#comment-15413521 ] ASF GitHub Bot commented on FLINK-2090: --- Github user asfgit closed the pull request

[GitHub] flink issue #2002: Support for bz2 compression in flink-core

2016-08-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2002 I think this looks good. +1 to merge. As a followup, we can upgrade the `commons-compression` version via dependency management (if it is backwards compatible, which apache commons libs

[GitHub] flink issue #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2345 @gyfora You mean if the "full async" is slower than the "semi async"? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413544#comment-15413544 ] ASF GitHub Bot commented on FLINK-4340: --- Github user StephanEwen commented on the is

[GitHub] flink issue #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2345 The "full async" takes more time but runs completely in the background, so performs better in most cases than "semi async". --- If your project is set up for it, you can reply to this email and

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413537#comment-15413537 ] ASF GitHub Bot commented on FLINK-4340: --- Github user gyfora commented on the issue:

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413542#comment-15413542 ] ASF GitHub Bot commented on FLINK-4340: --- Github user StephanEwen commented on the is

[GitHub] flink issue #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread gyfora
Github user gyfora commented on the issue: https://github.com/apache/flink/pull/2345 Some of the benefits we lose on restore. Especially for very large states this can be pretty serious. Maybe this is required for the sharding to some extent but I don't see this as completely

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413547#comment-15413547 ] ASF GitHub Bot commented on FLINK-4340: --- Github user gyfora commented on the issue:

[GitHub] flink issue #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread gyfora
Github user gyfora commented on the issue: https://github.com/apache/flink/pull/2345 But you are right it is probably more important to keep the latency down for the running programs, and for that the fully async seems to be strictly better --- If your project is set up for it, you

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413548#comment-15413548 ] ASF GitHub Bot commented on FLINK-4340: --- Github user gyfora commented on the issue:

[GitHub] flink issue #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2345 +1 for this There seems to be an issue with the RocksDB backup engine, to we should probably discourage that mode even in current releases. I would also remove the `HDFSCopyFromL

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413536#comment-15413536 ] ASF GitHub Bot commented on FLINK-4340: --- Github user StephanEwen commented on the is

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413563#comment-15413563 ] ASF GitHub Bot commented on FLINK-4340: --- Github user gyfora commented on the issue:

[jira] [Commented] (FLINK-4326) Flink start-up scripts should optionally start services on the foreground

2016-08-09 Thread Greg Hogan (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413551#comment-15413551 ] Greg Hogan commented on FLINK-4326: --- Using {{exec}} solves the problem of not knowing th

[GitHub] flink issue #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/2345 Jip, that's also good. 😃 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413575#comment-15413575 ] ASF GitHub Bot commented on FLINK-4340: --- Github user aljoscha commented on the issue

[GitHub] flink issue #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread gyfora
Github user gyfora commented on the issue: https://github.com/apache/flink/pull/2345 Good thing about the way fully async checkpoints are restored though is that it is very trivial to insert some state adaptor code :) --- If your project is set up for it, you can reply to this email

[GitHub] flink issue #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread gyfora
Github user gyfora commented on the issue: https://github.com/apache/flink/pull/2345 But I wonder what would happen in a scenario with a lot of states: Semi async: short local copy time at every snapshot + very fast restore Fully async: no copy time + very slow restore (put

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413579#comment-15413579 ] ASF GitHub Bot commented on FLINK-4340: --- Github user gyfora commented on the issue:

[GitHub] flink issue #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread gyfora
Github user gyfora commented on the issue: https://github.com/apache/flink/pull/2345 Hi, Isn't this way of checkpointing is much much slower then the semi async version? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] flink issue #2345: [FLINK-4340] Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread wenlong88
Github user wenlong88 commented on the issue: https://github.com/apache/flink/pull/2345 @aljoscha we use rocksdb checkpoint machanism to do the semi-async checkpoint, which use hard link to make checkpoint, cost quite a little IO and time in synchronized phrase. This works well even w

[jira] [Commented] (FLINK-4340) Remove RocksDB Semi-Async Checkpoint Mode

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413596#comment-15413596 ] ASF GitHub Bot commented on FLINK-4340: --- Github user wenlong88 commented on the issu

[GitHub] flink pull request #2341: [FLINK-4334] [quickstarts] Correctly exclude hadoo...

2016-08-09 Thread rehevkor5
Github user rehevkor5 closed the pull request at: https://github.com/apache/flink/pull/2341 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[jira] [Commented] (FLINK-4334) Shaded Hadoop1 jar not fully excluded in Quickstart

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413638#comment-15413638 ] ASF GitHub Bot commented on FLINK-4334: --- Github user rehevkor5 closed the pull reque

[GitHub] flink issue #2341: [FLINK-4334] [quickstarts] Correctly exclude hadoop1 in q...

2016-08-09 Thread rehevkor5
Github user rehevkor5 commented on the issue: https://github.com/apache/flink/pull/2341 You got it @StephanEwen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[jira] [Commented] (FLINK-4334) Shaded Hadoop1 jar not fully excluded in Quickstart

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413637#comment-15413637 ] ASF GitHub Bot commented on FLINK-4334: --- Github user rehevkor5 commented on the issu

[GitHub] flink issue #2305: [FLINK-4271] [DataStreamAPI] Enable CoGroupedStreams and ...

2016-08-09 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2305 I just thought of a different trick: We could add a second variant of the `apply(..)` function (for example called `with(...)` as in the DataSet API) and have the proper return type there (callin

[GitHub] flink issue #2344: [hotfix][build] Remove Scala suffix from Hadoop1 shading ...

2016-08-09 Thread uce
Github user uce commented on the issue: https://github.com/apache/flink/pull/2344 Duplicate of #2343? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[jira] [Commented] (FLINK-4271) There is no way to set parallelism of operators produced by CoGroupedStreams

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413641#comment-15413641 ] ASF GitHub Bot commented on FLINK-4271: --- Github user StephanEwen commented on the is

[GitHub] flink issue #2344: [hotfix][build] Remove Scala suffix from Hadoop1 shading ...

2016-08-09 Thread rmetzger
Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/2344 True. Thank you, I'll close this one. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] flink pull request #2344: [hotfix][build] Remove Scala suffix from Hadoop1 s...

2016-08-09 Thread rmetzger
Github user rmetzger closed the pull request at: https://github.com/apache/flink/pull/2344 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[jira] [Commented] (FLINK-4282) Add Offset Parameter to WindowAssigners

2016-08-09 Thread Aljoscha Krettek (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413645#comment-15413645 ] Aljoscha Krettek commented on FLINK-4282: - The {{WindowAssigner}} should not try t

[GitHub] flink pull request #2051: [FLINK-3779] Add support for queryable state

2016-08-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/2051 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (FLINK-3779) Add support for queryable state

2016-08-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413647#comment-15413647 ] ASF GitHub Bot commented on FLINK-3779: --- Github user asfgit closed the pull request

[GitHub] flink issue #2305: [FLINK-4271] [DataStreamAPI] Enable CoGroupedStreams and ...

2016-08-09 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/2305 Wouldn't this suffer from the same problem as the "casting solution"? People would use `apply` and then wonder why there is no `setParallelism`, not bothering to read the Javadoc to find out that th

  1   2   >