[jira] [Commented] (FLINK-4627) Use Flink's PropertiesUtil in Kinesis connector to extract typed values from config properties

2016-10-12 Thread Tzu-Li (Gordon) Tai (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570772#comment-15570772 ] Tzu-Li (Gordon) Tai commented on FLINK-4627: Hi Neelesh, I'm not aware of anyone working on

[jira] [Comment Edited] (FLINK-4627) Use Flink's PropertiesUtil in Kinesis connector to extract typed values from config properties

2016-10-12 Thread Tzu-Li (Gordon) Tai (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570772#comment-15570772 ] Tzu-Li (Gordon) Tai edited comment on FLINK-4627 at 10/13/16 4:13 AM: --

[jira] [Commented] (FLINK-4280) New Flink-specific option to set starting position of Kafka consumer without respecting external offsets in ZK / Broker

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570769#comment-15570769 ] ASF GitHub Bot commented on FLINK-4280: --- Github user tzulitai commented on the issue:

[GitHub] flink issue #2509: [FLINK-4280][kafka-connector] Explicit start position con...

2016-10-12 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2509 Rebasing + adding tests for the new functions now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[jira] [Commented] (FLINK-4280) New Flink-specific option to set starting position of Kafka consumer without respecting external offsets in ZK / Broker

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570766#comment-15570766 ] ASF GitHub Bot commented on FLINK-4280: --- Github user tzulitai commented on a diff in the pull

[GitHub] flink pull request #2509: [FLINK-4280][kafka-connector] Explicit start posit...

2016-10-12 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/2509#discussion_r83141867 --- Diff: flink-streaming-connectors/flink-connector-kafka-0.8/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer08.java --- @@

[jira] [Commented] (FLINK-4280) New Flink-specific option to set starting position of Kafka consumer without respecting external offsets in ZK / Broker

2016-10-12 Thread Tzu-Li (Gordon) Tai (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570763#comment-15570763 ] Tzu-Li (Gordon) Tai commented on FLINK-4280: I think using the {{KafkaTopicPartition}} is a

[jira] [Commented] (FLINK-3037) Make the behavior of the Kafka consumer configurable if the offsets to restore from are not available

2016-10-12 Thread Tzu-Li (Gordon) Tai (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570751#comment-15570751 ] Tzu-Li (Gordon) Tai commented on FLINK-3037: I'm a little bit confused here. On checkpoints /

[GitHub] flink issue #2580: [FLINK-4723] [kafka-connector] Unify committed offsets to...

2016-10-12 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2580 Thanks for the review @rmetzger. I've created several local branches to test out the new IT tests stability as @StephanEwen suggested, and they seem to be fine. I'll rebase this, address

[jira] [Commented] (FLINK-4723) Unify behaviour of committed offsets to Kafka / ZK for Kafka 0.8 and 0.9 consumer

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570730#comment-15570730 ] ASF GitHub Bot commented on FLINK-4723: --- Github user tzulitai commented on the issue:

[jira] [Commented] (FLINK-4727) Kafka 0.9 Consumer should also checkpoint auto retrieved offsets even when no data is read

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570678#comment-15570678 ] ASF GitHub Bot commented on FLINK-4727: --- Github user tzulitai commented on a diff in the pull

[GitHub] flink pull request #2585: [FLINK-4727] [kafka-connector] Set missing initial...

2016-10-12 Thread tzulitai
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/2585#discussion_r83138654 --- Diff: flink-streaming-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/Kafka09Fetcher.java ---

[GitHub] flink issue #2627: Kafka 0.10 follow-up fixes

2016-10-12 Thread tzulitai
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2627 +1, looks good to me 👍 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[jira] [Commented] (FLINK-3722) The divisions in the InMemorySorters' swap/compare methods hurt performance

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570510#comment-15570510 ] ASF GitHub Bot commented on FLINK-3722: --- GitHub user greghogan opened a pull request:

[GitHub] flink pull request #2628: [FLINK-3722] [runtime] Don't / and % when sorting

2016-10-12 Thread greghogan
GitHub user greghogan opened a pull request: https://github.com/apache/flink/pull/2628 [FLINK-3722] [runtime] Don't / and % when sorting Replace division and modulus with addition and subtraction. The timing chart below has three columns for two Gelly algorithms with

[jira] [Created] (FLINK-4820) Slf4j / log4j version upgrade to support dynamic change of log levels.

2016-10-12 Thread Zhenzhong Xu (JIRA)
Zhenzhong Xu created FLINK-4820: --- Summary: Slf4j / log4j version upgrade to support dynamic change of log levels. Key: FLINK-4820 URL: https://issues.apache.org/jira/browse/FLINK-4820 Project: Flink

[jira] [Created] (FLINK-4819) Checkpoint metadata+data inspection tool (view / update)

2016-10-12 Thread Zhenzhong Xu (JIRA)
Zhenzhong Xu created FLINK-4819: --- Summary: Checkpoint metadata+data inspection tool (view / update) Key: FLINK-4819 URL: https://issues.apache.org/jira/browse/FLINK-4819 Project: Flink Issue

[GitHub] flink issue #2627: Kafka 0.10 follow-up fixes

2016-10-12 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2627 Thanks a lot for fixing this. +1 from my side --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[jira] [Updated] (FLINK-4816) Executions failed from "DEPLOYING" should retain restored checkpoint information

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen updated FLINK-4816: Summary: Executions failed from "DEPLOYING" should retain restored checkpoint information (was:

[jira] [Created] (FLINK-4818) RestartStrategy should track how many failed restore attempts the same checkpoint has and fall back to earlier checkpoints

2016-10-12 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-4818: --- Summary: RestartStrategy should track how many failed restore attempts the same checkpoint has and fall back to earlier checkpoints Key: FLINK-4818 URL:

[GitHub] flink pull request #2606: [FLINK-3888] Allow custom convergence criterion in...

2016-10-12 Thread greghogan
Github user greghogan commented on a diff in the pull request: https://github.com/apache/flink/pull/2606#discussion_r83071641 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/iterative/task/IterationSynchronizationSinkTask.java --- @@ -56,11 +56,15 @@

[jira] [Created] (FLINK-4817) Checkpoint Coordinator should be called to restore state with a specific checkpoint ID

2016-10-12 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-4817: --- Summary: Checkpoint Coordinator should be called to restore state with a specific checkpoint ID Key: FLINK-4817 URL: https://issues.apache.org/jira/browse/FLINK-4817

[jira] [Commented] (FLINK-3888) Custom Aggregator with Convergence can't be registered directly with DeltaIteration

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569632#comment-15569632 ] ASF GitHub Bot commented on FLINK-3888: --- Github user greghogan commented on a diff in the pull

[jira] [Commented] (FLINK-3888) Custom Aggregator with Convergence can't be registered directly with DeltaIteration

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569631#comment-15569631 ] ASF GitHub Bot commented on FLINK-3888: --- Github user greghogan commented on a diff in the pull

[GitHub] flink pull request #2606: [FLINK-3888] Allow custom convergence criterion in...

2016-10-12 Thread greghogan
Github user greghogan commented on a diff in the pull request: https://github.com/apache/flink/pull/2606#discussion_r83069609 --- Diff: flink-optimizer/src/main/java/org/apache/flink/optimizer/plantranslate/JobGraphGenerator.java --- @@ -1513,14 +1513,21 @@ private void

[jira] [Commented] (FLINK-3888) Custom Aggregator with Convergence can't be registered directly with DeltaIteration

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569630#comment-15569630 ] ASF GitHub Bot commented on FLINK-3888: --- Github user greghogan commented on a diff in the pull

[GitHub] flink pull request #2606: [FLINK-3888] Allow custom convergence criterion in...

2016-10-12 Thread greghogan
Github user greghogan commented on a diff in the pull request: https://github.com/apache/flink/pull/2606#discussion_r83075316 --- Diff: flink-tests/src/test/java/org/apache/flink/test/iterative/aggregators/AggregatorConvergenceITCase.java --- @@ -137,42 +138,51 @@ public void

[jira] [Created] (FLINK-4816) Executions from "DEPLOYING" should retain restored checkpoint information

2016-10-12 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-4816: --- Summary: Executions from "DEPLOYING" should retain restored checkpoint information Key: FLINK-4816 URL: https://issues.apache.org/jira/browse/FLINK-4816 Project: Flink

[GitHub] flink issue #2288: Feature/s3 a fix

2016-10-12 Thread cresny
Github user cresny commented on the issue: https://github.com/apache/flink/pull/2288 I was hoping that when the 1.2-SNAPSHOT dust cleared it would no longer be necessary. At this point it seems it's only necessary for staging a YARN session for an S3A configured FileSystem,

[jira] [Created] (FLINK-4815) Automatic fallback to earlier checkpoints when checkpoint restore fails

2016-10-12 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-4815: --- Summary: Automatic fallback to earlier checkpoints when checkpoint restore fails Key: FLINK-4815 URL: https://issues.apache.org/jira/browse/FLINK-4815 Project: Flink

[jira] [Commented] (FLINK-4632) when yarn nodemanager lost, flink hung

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569490#comment-15569490 ] Stephan Ewen commented on FLINK-4632: - Do you have the logs of the JobManager from when the

[GitHub] flink issue #2288: Feature/s3 a fix

2016-10-12 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2288 @cresny Do you want to continue on this? I think it is a good fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[jira] [Commented] (FLINK-4651) Re-register processing time timers at the WindowOperator upon recovery.

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569468#comment-15569468 ] Stephan Ewen commented on FLINK-4651: - How will this be resolved? > Re-register processing time

[jira] [Commented] (FLINK-4651) Re-register processing time timers at the WindowOperator upon recovery.

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569467#comment-15569467 ] Stephan Ewen commented on FLINK-4651: - Fixed in 1.2 as part of

[jira] [Commented] (FLINK-3154) Update Kryo version from 2.24.0 to 3.0.3

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569461#comment-15569461 ] Stephan Ewen commented on FLINK-3154: - I would like to close this as "won't fix". This can only be

[jira] [Resolved] (FLINK-4701) Unprotected access to cancelables in StreamTask

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-4701. - Resolution: Not A Problem Not applicable any more to the latest code, where the cancelables have

[jira] [Closed] (FLINK-4701) Unprotected access to cancelables in StreamTask

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-4701. --- > Unprotected access to cancelables in StreamTask > --- > >

[jira] [Resolved] (FLINK-3253) deserializeObject exception on WindowOperator

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-3253. - Resolution: Unresolved The code for this part of the system changed completely, so that this

[jira] [Closed] (FLINK-3253) deserializeObject exception on WindowOperator

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-3253. --- > deserializeObject exception on WindowOperator > - > >

[jira] [Resolved] (FLINK-4102) Test failure with checkpoint barriers

2016-10-12 Thread Greg Hogan (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Hogan resolved FLINK-4102. --- Resolution: Cannot Reproduce > Test failure with checkpoint barriers >

[jira] [Closed] (FLINK-3354) RocksDB should compute checkpoint size based on backup file size

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-3354. --- > RocksDB should compute checkpoint size based on backup file size >

[jira] [Resolved] (FLINK-3354) RocksDB should compute checkpoint size based on backup file size

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-3354. - Resolution: Fixed Fixed in b17632ddbbb58f8b3e3d1a31d6658e59258918c7 > RocksDB should compute

[jira] [Commented] (FLINK-4102) Test failure with checkpoint barriers

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569440#comment-15569440 ] Stephan Ewen commented on FLINK-4102: - Can we close that as "could not reproduce" and reopen it once

[jira] [Resolved] (FLINK-2624) RabbitMQ source / sink should participate in checkpointing

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-2624. - Resolution: Fixed Fixed in 9215b72422d3e638fe950b61fa01f2e4e04981a0 > RabbitMQ source / sink

[jira] [Closed] (FLINK-2624) RabbitMQ source / sink should participate in checkpointing

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-2624. --- > RabbitMQ source / sink should participate in checkpointing >

[jira] [Commented] (FLINK-4410) Report more information about operator checkpoints

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569405#comment-15569405 ] Stephan Ewen commented on FLINK-4410: - Code that gathers additional metrics is integrated. What is

[jira] [Updated] (FLINK-4410) Report more information about operator checkpoints

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen updated FLINK-4410: Assignee: (was: Stephan Ewen) > Report more information about operator checkpoints >

[jira] [Closed] (FLINK-3594) StreamTask may fail when checkpoint is concurrent to regular termination

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-3594. --- > StreamTask may fail when checkpoint is concurrent to regular termination >

[jira] [Resolved] (FLINK-3594) StreamTask may fail when checkpoint is concurrent to regular termination

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-3594. - Resolution: Fixed Fix Version/s: 1.1.0 Fixed via 34110fefc6e49bcbe1ae8de090fa43a5f6ef5fec

[jira] [Commented] (FLINK-2221) Checkpoints to "file://" are not cleaned up

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569386#comment-15569386 ] Stephan Ewen commented on FLINK-2221: - I would like to close this as "won't fix", because it is

[jira] [Closed] (FLINK-3844) Checkpoint failures should not always lead to job failures

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-3844. --- > Checkpoint failures should not always lead to job failures >

[jira] [Resolved] (FLINK-3844) Checkpoint failures should not always lead to job failures

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen resolved FLINK-3844. - Resolution: Duplicate Duplicate of FLINK-4809 The other issue has already fine grained subtasks

[jira] [Created] (FLINK-4814) Remove extra storage location for externalized checkpoint metadata

2016-10-12 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-4814: -- Summary: Remove extra storage location for externalized checkpoint metadata Key: FLINK-4814 URL: https://issues.apache.org/jira/browse/FLINK-4814 Project: Flink

[GitHub] flink pull request #2627: Kafka 0.10 follow-up fixes

2016-10-12 Thread rmetzger
GitHub user rmetzger opened a pull request: https://github.com/apache/flink/pull/2627 Kafka 0.10 follow-up fixes After merging https://github.com/apache/flink/pull/2369, there was some follow-up feedback in the final commit:

[jira] [Commented] (FLINK-4512) Add option for persistent checkpoints

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569028#comment-15569028 ] ASF GitHub Bot commented on FLINK-4512: --- Github user StephanEwen commented on the issue:

[GitHub] flink issue #2608: [FLINK-4512] [FLIP-10] Add option to persist periodic che...

2016-10-12 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2608 +1 to the general approach and the code Some suggestions for name polishing: - How about renaming `DISCARD_ON_CANCELLATION` to `DELETE_ON_CANCELLATION`? That would sound more

[jira] [Commented] (FLINK-4803) Job Cancel can hang forever waiting for OutputFormat.close()

2016-10-12 Thread Shannon Carey (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569025#comment-15569025 ] Shannon Carey commented on FLINK-4803: -- Yes, that's right. cancel() blocks on close(), and therefore

[jira] [Commented] (FLINK-4604) Add support for standard deviation/variance

2016-10-12 Thread Timo Walther (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569021#comment-15569021 ] Timo Walther commented on FLINK-4604: - You cannot use

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83017556 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/ContinuousFileProcessingTests.java --- @@ -336,237 +348,294 @@ public int compare(String o1,

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83014810 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/ContinuousFileProcessingTests.java --- @@ -336,237 +348,294 @@ public int compare(String o1,

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83016929 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/ContinuousFileProcessingTests.java --- @@ -336,237 +348,294 @@ public int compare(String o1,

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83005668 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/ContinuousFileProcessingTests.java --- @@ -111,8 +114,8 @@ public void

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83019574 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/ContinuousFileMonitoringFunction.java --- @@ -295,7 +256,7 @@

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83016682 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/ContinuousFileProcessingTests.java --- @@ -336,237 +348,294 @@ public int compare(String o1,

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83019658 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/ContinuousFileMonitoringFunction.java --- @@ -107,10 +113,10 @@

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83007178 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/ContinuousFileProcessingTests.java --- @@ -336,237 +348,294 @@ public int compare(String o1,

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83022623 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/ContinuousFileProcessingCheckpointITCase.java --- @@ -257,74 +190,158 @@ public void

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83005541 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/ContinuousFileProcessingTests.java --- @@ -54,8 +55,10 @@ import java.util.List;

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83019134 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/ContinuousFileMonitoringFunction.java --- @@ -146,102 +152,57 @@

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83004834 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/ContinuousFileProcessingITCase.java --- @@ -111,174 +115,191 @@ protected void testProgram()

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83014794 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/ContinuousFileProcessingTests.java --- @@ -578,18 +647,28 @@ public void

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83015686 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/ContinuousFileProcessingTests.java --- @@ -578,18 +647,28 @@ public void

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83006923 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/ContinuousFileProcessingTests.java --- @@ -336,237 +348,294 @@ public int compare(String o1,

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83022754 --- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/ContinuousFileProcessingCheckpointITCase.java --- @@ -257,74 +190,158 @@ public void

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83020529 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/ContinuousFileMonitoringFunction.java --- @@ -60,26 +63,27 @@

[GitHub] flink pull request #2618: Refactoring the Continuous File Monitoring Functio...

2016-10-12 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/2618#discussion_r83005706 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/ContinuousFileProcessingTests.java --- @@ -256,11 +265,11 @@ public int compare(String o1,

[jira] [Commented] (FLINK-4283) ExecutionGraphRestartTest fails

2016-10-12 Thread Alexander Shoshin (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568940#comment-15568940 ] Alexander Shoshin commented on FLINK-4283: -- It is still relevant. I have just strated researching

[jira] [Updated] (FLINK-4635) Implement Data Transfer Authentication using shared secret configuration

2016-10-12 Thread Robert Metzger (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Metzger updated FLINK-4635: -- Issue Type: Sub-task (was: Task) Parent: FLINK-3930 > Implement Data Transfer

[jira] [Commented] (FLINK-4283) ExecutionGraphRestartTest fails

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568819#comment-15568819 ] Stephan Ewen commented on FLINK-4283: - Is this issue still relevant, or has this instability been

[jira] [Commented] (FLINK-4813) Having flink-test-utils as a dependency outside Flink fails the build

2016-10-12 Thread Stephan Ewen (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568803#comment-15568803 ] Stephan Ewen commented on FLINK-4813: - +1 > Having flink-test-utils as a dependency outside Flink

[GitHub] flink issue #2439: [FLINK-4450]update storm verion to 1.0.0 in flink-storm a...

2016-10-12 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2439 Okay, then we should re-add all the dependency exclusions that were removed from the `flink-storm/pom.xml`. --- If your project is set up for it, you can reply to this email and have your reply

[jira] [Commented] (FLINK-2608) Arrays.asList(..) does not work with CollectionInputFormat

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568798#comment-15568798 ] ASF GitHub Bot commented on FLINK-2608: --- Github user StephanEwen commented on the issue:

[GitHub] flink issue #2623: [FLINK-2608] Updated Twitter Chill version.

2016-10-12 Thread StephanEwen
Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/2623 As far as I read it, Kryo 3.x is not strictly serialization compatible with 2.x, hence the major version number bump. If the interfaces are still stable, then it should be fine to bump

[GitHub] flink pull request #2562: [FLINK-4691] [table] Add group-windows for streami...

2016-10-12 Thread twalthr
Github user twalthr commented on a diff in the pull request: https://github.com/apache/flink/pull/2562#discussion_r83009444 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/runtime/aggregate/PropertyRead.scala --- @@ -0,0 +1,33 @@ +/* + *

[jira] [Commented] (FLINK-2608) Arrays.asList(..) does not work with CollectionInputFormat

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568745#comment-15568745 ] ASF GitHub Bot commented on FLINK-2608: --- Github user rmetzger commented on the issue:

[jira] [Commented] (FLINK-4691) Add group-windows for streaming tables

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568789#comment-15568789 ] ASF GitHub Bot commented on FLINK-4691: --- Github user twalthr commented on a diff in the pull

[jira] [Commented] (FLINK-4691) Add group-windows for streaming tables

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568768#comment-15568768 ] ASF GitHub Bot commented on FLINK-4691: --- Github user twalthr commented on a diff in the pull

[jira] [Commented] (FLINK-4450) update storm version to 1.0.0

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568782#comment-15568782 ] ASF GitHub Bot commented on FLINK-4450: --- Github user StephanEwen commented on the issue:

[GitHub] flink pull request #2562: [FLINK-4691] [table] Add group-windows for streami...

2016-10-12 Thread twalthr
Github user twalthr commented on a diff in the pull request: https://github.com/apache/flink/pull/2562#discussion_r83008188 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/typeutils/RowIntervalTypeInfo.scala --- @@ -0,0 +1,108 @@ +/* + *

[GitHub] flink issue #2623: [FLINK-2608] Updated Twitter Chill version.

2016-10-12 Thread rmetzger
Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/2623 This change adds Kryo-shaded to our dependency tree: ``` [INFO] | +- com.twitter:chill_2.10:jar:0.8.1:compile [INFO] | | +- com.twitter:chill-java:jar:0.8.1:compile [INFO] |

[GitHub] flink issue #2619: [FLINK-4108] [scala] Consider ResultTypeQueryable for inp...

2016-10-12 Thread albertoRamon
Github user albertoRamon commented on the issue: https://github.com/apache/flink/pull/2619 Works OK ¡¡ [Capture Output](http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/jdbc-JDBCInputFormat-tp9393p9489.html) Thanks, Alb --- If your project is

[jira] [Commented] (FLINK-4108) NPE in Row.productArity

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568712#comment-15568712 ] ASF GitHub Bot commented on FLINK-4108: --- Github user albertoRamon commented on the issue:

[jira] [Updated] (FLINK-3037) Make the behavior of the Kafka consumer configurable if the offsets to restore from are not available

2016-10-12 Thread Robert Metzger (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Metzger updated FLINK-3037: -- Description: Currently, if the {{FlinkKafkaConsumer}} is restoring a checkpoint and the offset

[GitHub] flink pull request #2509: [FLINK-4280][kafka-connector] Explicit start posit...

2016-10-12 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/2509#discussion_r83003348 --- Diff: flink-streaming-connectors/flink-connector-kafka-0.8/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer08.java --- @@

[jira] [Commented] (FLINK-4280) New Flink-specific option to set starting position of Kafka consumer without respecting external offsets in ZK / Broker

2016-10-12 Thread Robert Metzger (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568701#comment-15568701 ] Robert Metzger commented on FLINK-4280: --- This issue is related:

[jira] [Commented] (FLINK-4280) New Flink-specific option to set starting position of Kafka consumer without respecting external offsets in ZK / Broker

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568703#comment-15568703 ] ASF GitHub Bot commented on FLINK-4280: --- Github user rmetzger commented on a diff in the pull

[jira] [Commented] (FLINK-4691) Add group-windows for streaming tables

2016-10-12 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568669#comment-15568669 ] ASF GitHub Bot commented on FLINK-4691: --- Github user fhueske commented on a diff in the pull

[GitHub] flink pull request #2562: [FLINK-4691] [table] Add group-windows for streami...

2016-10-12 Thread fhueske
Github user fhueske commented on a diff in the pull request: https://github.com/apache/flink/pull/2562#discussion_r83000595 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/api/java/table/windows/Session.java --- @@ -0,0 +1,48 @@ +/* + * Licensed to

[jira] [Commented] (FLINK-4604) Add support for standard deviation/variance

2016-10-12 Thread Anton Mushin (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568664#comment-15568664 ] Anton Mushin commented on FLINK-4604: - I tried check function in

[jira] [Commented] (FLINK-4813) Having flink-test-utils as a dependency outside Flink fails the build

2016-10-12 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/FLINK-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568657#comment-15568657 ] Maximilian Michels commented on FLINK-4813: --- +1 That sounds like a good solution. > Having

  1   2   3   >