Re: [VOTE] Release Apache Flink 1.1.0 (RC1)

@Ufuk: 3.3.9, that's probably it because that messes with the shading, right? @Stephan: Yes, even did a "rm -r .m2/repository". But the maven version is most likely the reason. On Mon, 1 Aug 2016 at 10:59 Stephan Ewen wrote: > @Aljoscha: Have you made sure you have a clean

Re: Map Reduce Sorting

Hi Hilmi, the results of the combiner are usually not completely sorted and if they are this property is not leveraged. This is due to the following reasons: 1) a sort-combiner only sorts as much data as fits into memory. If there is more data, the result consists of multiple sorted sequences. 2)

Re: [VOTE] Release Apache Flink 1.1.0 (RC1)

@Aljoscha: Have you made sure you have a clean maven cache (remove the .m2/repository/org/apache/flink folder)? On Mon, Aug 1, 2016 at 5:56 PM, Aljoscha Krettek wrote: > I tried it again now. I did: > > rm -r .m2/repository > mvn clean verify -Dhadoop.version=2.6.0 > >

[jira] [Created] (FLINK-4303) Add CEP examples

Timo Walther created FLINK-4303: --- Summary: Add CEP examples Key: FLINK-4303 URL: https://issues.apache.org/jira/browse/FLINK-4303 Project: Flink Issue Type: Improvement Components:

Re: [VOTE] Release Apache Flink 1.1.0 (RC1)

Which Maven version are you using? On Mon, Aug 1, 2016 at 5:56 PM, Aljoscha Krettek wrote: > I tried it again now. I did: > > rm -r .m2/repository > mvn clean verify -Dhadoop.version=2.6.0 > > failed again. Also with versions 2.6.1 and 2.6.3. > > On Mon, 1 Aug 2016 at 08:23

Map Reduce Sorting

Hi, I have a question regarding when data points are sorted when applying a simple Map Reduce Job. I have the following code: data = readFromSource() data.map().groupBy(0).reduce(...) This code will be translated into the following execution plan: map -> combiner -> hash partitioning

Re: [VOTE] Release Apache Flink 1.1.0 (RC1)

I think that FLINK-4094 is nice to fix but not a release blocker since we know how to prevent this situation (setting preallocation to true). On Mon, Aug 1, 2016 at 11:56 PM, Aljoscha Krettek wrote: > I tried it again now. I did: > > rm -r .m2/repository > mvn clean verify

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

Alright, that seems reasonable. I updated the doc to add the Collector to the method signature again. On Mon, 1 Aug 2016 at 00:59 Stephan Ewen wrote: > The Collector is pretty integral in all the other functions that return > multiple elements. I honestly don't see us

Re: [VOTE] Release Apache Flink 1.1.0 (RC1)

I tried it again now. I did: rm -r .m2/repository mvn clean verify -Dhadoop.version=2.6.0 failed again. Also with versions 2.6.1 and 2.6.3. On Mon, 1 Aug 2016 at 08:23 Maximilian Michels wrote: > This is also a major issue for batch with off-heap memory and memory >

Re: [DISCUSS] FLIP-5 Only send data to each taskmanager once for broadcasts

Hi Felix! Hope this helps_ Concerning (1.1) - The producer does not think in term of number of target TaskManagers. That number can, after all, change in the presence of a failure and recovery. The producer should, for its own result, not care how many consumers it will have (Tasks), but produce

Re: [VOTE] Release Apache Flink 1.1.0 (RC1)

This is also a major issue for batch with off-heap memory and memory preallocation turned off: https://issues.apache.org/jira/browse/FLINK-4094 Not hard to fix though as we simply need to reliably clear the direct memory instead of relying on garbage collection. Another possible fix is to maintain

[jira] [Created] (FLINK-4300) Improve error message for different Scala versions of JM and Client

Timo Walther created FLINK-4300: --- Summary: Improve error message for different Scala versions of JM and Client Key: FLINK-4300 URL: https://issues.apache.org/jira/browse/FLINK-4300 Project: Flink

[jira] [Created] (FLINK-4299) Show loss of job manager in Client

Ufuk Celebi created FLINK-4299: -- Summary: Show loss of job manager in Client Key: FLINK-4299 URL: https://issues.apache.org/jira/browse/FLINK-4299 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-4298) Clean up Storm Compatibility Dependencies

Stephan Ewen created FLINK-4298: --- Summary: Clean up Storm Compatibility Dependencies Key: FLINK-4298 URL: https://issues.apache.org/jira/browse/FLINK-4298 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-4297) Yarn client can't determine fat jar location if path contains spaces

Maximilian Michels created FLINK-4297: - Summary: Yarn client can't determine fat jar location if path contains spaces Key: FLINK-4297 URL: https://issues.apache.org/jira/browse/FLINK-4297

Re: [VOTE] Release Apache Flink 1.1.0 (RC1)

Thanks for the new release candidate Ufuk! Found two issues during testing: 1) Scheduling: The Flink scheduler accepts (it shouldn't) jobs with parallelism > total number of task slots, schedules tasks in all available task slots, and leaves the remaining tasks lingering forever. Haven't had

[jira] [Created] (FLINK-4296) Scheduler accepts more tasks than it has task slots available

Maximilian Michels created FLINK-4296: - Summary: Scheduler accepts more tasks than it has task slots available Key: FLINK-4296 URL: https://issues.apache.org/jira/browse/FLINK-4296 Project: Flink

[jira] [Created] (FLINK-4295) CoGroupSortTranslationTest#testGroupSortTuplesDefaultCoGroup fails to run

Chesnay Schepler created FLINK-4295: --- Summary: CoGroupSortTranslationTest#testGroupSortTuplesDefaultCoGroup fails to run Key: FLINK-4295 URL: https://issues.apache.org/jira/browse/FLINK-4295

[jira] [Created] (FLINK-4294) Allow access of composite type fields

Timo Walther created FLINK-4294: --- Summary: Allow access of composite type fields Key: FLINK-4294 URL: https://issues.apache.org/jira/browse/FLINK-4294 Project: Flink Issue Type: New Feature

Re: Introduction

Welcome to the community Neelesh :-) On Mon, Aug 1, 2016 at 3:53 PM, Kevin Jacobs wrote: > Hi! > > Welcome to the community :-)! > > > > On 01.08.2016 09:51, Ufuk Celebi wrote: > >> On Sun, Jul 31, 2016 at 8:07 PM, Neelesh Salian >> wrote: >> >>> I

[jira] [Created] (FLINK-4293) Malformatted Apache Haders

Stephan Ewen created FLINK-4293: --- Summary: Malformatted Apache Haders Key: FLINK-4293 URL: https://issues.apache.org/jira/browse/FLINK-4293 Project: Flink Issue Type: Bug Affects Versions:

[jira] [Created] (FLINK-4292) HCatalog project incorrectly set up

Stephan Ewen created FLINK-4292: --- Summary: HCatalog project incorrectly set up Key: FLINK-4292 URL: https://issues.apache.org/jira/browse/FLINK-4292 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-4291) No log entry for unscheduled reporters

Chesnay Schepler created FLINK-4291: --- Summary: No log entry for unscheduled reporters Key: FLINK-4291 URL: https://issues.apache.org/jira/browse/FLINK-4291 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-4290) CassandraConnectorTest deadlocks

Stephan Ewen created FLINK-4290: --- Summary: CassandraConnectorTest deadlocks Key: FLINK-4290 URL: https://issues.apache.org/jira/browse/FLINK-4290 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-4289) Source files have executable flag set

Ufuk Celebi created FLINK-4289: -- Summary: Source files have executable flag set Key: FLINK-4289 URL: https://issues.apache.org/jira/browse/FLINK-4289 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-4288) Make it possible to unregister tables

Timo Walther created FLINK-4288: --- Summary: Make it possible to unregister tables Key: FLINK-4288 URL: https://issues.apache.org/jira/browse/FLINK-4288 Project: Flink Issue Type: Improvement

Re: [VOTE] Release Apache Flink 1.1.0 (RC1)

Just tried to reproduce the error reported by Aljoscha, but could not. I used a clean checkpoint of the RC1 code and cleaned all local maven caches before the testing. @Aljoscha: Can you reproduce this on your machine? Can you try and clean the maven caches? On Sun, Jul 31, 2016 at 7:31 PM, Ufuk

[jira] [Created] (FLINK-4287) Unable to access secured HBase from a yarn-session.

Niels Basjes created FLINK-4287: --- Summary: Unable to access secured HBase from a yarn-session. Key: FLINK-4287 URL: https://issues.apache.org/jira/browse/FLINK-4287 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-2 Extending Window Function Metadata

The Collector is pretty integral in all the other functions that return multiple elements. I honestly don't see us switching away from it, given that it is such a core part of the API. The close() method has, to my best knowledge, not caused issues, yet. I cannot recall anyone mentioning that the

Re: Introduction

On Sun, Jul 31, 2016 at 8:07 PM, Neelesh Salian wrote: > I am Neelesh Salian; I recently joined the Flink community and I wanted to > take this opportunity to formally introduce myself. Thanks and welcome! :-)

[jira] [Created] (FLINK-4286) Have Kafka examples that use the Kafka 0.9 connector

Tzu-Li (Gordon) Tai created FLINK-4286: -- Summary: Have Kafka examples that use the Kafka 0.9 connector Key: FLINK-4286 URL: https://issues.apache.org/jira/browse/FLINK-4286 Project: Flink

[jira] [Created] (FLINK-4285) Non-existing example in Flink quickstart setup documentation

Tzu-Li (Gordon) Tai created FLINK-4285: -- Summary: Non-existing example in Flink quickstart setup documentation Key: FLINK-4285 URL: https://issues.apache.org/jira/browse/FLINK-4285 Project:

Re: Introduction

Welcome Neelesh! On Mon, Aug 1, 2016 at 4:47 AM, Márton Balassi wrote: > Welcome Neelesh, great to have you here. :) > > On Sun, Jul 31, 2016, 11:08 Neelesh Salian wrote: > > > Hello folks, > > > > I am Neelesh Salian; I recently joined the