[RESULT][VOTE] Release 1.3.2, release candidate #3

2017-08-04 Thread Aljoscha Krettek
The voting time has passed and I'm happy to announce that we've collected 
enough votes to release this RC as Flink 1.3.2.

+1 votes:
- Stefan (non-binding)
- Till (binding)
- Gordon (binding)
- Piotr (non-binding)
- Fabian (binding)
- Aljoscha (binding)
- Greg (binding)

That's 7 votes, 5 binding. No 0 or -1 votes.

> On 5. Aug 2017, at 07:49, Aljoscha Krettek  wrote:
> 
> Hi Bowen,
> 
> You're right, I didn't give a lot of thought to Kinesis because there were 
> critical bugs in Kafka and incremental checkpointing that needed to be fixed 
> quickly. It's not good to neglect another important connector for this, 
> though. We should start focusing on the Kinesis issues next week and soon do 
> a 1.3.3 release that will then mostly contain the Kinesis fixes. What do you 
> think?
> 
> Best,
> Aljoscha
> 
>> On 5. Aug 2017, at 03:38, Greg Hogan  wrote:
>> 
>> Thanks Aljoscha!
>> 
>> +1 (binding)
>> 
>> - verified source and binary signatures
>> - verified source and binary checksums
>> - verified LICENSEs
>> - verified NOTICEs
>> - built from source
>> 
>> Greg
>> 
>>> On Aug 4, 2017, at 2:00 AM, Aljoscha Krettek  wrote:
>>> 
>>> Hi everyone,
>>> 
>>> Please review and vote on the release candidate #3 for the version 1.3.2, 
>>> as follows:
>>> [ ] +1, Approve the release
>>> [ ] -1, Do not approve the release (please provide specific comments)
>>> 
>>> 
>>> The complete staging area is available for your review, which includes:
>>> * JIRA release notes [1],
>>> * the official Apache source release and binary convenience releases to be 
>>> deployed to dist.apache.org [2], which is signed with the key with 
>>> fingerprint 0xA8F4FD97121D7293 [3],
>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>> * source code tag "release-1.3.2-rc2" [5],
>>> * website pull request listing the new release and adding announcement blog 
>>> post [6]. 
>>> 
>>> The only change in this RC compared to the last is this commit that fixes 
>>> copying of the Gelly examples jar: 
>>> https://github.com/apache/flink/commit/fda455e23a6192200b85b84d3aee312c9be40c99.
>>>  I would therefore like to propose a shorter voting period because the last 
>>> RC was seemingly good except for this bug. Please voice your concerns about 
>>> this if you have any. The vote will be open for at least 24 hours. It is 
>>> adopted by majority approval, with at least 3 PMC affirmative votes.
>>> 
>>> Please use the provided document, as discussed before, for coordinating the 
>>> testing efforts: [7]. I have copied over the cluster testing efforts and 
>>> functional testing efforts since the code of this RC is exactly the same as 
>>> RC2.
>>> 
>>> Thanks,
>>> Aljoscha
>>> 
>>> [1] 
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12340984
>>> [2] http://people.apache.org/~aljoscha/flink-1.3.2-rc3/
>>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
>>> [4] https://repository.apache.org/content/repositories/orgapacheflink-1135/
>>> [5] 
>>> https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h=d0f7528a69a93045f4af686347753f8737ca0b2b
>>> [6] https://github.com/apache/flink-web/pull/75
>>> [7] 
>>> https://docs.google.com/document/d/1OJAE6scAZXbSBaGNNEpU1qhr3Jb7bwbec7keEnT9hmA/edit?usp=sharing
>> 
> 



Re: [VOTE] Release 1.3.2, release candidate #3

2017-08-04 Thread Aljoscha Krettek
With 24 hours now having passed, this vote is now complete. I'll summarise the 
results in an extra thread.

Best,
Aljoscha
> On 5. Aug 2017, at 07:49, Aljoscha Krettek  wrote:
> 
> Hi Bowen,
> 
> You're right, I didn't give a lot of thought to Kinesis because there were 
> critical bugs in Kafka and incremental checkpointing that needed to be fixed 
> quickly. It's not good to neglect another important connector for this, 
> though. We should start focusing on the Kinesis issues next week and soon do 
> a 1.3.3 release that will then mostly contain the Kinesis fixes. What do you 
> think?
> 
> Best,
> Aljoscha
> 
>> On 5. Aug 2017, at 03:38, Greg Hogan  wrote:
>> 
>> Thanks Aljoscha!
>> 
>> +1 (binding)
>> 
>> - verified source and binary signatures
>> - verified source and binary checksums
>> - verified LICENSEs
>> - verified NOTICEs
>> - built from source
>> 
>> Greg
>> 
>>> On Aug 4, 2017, at 2:00 AM, Aljoscha Krettek  wrote:
>>> 
>>> Hi everyone,
>>> 
>>> Please review and vote on the release candidate #3 for the version 1.3.2, 
>>> as follows:
>>> [ ] +1, Approve the release
>>> [ ] -1, Do not approve the release (please provide specific comments)
>>> 
>>> 
>>> The complete staging area is available for your review, which includes:
>>> * JIRA release notes [1],
>>> * the official Apache source release and binary convenience releases to be 
>>> deployed to dist.apache.org [2], which is signed with the key with 
>>> fingerprint 0xA8F4FD97121D7293 [3],
>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>> * source code tag "release-1.3.2-rc2" [5],
>>> * website pull request listing the new release and adding announcement blog 
>>> post [6]. 
>>> 
>>> The only change in this RC compared to the last is this commit that fixes 
>>> copying of the Gelly examples jar: 
>>> https://github.com/apache/flink/commit/fda455e23a6192200b85b84d3aee312c9be40c99.
>>>  I would therefore like to propose a shorter voting period because the last 
>>> RC was seemingly good except for this bug. Please voice your concerns about 
>>> this if you have any. The vote will be open for at least 24 hours. It is 
>>> adopted by majority approval, with at least 3 PMC affirmative votes.
>>> 
>>> Please use the provided document, as discussed before, for coordinating the 
>>> testing efforts: [7]. I have copied over the cluster testing efforts and 
>>> functional testing efforts since the code of this RC is exactly the same as 
>>> RC2.
>>> 
>>> Thanks,
>>> Aljoscha
>>> 
>>> [1] 
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12340984
>>> [2] http://people.apache.org/~aljoscha/flink-1.3.2-rc3/
>>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
>>> [4] https://repository.apache.org/content/repositories/orgapacheflink-1135/
>>> [5] 
>>> https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h=d0f7528a69a93045f4af686347753f8737ca0b2b
>>> [6] https://github.com/apache/flink-web/pull/75
>>> [7] 
>>> https://docs.google.com/document/d/1OJAE6scAZXbSBaGNNEpU1qhr3Jb7bwbec7keEnT9hmA/edit?usp=sharing
>> 
> 



Re: [VOTE] Release 1.3.2, release candidate #3

2017-08-04 Thread Aljoscha Krettek
Hi Bowen,

You're right, I didn't give a lot of thought to Kinesis because there were 
critical bugs in Kafka and incremental checkpointing that needed to be fixed 
quickly. It's not good to neglect another important connector for this, though. 
We should start focusing on the Kinesis issues next week and soon do a 1.3.3 
release that will then mostly contain the Kinesis fixes. What do you think?

Best,
Aljoscha

> On 5. Aug 2017, at 03:38, Greg Hogan  wrote:
> 
> Thanks Aljoscha!
> 
> +1 (binding)
> 
> - verified source and binary signatures
> - verified source and binary checksums
> - verified LICENSEs
> - verified NOTICEs
> - built from source
> 
> Greg
> 
>> On Aug 4, 2017, at 2:00 AM, Aljoscha Krettek  wrote:
>> 
>> Hi everyone,
>> 
>> Please review and vote on the release candidate #3 for the version 1.3.2, as 
>> follows:
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>> 
>> 
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release and binary convenience releases to be 
>> deployed to dist.apache.org [2], which is signed with the key with 
>> fingerprint 0xA8F4FD97121D7293 [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag "release-1.3.2-rc2" [5],
>> * website pull request listing the new release and adding announcement blog 
>> post [6]. 
>> 
>> The only change in this RC compared to the last is this commit that fixes 
>> copying of the Gelly examples jar: 
>> https://github.com/apache/flink/commit/fda455e23a6192200b85b84d3aee312c9be40c99.
>>  I would therefore like to propose a shorter voting period because the last 
>> RC was seemingly good except for this bug. Please voice your concerns about 
>> this if you have any. The vote will be open for at least 24 hours. It is 
>> adopted by majority approval, with at least 3 PMC affirmative votes.
>> 
>> Please use the provided document, as discussed before, for coordinating the 
>> testing efforts: [7]. I have copied over the cluster testing efforts and 
>> functional testing efforts since the code of this RC is exactly the same as 
>> RC2.
>> 
>> Thanks,
>> Aljoscha
>> 
>> [1] 
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12340984
>> [2] http://people.apache.org/~aljoscha/flink-1.3.2-rc3/
>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
>> [4] https://repository.apache.org/content/repositories/orgapacheflink-1135/
>> [5] 
>> https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h=d0f7528a69a93045f4af686347753f8737ca0b2b
>> [6] https://github.com/apache/flink-web/pull/75
>> [7] 
>> https://docs.google.com/document/d/1OJAE6scAZXbSBaGNNEpU1qhr3Jb7bwbec7keEnT9hmA/edit?usp=sharing
> 



Re: [VOTE] Release 1.3.2, release candidate #2

2017-08-04 Thread Aljoscha Krettek
Hi,

I think FLINK-7352 is only a testing instability. We should definitely fix that 
but I don't think it should block the release, which has quite a few critical 
fixes.

Best,
Aljoscha

> On 5. Aug 2017, at 00:05, Ted Yu  wrote:
> 
> When I ran integration tests, I bumped into:
> FLINK-7352
> 
> Should this be resolved for 1.3.2 ?
> 
> Thanks
> 
> On Sun, Jul 30, 2017 at 12:07 AM, Aljoscha Krettek 
> wrote:
> 
>> Hi everyone,
>> 
>> Please review and vote on the release candidate #2 for the version 1.3.2,
>> as follows:
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>> 
>> 
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release and binary convenience releases to be
>> deployed to dist.apache.org [2], which is signed with the key with
>> fingerprint 0xA8F4FD97121D7293 [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag "release-1.3.2-rc2" [5],
>> * website pull request listing the new release and adding announcement
>> blog post [6].
>> 
>> The vote will be open for at least 72 hours (excluding this current
>> weekend). It is adopted by majority approval, with at least 3 PMC
>> affirmative votes.
>> 
>> Please use the provided document, as discussed before, for coordinating
>> the testing efforts: [7]
>> 
>> Thanks,
>> Aljoscha
>> 
>> [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>> projectId=12315522&version=12340984
>> [2] http://people.apache.org/~aljoscha/flink-1.3.2-rc2/
>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
>> [4] https://repository.apache.org/content/repositories/
>> orgapacheflink-1133/
>> [5] https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h=
>> e38825d0c8e7fe2191a4c657984d9939ed8dd0ad
>> [6] https://github.com/apache/flink-web/pull/75
>> [7] https://docs.google.com/document/d/1dN9AM9FUPizIu4hTKAXJSbbAORRdr
>> ce-BqQ8AUHlOqE/edit?usp=sharing



Re: [VOTE] Release 1.3.2, release candidate #3

2017-08-04 Thread Greg Hogan
Thanks Aljoscha!

+1 (binding)

- verified source and binary signatures
- verified source and binary checksums
- verified LICENSEs
- verified NOTICEs
- built from source

Greg

> On Aug 4, 2017, at 2:00 AM, Aljoscha Krettek  wrote:
> 
> Hi everyone,
> 
> Please review and vote on the release candidate #3 for the version 1.3.2, as 
> follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
> 
> 
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release and binary convenience releases to be 
> deployed to dist.apache.org [2], which is signed with the key with 
> fingerprint 0xA8F4FD97121D7293 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "release-1.3.2-rc2" [5],
> * website pull request listing the new release and adding announcement blog 
> post [6]. 
> 
> The only change in this RC compared to the last is this commit that fixes 
> copying of the Gelly examples jar: 
> https://github.com/apache/flink/commit/fda455e23a6192200b85b84d3aee312c9be40c99.
>  I would therefore like to propose a shorter voting period because the last 
> RC was seemingly good except for this bug. Please voice your concerns about 
> this if you have any. The vote will be open for at least 24 hours. It is 
> adopted by majority approval, with at least 3 PMC affirmative votes.
> 
> Please use the provided document, as discussed before, for coordinating the 
> testing efforts: [7]. I have copied over the cluster testing efforts and 
> functional testing efforts since the code of this RC is exactly the same as 
> RC2.
> 
> Thanks,
> Aljoscha
> 
> [1] 
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12340984
> [2] http://people.apache.org/~aljoscha/flink-1.3.2-rc3/
> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> [4] https://repository.apache.org/content/repositories/orgapacheflink-1135/
> [5] 
> https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h=d0f7528a69a93045f4af686347753f8737ca0b2b
> [6] https://github.com/apache/flink-web/pull/75
> [7] 
> https://docs.google.com/document/d/1OJAE6scAZXbSBaGNNEpU1qhr3Jb7bwbec7keEnT9hmA/edit?usp=sharing



[jira] [Created] (FLINK-7372) Remove ActorGateway from JobGraph

2017-08-04 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-7372:


 Summary: Remove ActorGateway from JobGraph
 Key: FLINK-7372
 URL: https://issues.apache.org/jira/browse/FLINK-7372
 Project: Flink
  Issue Type: Improvement
  Components: Distributed Coordination
Affects Versions: 1.4.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Priority: Minor


As a preliminary step for easier Flip-6 integration we should try to decouple 
as many components from the underlying RPC abstraction as possible. One of 
these components is the {{JobGraph}} which has a dependency on {{ActorGateway}} 
via its {{JobGraph#uploadUserJars}} method.

I propose to get rid of the {{ActorGateway}} parameter and passing instead the 
BlobServer's address as an {{InetSocketAddress}} instance.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Re: [VOTE] Release 1.3.2, release candidate #2

2017-08-04 Thread Ted Yu
When I ran integration tests, I bumped into:
FLINK-7352

Should this be resolved for 1.3.2 ?

Thanks

On Sun, Jul 30, 2017 at 12:07 AM, Aljoscha Krettek 
wrote:

> Hi everyone,
>
> Please review and vote on the release candidate #2 for the version 1.3.2,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release and binary convenience releases to be
> deployed to dist.apache.org [2], which is signed with the key with
> fingerprint 0xA8F4FD97121D7293 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "release-1.3.2-rc2" [5],
> * website pull request listing the new release and adding announcement
> blog post [6].
>
> The vote will be open for at least 72 hours (excluding this current
> weekend). It is adopted by majority approval, with at least 3 PMC
> affirmative votes.
>
> Please use the provided document, as discussed before, for coordinating
> the testing efforts: [7]
>
> Thanks,
> Aljoscha
>
> [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12315522&version=12340984
> [2] http://people.apache.org/~aljoscha/flink-1.3.2-rc2/
> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> [4] https://repository.apache.org/content/repositories/
> orgapacheflink-1133/
> [5] https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h=
> e38825d0c8e7fe2191a4c657984d9939ed8dd0ad
> [6] https://github.com/apache/flink-web/pull/75
> [7] https://docs.google.com/document/d/1dN9AM9FUPizIu4hTKAXJSbbAORRdr
> ce-BqQ8AUHlOqE/edit?usp=sharing


Re: [VOTE] Release 1.3.2, release candidate #3

2017-08-04 Thread Bowen Li
Hi everyone,

I read the doc, and found that we are not even testing Flink against
Kinesis with *flink-connector-kinesis*.

According to our experience, *flink-connector-kinesis *releases tend to
have more issues w.r.t. dependency version conflicts, performance,
capability, and stability. We've reported 5 issues (FLINK-6365, 6951, 7223,
7366, 7367) so far and resolved one (FLINK-6365). I guess those issues are
resulting from 1) flink-connector-kinesis is not built as part of Flink's
building process (because of AWS licensing issue), and 2) release managers
are not testing flink-connector-kinesis in release testing pipeline. Thus,
we cannot detect those issues before releasing.

@StephanEwen and I talked about the popularity of Kinesis within Flink
users, Stephan said Kinesis is quite *popular*. If it's a popular one,
shall we include it in the testing plan from now on?

Thanks,
Bowen



On Fri, Aug 4, 2017 at 8:36 AM, Aljoscha Krettek 
wrote:

> +1
>
>  - checked checksums and signatures
>  - did extensive cluster testing on GCE with dataproc and a lot of
> different combinations of backends and configuration settings
>
> Tested configuration combinations (from the testing coordination document):
>  - (aljoscha, GCE, dataproc, Rocks(incremental), YARN per-job (no HA), no
> kerberos, Kafka 0.10.x, OK)
>  - (aljoscha, GCE, dataproc, Rocks(no incremental), YARN per-job (no HA),
> no kerberos, Kafka 0.10.x, OK)
>  - (aljoscha, GCE, dataproc, Rocks(incremental), YARN per-job (HA), no
> kerberos, Kafka 0.10.x, OK)
>  - (aljoscha, GCE, dataproc, Rocks(no incremental), YARN per-job (HA), no
> kerberos, Kafka 0.10.x, OK)
>  - (aljoscha, GCE, dataproc, FileBackend(sync), YARN per-job (no HA), no
> kerberos, Kafka 0.10.x, ONGOING)
>  - (aljoscha, GCE, dataproc, FileBackend(async), YARN per-job (HA), no
> kerberos, Kafka 0.10.x, ONGOING)
>
> > On 4. Aug 2017, at 16:57, Fabian Hueske  wrote:
> >
> > +1
> >
> > - checked added and modified Maven dependencies since Flink 1.3.1
> >  - io.dropwizard.metrics/metrics-core was upgraded to 3.1.5 and is ASL
> >  - org.xerial.snappy/snappy-java/1.0.5 was added and is ASL
> > - checked flink-runtime-web for new files with non-Apache licenses
> >
> > Cheers, Fabian
> >
> > 2017-08-04 13:32 GMT+02:00 Tzu-Li (Gordon) Tai :
> >
> >> +1
> >>
> >> - Functional cluster test results carried over from RC2 (did not retest
> >> those for RC3 as the additional commit fix for Gelly should be
> irrelevant)
> >> - Built from source with Scala 2.10 and 2.11 (MacOSX), successful
> >> - Checked to-be-released Maven artifacts
> >>
> >> Cheers,
> >> Gordon
> >>
> >> On 4 August 2017 at 5:43:09 PM, Till Rohrmann (trohrm...@apache.org)
> >> wrote:
> >>
> >> +1
> >>
> >> - Checked checksums and GPG files
> >> - Sources contain no binaries
> >> - Check all POM files point to the same version
> >> - Read readme file
> >> - Build from source archive with Hadoop 2.8.1
> >> - Test start up scripts for local and standalone mode
> >> - Test job submission for multi-node cluster
> >> - Plan-visualizer
> >> - Verified quickstarts for IntelliJ and Eclipse
> >> - Tested SBT
> >>
> >> Caveat but not a blocker:
> >> - Building with Maven 3.3+ with Hadoop 2.7+ fails because Hadoop 2.7+
> pulls
> >> in Curator 2.7.1 which makes the ZooKeeperTests fail with
> >> NoSuchMethodError. The problem are the immutable reactor builds.
> Installing
> >> first the Hadoop-shaded dependencies and then running the build again
> >> solves the problem.
> >>
> >> Cheers,
> >> Till
> >>
> >> On Fri, Aug 4, 2017 at 10:50 AM, Stefan Richter <
> >> s.rich...@data-artisans.com
> >>> wrote:
> >>
> >>> +1
> >>>
> >>> 1. Cluster tests
> >>>
> >>> Tested with the stateful state machine job on the following settings:
> >>> - Cloud env: AWS
> >>> - Distributions: EMR 5.7.0
> >>> - Flink deployment method: YARN (Hadoop 2.7.2)
> >>> - HA: enabled
> >>> - Kerberos: disabled
> >>> - Kafka version: 0.10, 0.11
> >>> - State Backends: Heap (Sync / Async) & RocksDB (incremental / full)
> >>> - Filesystem: S3 and HDFS (Hadoop 2.7.2)
> >>> - Externalized checkpoints: enabled & disabled
> >>>
> >>> 2. Building with Scala 2.11 works
> >>>
> >>> 3. Building against Hadoop version works
> >>>
>  Am 04.08.2017 um 08:00 schrieb Aljoscha Krettek  >:
> 
>  Hi everyone,
> 
>  Please review and vote on the release candidate #3 for the version
> >>> 1.3.2, as follows:
>  [ ] +1, Approve the release
>  [ ] -1, Do not approve the release (please provide specific comments)
> 
> 
>  The complete staging area is available for your review, which
> includes:
>  * JIRA release notes [1],
>  * the official Apache source release and binary convenience releases
> to
> >>> be deployed to dist.apache.org [2], which is signed with the key with
> >>> fingerprint 0xA8F4FD97121D7293 [3],
>  * all artifacts to be deployed to the Maven Central Repository [4],
>  * source code tag "release-1.3.2-rc2" [5],
>  * website pull req

Re: [VOTE] Release 1.3.2, release candidate #3

2017-08-04 Thread Aljoscha Krettek
+1

 - checked checksums and signatures
 - did extensive cluster testing on GCE with dataproc and a lot of different 
combinations of backends and configuration settings

Tested configuration combinations (from the testing coordination document):
 - (aljoscha, GCE, dataproc, Rocks(incremental), YARN per-job (no HA), no 
kerberos, Kafka 0.10.x, OK) 
 - (aljoscha, GCE, dataproc, Rocks(no incremental), YARN per-job (no HA), no 
kerberos, Kafka 0.10.x, OK) 
 - (aljoscha, GCE, dataproc, Rocks(incremental), YARN per-job (HA), no 
kerberos, Kafka 0.10.x, OK) 
 - (aljoscha, GCE, dataproc, Rocks(no incremental), YARN per-job (HA), no 
kerberos, Kafka 0.10.x, OK) 
 - (aljoscha, GCE, dataproc, FileBackend(sync), YARN per-job (no HA), no 
kerberos, Kafka 0.10.x, ONGOING) 
 - (aljoscha, GCE, dataproc, FileBackend(async), YARN per-job (HA), no 
kerberos, Kafka 0.10.x, ONGOING) 

> On 4. Aug 2017, at 16:57, Fabian Hueske  wrote:
> 
> +1
> 
> - checked added and modified Maven dependencies since Flink 1.3.1
>  - io.dropwizard.metrics/metrics-core was upgraded to 3.1.5 and is ASL
>  - org.xerial.snappy/snappy-java/1.0.5 was added and is ASL
> - checked flink-runtime-web for new files with non-Apache licenses
> 
> Cheers, Fabian
> 
> 2017-08-04 13:32 GMT+02:00 Tzu-Li (Gordon) Tai :
> 
>> +1
>> 
>> - Functional cluster test results carried over from RC2 (did not retest
>> those for RC3 as the additional commit fix for Gelly should be irrelevant)
>> - Built from source with Scala 2.10 and 2.11 (MacOSX), successful
>> - Checked to-be-released Maven artifacts
>> 
>> Cheers,
>> Gordon
>> 
>> On 4 August 2017 at 5:43:09 PM, Till Rohrmann (trohrm...@apache.org)
>> wrote:
>> 
>> +1
>> 
>> - Checked checksums and GPG files
>> - Sources contain no binaries
>> - Check all POM files point to the same version
>> - Read readme file
>> - Build from source archive with Hadoop 2.8.1
>> - Test start up scripts for local and standalone mode
>> - Test job submission for multi-node cluster
>> - Plan-visualizer
>> - Verified quickstarts for IntelliJ and Eclipse
>> - Tested SBT
>> 
>> Caveat but not a blocker:
>> - Building with Maven 3.3+ with Hadoop 2.7+ fails because Hadoop 2.7+ pulls
>> in Curator 2.7.1 which makes the ZooKeeperTests fail with
>> NoSuchMethodError. The problem are the immutable reactor builds. Installing
>> first the Hadoop-shaded dependencies and then running the build again
>> solves the problem.
>> 
>> Cheers,
>> Till
>> 
>> On Fri, Aug 4, 2017 at 10:50 AM, Stefan Richter <
>> s.rich...@data-artisans.com
>>> wrote:
>> 
>>> +1
>>> 
>>> 1. Cluster tests
>>> 
>>> Tested with the stateful state machine job on the following settings:
>>> - Cloud env: AWS
>>> - Distributions: EMR 5.7.0
>>> - Flink deployment method: YARN (Hadoop 2.7.2)
>>> - HA: enabled
>>> - Kerberos: disabled
>>> - Kafka version: 0.10, 0.11
>>> - State Backends: Heap (Sync / Async) & RocksDB (incremental / full)
>>> - Filesystem: S3 and HDFS (Hadoop 2.7.2)
>>> - Externalized checkpoints: enabled & disabled
>>> 
>>> 2. Building with Scala 2.11 works
>>> 
>>> 3. Building against Hadoop version works
>>> 
 Am 04.08.2017 um 08:00 schrieb Aljoscha Krettek :
 
 Hi everyone,
 
 Please review and vote on the release candidate #3 for the version
>>> 1.3.2, as follows:
 [ ] +1, Approve the release
 [ ] -1, Do not approve the release (please provide specific comments)
 
 
 The complete staging area is available for your review, which includes:
 * JIRA release notes [1],
 * the official Apache source release and binary convenience releases to
>>> be deployed to dist.apache.org [2], which is signed with the key with
>>> fingerprint 0xA8F4FD97121D7293 [3],
 * all artifacts to be deployed to the Maven Central Repository [4],
 * source code tag "release-1.3.2-rc2" [5],
 * website pull request listing the new release and adding announcement
>>> blog post [6].
 
 The only change in this RC compared to the last is this commit that
>>> fixes copying of the Gelly examples jar: https://github.com/apache/
>>> flink/commit/fda455e23a6192200b85b84d3aee312c9be40c99. I would therefore
>>> like to propose a shorter voting period because the last RC was seemingly
>>> good except for this bug. Please voice your concerns about this if you
>> have
>>> any. The vote will be open for at least 24 hours. It is adopted by
>> majority
>>> approval, with at least 3 PMC affirmative votes.
 
 Please use the provided document, as discussed before, for coordinating
>>> the testing efforts: [7]. I have copied over the cluster testing efforts
>>> and functional testing efforts since the code of this RC is exactly the
>>> same as RC2.
 
 Thanks,
 Aljoscha
 
 [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>>> projectId=12315522&version=12340984
 [2] http://people.apache.org/~aljoscha/flink-1.3.2-rc3/
 [3] https://dist.apache.org/repos/dist/release/flink/KEYS
 [4] https://repository.apach

Re: [VOTE] Release 1.3.2, release candidate #3

2017-08-04 Thread Fabian Hueske
+1

- checked added and modified Maven dependencies since Flink 1.3.1
  - io.dropwizard.metrics/metrics-core was upgraded to 3.1.5 and is ASL
  - org.xerial.snappy/snappy-java/1.0.5 was added and is ASL
- checked flink-runtime-web for new files with non-Apache licenses

Cheers, Fabian

2017-08-04 13:32 GMT+02:00 Tzu-Li (Gordon) Tai :

> +1
>
> - Functional cluster test results carried over from RC2 (did not retest
> those for RC3 as the additional commit fix for Gelly should be irrelevant)
> - Built from source with Scala 2.10 and 2.11 (MacOSX), successful
> - Checked to-be-released Maven artifacts
>
> Cheers,
> Gordon
>
> On 4 August 2017 at 5:43:09 PM, Till Rohrmann (trohrm...@apache.org)
> wrote:
>
> +1
>
> - Checked checksums and GPG files
> - Sources contain no binaries
> - Check all POM files point to the same version
> - Read readme file
> - Build from source archive with Hadoop 2.8.1
> - Test start up scripts for local and standalone mode
> - Test job submission for multi-node cluster
> - Plan-visualizer
> - Verified quickstarts for IntelliJ and Eclipse
> - Tested SBT
>
> Caveat but not a blocker:
> - Building with Maven 3.3+ with Hadoop 2.7+ fails because Hadoop 2.7+ pulls
> in Curator 2.7.1 which makes the ZooKeeperTests fail with
> NoSuchMethodError. The problem are the immutable reactor builds. Installing
> first the Hadoop-shaded dependencies and then running the build again
> solves the problem.
>
> Cheers,
> Till
>
> On Fri, Aug 4, 2017 at 10:50 AM, Stefan Richter <
> s.rich...@data-artisans.com
> > wrote:
>
> > +1
> >
> > 1. Cluster tests
> >
> > Tested with the stateful state machine job on the following settings:
> > - Cloud env: AWS
> > - Distributions: EMR 5.7.0
> > - Flink deployment method: YARN (Hadoop 2.7.2)
> > - HA: enabled
> > - Kerberos: disabled
> > - Kafka version: 0.10, 0.11
> > - State Backends: Heap (Sync / Async) & RocksDB (incremental / full)
> > - Filesystem: S3 and HDFS (Hadoop 2.7.2)
> > - Externalized checkpoints: enabled & disabled
> >
> > 2. Building with Scala 2.11 works
> >
> > 3. Building against Hadoop version works
> >
> > > Am 04.08.2017 um 08:00 schrieb Aljoscha Krettek :
> > >
> > > Hi everyone,
> > >
> > > Please review and vote on the release candidate #3 for the version
> > 1.3.2, as follows:
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific comments)
> > >
> > >
> > > The complete staging area is available for your review, which includes:
> > > * JIRA release notes [1],
> > > * the official Apache source release and binary convenience releases to
> > be deployed to dist.apache.org [2], which is signed with the key with
> > fingerprint 0xA8F4FD97121D7293 [3],
> > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > * source code tag "release-1.3.2-rc2" [5],
> > > * website pull request listing the new release and adding announcement
> > blog post [6].
> > >
> > > The only change in this RC compared to the last is this commit that
> > fixes copying of the Gelly examples jar: https://github.com/apache/
> > flink/commit/fda455e23a6192200b85b84d3aee312c9be40c99. I would therefore
> > like to propose a shorter voting period because the last RC was seemingly
> > good except for this bug. Please voice your concerns about this if you
> have
> > any. The vote will be open for at least 24 hours. It is adopted by
> majority
> > approval, with at least 3 PMC affirmative votes.
> > >
> > > Please use the provided document, as discussed before, for coordinating
> > the testing efforts: [7]. I have copied over the cluster testing efforts
> > and functional testing efforts since the code of this RC is exactly the
> > same as RC2.
> > >
> > > Thanks,
> > > Aljoscha
> > >
> > > [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > projectId=12315522&version=12340984
> > > [2] http://people.apache.org/~aljoscha/flink-1.3.2-rc3/
> > > [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> > > [4] https://repository.apache.org/content/repositories/
> > orgapacheflink-1135/
> > > [5] https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h=
> > d0f7528a69a93045f4af686347753f8737ca0b2b
> > > [6] https://github.com/apache/flink-web/pull/75
> > > [7] https://docs.google.com/document/d/1OJAE6scAZXbSBaGNNEpU1qhr3Jb7b
> > wbec7keEnT9hmA/edit?usp=sharing
> >
> >
>


Re: [VOTE] Release 1.3.2, release candidate #3

2017-08-04 Thread Piotr Nowojski
+1 

- Sources contain no binaries
- Build and test (men clean verify) from source archive, found some issues 
previously mentioned by Nico
- Build with Scala 2.10/Scala 2.11
- Test start up scripts for local and standalone mode

Cluster tests:
- verified that FlinkKafkaProducer010 provides at-least-once semantic (or at 
least that it provides it some cases that were not working in 1.3.1)

Piotrek

> On Aug 4, 2017, at 1:32 PM, Tzu-Li (Gordon) Tai  wrote:
> 
> +1
> 
> - Functional cluster test results carried over from RC2 (did not retest those 
> for RC3 as the additional commit fix for Gelly should be irrelevant)
> - Built from source with Scala 2.10 and 2.11 (MacOSX), successful
> - Checked to-be-released Maven artifacts
> 
> Cheers,
> Gordon
> 
> On 4 August 2017 at 5:43:09 PM, Till Rohrmann (trohrm...@apache.org) wrote:
> 
> +1 
> 
> - Checked checksums and GPG files 
> - Sources contain no binaries 
> - Check all POM files point to the same version 
> - Read readme file 
> - Build from source archive with Hadoop 2.8.1 
> - Test start up scripts for local and standalone mode 
> - Test job submission for multi-node cluster 
> - Plan-visualizer 
> - Verified quickstarts for IntelliJ and Eclipse 
> - Tested SBT 
> 
> Caveat but not a blocker: 
> - Building with Maven 3.3+ with Hadoop 2.7+ fails because Hadoop 2.7+ pulls 
> in Curator 2.7.1 which makes the ZooKeeperTests fail with 
> NoSuchMethodError. The problem are the immutable reactor builds. Installing 
> first the Hadoop-shaded dependencies and then running the build again 
> solves the problem. 
> 
> Cheers, 
> Till 
> 
> On Fri, Aug 4, 2017 at 10:50 AM, Stefan Richter > wrote: 
> 
>> +1 
>> 
>> 1. Cluster tests 
>> 
>> Tested with the stateful state machine job on the following settings: 
>> - Cloud env: AWS 
>> - Distributions: EMR 5.7.0 
>> - Flink deployment method: YARN (Hadoop 2.7.2) 
>> - HA: enabled 
>> - Kerberos: disabled 
>> - Kafka version: 0.10, 0.11 
>> - State Backends: Heap (Sync / Async) & RocksDB (incremental / full) 
>> - Filesystem: S3 and HDFS (Hadoop 2.7.2) 
>> - Externalized checkpoints: enabled & disabled 
>> 
>> 2. Building with Scala 2.11 works 
>> 
>> 3. Building against Hadoop version works 
>> 
>>> Am 04.08.2017 um 08:00 schrieb Aljoscha Krettek : 
>>> 
>>> Hi everyone, 
>>> 
>>> Please review and vote on the release candidate #3 for the version 
>> 1.3.2, as follows: 
>>> [ ] +1, Approve the release 
>>> [ ] -1, Do not approve the release (please provide specific comments) 
>>> 
>>> 
>>> The complete staging area is available for your review, which includes: 
>>> * JIRA release notes [1], 
>>> * the official Apache source release and binary convenience releases to 
>> be deployed to dist.apache.org [2], which is signed with the key with 
>> fingerprint 0xA8F4FD97121D7293 [3], 
>>> * all artifacts to be deployed to the Maven Central Repository [4], 
>>> * source code tag "release-1.3.2-rc2" [5], 
>>> * website pull request listing the new release and adding announcement 
>> blog post [6]. 
>>> 
>>> The only change in this RC compared to the last is this commit that 
>> fixes copying of the Gelly examples jar: https://github.com/apache/ 
>> flink/commit/fda455e23a6192200b85b84d3aee312c9be40c99. I would therefore 
>> like to propose a shorter voting period because the last RC was seemingly 
>> good except for this bug. Please voice your concerns about this if you have 
>> any. The vote will be open for at least 24 hours. It is adopted by majority 
>> approval, with at least 3 PMC affirmative votes. 
>>> 
>>> Please use the provided document, as discussed before, for coordinating 
>> the testing efforts: [7]. I have copied over the cluster testing efforts 
>> and functional testing efforts since the code of this RC is exactly the 
>> same as RC2. 
>>> 
>>> Thanks, 
>>> Aljoscha 
>>> 
>>> [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa? 
>> projectId=12315522&version=12340984 
>>> [2] http://people.apache.org/~aljoscha/flink-1.3.2-rc3/ 
>>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS 
>>> [4] https://repository.apache.org/content/repositories/ 
>> orgapacheflink-1135/ 
>>> [5] https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h= 
>> d0f7528a69a93045f4af686347753f8737ca0b2b 
>>> [6] https://github.com/apache/flink-web/pull/75 
>>> [7] https://docs.google.com/document/d/1OJAE6scAZXbSBaGNNEpU1qhr3Jb7b 
>> wbec7keEnT9hmA/edit?usp=sharing 
>> 
>> 



[jira] [Created] (FLINK-7371) user defined aggregator assumes nr of arguments smaller or equal than number of row fields

2017-08-04 Thread Stefano Bortoli (JIRA)
Stefano Bortoli created FLINK-7371:
--

 Summary: user defined aggregator assumes nr of arguments smaller 
or equal than number of row fields
 Key: FLINK-7371
 URL: https://issues.apache.org/jira/browse/FLINK-7371
 Project: Flink
  Issue Type: Bug
  Components: Table API & SQL
Affects Versions: 1.3.1
Reporter: Stefano Bortoli


The definition of user define aggregations with a number of parameters larger 
than the row fields causes ArrayIndexOutOfBoundsException because the indexing 
is based on a linear iteration over row fields. This does not consider cases 
where fields can be used more than once and constant values are passed to the 
aggregation function.

for example:
{code}
window(partition {} order by [2] rows between $5 PRECEDING and CURRENT ROW aggs 
[myAgg($0, $1, $3, $0, $4)])
{code}

where $3 and $4 are reference to constants, and $0 and $1 are fields causes:

{code}
java.lang.ArrayIndexOutOfBoundsException: 4
at 
org.apache.flink.table.plan.schema.RowSchema.mapIndex(RowSchema.scala:134)
at 
org.apache.flink.table.plan.schema.RowSchema$$anonfun$mapAggregateCall$1.apply(RowSchema.scala:147)
at 
org.apache.flink.table.plan.schema.RowSchema$$anonfun$mapAggregateCall$1.apply(RowSchema.scala:147)
at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at 
org.apache.flink.table.plan.schema.RowSchema.mapAggregateCall(RowSchema.scala:147)
at 
org.apache.flink.table.plan.nodes.datastream.DataStreamOverAggregate$$anonfun$9.apply(DataStreamOverAggregate.scala:362)
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Re: [VOTE] Release 1.3.2, release candidate #3

2017-08-04 Thread Tzu-Li (Gordon) Tai
+1

- Functional cluster test results carried over from RC2 (did not retest those 
for RC3 as the additional commit fix for Gelly should be irrelevant)
- Built from source with Scala 2.10 and 2.11 (MacOSX), successful
- Checked to-be-released Maven artifacts

Cheers,
Gordon

On 4 August 2017 at 5:43:09 PM, Till Rohrmann (trohrm...@apache.org) wrote:

+1 

- Checked checksums and GPG files 
- Sources contain no binaries 
- Check all POM files point to the same version 
- Read readme file 
- Build from source archive with Hadoop 2.8.1 
- Test start up scripts for local and standalone mode 
- Test job submission for multi-node cluster 
- Plan-visualizer 
- Verified quickstarts for IntelliJ and Eclipse 
- Tested SBT 

Caveat but not a blocker: 
- Building with Maven 3.3+ with Hadoop 2.7+ fails because Hadoop 2.7+ pulls 
in Curator 2.7.1 which makes the ZooKeeperTests fail with 
NoSuchMethodError. The problem are the immutable reactor builds. Installing 
first the Hadoop-shaded dependencies and then running the build again 
solves the problem. 

Cheers, 
Till 

On Fri, Aug 4, 2017 at 10:50 AM, Stefan Richter  wrote: 

> +1 
> 
> 1. Cluster tests 
> 
> Tested with the stateful state machine job on the following settings: 
> - Cloud env: AWS 
> - Distributions: EMR 5.7.0 
> - Flink deployment method: YARN (Hadoop 2.7.2) 
> - HA: enabled 
> - Kerberos: disabled 
> - Kafka version: 0.10, 0.11 
> - State Backends: Heap (Sync / Async) & RocksDB (incremental / full) 
> - Filesystem: S3 and HDFS (Hadoop 2.7.2) 
> - Externalized checkpoints: enabled & disabled 
> 
> 2. Building with Scala 2.11 works 
> 
> 3. Building against Hadoop version works 
> 
> > Am 04.08.2017 um 08:00 schrieb Aljoscha Krettek : 
> > 
> > Hi everyone, 
> > 
> > Please review and vote on the release candidate #3 for the version 
> 1.3.2, as follows: 
> > [ ] +1, Approve the release 
> > [ ] -1, Do not approve the release (please provide specific comments) 
> > 
> > 
> > The complete staging area is available for your review, which includes: 
> > * JIRA release notes [1], 
> > * the official Apache source release and binary convenience releases to 
> be deployed to dist.apache.org [2], which is signed with the key with 
> fingerprint 0xA8F4FD97121D7293 [3], 
> > * all artifacts to be deployed to the Maven Central Repository [4], 
> > * source code tag "release-1.3.2-rc2" [5], 
> > * website pull request listing the new release and adding announcement 
> blog post [6]. 
> > 
> > The only change in this RC compared to the last is this commit that 
> fixes copying of the Gelly examples jar: https://github.com/apache/ 
> flink/commit/fda455e23a6192200b85b84d3aee312c9be40c99. I would therefore 
> like to propose a shorter voting period because the last RC was seemingly 
> good except for this bug. Please voice your concerns about this if you have 
> any. The vote will be open for at least 24 hours. It is adopted by majority 
> approval, with at least 3 PMC affirmative votes. 
> > 
> > Please use the provided document, as discussed before, for coordinating 
> the testing efforts: [7]. I have copied over the cluster testing efforts 
> and functional testing efforts since the code of this RC is exactly the 
> same as RC2. 
> > 
> > Thanks, 
> > Aljoscha 
> > 
> > [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa? 
> projectId=12315522&version=12340984 
> > [2] http://people.apache.org/~aljoscha/flink-1.3.2-rc3/ 
> > [3] https://dist.apache.org/repos/dist/release/flink/KEYS 
> > [4] https://repository.apache.org/content/repositories/ 
> orgapacheflink-1135/ 
> > [5] https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h= 
> d0f7528a69a93045f4af686347753f8737ca0b2b 
> > [6] https://github.com/apache/flink-web/pull/75 
> > [7] https://docs.google.com/document/d/1OJAE6scAZXbSBaGNNEpU1qhr3Jb7b 
> wbec7keEnT9hmA/edit?usp=sharing 
> 
> 


Re: [VOTE] Release 1.3.2, release candidate #3

2017-08-04 Thread Till Rohrmann
+1

- Checked checksums and GPG files
- Sources contain no binaries
- Check all POM files point to the same version
- Read readme file
- Build from source archive with Hadoop 2.8.1
- Test start up scripts for local and standalone mode
- Test job submission for multi-node cluster
- Plan-visualizer
- Verified quickstarts for IntelliJ and Eclipse
- Tested SBT

Caveat but not a blocker:
- Building with Maven 3.3+ with Hadoop 2.7+ fails because Hadoop 2.7+ pulls
in Curator 2.7.1 which makes the ZooKeeperTests fail with
NoSuchMethodError. The problem are the immutable reactor builds. Installing
first the Hadoop-shaded dependencies and then running the build again
solves the problem.

Cheers,
Till

On Fri, Aug 4, 2017 at 10:50 AM, Stefan Richter  wrote:

> +1
>
> 1. Cluster tests
>
> Tested with the stateful state machine job on the following settings:
> - Cloud env: AWS
> - Distributions: EMR 5.7.0
> - Flink deployment method: YARN (Hadoop 2.7.2)
> - HA: enabled
> - Kerberos: disabled
> - Kafka version: 0.10, 0.11
> - State Backends: Heap (Sync / Async) & RocksDB (incremental / full)
> - Filesystem: S3 and HDFS (Hadoop 2.7.2)
> - Externalized checkpoints: enabled & disabled
>
> 2. Building with Scala 2.11 works
>
> 3. Building against Hadoop version works
>
> > Am 04.08.2017 um 08:00 schrieb Aljoscha Krettek :
> >
> > Hi everyone,
> >
> > Please review and vote on the release candidate #3 for the version
> 1.3.2, as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> >
> > The complete staging area is available for your review, which includes:
> > * JIRA release notes [1],
> > * the official Apache source release and binary convenience releases to
> be deployed to dist.apache.org [2], which is signed with the key with
> fingerprint 0xA8F4FD97121D7293 [3],
> > * all artifacts to be deployed to the Maven Central Repository [4],
> > * source code tag "release-1.3.2-rc2" [5],
> > * website pull request listing the new release and adding announcement
> blog post [6].
> >
> > The only change in this RC compared to the last is this commit that
> fixes copying of the Gelly examples jar: https://github.com/apache/
> flink/commit/fda455e23a6192200b85b84d3aee312c9be40c99. I would therefore
> like to propose a shorter voting period because the last RC was seemingly
> good except for this bug. Please voice your concerns about this if you have
> any. The vote will be open for at least 24 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
> >
> > Please use the provided document, as discussed before, for coordinating
> the testing efforts: [7]. I have copied over the cluster testing efforts
> and functional testing efforts since the code of this RC is exactly the
> same as RC2.
> >
> > Thanks,
> > Aljoscha
> >
> > [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12315522&version=12340984
> > [2] http://people.apache.org/~aljoscha/flink-1.3.2-rc3/
> > [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> > [4] https://repository.apache.org/content/repositories/
> orgapacheflink-1135/
> > [5] https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h=
> d0f7528a69a93045f4af686347753f8737ca0b2b
> > [6] https://github.com/apache/flink-web/pull/75
> > [7] https://docs.google.com/document/d/1OJAE6scAZXbSBaGNNEpU1qhr3Jb7b
> wbec7keEnT9hmA/edit?usp=sharing
>
>


[jira] [Created] (FLINK-7370) rework operator documentation

2017-08-04 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-7370:
--

 Summary: rework operator documentation
 Key: FLINK-7370
 URL: https://issues.apache.org/jira/browse/FLINK-7370
 Project: Flink
  Issue Type: Improvement
  Components: Documentation, Streaming
Affects Versions: 1.3.0, 1.4.0
Reporter: Nico Kruber
Assignee: Nico Kruber


The structure of the operator documentation could be improved the following way:

- Create category {{Streaming/Operators}}.
- Move {{Streaming/Overview/DataStream Transformations}} to 
{{Streaming/Operators/Overview}}.
- Move {{ProcessFunction}}, {{Windows}}, and {{Async IO}} to 
{{Streaming/Operators}}
- create any necessary redirects for old URLs



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Re: [VOTE] Release 1.3.2, release candidate #3

2017-08-04 Thread Stefan Richter
+1

1. Cluster tests

Tested with the stateful state machine job on the following settings:
- Cloud env: AWS
- Distributions: EMR 5.7.0
- Flink deployment method: YARN (Hadoop 2.7.2)
- HA: enabled
- Kerberos: disabled
- Kafka version: 0.10, 0.11
- State Backends: Heap (Sync / Async) & RocksDB (incremental / full)
- Filesystem: S3 and HDFS (Hadoop 2.7.2)
- Externalized checkpoints: enabled & disabled

2. Building with Scala 2.11 works

3. Building against Hadoop version works

> Am 04.08.2017 um 08:00 schrieb Aljoscha Krettek :
> 
> Hi everyone,
> 
> Please review and vote on the release candidate #3 for the version 1.3.2, as 
> follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
> 
> 
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release and binary convenience releases to be 
> deployed to dist.apache.org [2], which is signed with the key with 
> fingerprint 0xA8F4FD97121D7293 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "release-1.3.2-rc2" [5],
> * website pull request listing the new release and adding announcement blog 
> post [6]. 
> 
> The only change in this RC compared to the last is this commit that fixes 
> copying of the Gelly examples jar: 
> https://github.com/apache/flink/commit/fda455e23a6192200b85b84d3aee312c9be40c99.
>  I would therefore like to propose a shorter voting period because the last 
> RC was seemingly good except for this bug. Please voice your concerns about 
> this if you have any. The vote will be open for at least 24 hours. It is 
> adopted by majority approval, with at least 3 PMC affirmative votes.
> 
> Please use the provided document, as discussed before, for coordinating the 
> testing efforts: [7]. I have copied over the cluster testing efforts and 
> functional testing efforts since the code of this RC is exactly the same as 
> RC2.
> 
> Thanks,
> Aljoscha
> 
> [1] 
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12340984
> [2] http://people.apache.org/~aljoscha/flink-1.3.2-rc3/
> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> [4] https://repository.apache.org/content/repositories/orgapacheflink-1135/
> [5] 
> https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h=d0f7528a69a93045f4af686347753f8737ca0b2b
> [6] https://github.com/apache/flink-web/pull/75
> [7] 
> https://docs.google.com/document/d/1OJAE6scAZXbSBaGNNEpU1qhr3Jb7bwbec7keEnT9hmA/edit?usp=sharing



[jira] [Created] (FLINK-7369) Add more information for `Key group index out of range of key group range` exception

2017-08-04 Thread Benedict Jin (JIRA)
Benedict Jin created FLINK-7369:
---

 Summary: Add more information for `Key group index out of range of 
key group range` exception
 Key: FLINK-7369
 URL: https://issues.apache.org/jira/browse/FLINK-7369
 Project: Flink
  Issue Type: Improvement
  Components: Local Runtime
Reporter: Benedict Jin
Assignee: Benedict Jin


When i got the following exception log, it make me confused that the index is 
more than `32` or less than `16`. So, we should add more information for it.

```java
java.lang.IllegalArgumentException: Key group index out of range of key group 
range [16, 32).
at 
org.apache.flink.runtime.state.heap.NestedMapsStateTable.setMapForKeyGroup(NestedMapsStateTable.java:104)
at 
org.apache.flink.runtime.state.heap.NestedMapsStateTable.putAndGetOld(NestedMapsStateTable.java:218)
at 
org.apache.flink.runtime.state.heap.NestedMapsStateTable.put(NestedMapsStateTable.java:207)
at 
org.apache.flink.runtime.state.heap.NestedMapsStateTable.put(NestedMapsStateTable.java:145)
at 
org.apache.flink.runtime.state.heap.HeapValueState.update(HeapValueState.java:72)
at 
org.apache.flink.cep.operator.AbstractKeyedCEPPatternOperator.updateNFA(AbstractKeyedCEPPatternOperator.java:276)
at 
org.apache.flink.cep.operator.AbstractKeyedCEPPatternOperator.processElement(AbstractKeyedCEPPatternOperator.java:171)
at 
org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:206)
at 
org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:69)
at 
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:263)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
at java.lang.Thread.run(Thread.java:745)
```



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)