Re: Hanging Jenkins builds.

2017-04-22 Thread Ted Yu 
Looks like this might be the cause for the failed build (
https://builds.apache.org/view/Beam/job/beam_PreCommit_Java_MavenInstall/9927/console
):

/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_MavenInstall/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/CombineFns.java:145:
error: reference notfound
   * See {@link #compose()} or {@link #composeKeyed()}) for details.
 ^
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_MavenInstall/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/CombineFns.java:147:
warning - Tag @link:can't find composeKeyed() in
org.apache.beam.sdk.transforms.CombineFns.CoCombineResult

FYI

On Fri, Apr 21, 2017 at 11:17 PM, Aviem Zur  wrote:

> Hi all,
>
> Please be aware that Beam builds (precommit + postcommit validations) are
> hanging since a few hours ago.
>
> This seems to be a problem in builds of other projects as well (for
> example, Kafka).
>
> I've opened an INFRA ticket:
> https://issues.apache.org/jira/browse/INFRA-13949
>

Re: [DISCUSSION] Encouraging more contributions

2017-04-22 Thread tarush grover 
Hi All,

I can take few things. I have planned to contribute towards beam SQL DSL
but members can assign more things and will be happy to contribute towards
those tasks.

Regards,
Tarush

On Sat, 22 Apr 2017 at 8:40 PM, Mingmin Xu  wrote:

> Good point, could also disable the auto assignment when creating JIRA
> ticket. Now it goes to component leader directly.
>
> Sent from my iPhone
>
> > On Apr 22, 2017, at 7:34 AM, Ted Yu  wrote:
> >
> > +1
> >
> >> On Sat, Apr 22, 2017 at 7:31 AM, Aviem Zur  wrote:
> >>
> >> Hi all,
> >>
> >> I wanted to start a discussion about actions we can take to encourage
> more
> >> contributions to the project.
> >>
> >> A few points I've been thinking of:
> >>
> >> 1. Have people unassign themselves from issues they're not actively
> working
> >> on.
> >> 2. Have the community engage more in triage, improving tickets
> descriptions
> >> and raising concerns.
> >> 3. Clean house - apply (2) to currently open issues (over 800). Perhaps
> >> some can be closed.
> >>
> >> Thoughts? Ideas?
> >>
>

Re: [DISCUSSION] Encouraging more contributions

2017-04-22 Thread Mingmin Xu 
Good point, could also disable the auto assignment when creating JIRA ticket. 
Now it goes to component leader directly.

Sent from my iPhone

> On Apr 22, 2017, at 7:34 AM, Ted Yu  wrote:
> 
> +1
> 
>> On Sat, Apr 22, 2017 at 7:31 AM, Aviem Zur  wrote:
>> 
>> Hi all,
>> 
>> I wanted to start a discussion about actions we can take to encourage more
>> contributions to the project.
>> 
>> A few points I've been thinking of:
>> 
>> 1. Have people unassign themselves from issues they're not actively working
>> on.
>> 2. Have the community engage more in triage, improving tickets descriptions
>> and raising concerns.
>> 3. Clean house - apply (2) to currently open issues (over 800). Perhaps
>> some can be closed.
>> 
>> Thoughts? Ideas?
>>

Re: [DISCUSSION] Encouraging more contributions

2017-04-22 Thread Ted Yu 
+1

On Sat, Apr 22, 2017 at 7:31 AM, Aviem Zur  wrote:

> Hi all,
>
> I wanted to start a discussion about actions we can take to encourage more
> contributions to the project.
>
> A few points I've been thinking of:
>
> 1. Have people unassign themselves from issues they're not actively working
> on.
> 2. Have the community engage more in triage, improving tickets descriptions
> and raising concerns.
> 3. Clean house - apply (2) to currently open issues (over 800). Perhaps
> some can be closed.
>
> Thoughts? Ideas?
>

Re: [PROPOSAL] Remove KeyedCombineFn

2017-04-22 Thread Pei HE 
+1

On Sat, Apr 22, 2017 at 12:16 PM, Jean-Baptiste Onofré 
wrote:

> +1
>
> Regards
> JB
>
>
> On 04/21/2017 07:24 PM, Kenneth Knowles wrote:
>
>> Hi all,
>>
>> I propose that we remove KeyedCombineFn before the first stable release.
>>
>> I don't think it adds enough value for the complexity it adds to e.g.
>> CombineWithContext [1] and state [2, 3], and it doesn't seem to me that
>> users really use it when we might expect. I am happy to be demonstrated
>> wrong.
>>
>> It is very likely that you have never written [4, 5] or thought about
>> KeyedCombineFn. So for context, here are excepts from signatures just to
>> show the difference from CombineFn:
>>
>> CombineFn {
>>   AccumT createAccumulator();
>>   AccumT addInput(AccumT accum, InputT input);
>>   AccumT mergeAccumulators(Iterable accums);
>>   OutputT extractOutput(AccumT accum);
>> }
>>
>> KeyedCombineFn {
>>   AccumT createAccumulator(K key);
>>   AccumT addInput(K key, AccumT accum, InputT input);
>>   AccumT mergeAccumulators(K key, Iterable accums);
>>   OutputT extractOutput(K key, AccumT accum);
>> }
>>
>> So what are the particular reasons for this, versus a CombineFn that has
>> KVs as its input and accumulator types?
>>
>>  - There are some performance improvements potentially from not passing
>> keys around, based on the assumption they are always available.
>>
>>  - There is also a spec difference because it only has to be associative
>> and commutative per key, cannot be applied in a global combine, and
>> addInput is automatically key preserving.
>>
>> But in fact, in all of my code crawling the class is almost never used
>> (even over the course of its history at Google) and even the few uses I
>> found were often mistakes where the key is totally ignored, probably
>> because a user thinks "I am doing a keyed combine so I need a keyed
>> combine
>> function". So the number of users actually affected is about zero.
>>
>> I would be curious if anyone has a compelling case for keeping
>> KeyedCombineFn.
>>
>> Kenn
>>
>> [1]
>> https://github.com/yafengguo/Apache-beam/blob/master/sdks/ja
>> va/core/src/main/java/org/apache/beam/sdk/transforms/Combine
>> WithContext.java
>> [2] https://issues.apache.org/jira/browse/BEAM-1336
>> [3] https://github.com/apache/beam/pull/2627
>> [4]
>> https://github.com/search?l=Java=KeyedCombineFn=advsea
>> rch=Code=%E2%9C%93
>> [5] https://www.google.com/search?q=KeyedCombineFn
>>
>>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Hanging Jenkins builds.

2017-04-22 Thread Aviem Zur 
Hi all,

Please be aware that Beam builds (precommit + postcommit validations) are
hanging since a few hours ago.

This seems to be a problem in builds of other projects as well (for
example, Kafka).

I've opened an INFRA ticket:
https://issues.apache.org/jira/browse/INFRA-13949