reading from s3 file in aws

2017-06-21 Thread Jyotirmoy Sundi
Hi Folks, Is there any way to read from s3 buckets in beam, Trace: Exception in thread "main" java.lang.IllegalStateException: Failed to validate s3://my_bucket/path/to/input-*.csv at org.apache.beam.sdk.io.TextIO$Read$Bound.expand(TextIO.java:254) at org.apache.beam.sdk.io.TextIO$Read$Bound.e

Re: [DISCUSS] Apache Beam 2.1.0 release next week ?

2017-06-21 Thread Ismaël Mejía
Thahks JB for keeping the time based release agenda. I really don't have any blocker but I would like to have the hadoop version alignment PR merged before this one and probably also Nexmark (considering that Etienne fixed most of the issues and we have already the LGTM, we are just waiting for a l

Re: [DISCUSS] Apache Beam 2.1.0 release next week ?

2017-06-21 Thread Reuven Lax
Does mean that value-dependent FileBasedSink will miss 2.1.0, but I guess it will make 2.2.0 then. On Wed, Jun 21, 2017 at 7:23 PM, Jean-Baptiste Onofré wrote: > Hi guys, > > As we released 2.0.0 (first stable release) last month during ApacheCon, > and to maintain our release pace, I would like

Re: [DISCUSS] Apache Beam 2.1.0 release next week ?

2017-06-21 Thread Mingmin Xu
SQL would be ready in the next 2.2.0 version I think. There're still some tasks being developed, and may not finish next week. On Wed, Jun 21, 2017 at 7:58 PM, JingsongLee wrote: > Very nice to see this release. Include the merge of DSL_SQL? > Pleased to see BEAM-1612 can be completed. (Not bloc

Re: [DISCUSS] Apache Beam 2.1.0 release next week ?

2017-06-21 Thread JingsongLee
Very nice to see this release. Include the merge of DSL_SQL? Pleased to see BEAM-1612 can be completed. (Not blocker)https://issues.apache.org/jira/browse/BEAM-1612 Best, JingsongLee --From:Kenneth Knowles Time:2017 Jun 22 (Thu) 10:4

Re: [DISCUSS] Apache Beam 2.1.0 release next week ?

2017-06-21 Thread Kenneth Knowles
This sounds good to me. I have tagged some of my JIRAs with 2.1.0 as they are known issues users have hit since the release that should just take a minute to address. This is to pressure me, not to block the release :-) On Wed, Jun 21, 2017 at 7:23 PM, Jean-Baptiste Onofré wrote: > Hi guys, > >

Re: Reduced Availability from 17.6. - 24.6

2017-06-21 Thread JingsongLee
Have a good time~ Best, JingsongLee--From:Jean-Baptiste Onofré Time:2017 Jun 22 (Thu) 10:21To:dev Subject:Re: Reduced Availability from 17.6. - 24.6 Enjoy !! Well deserved ! Regards JB On 06/21/2017 11:37 PM, Davor Bonaci wrote: >

Re: Fwd: [Report] Eagle - June 2017

2017-06-21 Thread Jean-Baptiste Onofré
Interesting to see that. I will ping on their mailing list. Regards JB On 06/22/2017 12:09 AM, Davor Bonaci wrote: From this month's Eagle report: "Community is working on Alert engine based Apache Beam, led by YHD.com. https://github.com/1haodian/eagle/tree/alertenginebeam"; This could be a

[DISCUSS] Apache Beam 2.1.0 release next week ?

2017-06-21 Thread Jean-Baptiste Onofré
Hi guys, As we released 2.0.0 (first stable release) last month during ApacheCon, and to maintain our release pace, I would like to release 2.1.0 next week. This release would include lot of bug fixes and some new features: https://issues.apache.org/jira/projects/BEAM/versions/12340528 I'm v

Re: Fwd: [Report] Eagle - June 2017

2017-06-21 Thread Jyotirmoy Sundi
Would like to help have worked on we beam apps On Wed, Jun 21, 2017 at 7:14 PM JingsongLee wrote: > I'd like participate in doing some work. > > Best, JingsongLee > --From:Davor > Bonaci Time:2017 Jun 22 (Thu) 06:10To:dev < > dev@be

Re: Reduced Availability from 17.6. - 24.6

2017-06-21 Thread Jean-Baptiste Onofré
Enjoy !! Well deserved ! Regards JB On 06/21/2017 11:37 PM, Davor Bonaci wrote: To tag along on the same thread, I'm out through the Independence Day holiday. Back on 7/5. On Fri, Jun 16, 2017 at 10:03 PM, Aljoscha Krettek wrote: Hi, I’ll be on vacation next week, just in case anyone is wo

Re: Fwd: [Report] Eagle - June 2017

2017-06-21 Thread JingsongLee
I'd like participate in doing some work. Best, JingsongLee --From:Davor Bonaci Time:2017 Jun 22 (Thu) 06:10To:dev Subject:Fwd: [Report] Eagle - June 2017 From this month's Eagle report: "Community is working on Alert engine based Ap

Fwd: [Report] Eagle - June 2017

2017-06-21 Thread Davor Bonaci
>From this month's Eagle report: "Community is working on Alert engine based Apache Beam, led by YHD.com. https://github.com/1haodian/eagle/tree/alertenginebeam"; This could be an opportunity for someone in the Beam project to reach out, and help out with the cross-project integration. Any volunt

Re: Reduced Availability from 17.6. - 24.6

2017-06-21 Thread Davor Bonaci
To tag along on the same thread, I'm out through the Independence Day holiday. Back on 7/5. On Fri, Jun 16, 2017 at 10:03 PM, Aljoscha Krettek wrote: > Hi, > > I’ll be on vacation next week, just in case anyone is wondering why I’m > not responding. :-) > > Best, > Aljoscha