Fwd: Builds Meeting this Thursday

Hi all, I've found out about this presentation on Apache Builds. Sharing with dev@ in case it's of interest to anyone. See the agenda: https://cwiki.apache.org/confluence/display/INFRA/ASF+Builds+Agenda+2021-01-14 Best -P. -- Forwarded message - From: Jarek Potiuk Date: Tue, Jan 1

Re: Planning a freeze on website changes to merge new designs

OK, so the current status is that changes are being reviewed and merged into a separate website-revamp branch. There are several reasons we could want to do this (e.g. could the changes not be made incrementally, or was there a strong desire to launch the new site "all at once"), but at some point

Re: Planning a freeze on website changes to merge new designs

On Wed, Jan 13, 2021 at 9:09 AM Jakub Sadowski wrote: > Hi guys, > unfortunately option 1 is not really possible, because most of the new > content files are importing shortcodes files so if we merge only content > without layouts, these pages won't compile. > You could add the shortcode files o

Re: Can composite transforms have zero subtransforms?

I agree there's value in this. The cause of BEAM-11614 is a good example of your first instantiation. An optimization was added to the Convert transform to make it recognize when it's unneeded and short-circuit itself. This short-circuit was triggered (twice) within the SqlTransform used in sql_tax

Re: PTransform Annotations Proposal

Wrote a design draft for resource-related annotations. Please have a look: https://docs.google.com/document/d/1phExeGD1gdDI9M8LK4ZG57UGa7dswpB8Aj6jxWj4uQk/edit?usp=sharing Cheers, Mirac On 2020/11/26 20:20:09, Mirac Vuslat Basaran wrote: > Created a PR without unit tests at https://github.com/a

Re: [PROPOSAL] Preparing for Beam 2.28.0 release

Thank you Cham! On Wed, Jan 13, 2021 at 12:44 PM Rui Wang wrote: > Thanks Cham for working on this! > > > -Rui > > On Wed, Jan 13, 2021 at 11:32 AM Kyle Weaver wrote: > >> Thanks for stepping up Cham! >> >> Remember to mark critical JIRA issues as release blockers everybody! >> >> On Wed, Jan 1

Re: [PROPOSAL] Preparing for Beam 2.28.0 release

Thanks Cham for working on this! -Rui On Wed, Jan 13, 2021 at 11:32 AM Kyle Weaver wrote: > Thanks for stepping up Cham! > > Remember to mark critical JIRA issues as release blockers everybody! > > On Wed, Jan 13, 2021 at 11:25 AM Chamikara Jayalath > wrote: > >> Hi All, >> >> Beam 2.28.0 rel

Re: Can composite transforms have zero subtransforms?

I just want to highlight that "just returns its inputs" has a couple of non-useless instantiations: - programmatically generating a transform (as in SqlTransform) - rearranging the Java or Python data structures (like PCollectionList and PCollectionTuple) that are noops in the protocol buffers

JIRA Ticket triage - Beam Summit Website

Hi folks, I wanted to give a heads up that I am working on completing a JIRA ticket triage. I am starting with the Beam Summit Website tickets older than 2019. I will kick off the process by reaching out via a comment on the ticket or email to reporters who created tickets with a spreadsheet of a

Re: [PROPOSAL] Preparing for Beam 2.28.0 release

Thanks for stepping up Cham! Remember to mark critical JIRA issues as release blockers everybody! On Wed, Jan 13, 2021 at 11:25 AM Chamikara Jayalath wrote: > Hi All, > > Beam 2.28.0 release is scheduled to be cut on January 27th according to > the release calendar [1] > > I'd like to volunteer

[PROPOSAL] Preparing for Beam 2.28.0 release

Hi All, Beam 2.28.0 release is scheduled to be cut on January 27th according to the release calendar [1] I'd like to volunteer myself to be the release manager for this release. I plan on cutting the release branch on the scheduled date. Any comments or objections ? Thanks, Cham [1] https://ca

Re: Contributor permission for Beam Jira

Done. Welcome! On Wed, Jan 13, 2021 at 1:56 AM Raphael Sanamyan < raphael.sanam...@akvelon.com> wrote: > Hi, I am Raphael from Akvelon. > > I would like to be a beam contributor. > Could you please add me as a contributor to Beam's Jira issue tracker? I > would like to create/assign tickets? > >

Re: Planning a freeze on website changes to merge new designs

Hi guys, unfortunately option 1 is not really possible, because most of the new content files are importing shortcodes files so if we merge only content without layouts, these pages won't compile. The only reasonable option is to freeze the website and merge it then. My proposition is to merge it a

Re: Can composite transforms have zero subtransforms?

Thanks Robert. Docs PR is up: https://github.com/apache/beam/pull/13744 Brian On Tue, Jan 12, 2021 at 2:04 PM Robert Bradshaw wrote: > Yes, a PTansform can have no sub-transforms, as long as it only returns > its inputs. Updating the docs would be a good idea. > > On Tue, Jan 12, 2021 at 1:04 P

Re: ElasticsearchIO.Write() dynamic ES indices

It looks like you should be able to accomplish this with withIndexFn [1]. Does that work for you? Brian [1] https://beam.apache.org/releases/javadoc/2.27.0/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.Write.html#withIndexFn-org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO.Write.FieldV

Contributor permission for Beam Jira

Hi, I am Raphael from Akvelon. I would like to be a beam contributor. Could you please add me as a contributor to Beam's Jira issue tracker? I would like to create/assign tickets? My Jira username: raphael.sanamyan Thanks, Raphael

ElasticsearchIO.Write() dynamic ES indices

Hi all, I am writing to Elasticsearch using Beam (Google Dataflow). At the moment I pass index name to ElasticsearchIO.Write() as variable when deploying the dataflow. Instead of having one static index, I want to insert to multiple Elasticsearch indices, I have the index name on Pub/Sub topic, so