Re: Gradle Status [April 11]

just created one https://issues.apache.org/jira/browse/BEAM-4057 Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-12 18:15 GMT+02:00 Ahmet Altay <al...@google.com>: >> Found another blocker in current artifacts creations: there is n

Re: [PROPOSAL] Preparing 2.5.0 release next week

created https://issues.apache.org/jira/browse/BEAM-4057 Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-12 18:11 GMT+02:00 Scott Wegner <sweg...@google.com>: > Romain, could you please open a JIRA describing the requirements for the > gen

grade test setup different than maven one?

are not but of the setup). Any inputs or in progress todo i missed? Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book

Re: Gradle Status [April 11]

Found another blocker in current artifacts creations: there is not pom.xml and pom.properties in META-INF. This is used by tools + libraries + integrations so it is quite important to not break it. Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-12 0:59 GMT

Re: Updated [Proposal] Apache Beam Fn API : Defining and adding SDK Metrics

Maybe leave it out until proven it is needed. ATM counters are used a lot but others are less mainstream so being too fine from the start can just add complexity and bugs in impls IMHO. Le 12 avr. 2018 08:06, "Robert Bradshaw" a écrit : > By "type" of metric, I mean both

Re: [PROPOSAL] Preparing 2.5.0 release next week

e more than a week to > > finish that release. Even if that is not enough time to finish a release > it > > will put is in the proximity of the 5/5 target date. I prefer that > option, > > to another potential release with Maven. > > > > On Wed, Apr 11, 2018 at 1:

Re: [PROPOSAL] Preparing 2.5.0 release next week

Any hope the release is on central before the 30th? Le 11 avr. 2018 22:02, "Robert Bradshaw" a écrit : > +1 to keeping up the regular release cycle, but I don't think it makes > sense to cut until we're ready to actively work on the release. A cut date > two weeks from now

Re: Gradle Status [April 6]

while writing it so all the instructions were tested on that > version. I'll try testing the instructions on 2018 to see if I can > reproduce the issues people are having. > > On Wed, Apr 11, 2018 at 9:51 AM Lukasz Cwik <lc...@google.com> wrote: > >> I use 2017.3 and it ha

Re: Development of new HTTP source

? > > Thank you. > > 2018-04-11 10:50 GMT-03:00 Romain Manni-Bucau <rmannibu...@gmail.com>: > >> Hi Daniel, >> >> I know JB has a started a RestIO, it is not yet complete and needs >> some love but probably some opportunity for converge and collaboration

Re: Gradle Status [April 6]

Any of you using the idea 2018? the import works for me but then it is not as smooth as it seems for you. I'm just trying to see if it is a procedure thing or a version issue. Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-11 17:28 GMT+02:00 Kenneth

Re: Development of new HTTP source

Hi Daniel, I know JB has a started a RestIO, it is not yet complete and needs some love but probably some opportunity for converge and collaboration here Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-11 15:42 GMT+02:00 Daniel Salerno <sale

Re: Gradle Status [April 6]

configuration (properties and or plugin) like the ones defining the java versions I didn't check the profiles but it makes already some work ;) Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-11 6:50 GMT+02:00 Romain Manni-Bucau <rmannibu...@gmail.

Re: org.apache.beam.sdk.values.TupleTag#genId and stacktraces?

Le 10 avr. 2018 22:59, "Robert Bradshaw" <rober...@google.com> a écrit : On Tue, Apr 10, 2018 at 1:49 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > > Le 10 avr. 2018 21:25, "Robert Bradshaw" <rober...@google.com> a écrit : > > O

Re: Gradle Status [April 6]

would then need to use Maven to rebuild the whole project. It would be cool if Gradle could do this more reliably than Maven did. Hmm, i exactly experiment the opposite. Testes with idea 2017 and 2018. Reuven On Tue, Apr 10, 2018 at 8:46 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote

Re: org.apache.beam.sdk.values.TupleTag#genId and stacktraces?

Le 10 avr. 2018 21:25, "Robert Bradshaw" <rober...@google.com> a écrit : On Tue, Apr 10, 2018 at 12:10 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > This is interesting cause it leads to "why do the workers need to do it > again instead of reusin

Re: Gradle Status [April 6]

that I was messing with the build.gradle files. Anyhow it doesn't >> > really add much value. >> > >> > The gradle runner _is_ able to use submodules and run individual tests >> > methods, and all that. >> > >> > Kenn >> > >> > >> >

Re: Gradle Status [April 6]

are to just run the module...and i hope i dont need a 50chars long command specific each time to do it, just "build" would be good. Kenn On Tue, Apr 10, 2018 at 9:31 AM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Runner a test doesnt have the right classpath (idea uses o

Re: org.apache.beam.sdk.values.TupleTag#genId and stacktraces?

the machines agreed on the same > tuple ID, which could be accomplished with serialization. Serialization, > however, doesn't work well with static initializers, since those will have > been called to initialize the class at load time. > > On Tue, Apr 10, 2018 at 10:27 AM Romain Manni-Bucau

Re: org.apache.beam.sdk.values.TupleTag#genId and stacktraces?

, "Robert Bradshaw" <rober...@google.com> a écrit : If it's too slow perhaps you could use the constructor where you pass an explicit id (though in my experience walking the stack isn't that slow). On Tue, Apr 10, 2018 at 10:09 AM Romain Manni-Bucau <rmannibu...@gmail.com> wro

Re: org.apache.beam.sdk.values.TupleTag#genId and stacktraces?

time, you still go through the whole project and check all upstream deps it seems. It is <1s with maven vs 5-6s with gradle. On Tue, Apr 10, 2018 at 9:33 AM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Hi > > does org.apache.beam.sdk.values.TupleTag#genId need to get

Re: org.apache.beam.sdk.values.TupleTag#genId and stacktraces?

Oops cross post sorry. Issue i hit on this thread is it is used a lot in tests abd it slows down tests for nothing like with generatesequence ones Le 10 avr. 2018 19:00, "Romain Manni-Bucau" <rmannibu...@gmail.com> a écrit : > > > Le 10 avr. 2018 18:40, "Robert

org.apache.beam.sdk.values.TupleTag#genId and stacktraces?

Hi does org.apache.beam.sdk.values.TupleTag#genId need to get the stacktrace or can we use any id generator (like UUID.random().toString())? Using traces is quite slow under load and environments where the root stack is not just the "next" level so skipping it would be nice. Romain M

Re: Gradle Status [April 6]

with the right classpath and a fast compile step if needed. I lost literally 1h for something simple with that tooling, this is way too much to be acceptable on my side since I'm sadly not paid to work on beam (one day maybe ;)). Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github

Re: Gradle Status [April 6]

Ok, didn't find a way to make it working properly (only workaround with direct commands and no good idea integration for debugging). I'm back with maven, if anyone knows how to properly solve it let's do it. If not I think JB point is to consider more than any other criteria. Romain Manni-Bucau

Re: Gradle Status [April 6]

side note: do NOT use auto-import until you are sure you can, it locks regularly on beam (pby too big for idea?) and makes idea ready to be killed :( Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré <j...@nanthrax.

Re: Gradle Status [April 6]

: java.lang.NoClassDefFoundError: net/bytebuddy/NamingStrategy ? Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-10 15:13 GMT+02:00 Lukasz Cwik <lc...@google.com>: > I have found that the simplest setup is to delegate the build/test actions &g

Re: Gradle Status [April 6]

need like some SPI registration which are used by all our registrar => no way to run tests in idea without hacking the configuration quite deeply :( Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-10 10:08 GMT+02:00 Etienne Chauchot <echauc...@apac

Re: [PROPOSAL] Preparing 2.5.0 release next week

tar ball uploaded to www.apache.org/dist/ > > <http://www.apache.org/dist/> will contain the gradle build files > allowing one > > to reproduce the artifacts (jars (source/test/javadoc)/poms). > > > > On Mon, Apr 9, 2018 at 3:44 PM Romain Manni-Bucau <rmannibu.

Re: Gradle Status [April 6]

ifiable. We now have a lot of configurations not making much sense too until you scripted yourself the script or know beam well. This doesnt enable contribs :(. Hope it is clearer On Mon, Apr 9, 2018 at 2:49 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > +1 gradle not being mains

Re: Gradle Status [April 6]

pful. >>>> >>>> The goal is really to simplify the contribution. >>>> >>>> Do you agree if I add a Jira about "Gradle polish" ? >>>> Thoughts ? >>>> >>>> Regards >>>> JB >>>

Re: [PROPOSAL] Preparing 2.5.0 release next week

I got the same with that PR applied and the previous command. Is using your fork needed? Is there any PR to import it? In any case master is not ready to be released with that yet - to come back to the actual topic. Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book

Re: [PROPOSAL] Preparing 2.5.0 release next week

@Lukasz: same with gradlew and release option, pom is empty (no parent, no dependencies, no more description - needed since central poms use that for doc purposes). Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Ol

Re: [PROPOSAL] Preparing 2.5.0 release next week

http://maven.apache.org/xsd/maven-4.0.0.xsd; xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance;> 4.0.0 org.apache.beam beam-sdks-java-core 2.5.0-SNAPSHOT Apache Beam :: SDKs :: Java :: Core Doesn't seem that ready ;) Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau

Re: [PROPOSAL] Preparing 2.5.0 release next week

I will check now what's the pom status, if they are ok it can be worth testing gradle Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rm

Re: building on top of filesystem, can beam help?

ath forward here? Then you can build on top of VFS instead, and simply layer VfsFilesystem on top of it when running on Beam. On Fri, Apr 6, 2018 at 1:23 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Partially. Will run with beam in half of the cases or without in the > rem

Re: building on top of filesystem, can beam help?

.com> a écrit : > So is this project of yours also built on top of Beam, or is it unrelated? > > On Fri, Apr 6, 2018 at 1:12 PM Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > >> Issues forking are: >> >> 1. I have to drop beam FileIO (in all its f

Re: building on top of filesystem, can beam help?

eam > might make changes to it that cause problems for your other project. I > would recommend starting by forking if it serves your needs. > > Reuven > > On Fri, Apr 6, 2018 at 8:17 AM Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > >> Hi guys, >> >

Re: Gradle Status [April 6]

le in a pom file to be clearer? On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Hi Scott, > > is it right that 2 doesn't handle the hierachy anymore and that it doesn't > handle profiles for runners as it is currently with maven? > > &g

building on top of filesystem, can beam help?

subproject (or at least submodule) and add the configuration hint in the API? Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibuc

Re: [PROPOSAL] Preparing 2.5.0 release next week

+1 to get 2.5 out asap (fixes blockers so always good to let it be upgraded) +1000 to split beam releases (and even repos) by concerns Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibuca

Re: Gradle migration fixit: April 3

pick up the pace. > > Reuven > > On Wed, Apr 4, 2018 at 5:52 AM Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > >> Hi guys, >> >> Anyone has global a view of where we are? When I left yesterday we were >> far to be consummable by maven us

Re: Gradle migration fixit: April 3

inks issue with gradle? If i dont gradle clean maven doesnt build anymore cause of it (ending up in OOME cause it loops on sym links). Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com>

Re: Gradle migration fixit: April 3

+1 also ensuring each optional task has a switch off flag will be important (currently i cant use my computer while building with gradle which is a big lost of time in a day) Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/>

Re: Gradle migration fixit: April 3

:33 AM Reuven Lax <re...@google.com> wrote: > >> Here is the Kanban board tracking all the current tasks. It looks like 7 >> of them have already been closed over the past two days, so we're down to >> 19! >> >> https://issues.apache.org/jira/secure/RapidBoar

Re: Gradle migration fixit: April 3

Yep - sorry if it was unclear. I know linux distro often do it (never understood why though). Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://g

Re: Gradle migration fixit: April 3

adle. > > That being said, if this is interesting to users@ we can notify them as > well. > > Reuven > > On Thu, Mar 29, 2018 at 1:45 PM Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > >> >> >> Le 29 mars 2018 21:20, "Reuven Lax" <re...@google.com

Re: Gradle migration fixit: April 3

Le 29 mars 2018 21:20, "Reuven Lax" <re...@google.com> a écrit : On Thu, Mar 29, 2018 at 12:17 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > > > Le 29 mars 2018 20:35, "Reuven Lax" <re...@google.com> a écrit : > > 1. As L

Re: Gradle migration fixit: April 3

one or two weeks to delete the poms (assuming that Beam is ready at the end of the day). Can you try to put a date of that and we will communicate on it publicly if anyone else does (i dont think but i didnt expect you to do it too ;)). On Thu, Mar 29, 2018 at 5:20 AM Romain Manni-Bucau <rmannibu.

Re: Gradle migration fixit: April 3

Le 29 mars 2018 18:44, "Lukasz Cwik" <lc...@google.com> a écrit : On Thu, Mar 29, 2018 at 5:20 AM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Hi Reuven, a few questions: > > 1. any inputs on how we can work on the jenkins part? Do we test it live > wi

Re: Gradle migration fixit: April 3

Hi Reuven, a few questions: 1. any inputs on how we can work on the jenkins part? Do we test it live wiht "fake" PRs? 2. What's the rational to not start by deleting the poms? Sounds like it will be a day working on gradle and on the 4th we'll be back on maven Romain Manni-Bucau @r

waitUntilFinish@direct runner

Hi guys, any objection to merge https://github.com/apache/beam/pull/4790? Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibuc

Re: All SDKs on all Runners (portability virtual team)

ave replacements/optims but by default it shouldnt require anything to use a 100% java runner IMHO. There is still code which is specific that integrates those components onto the runners native execution plan. On Mon, Mar 26, 2018 at 1:21 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: &g

Re: All SDKs on all Runners (portability virtual team)

Hi Lukasz, Did you started to do it generically (ie outside a particular runner impl) using pipeline pre visitor or so, or is it still hardcoded in runner? It is key to not bind it to runners since this is a transversal feature which doesnt require more than the primitives the runner supports to

Re: Releases and user support

Is it something we can add now the 2.4 is out? Le 13 mars 2018 16:02, "Jean-Baptiste Onofré" <j...@nanthrax.net> a écrit : > You can also take a look on karaf download page about the schedule and > active table. > > Regards > JB > Le 13 mars 2018, à 0

Re: Migrating to Gradle: Community Fixit day

Hi Reuven I can try to help on the 3rd (don't forget you are in the future for me so you can need to launch it on the 2nd for you maybe) but not on the 28th :(. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Ol

Re: Apache Beam 2.4.0 release process retrospective and automation possibilities

eeds to sign, there could be a script for > one to download artifacts, sign them, and push the signatures. On success, > an email body (with all the links and details) would be created, to be sent > out by a human. > > More responses inline below (and thanks for the feedback!). >

Re: Beam Summit - IO brainstorming

+1000 for record metadata (camel headers) For python it can be interesting to "just" generate python styled IO API and use jython under the hood to let python users code as they know but reuse all the beam ecosystem - including runners! The other way around implies a lot of work for the community

Re: Gradle status

Can work Reuven, where is the "todo" list? Thought we were done (= the replacement was not blocking the dev) multiple times due to other threads but reading this week mails it sounds like we are not yet here. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau>

Re: Apache Beam 2.4.0 release process retrospective and automation possibilities

Hi Le 23 mars 2018 04:29, "Alan Myrvold" a écrit : Robert explained his experience with the release process as the release engineer for 2.4.0, and we discussed the prototype shell script for checking release progress in

Re: Gradle status

the cost to move to gradle, it doesnt work for the project do please another option (rollbacking gradle or removing maven, all other options are negative for the project and a pain for committers and contributors whatever you think). Thanks, Dan On Thu, Mar 22, 2018 at 10:30 AM, Romain Manni

Re: Gradle status

tes the release > artifacts, although it is not hooked up yet with authentication. > > I expect to help make incremental progress over the next month converting > some of the integration tests, but welcome incremental improvements from > others. > > > > On Thu, Mar 22, 20

Re: Gradle status

ill alone) > > > On Thu, Mar 22, 2018 at 9:32 AM Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > >> @Valentyn: concretely any user can PR and be part of that process so >> anyone can do it wrong (me first) >> @Luskasz, Hennking: fine but what

Re: Gradle status

Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book <htt

Re: Gradle status

pain point. > > > On Thu, Mar 22, 2018 at 9:30 AM, Romain Manni-Bucau > <rmannibu...@gmail.com> wrote: > > hey guys, > > > > 2.4 is out, do we plan to drop all maven descriptors tomorrow or on > monday? > > > > > > Romain Manni-Bucau > &g

Re: [ANNOUNCE] Apache Beam 2.4.0 released

congrats guys Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in

Re: Gradle status

hey guys, 2.4 is out, do we plan to drop all maven descriptors tomorrow or on monday? Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rm

Re: beam 3?

Probably too quickly done but can be used as a start here is a first list: https://gist.github.com/rmannibucau/ab7543c23b6f57af921d98639fbcd436 Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Ol

Re: beam 3?

what the team is thinking. > > Regards > JB > > On 03/21/2018 09:26 AM, Romain Manni-Bucau wrote: > > > > > > 2018-03-21 9:13 GMT+01:00 Jean-Baptiste Onofré <j...@nanthrax.net > > <mailto:j...@nanthrax.net>>: > > > > Hi Romain,

Re: beam 3?

to mimic the Message Headers in > Apache > Camel. It would allow us to have more dynamic IOs and transforms, and give > some > additional statements to the runners. > +1 > > I'm proposing to start a vote to create the 2.x branch and move master to > Beam > 3.0.0-SNAP

beam 3?

at all) 4. make runners decorable properly and not just forked each time you need to modify some behavior for a particular case (+ indeed all the issues we hit and saw on the list) Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.ne

Re: PipelineOptions strict mode broken?

e would still be silent and not obvious if we dont split the configs in namespaces which are easier to understand. That said it wouldnt be a blocker anymore, you are right. > > > On Tue, Mar 20, 2018 at 9:26 AM Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > >> Hi gu

PipelineOptions strict mode broken?

ptions are requested they would validate the prefix spark.*. This is not perfect but current state with a single eager registry for the validation is very hardly usable as soon as you try to industrialize beam in something else than a main :(. Any one has an idea to not break the API? Romain Manni

Re: Implementing @OnWindowExpiration in StatefulParDo [BEAM-1589]

Hi Huygaa, Cant it be predefined timers? Romain Le 20 mars 2018 00:52, "Huygaa Batsaikhan" a écrit : Hi everyone, I am working on BEAM-1589 . In short, currently, there is no default way of saving/flushing state before a

Re: [VOTE] Release 2.4.0, release candidate #3

-0 (cause of the teardown issue which is still a blocker), otherwise spark/direct runners work in my projects Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github &

Re: slack @the-asf?

Hi guys, finally managed to send the invite to everybody, time to move guys :) Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibuc

Re: (java) stream & beam?

while it's true that raw GBK can't be fluent (due to constraint on >> element type). once we have schema support we can introduce groupByField, >> and that can be fluent. >> >> >> On Wed, Mar 14, 2018 at 11:50 PM Robert Bradshaw <rober...@google.com> >> wrote: >&g

Re: (java) stream & beam?

Le 15 mars 2018 07:50, "Robert Bradshaw" <rober...@google.com> a écrit : On Wed, Mar 14, 2018 at 11:04 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Le 15 mars 2018 06:52, "Robert Bradshaw" <rober...@google.com> a écrit : >> The stream AP

Re: Beam 2.4.0

Done, thanks. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in

Re: Beam 2.4.0

up, know it missed the 2.4 but can https://github.com/apache/beam/pull/4790 have some love? it really makes beam pretty unusable with direct runner, I start to have "// workaround for BEAM-3409" everywhere in my codebase which is quite bothering after 3 months :( Romain Manni-Bucau @r

Re: slack @the-asf?

have a big button, you enter your apache.org mail and can log in directly. What did you see? > > On Wed, Mar 14, 2018 at 9:19 AM, Romain Manni-Bucau <rmannibu...@gmail.com > > wrote: > >> Can help but I have no idea how to do it yet, don't hesitate to ping me >

Re: slack @the-asf?

updated the slack message (feel free to rephrase/reformat it, the length is quite limited so it is ok) and will send the message just after this one on #general will need help for the website update on the 26th ;) Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog

slack @the-asf?

et history on slack as well which would be beneficial for beam too. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |

Re: (java) stream & beam?

/9c4ea86a59ae3b899498f389b5459d67c2b4cdcd/hazelcast-jet-core/src/main/java/com/hazelcast/jet/pipeline/StreamStage.java Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com&g

Re: (java) stream & beam?

her > > That said, the Collector API has some minimal, cosmetic differences, such > as CombineFn.addInput may either mutate the accumulator or return it. The > Collector accumulator method is a BiConsumer, meaning it must modify. > > On Tue, Mar 13, 2018 at 11:39 AM Romain Manni-Buca

Re: (java) stream & beam?

an entirely new DSL or SDK. > > [1] https://docs.oracle.com/javase/8/docs/api/java/util/ > stream/Collectors.html#groupingBy-java.util.function.Function- > > On Tue, Mar 13, 2018 at 11:06 AM Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > >> Yep >> &g

Re: Advice on parallelizing network calls in DoFn

for real pipelines which are way under 10ops/s/thread in general. On Tue, Mar 13, 2018 at 2:43 AM, Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Here are some figures (small warn is I did priviledge beam a lot in this > benchmark, a bit more than it should in a real impl,

Re: (java) stream & beam?

ettek" <aljos...@apache.org> a écrit : > https://beam.apache.org/blog/2016/05/27/where-is-my- > pcollection-dot-map.html > > On 11. Mar 2018, at 22:21, Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > > > > Le 12 mars 2018 00:16, "Reuven

Re: Releases and user support

Just to illustrate what I was looking for @beam: this kind of page https://tomcat.apache.org/tomcat-80-eol.html but maybe not that fine grained. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Ol

Re: Releases and user support

sounds good, let's wait this day (so tomorrow with the timezone issue) to see if there is any other proposals and if not start a vote tmr. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibuca

Re: Releases and user support

Works for me. What's the procedure to add it on the website (and where can we add it)? Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rm

Re: Releases and user support

on't forget big data is not used by "small" projects but mainly by companies only for obvious reasons ;)). If we think in terms of active branches, what does it mean? 2 max active branches at a time? > > My €0.01 > > Regards > JB > Le 13 mars 2018, à 01:23,

Re: Advice on parallelizing network calls in DoFn

(my username being rmannibucau): @Benchmark public void stringLength(final Blackhole blackhole) { blackhole.consume(System.getProperty("user.name").length()); } the execution is slower than completionfuture in beam case: Comparison.stringLength thrpt5 42428432,508 ±

Re: Releases and user support

osed is wrong - it is likely - but what is your alternative to try to make beam 1. better percieved, 2. usable by companies (integrable in a software plan)? > > > > On Tue, Mar 13, 2018 at 1:23 AM Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > >> Up? >> &g

Re: Releases and user support

. This is the (core dev) drawback but the advantage for the communities and companies using beam is that they know they can rely on it and plan migrations as needed to never be on a no more maintained version. wdyt? Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog

Re: Configuring file-based transforms with different options

Le 12 mars 2018 23:05, "Chamikara Jayalath" <chamik...@google.com> a écrit : On Mon, Mar 12, 2018 at 2:36 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > > > Le 12 mars 2018 22:22, "Chamikara Jayalath" <chamik...@google.com> a > écrit

Re: Configuring file-based transforms with different options

Le 12 mars 2018 22:22, "Chamikara Jayalath" <chamik...@google.com> a écrit : On Mon, Mar 12, 2018 at 12:42 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > > > Le 12 mars 2018 18:56, "Chamikara Jayalath" <chamik...@google.com> a > é

Re: Configuring file-based transforms with different options

erstood and it would be best if the way a user defines > these filesystems is the same in all languages because it would be annoying > to provide the same configuration (in slightly different ways) for Java, > Python, Go, ... > > On Fri, Mar 9, 2018 at 2:01 PM, Romain Manni-Bucau <rmannibu..

Re: Advice on parallelizing network calls in DoFn

.com> a écrit : > Do you have data that supports this? > > Note that in reality for something like passing an element between DoFns, > the constant in o(1) actually matters. Decreasing SDK harness overhead is a > good thing though. > > On Mon, Mar 12, 2018 at 10:1

Re: Configuring file-based transforms with different options

it would be best if the way a user defines > these filesystems is the same in all languages because it would be annoying > to provide the same configuration (in slightly different ways) for Java, > Python, Go, ... > > On Fri, Mar 9, 2018 at 2:01 PM, Romain Manni-Bucau <rmannibu...@gmail

Re: Advice on parallelizing network calls in DoFn

u the best of both worlds. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibu

<    1   2   3   4   5   >