+1 (non-binding) Really excited for this! Looking forward to help on new
view designs and such in #sig-ui as well :D
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Wed, Mar 3, 2021 at 11:22 AM Ian Stanton
wrote:
> +1 (non-binding). Excellent work!
&
Same here. I personally really liked the aggregation of articles from the
community talking about Airflow. I spent some time reading them.
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Wed, Feb 17, 2021 at 10:33 AM Tomasz Urbaszek
wrote:
> Per
Yass! 🎉 🎉 🎉 🎉 🎉 🎉
Great news!
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Thu, Dec 17, 2020 at 11:00 AM Tomasz Urbaszek
wrote:
> There's official Apache Airflow blogpost with similar content to Ash mail:
> https://airflow.apache.org
+1 (non-binding) !
Tested taskflow api tutorial in basic local setup. Tested a simple internal
TFX example on Airflow in the same setup.
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Wed, Dec 16, 2020 at 6:00 PM Maxime Beauchemin <
maxi
Fair enough.
I would argue then to have a small example (like the one pandas to csv and
GCS) added to core as an example one maybe? Its fine to leave it in
examples folder otherwise
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Wed, Dec 2, 2020 at 11
while providing extensibility.
What do you all think?
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Wed, Dec 2, 2020 at 10:25 AM Tomasz Urbaszek
wrote:
> > Then you could have XComBackendSerializationBackend
>
> That's definitely som
Would be interested to also know possible ways to do what Luciano
described. Hopefully w the serialized DAG and the new API we can start just
pushing the DAG to the DB (wishful thinking)?
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Mon, Nov 2, 2020 a
ve SubDags, would be curious
on people usages of SubDags and to check if there's any use case that does
not fit TaskGroups.
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Fri, Sep 18, 2020 at 6:34 AM Kaxil Naik wrote:
> Hi all,
>
> One of the t
Agree on keeping those separate, just intervened as I believe its a great
idea. But lets keep @beam and @spark to a separate thread.
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Tue, Sep 1, 2020 at 2:14 PM Tomasz Urbaszek wrote:
> Daniel is righ
I would be highly in favour of having a generic Beam operator. Similar
to @spark_task decorator. Something where you can easily define and wrap a
beam pipeline and convert it to an Airflow operator.
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Tue,
would need to ask the user to parametrize
task_id and that's a bit counter intuitive as Airflow task_id are not
templatized. Another option is to make this behaviour customizable and have
a flag that activates it on the TaskGroup.
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twi
Formatting wise, it seems it got formatted nicely just copy pasting so I
don't think I will modify this.
Thanks everyone for your contributions!
Best,
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Thu, Aug 27, 2020 at 12:08 PM Kaxil Naik wrote:
>
Okay, then if noone objects before Monday 8AM MST, I'll assume lazy
consensus and bring the document text into the main CWiki page for AIP.
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Thu, Aug 27, 2020 at 2:21 AM Jarek Potiuk
wrote:
> LGTM
&
migrate, but I would be a +1 on that vote.
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Tue, Aug 25, 2020 at 12:20 PM Jarek Potiuk
wrote:
> I also prefer Github Issues than either CWiki or PR.
>
> I think there is little contention and usually,
In that direction, I would prefer having markdown docs and PRs instead
actually, similarly to how github.com/tensorflow/community does it.
Thoughts?
Will do a second pass to the later today :D
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Tue, Aug 25
+ 1 (non-binding)
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Tue, Aug 25, 2020 at 7:18 AM Felix Uellendall
wrote:
> +1 (binding)
>
> Felix
>
> On Tue, Aug 25, 2020 at 15:15, James Coder wrote:
>
> > +1 (non-binding)
>
by Max tomorrow and add details based on
that as needed. If someone wants to do that before tomorrow afternoon,
please let me know!
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Mon, Aug 24, 2020 at 3:53 PM Maxime Beauchemin <
maximebeauche...@
I do have access, I will add it there once there's consensus (not sure if
we should submit this to vote).
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Fri, Aug 21, 2020 at 12:09 PM Ry Walker wrote:
> Proposal looks good to me Gerard - than
me know what you think about this :D
Best,
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
%3Cdev.airflow.apache.org%3E
Best,
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Thu, Aug 20, 2020 at 7:10 PM Yu Qian wrote:
> Hi, Gerard, yes I agree it's possible to do this at UI level without any
> fundamental change to the implementation. If ex
operator since it will introduce delays on
operator execution (as it will need to execute 1 dummy operator and that
can be expensive imo).
Overall though proposal looks good, unless anyone opposes it, I would move
this to vote mode :D
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.
Re graph times. That makes sense. Let me know what you find. We may be able
to contribute on the lazy loading part.
Looking forward to see the updated AIP!
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casassaez>
On Fri, Aug 14, 2020 at 6:14 AM Kaxil Naik
being an issue here. It may be
easily solvable by implementing lazy loading of the graph. Not sure how
easy to implement/add to the UI extension (and dont want to push for early
optimization as its the root of all evil).
Gerard Casas Saez
Twitter | Cortex | @casassaez <http://twitter.com/casass
Closing this vote after 48h as announced.
+5 (bindings) +2 (non-binding)
No negative votes or objections.
We will proceed with the current updated AIP. Thank you everyone!
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Jun 18, 2020, 11:02 AM -0600, Jarek Potiuk , wrote:
> +1 (binding)
to be changed to accommodate this and we will need to
find a way to have multiple task instances of a single task_id + lineage
visualization for understanding what event/task instance generated what
execution of a task instance.
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Jun 17, 2020, 10:
opers to comment/bring up objections
on the change.
If no objection is raised by Thursday June 18th at 1PM MST, we will move
forward with the change  and merge https://github.com/apache/airflow/pull/8962Â
PR.
Thanks everyone for all the work on this change! 🎉
Gerard Casas Saez
Twitter
How should we go about this? Is an AIP needed? GitHub issues?
Given most implementation for the backend seems to be done, it may be just
needed to do a few issues on GitHub and work on them.
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Jun 16, 2020, 2:07 AM -0600, Tomasz Urbaszek , wrote
relationship between upstream tasks to
downstream tasks is no longer 1:1 but 1:many.
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Jun 16, 2020, 7:00 AM -0600, 蒋晓峰 , wrote:
> Hello everyone,
>
> Sending a message to everyone and collecting feedback on the AIP-35 on
> adding signal-base
By XCom support before XComArg I meant as XCom parameters for operators. You
needed to use {{contex[‘ti’].xcom_pull(…)}} instead of using XComArg objects
like you can do as latest master.
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Jun 15, 2020, 5:02 PM -0600, Kaxil Naik , wrote:
> Is
face those dag.params in the Trigger Run UI
instead of asking the user to paste a JSON blob to overwrite them?
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Jun 15, 2020, 5:02 PM -0600, Kaxil Naik , wrote:
> Isn't it already possible using params (
> https://github.com/apache/airflo
topology is definitely an easier step to
focus on and will probs bring enough value.
Curious what are other people thoughts on this?
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Jun 12, 2020, 10:00 AM -0600, Dan Davydov ,
wrote:
> I think this is a great idea! One thing that I think we sho
nesParams + pipeline decorator.
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
how to best work on
this:Â https://apache-airflow.slack.com/archives/CCPRP7943/p1587600333439600
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Apr 21, 2020, 6:02 PM -0600, Kaxil Naik , wrote:
> I think if we plan to have call let's make it public so whoever else wants
> to work on
take some of the work myself.
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Apr 21, 2020, 2:58 PM -0600, Ash Berlin-Taylor , wrote:
> Before anyone starts cutting code for this, checkout the
> proof-of-concept that Gerard made here https://github.com/casassg/corrent/
>
> -a
>
&
Hi Kaxil,
Sorry, mainly blocked w other work at the moment. Hope to be able to take on
this soonish rather than later. If someone has more bandwidth and wants to go
ahead and start implementing, that would be great!
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Apr 21, 2020, 5:40 AM -0600
I wish, it seems I have no permissions to assign issues to myself.
I was thinking on taking from 8052 to 8055 since they are all closely related.
I can take on more once those are finished.
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Apr 1, 2020, 9:35 PM -0600, Kaxil Naik , wrote
them would be appreciated :D
Also, I would create a Milestone to track them together (just a suggestion
though).
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Mar 30, 2020, 11:55 AM -0600, Tomasz Urbaszek ,
wrote:
> Hi all,
>
> This vote has passed with:
> - 9 +1 votes (bindi
I did not cover. Otherwise, I will submit it for vote and start
breaking down the work in issues (I guess now GitHub issues?).
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Mar 3, 2020, 3:21 PM -0700, Evgeny Shulman ,
wrote:
> Hey Everybody
>
> I am happy to contribute to this di
Really like the proposal, updated AIP doc for now.
Updates:
• Changed name (as pointed by Ash)
• Updated example with new decorator interface
• Updated description of decorator to match the new proposed interface
• Added link to Airflow Slack thread into AIP for clarity
Gerard Casas Saez
tasks after that. Haven’t been able to find a good guideline for
this process, mainly looking at other completed AIP. Let me know if I should
wait more.
Best,
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Feb 24, 2020, 10:31 AM -0700, Jarek Potiuk , wrote:
> Ah yeah... I totally forgot ab
tant we get the interface 95%+ right from the
> > get-go since it could be hard to change later once users start depending on
> > it.
> >
> > On Fri, Feb 21, 2020 at 12:07 PM Gerard Casas Saez
> > wrote:
> >
> > > Hi everyone,
> > >
> > >
GitHub PRs with the mentioned changes.
Please let me know if there’s any aspect that people feel strongly opinionatedÂ
against or aspects that are not clear and I should work on expanding further.
Best,
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
meanwhile. And
also allow for more visibility next week once published on Confluence.
Let me know if this seems appropriate or not :D
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Feb 5, 2020, 2:46 PM -0700, Gerard Casas Saez ,
wrote:
> Wrote this small proposalÂ
> https://docs.goog
frameworks that inspired the
proposal.
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Feb 5, 2020, 2:36 PM -0700, Dan Davydov ,
wrote:
> Ok let's try it, don't know if we're violating some Apache process here but
> I guess we'll find out :).
>
> On Wed, Feb 5,
Happy to drive this. What would be a good place to put this design doc?
Guessing confluence, not sure under what directory though.
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Feb 4, 2020, 1:18 PM -0700, Jarek Potiuk , wrote:
> +1 short design doc would be cool.
>
> wt., 4 lut 20
efit the Airflow community to grow. What do
you think? Any thoughts on functional DAGs? Would love to hear what are pros
and cons of this expansion. To me, it seems highly linked to the Lineage
discussion happening at the same time, as both discussions are around improving
data pipelines.
Thanks!
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
er than words, so here’s my
experimental code: https://github.com/casassg/corrent (ignore the awful name
and the injection part).
Gerard Casas Saez
Twitter | Cortex |Â @casassaez
On Jan 22, 2020, 8:40 PM -0700, Tao Feng , wrote:
> Thanks Bolke. For those that are not aware, my team is worki
47 matches
Mail list logo
