I really do not think we should drop sqlite support. We use sqlite for local testing/development; I used it when doing my initial evaluation of Airflow; and there are regular comments in this group about people using sqlite locally for their workflows. It feels like a critical feature for a lot of use cases.
I also feel like we're kind of jumping the gun on pushing for 2.0 (is it just because 9 is the last single digit?). There is a lot of the code base that's pretty hairy and a number of things that are fairly non-performant / buggy. I don't think that we NEED to break backward compatibility to clean things up and fix things. I don't particularly like pickle or the old import style but I don't feel like supporting them is terribly onerous (I could be wrong). New webserver, releasing the API, and even timezones are all things that can be released in a non-breaking fashion. --George On Thu, Dec 14, 2017 at 2:20 PM Alek Storm <alek.st...@gmail.com> wrote: > We use sqlite for developing Airflow DAGs locally, and doing basic checks > for syntax/import errors in our CI/CD pipeline. > > Alek > > On Thu, Dec 14, 2017 at 4:19 PM, Chris Riccomini <criccom...@apache.org> > wrote: > > > Agree with Bolke that it's a good idea, but major work. I will push back > on > > this for 2.0 mostly due to time concerns. I don't want 2.0 to take months > > and months to get out. > > > > > Btw: what about dropping sqlite support? > > > > I'm fine with this, but isn't this really useful for demos? Seems to me > > that the out of the box experience from airflow is one of the things that > > make sit so compelling. > > > > On Thu, Dec 14, 2017 at 2:12 PM, Bolke de Bruin <bdbr...@gmail.com> > wrote: > > > > > That will take quite some work. It is a a good idea but also a major > > > change. Not sure if we should target that. > > > > > > Btw: what about dropping sqlite support? > > > > > > Verstuurd vanaf mijn iPad > > > > > > > Op 14 dec. 2017 om 21:19 heeft Gael Magnan <gaelmag...@gmail.com> > het > > > volgende geschreven: > > > > > > > > Hi, > > > > > > > > haven't been following much lately but on the import side of things, > > > isn't > > > > Airflow 2 the best moment to change to a pip plugin system for > imports > > of > > > > third party stuff? > > > > I.E being able to add a new type of credentials, operator etc.. > without > > > > touching to the airflow code itself or having them in a special > folder. > > > > > > > > Regards > > > > Gael > > > > > > > > > > > > > > > > Le jeu. 14 déc. 2017 à 14:17, Chris Riccomini <criccom...@apache.org > > > > a > > > > écrit : > > > > > > > >> I'm fine with sensor refactor. Added to Wiki. > > > >> > > > >> On Thu, Dec 14, 2017 at 11:16 AM, Chris Riccomini < > > > criccom...@apache.org> > > > >> wrote: > > > >> > > > >>> @Bolke, > > > >>> > > > >>>> Should we, before 2.0, start the graduation from the incubator? > > > >>> > > > >>> No, I'd rather keep them separate. We can certainly start > graduation, > > > but > > > >>> I don't want to block 2.0. Can pursue them in parallel. > > > >>> > > > >>> On Thu, Dec 14, 2017 at 11:15 AM, Andy Hadjigeorgiou < > > > >> andyxha...@gmail.com > > > >>>> wrote: > > > >>> > > > >>>> Does it make sense to include sensors.py refactor in 2.0, so we > can > > > >>>> retire the old import structure easily and support the new sensors > > > >> package > > > >>>> import structure? > > > >>>> > > > >>>> - Andy > > > >>>> > > > >>>> On Thu, Dec 14, 2017 at 2:12 PM, Driesprong, Fokko > > > <fo...@driesprong.frl > > > >>> > > > >>>> wrote: > > > >>>> > > > >>>>> Hi all, > > > >>>>> > > > >>>>> Good initiative. I would be happy to refactor the sensors > package. > > I > > > >>>>> started on it but it changes a lot, all the imports will break. > > > >>>>> > > > >>>>> https://github.com/apache/incubator-airflow/pull/2875 > > > >>>>> > > > >>>>> What do you guys think? > > > >>>>> > > > >>>>> Cheers, Fokko > > > >>>>> > > > >>>>> 2017-12-14 20:09 GMT+01:00 Chris Riccomini < > criccom...@apache.org > > >: > > > >>>>> > > > >>>>>> I have created a wiki here: > > > >>>>>> > > > >>>>>> https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0 > > > >>>>>> > > > >>>>>> To track features and progress. > > > >>>>>> > > > >>>>>> On Thu, Dec 14, 2017 at 11:08 AM, Chris Riccomini < > > > >>>>> criccom...@apache.org> > > > >>>>>> wrote: > > > >>>>>> > > > >>>>>>>> Re: #2: Is there a current ticket out for removing the legacy > > > >>>>> import > > > >>>>>>> style? > > > >>>>>>> > > > >>>>>>> No, I don't think so, but you can create one! :) > > > >>>>>>> > > > >>>>>>> On Thu, Dec 14, 2017 at 11:06 AM, Andy Hadjigeorgiou < > > > >>>>>> andyxha...@gmail.com > > > >>>>>>>> wrote: > > > >>>>>>> > > > >>>>>>>> This sounds great, something I'd like to see updated for 2.0 > > > >>>>> release (or > > > >>>>>>>> before) is the Airflow documentation > > > >>>>>>>> <http://airflow.readthedocs.io/en/latest/installation.html> ( > > > >>>>>>>> http://airflow.readthedocs.io/en/latest/installation.html). > It > > > >>>>> seems > > > >>>>>> that > > > >>>>>>>> updating the repo does not update this site - and given that > we > > > >>>>> will be > > > >>>>>>>> removing certain deprecated features I imagine the docs will > > > >> change > > > >>>>>>>> substantially. > > > >>>>>>>> > > > >>>>>>>> Re: #2: Is there a current ticket out for removing the legacy > > > >> import > > > >>>>>>>> style? > > > >>>>>>>> I'm happy to help drive that forward. > > > >>>>>>>> > > > >>>>>>>> - Andy > > > >>>>>>>> > > > >>>>>>>> On Thu, Dec 14, 2017 at 1:55 PM, Chris Riccomini < > > > >>>>> criccom...@apache.org > > > >>>>>>> > > > >>>>>>>> wrote: > > > >>>>>>>> > > > >>>>>>>>> Hey all, > > > >>>>>>>>> > > > >>>>>>>>> With 1.9.0 wrapping up soon (hopefully), there's been some > > > >>>>> discussion > > > >>>>>> on > > > >>>>>>>>> the having the next release be Airflow 2.0 (rather than > 1.10). > > > >>>>> This > > > >>>>>>>> would > > > >>>>>>>>> allow us to break compatibility, and clean up some stuff. > > > >> Proposed > > > >>>>>>>> things > > > >>>>>>>>> to include in 2.0 are: > > > >>>>>>>>> > > > >>>>>>>>> 1) New webserver that Joy Gao has been working on. > > > >>>>>>>>> 2) Remove the legacy import style that's been deprecated > since > > > >> at > > > >>>>>> least > > > >>>>>>>> 1.8 > > > >>>>>>>>> 3) New timzone feature > > > >>>>>>>>> 4) Move API out of experimental > > > >>>>>>>>> > > > >>>>>>>>> I want to keep the list fairly tight, preferably to things > that > > > >>>>> have > > > >>>>>>>>> already been done, so that we can ship it fairly quickly (in > > the > > > >>>>> next > > > >>>>>>>>> couple of months). > > > >>>>>>>>> > > > >>>>>>>>> Does this sound like a good list? > > > >>>>>>>>> > > > >>>>>>>>> Cheers, > > > >>>>>>>>> Chris > > > >> > > > > > >
