Hi all, I just tried to install a local Airflow instance within Docker, using Python 3.5, but I got an error on initdb. Anyone else experiencing this? It has to do with the schema migrations.
Cheers, Fokko 2018-07-09 22:56 GMT+02:00 Taylor Edmiston <tedmis...@gmail.com>: > We discussed the splitting out of hooks & operators from core internally at > Astronomer again today. I should be able to allocate some time to > splitting out hooks & operators for the purpose of speeding up the CI. A > faster CI on core would be hugely beneficial for all contributors. > > I created a Jira issue for this at https://issues.apache.org/ > jira/browse/AIRFLOW-2732. > > If anyone has pointers for this work, I'm all ears. I've done a lot with > Python unit tests in the past but this part of the Airflow codebase is new > to me. > > Taylor > > *Taylor Edmiston* > Blog <https://blog.tedmiston.com/> | CV > <https://stackoverflow.com/cv/taylor> | LinkedIn > <https://www.linkedin.com/in/tedmiston/> | AngelList > <https://angel.co/taylor> | Stack Overflow > <https://stackoverflow.com/users/149428/taylor-edmiston> > > > On Sun, Jul 1, 2018 at 6:02 AM, Bolke de Bruin <bdbr...@gmail.com> wrote: > > > Separating the tests is where the effort lies. So having that as a > > consequence of splitting the packages would be nice. It has come up a > > couple of times but it was not picked up unfortunately. > > > > B. > > > > > On 28 Jun 2018, at 08:32, Maxime Beauchemin < > maximebeauche...@gmail.com> > > wrote: > > > > > > It would be so nice to have a fast test suite. Having to wait for > Travis > > > for up to an hour makes many workflows (like working on a release) > super > > > painful. > > > > > > I spoke with folks at Astronomer recently about moving all operators > and > > > hooks to another Python package that airflow would import. This would > > allow > > > for independent test suites and to have a more regular release cadence > on > > > hooks and operators. What do you think? > > > > > > Max > > > > > > On Wed, Jun 27, 2018 at 11:18 PM Bolke de Bruin <bdbr...@gmail.com> > > wrote: > > > > > >> Arghhh. The downside of doing this late at night and wanting to go to > > >> bed... :-). Will make a new one > > >> > > >> Sent from my iPhone > > >> > > >>> On 28 Jun 2018, at 00:07, Chris Fei <cfe...@gmail.com> wrote: > > >>> > > >>> Great, thank you! I just took this for a quick spin and it looks like > > >>> there's DB migration task missing. The task you committed just > > recently, > > >>> 9635ae0956e7_index_faskfail.py, has a down_revision of 856955da8476 > > >>> which can't be found when running airflow initdb (seehttps:// > > >> github.com/apache/incubator-airflow/tree/v1-10-test/airflow/ > > migrations/versions > > >> ). > > >>> Chris > > >>> > > >>> > > >>>> On Wed, Jun 27, 2018, at 5:09 PM, Bolke de Bruin wrote: > > >>>> Hi All, > > >>>> > > >>>> I have created a sdist package that is available at: > > >>>> > > >>>> > > >> http://people.apache.org/~bolke/apache-airflow-1.10.0b3+incu > > bating.tar.gz > > >>>> < > > >> http://people.apache.org/~bolke/apache-airflow-1.10.0b3+incu > > bating.tar.gz>> > > >> > > >>>> In order to distinguish it from an actual (apache) release it is: > > >>>> > > >>>> 1. Marked as beta (python package managers do not install beta > > >>>> versions by default - PEP 440)> 2. It is not signed > > >>>> 3. It is not at an official apache distribution location > > >>>> > > >>>> You can also put something like this in a requirements.txt file: > > >>>> > > >>>> git+ > > >>>> > > >> https://github.com/apache/incubator-airflow@v1-10-test#egg= > > apache-airflow[celery,crypto,emr,hive,hdfs,ldap,mysql, > > postgres,redis,slack,s3 > > >>>> < > > >> https://github.com/apache/incubator-airflow@v1-10-test#egg= > > apache-airflow[celery,crypto,emr,hive,hdfs,ldap,mysql, > > postgres,redis,slack,s3 > > >>> > > >>>> ]> <https://github.com/rodrigc/incubator-airflow@master#egg=apache- > > >>>> airflow[celery,crypto,emr,hive,hdfs,ldap,mysql,postgres,redi > > s,slack,s- > > >>>> 3] > > >>>> < > > >> https://github.com/rodrigc/incubator-airflow@master#egg=apac > > he-airflow[celery,crypto,emr,hive,hdfs,ldap,mysql,postgres, > > redis,slack,s3][1] > > >>> > > >>>>>> > > >>>> and then "pip install -r requirements.txt”. > > >>>> > > >>>> I hope that after this beta we can go to RC and start voting on > 1.10.> > > >>>> Cheers > > >>>> Bolke > > >>> > > >>> > > >>> Links: > > >>> > > >>> 1. > > >> https://github.com/rodrigc/incubator-airflow@master#egg=apac > > he-airflow[celery,crypto,emr,hive,hdfs,ldap,mysql,postgres, > > redis,slack,s3]%20%3Chttps://github.com/rodrigc/incubator- > > airflow@master#egg=apache-airflow[celery,crypto,emr, > > hive,hdfs,ldap,mysql,postgres,redis,slack,s3] > > <https://github.com/rodrigc/incubator-airflow@master#egg= > apache-airflow[celery,crypto,emr,hive,hdfs,ldap,mysql, > postgres,redis,slack,s3]%20%3Chttps://github.com/rodrigc/ > incubator-airflow@master%23egg=apache-airflow[celery, > crypto,emr,hive,hdfs,ldap,mysql,postgres,redis,slack,s3]> > > >> <https://github.com/rodrigc/incubator-airflow@master#egg=apa > > che-airflow[celery,crypto,emr,hive,hdfs,ldap,mysql,postgres, > > redis,slack,s3]%20%3Chttps://github.com/rodrigc/incubator- > > airflow@master%23egg=apache-airflow[celery,crypto,emr, > > hive,hdfs,ldap,mysql,postgres,redis,slack,s3]> > > >> > > > > >
MacBook-Pro-van-Fokko:cardio-ai fokkodriesprong$ docker run -t -i -p 8080:8080 python:3.5 bash root@88775efd72a9:/# pip install http://people.apache.org/~bolke/apache-airflow-1.10.0b3+incubating.tar.gz Collecting http://people.apache.org/~bolke/apache-airflow-1.10.0b3+incubating.tar.gz Downloading http://people.apache.org/~bolke/apache-airflow-1.10.0b3+incubating.tar.gz (4.4MB) 100% |████████████████████████████████| 4.4MB 6.5MB/s Collecting alembic<0.9,>=0.8.3 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/f0/7d/7fcda63887d9726e0145e98802baf374ec8cf889325e469194cd7926c98e/alembic-0.8.10.tar.gz (976kB) 100% |████████████████████████████████| 983kB 19.2MB/s Collecting bleach==2.1.2 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/07/59/06f368f350cd7f226606c2bc7210c6b336312df62185de333c66d6180512/bleach-2.1.2-py2.py3-none-any.whl Collecting configparser<3.6.0,>=3.5.0 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/7c/69/c2ce7e91c89dc073eb1aa74c0621c3eefbffe8216b3f9af9d3885265c01c/configparser-3.5.0.tar.gz Collecting croniter<0.4,>=0.3.17 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/27/69/2426b76c3647d8cc959e88aadecfb7ad96cf2c40c0ea784b6bf43a844c73/croniter-0.3.24.tar.gz Collecting dill<0.3,>=0.2.2 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/6f/78/8b96476f4ae426db71c6e86a8e6a81407f015b34547e442291cd397b18f3/dill-0.2.8.2.tar.gz (150kB) 100% |████████████████████████████████| 153kB 12.1MB/s Collecting flask<0.13,>=0.12.4 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/2e/48/f1936dadac2326b3d73f2fe0a964a87d16be16eb9d7fc56f09c1bea3d17c/Flask-0.12.4-py2.py3-none-any.whl (81kB) 100% |████████████████████████████████| 81kB 18.2MB/s Collecting flask-appbuilder<2.0.0,>=1.11.1 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/56/24/9e02d295fdc280059b5d10fb131a746367842bac8535c0ca6bdce232f2de/Flask-AppBuilder-1.11.1.tar.gz (1.5MB) 100% |████████████████████████████████| 1.5MB 8.6MB/s Collecting flask-admin==1.4.1 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/39/65/eb6238c40cd9ec20279969ef9ee7ac412fc7c9a6681965bcd58a63eeac2b/Flask-Admin-1.4.1.tar.gz (922kB) 100% |████████████████████████████████| 931kB 22.3MB/s Collecting flask-caching<1.4.0,>=1.3.3 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/88/f1/7bbd68a4d79eb53c33863a17926f14268749b897c8a77ab589f2e9117d47/Flask_Caching-1.3.3-py2.py3-none-any.whl Collecting flask-login==0.2.11 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/42/3c/ead3f50b8a39b6dd3499ae6f0f5b13b955130c92a7479a287e2e07921faf/Flask-Login-0.2.11.tar.gz Collecting flask-swagger==0.2.13 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/68/97/4e31ac3dc4a44a4b7487eab8404a68c871b57a15811e189862d0bf0c5b55/flask-swagger-0.2.13.tar.gz Collecting flask-wtf<0.15,>=0.14.2 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/60/3a/58c629472d10539ae5167dc7c1fecfa95dd7d0b7864623931e3776438a24/Flask_WTF-0.14.2-py2.py3-none-any.whl Collecting funcsigs==1.0.0 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/09/8d/17528625d12ca90651dd1f7958fd0d32b23b15f2197023372669fd683321/funcsigs-1.0.0-py2.py3-none-any.whl Collecting future<0.17,>=0.16.0 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/00/2b/8d082ddfed935f3608cc61140df6dcbf0edea1bc3ab52fb6c29ae3e81e85/future-0.16.0.tar.gz (824kB) 100% |████████████████████████████████| 829kB 20.7MB/s Collecting gitpython>=2.0.2 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/ac/c9/96d7c86c623cb065976e58c0f4898170507724d6b4be872891d763d686f4/GitPython-2.1.10-py2.py3-none-any.whl (449kB) 100% |████████████████████████████████| 450kB 19.2MB/s Collecting gunicorn<20.0,>=19.4.0 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/8c/da/b8dd8deb741bff556db53902d4706774c8e1e67265f69528c14c003644e6/gunicorn-19.9.0-py2.py3-none-any.whl (112kB) 100% |████████████████████████████████| 122kB 7.5MB/s Collecting iso8601>=0.1.12 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/ef/57/7162609dab394d38bbc7077b7ba0a6f10fb09d8b7701ea56fa1edc0c4345/iso8601-0.1.12-py2.py3-none-any.whl Collecting jinja2<2.9.0,>=2.7.3 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/67/ea/92b1d9d8f2dc43302df7f5271b9500bbfc237386782343561a5f62beb306/Jinja2-2.8.1-py2.py3-none-any.whl (264kB) 100% |████████████████████████████████| 266kB 11.8MB/s Collecting lxml<4.0,>=3.6.0 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/23/3e/0b8d8e1a8244159f82d06a14734045e1f2eff199e1f20046214902857b73/lxml-3.8.0-cp35-cp35m-manylinux1_x86_64.whl (7.2MB) 100% |████████████████████████████████| 7.2MB 6.2MB/s Collecting markdown<3.0,>=2.5.2 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/6d/7d/488b90f470b96531a3f5788cf12a93332f543dbab13c423a5e7ce96a0493/Markdown-2.6.11-py2.py3-none-any.whl (78kB) 100% |████████████████████████████████| 81kB 15.2MB/s Collecting pandas<1.0.0,>=0.17.1 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/1d/cb/1cc27b57855a7f7c25965ab257e2bf251589bfe450e2cb23820b11a674a6/pandas-0.23.3-cp35-cp35m-manylinux1_x86_64.whl (8.7MB) 100% |████████████████████████████████| 8.7MB 5.6MB/s Collecting pendulum==1.4.4 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/cc/ba/a7ef16485be134f0acec118b6b9036741d664cc0042c7d6e48c613ea9d88/pendulum-1.4.4-cp35-cp35m-manylinux1_x86_64.whl (123kB) 100% |████████████████████████████████| 133kB 18.3MB/s Collecting psutil<5.0.0,>=4.2.0 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/6c/49/0f784a247868e167389f6ac76b8699b2f3d6f4e8e85685dfec43e58d1ed1/psutil-4.4.2.tar.gz (1.8MB) 100% |████████████████████████████████| 1.8MB 13.9MB/s Collecting pygments<3.0,>=2.0.1 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/02/ee/b6e02dc6529e82b75bb06823ff7d005b141037cb1416b10c6f00fc419dca/Pygments-2.2.0-py2.py3-none-any.whl (841kB) 100% |████████████████████████████████| 849kB 6.4MB/s Collecting python-daemon<2.2,>=2.1.1 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/f4/59/816004688f8e8602526553cd96226f34657ce4a86daa2240c3eebb0568a3/python_daemon-2.1.2-py2.py3-none-any.whl Collecting python-dateutil<3,>=2.3 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/cf/f5/af2b09c957ace60dcfac112b669c45c8c97e32f94aa8b56da4c6d1682825/python_dateutil-2.7.3-py2.py3-none-any.whl (211kB) 100% |████████████████████████████████| 215kB 6.6MB/s Collecting python-nvd3==0.15.0 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/0b/aa/97165daa6e319409c5c2582e62736a7353bda3c90d90fdcb0b11e116dd2d/python-nvd3-0.15.0.tar.gz Collecting requests<3,>=2.5.1 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl (91kB) 100% |████████████████████████████████| 92kB 16.7MB/s Collecting setproctitle<2,>=1.1.8 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/5a/0d/dc0d2234aacba6cf1a729964383e3452c52096dc695581248b548786f2b3/setproctitle-1.1.10.tar.gz Collecting sqlalchemy<1.2.0,>=1.1.15 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/cc/4d/96d93ff77cd67aca7618e402191eee3490d8f5f245d6ab7622d35fe504f4/SQLAlchemy-1.1.18.tar.gz (5.3MB) 100% |████████████████████████████████| 5.3MB 10.9MB/s Collecting sqlalchemy-utc>=0.9.0 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/7e/84/85cb683933ac5c2e1d5d3d33de06e72d761c9e172dc627e2b0b73a327d1d/SQLAlchemy_Utc-0.10.0-py2.py3-none-any.whl Collecting tabulate<0.8.0,>=0.7.5 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/a5/8d/86bf900d62216e2be7806d2ff4615cb7da54e13aeb7765549310c355cbae/tabulate-0.7.7-py2.py3-none-any.whl Collecting tenacity==4.8.0 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/fc/e9/5499018e0d420f8d03a215c310ee7bc6e1a7e84adaa63f6ea208e864bdb6/tenacity-4.8.0-py2.py3-none-any.whl Collecting thrift>=0.9.2 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/c6/b4/510617906f8e0c5660e7d96fbc5585113f83ad547a3989b80297ac72a74c/thrift-0.11.0.tar.gz (52kB) 100% |████████████████████████████████| 61kB 15.4MB/s Collecting tzlocal>=1.4 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/cb/89/e3687d3ed99bc882793f82634e9824e62499fdfdc4b1ae39e211c5b05017/tzlocal-1.5.1.tar.gz Collecting unicodecsv>=0.14.1 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/6f/a4/691ab63b17505a26096608cc309960b5a6bdf39e4ba1a793d5f9b1a53270/unicodecsv-0.14.1.tar.gz Collecting werkzeug<0.15.0,>=0.14.1 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/20/c4/12e3e56473e52375aa29c4764e70d1b8f3efa6682bef8d0aae04fe335243/Werkzeug-0.14.1-py2.py3-none-any.whl (322kB) 100% |████████████████████████████████| 327kB 10.4MB/s Collecting zope.deprecation<5.0,>=4.0 (from apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/ee/33/625098914ec59b3006adf2cdf44a721e9671f4836af9eeb8cbe14e485954/zope.deprecation-4.3.0-py2.py3-none-any.whl Collecting Mako (from alembic<0.9,>=0.8.3->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/eb/f3/67579bb486517c0d49547f9697e36582cd19dafb5df9e687ed8e22de57fa/Mako-1.0.7.tar.gz (564kB) 100% |████████████████████████████████| 573kB 21.2MB/s Collecting python-editor>=0.3 (from alembic<0.9,>=0.8.3->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/65/1e/adf6e000ea5dc909aa420352d6ba37f16434c8a3c2fa030445411a1ed545/python-editor-1.0.3.tar.gz Collecting six (from bleach==2.1.2->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl Collecting html5lib!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,>=0.99999999pre (from bleach==2.1.2->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/a5/62/bbd2be0e7943ec8504b517e62bab011b4946e1258842bc159e5dfde15b96/html5lib-1.0.1-py2.py3-none-any.whl (117kB) 100% |████████████████████████████████| 122kB 13.7MB/s Collecting itsdangerous>=0.21 (from flask<0.13,>=0.12.4->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/dc/b4/a60bcdba945c00f6d608d8975131ab3f25b22f2bcfe1dab221165194b2d4/itsdangerous-0.24.tar.gz (46kB) 100% |████████████████████████████████| 51kB 5.8MB/s Collecting click>=2.0 (from flask<0.13,>=0.12.4->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/34/c1/8806f99713ddb993c5366c362b2f908f18269f8d792aff1abfd700775a77/click-6.7-py2.py3-none-any.whl (71kB) 100% |████████████████████████████████| 71kB 11.6MB/s Collecting colorama==0.3.9 (from flask-appbuilder<2.0.0,>=1.11.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl Collecting Flask-Babel==0.11.1 (from flask-appbuilder<2.0.0,>=1.11.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/47/96/6013d4091fb4238e27e918aec4929f082942fa8c9489ae3aad2f18de4b5b/Flask-Babel-0.11.1.tar.gz (40kB) 100% |████████████████████████████████| 40kB 13.3MB/s Collecting Flask-OpenID==1.2.5 (from flask-appbuilder<2.0.0,>=1.11.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/d1/a2/9d1fba3287a65f81b9d1c09c4f7cb16f8ea4988b1bc97ffea0d60983338f/Flask-OpenID-1.2.5.tar.gz (43kB) 100% |████████████████████████████████| 51kB 19.5MB/s Collecting Flask-SQLAlchemy==2.1 (from flask-appbuilder<2.0.0,>=1.11.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/b3/52/227aaf4e8cebb153e239c518a9e916590b2fe0e4350e6b02d92b546b69b7/Flask-SQLAlchemy-2.1.tar.gz (95kB) 100% |████████████████████████████████| 102kB 18.4MB/s Collecting wtforms (from flask-admin==1.4.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/9f/c8/dac5dce9908df1d9d48ec0e26e2a250839fa36ea2c602cc4f85ccfeb5c65/WTForms-2.2.1-py2.py3-none-any.whl (166kB) 100% |████████████████████████████████| 174kB 14.4MB/s Collecting PyYAML>=3.0 (from flask-swagger==0.2.13->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/9e/a3/1d13970c3f36777c583f136c136f804d70f500168edc1edea6daa7200769/PyYAML-3.13.tar.gz (270kB) 100% |████████████████████████████████| 276kB 18.7MB/s Collecting ordereddict (from funcsigs==1.0.0->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/53/25/ef88e8e45db141faa9598fbf7ad0062df8f50f881a36ed6a0073e1572126/ordereddict-1.1.tar.gz Collecting gitdb2>=2.0.0 (from gitpython>=2.0.2->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/e0/95/c772c13b7c5740ec1a0924250e6defbf5dfdaee76a50d1c47f9c51f1cabb/gitdb2-2.0.3-py2.py3-none-any.whl (63kB) 100% |████████████████████████████████| 71kB 20.6MB/s Collecting MarkupSafe (from jinja2<2.9.0,>=2.7.3->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/4d/de/32d741db316d8fdb7680822dd37001ef7a448255de9699ab4bfcbdf4172b/MarkupSafe-1.0.tar.gz Collecting pytz>=2011k (from pandas<1.0.0,>=0.17.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/30/4e/27c34b62430286c6d59177a0842ed90dc789ce5d1ed740887653b898779a/pytz-2018.5-py2.py3-none-any.whl (510kB) 100% |████████████████████████████████| 512kB 13.8MB/s Collecting numpy>=1.9.0 (from pandas<1.0.0,>=0.17.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/43/17/cd9fa14492dbef2aaf22622db79dba087c10f125473e730cda2f2019c40b/numpy-1.14.5-cp35-cp35m-manylinux1_x86_64.whl (12.1MB) 100% |████████████████████████████████| 12.1MB 5.8MB/s Collecting pytzdata>=2018.3.0.0 (from pendulum==1.4.4->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/2b/b8/a007eedc118838b27247da7c31f25c2b9e68e68ed2ffafed3d340d379e84/pytzdata-2018.5-py2.py3-none-any.whl (981kB) 100% |████████████████████████████████| 983kB 19.6MB/s Collecting lockfile>=0.10 (from python-daemon<2.2,>=2.1.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/c8/22/9460e311f340cb62d26a38c419b1381b8593b0bb6b5d1f056938b086d362/lockfile-0.12.2-py2.py3-none-any.whl Requirement already satisfied: setuptools in /usr/local/lib/python3.5/site-packages (from python-daemon<2.2,>=2.1.1->apache-airflow==1.10.0b3+incubating) (39.1.0) Collecting docutils (from python-daemon<2.2,>=2.1.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/36/fa/08e9e6e0e3cbd1d362c3bbee8d01d0aedb2155c4ac112b19ef3cae8eed8d/docutils-0.14-py3-none-any.whl (543kB) 100% |████████████████████████████████| 552kB 7.6MB/s Collecting python-slugify>=1.2.5 (from python-nvd3==0.15.0->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/70/c1/98bfb2c981787dcec4613c5da2c17d6f54613935b0e3a877e87a9fa974e4/python-slugify-1.2.5.tar.gz Collecting urllib3<1.24,>=1.21.1 (from requests<3,>=2.5.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl (133kB) 100% |████████████████████████████████| 143kB 15.2MB/s Collecting chardet<3.1.0,>=3.0.2 (from requests<3,>=2.5.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB) 100% |████████████████████████████████| 143kB 17.3MB/s Collecting idna<2.8,>=2.5 (from requests<3,>=2.5.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl (58kB) 100% |████████████████████████████████| 61kB 17.8MB/s Collecting certifi>=2017.4.17 (from requests<3,>=2.5.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl (150kB) 100% |████████████████████████████████| 153kB 15.1MB/s Collecting monotonic>=0.6 (from tenacity==4.8.0->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl Collecting webencodings (from html5lib!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,>=0.99999999pre->bleach==2.1.2->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/f4/24/2a3e3df732393fed8b3ebf2ec078f05546de641fe1b667ee316ec1dcf3b7/webencodings-0.5.1-py2.py3-none-any.whl Collecting Babel>=2.3 (from Flask-Babel==0.11.1->flask-appbuilder<2.0.0,>=1.11.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/b8/ad/c6f60602d3ee3d92fbed87675b6fb6a6f9a38c223343ababdb44ba201f10/Babel-2.6.0-py2.py3-none-any.whl (8.1MB) 100% |████████████████████████████████| 8.1MB 5.6MB/s Collecting python3-openid>=2.0 (from Flask-OpenID==1.2.5->flask-appbuilder<2.0.0,>=1.11.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/bd/de/52c5699f52dcee3037db587196dcaf63ffedf5fbeba3183afe9b21a3a89f/python3_openid-3.1.0-py3-none-any.whl (130kB) 100% |████████████████████████████████| 133kB 16.6MB/s Collecting smmap2>=2.0.0 (from gitdb2>=2.0.0->gitpython>=2.0.2->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/e3/59/4e22f692e65f5f9271252a8e63f04ce4ad561d4e06192478ee48dfac9611/smmap2-2.0.3-py2.py3-none-any.whl Collecting Unidecode>=0.04.16 (from python-slugify>=1.2.5->python-nvd3==0.15.0->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/59/ef/67085e30e8bbcdd76e2f0a4ad8151c13a2c5bce77c85f8cad6e1f16fb141/Unidecode-1.0.22-py2.py3-none-any.whl (235kB) 100% |████████████████████████████████| 235kB 11.3MB/s Collecting defusedxml (from python3-openid>=2.0->Flask-OpenID==1.2.5->flask-appbuilder<2.0.0,>=1.11.1->apache-airflow==1.10.0b3+incubating) Downloading https://files.pythonhosted.org/packages/87/1c/17f3e3935a913dfe2a5ca85fa5ccbef366bfd82eb318b1f75dadbf0affca/defusedxml-0.5.0-py2.py3-none-any.whl Building wheels for collected packages: apache-airflow, alembic, configparser, croniter, dill, flask-appbuilder, flask-admin, flask-login, flask-swagger, future, psutil, python-nvd3, setproctitle, sqlalchemy, thrift, tzlocal, unicodecsv, Mako, python-editor, itsdangerous, Flask-Babel, Flask-OpenID, Flask-SQLAlchemy, PyYAML, ordereddict, MarkupSafe, python-slugify Running setup.py bdist_wheel for apache-airflow ... done Stored in directory: /root/.cache/pip/wheels/dd/62/34/af9d4cb301642c4227af52b8f8e16d31ff7596d0295331a6c9 Running setup.py bdist_wheel for alembic ... done Stored in directory: /root/.cache/pip/wheels/a1/ba/88/bbe3e48e2bdc60544b78e30d2b6e7576ff2db7007d51955de3 Running setup.py bdist_wheel for configparser ... done Stored in directory: /root/.cache/pip/wheels/a3/61/79/424ef897a2f3b14684a7de5d89e8600b460b89663e6ce9d17c Running setup.py bdist_wheel for croniter ... done Stored in directory: /root/.cache/pip/wheels/0f/30/ec/4dfd1f9ea3d306dc91b8280addf3520dfe6a95e7a4c5323117 Running setup.py bdist_wheel for dill ... done Stored in directory: /root/.cache/pip/wheels/e2/5d/17/f87cb7751896ac629b435a8696f83ee75b11029f5d6f6bda72 Running setup.py bdist_wheel for flask-appbuilder ... done Stored in directory: /root/.cache/pip/wheels/c7/d6/fb/d7a562d5261645f98a5dd52c39de53c789e67be61392bc2d18 Running setup.py bdist_wheel for flask-admin ... done Stored in directory: /root/.cache/pip/wheels/70/b5/a8/c923a58f1d074fb27b1d9dcfd937730481a8fb693d934594a5 Running setup.py bdist_wheel for flask-login ... done Stored in directory: /root/.cache/pip/wheels/94/98/8f/b357415afe27adcdf071306f3f981e61a0498b7101707471fc Running setup.py bdist_wheel for flask-swagger ... done Stored in directory: /root/.cache/pip/wheels/81/b7/f4/1b38146234576bec8e9275e8bc49387d33e6af2aaa17f0b099 Running setup.py bdist_wheel for future ... done Stored in directory: /root/.cache/pip/wheels/bf/c9/a3/c538d90ef17cf7823fa51fc701a7a7a910a80f6a405bf15b1a Running setup.py bdist_wheel for psutil ... done Stored in directory: /root/.cache/pip/wheels/87/c5/27/5100942de9befdd850a4e0b09f2e0d70758ae27429c0000d22 Running setup.py bdist_wheel for python-nvd3 ... done Stored in directory: /root/.cache/pip/wheels/88/62/32/46f3425abe3505837e6345f7225201b5eb928c0991c99f5e47 Running setup.py bdist_wheel for setproctitle ... done Stored in directory: /root/.cache/pip/wheels/e6/b1/a6/9719530228e258eba904501fef99d5d85c80d52bd8f14438a3 Running setup.py bdist_wheel for sqlalchemy ... done Stored in directory: /root/.cache/pip/wheels/6e/2a/d1/54c6865667d73c4d70a5bdffc2f8fb08f1b5a75c868fde617b Running setup.py bdist_wheel for thrift ... done Stored in directory: /root/.cache/pip/wheels/be/36/81/0f93ba89a1cb7887c91937948519840a72c0ffdd57cac0ae8f Running setup.py bdist_wheel for tzlocal ... done Stored in directory: /root/.cache/pip/wheels/15/ae/df/a67bf1ed84e9bf230187d36d8dcfd30072bea0236cb059ed91 Running setup.py bdist_wheel for unicodecsv ... done Stored in directory: /root/.cache/pip/wheels/a6/09/e9/e800279c98a0a8c94543f3de6c8a562f60e51363ed26e71283 Running setup.py bdist_wheel for Mako ... done Stored in directory: /root/.cache/pip/wheels/15/35/25/dbcb848832ccb1a4b4ad23f529badfd3bce9bf88017f7ca510 Running setup.py bdist_wheel for python-editor ... done Stored in directory: /root/.cache/pip/wheels/36/e0/98/ba386b125a00ea9dd52e2c16aa2ec0adbbd639b84bfe2e001d Running setup.py bdist_wheel for itsdangerous ... done Stored in directory: /root/.cache/pip/wheels/2c/4a/61/5599631c1554768c6290b08c02c72d7317910374ca602ff1e5 Running setup.py bdist_wheel for Flask-Babel ... done Stored in directory: /root/.cache/pip/wheels/2e/56/0c/ae3cc1279a4d1c5b7cec98634973266e3b4c06d026c2fc35cf Running setup.py bdist_wheel for Flask-OpenID ... done Stored in directory: /root/.cache/pip/wheels/3b/1e/c0/b941aa1a5954f1acabbf1671a64e27adc359c0b0e74b16f8b4 Running setup.py bdist_wheel for Flask-SQLAlchemy ... done Stored in directory: /root/.cache/pip/wheels/bd/18/c4/5b1ebaec15e2bb933089576ebf905ea29976b2f37ed629e1c3 Running setup.py bdist_wheel for PyYAML ... done Stored in directory: /root/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f Running setup.py bdist_wheel for ordereddict ... done Stored in directory: /root/.cache/pip/wheels/78/d3/a8/9f52c7a389a0ffba6f575a3886b5f7a55461fb6ec34aa5fd38 Running setup.py bdist_wheel for MarkupSafe ... done Stored in directory: /root/.cache/pip/wheels/33/56/20/ebe49a5c612fffe1c5a632146b16596f9e64676768661e4e46 Running setup.py bdist_wheel for python-slugify ... done Stored in directory: /root/.cache/pip/wheels/2a/21/79/f359bb1659ed86cf1a402bdea9f264f2bcf44489f200882fa0 Successfully built apache-airflow alembic configparser croniter dill flask-appbuilder flask-admin flask-login flask-swagger future psutil python-nvd3 setproctitle sqlalchemy thrift tzlocal unicodecsv Mako python-editor itsdangerous Flask-Babel Flask-OpenID Flask-SQLAlchemy PyYAML ordereddict MarkupSafe python-slugify Installing collected packages: sqlalchemy, MarkupSafe, Mako, python-editor, alembic, six, webencodings, html5lib, bleach, configparser, python-dateutil, croniter, dill, jinja2, itsdangerous, click, werkzeug, flask, colorama, pytz, Babel, Flask-Babel, flask-login, defusedxml, python3-openid, Flask-OpenID, Flask-SQLAlchemy, wtforms, flask-wtf, flask-appbuilder, flask-admin, flask-caching, PyYAML, flask-swagger, ordereddict, funcsigs, future, smmap2, gitdb2, gitpython, gunicorn, iso8601, lxml, markdown, numpy, pandas, tzlocal, pytzdata, pendulum, psutil, pygments, lockfile, docutils, python-daemon, Unidecode, python-slugify, python-nvd3, urllib3, chardet, idna, certifi, requests, setproctitle, sqlalchemy-utc, tabulate, monotonic, tenacity, thrift, unicodecsv, zope.deprecation, apache-airflow Successfully installed Babel-2.6.0 Flask-Babel-0.11.1 Flask-OpenID-1.2.5 Flask-SQLAlchemy-2.1 Mako-1.0.7 MarkupSafe-1.0 PyYAML-3.13 Unidecode-1.0.22 alembic-0.8.10 apache-airflow-1.10.0b3+incubating bleach-2.1.2 certifi-2018.4.16 chardet-3.0.4 click-6.7 colorama-0.3.9 configparser-3.5.0 croniter-0.3.24 defusedxml-0.5.0 dill-0.2.8.2 docutils-0.14 flask-0.12.4 flask-admin-1.4.1 flask-appbuilder-1.11.1 flask-caching-1.3.3 flask-login-0.2.11 flask-swagger-0.2.13 flask-wtf-0.14.2 funcsigs-1.0.0 future-0.16.0 gitdb2-2.0.3 gitpython-2.1.10 gunicorn-19.9.0 html5lib-1.0.1 idna-2.7 iso8601-0.1.12 itsdangerous-0.24 jinja2-2.8.1 lockfile-0.12.2 lxml-3.8.0 markdown-2.6.11 monotonic-1.5 numpy-1.14.5 ordereddict-1.1 pandas-0.23.3 pendulum-1.4.4 psutil-4.4.2 pygments-2.2.0 python-daemon-2.1.2 python-dateutil-2.7.3 python-editor-1.0.3 python-nvd3-0.15.0 python-slugify-1.2.5 python3-openid-3.1.0 pytz-2018.5 pytzdata-2018.5 requests-2.19.1 setproctitle-1.1.10 six-1.11.0 smmap2-2.0.3 sqlalchemy-1.1.18 sqlalchemy-utc-0.10.0 tabulate-0.7.7 tenacity-4.8.0 thrift-0.11.0 tzlocal-1.5.1 unicodecsv-0.14.1 urllib3-1.23 webencodings-0.5.1 werkzeug-0.14.1 wtforms-2.2.1 zope.deprecation-4.3.0 root@88775efd72a9:/# airflow initdb [2018-07-13 12:58:12,960] {__init__.py:51} INFO - Using executor SequentialExecutor DB: sqlite:////root/airflow/airflow.db [2018-07-13 12:58:13,155] {db.py:338} INFO - Creating tables INFO [alembic.runtime.migration] Context impl SQLiteImpl. INFO [alembic.runtime.migration] Will assume non-transactional DDL. /usr/local/lib/python3.5/site-packages/alembic/util/messaging.py:69: UserWarning: Revision 856955da8476 referenced from 856955da8476 -> 9635ae0956e7 (head), index-faskfail is not present warnings.warn(msg) Traceback (most recent call last): File "/usr/local/bin/airflow", line 32, in <module> args.func(args) File "/usr/local/lib/python3.5/site-packages/airflow/bin/cli.py", line 1002, in initdb db_utils.initdb(settings.RBAC) File "/usr/local/lib/python3.5/site-packages/airflow/utils/db.py", line 92, in initdb upgradedb() File "/usr/local/lib/python3.5/site-packages/airflow/utils/db.py", line 346, in upgradedb command.upgrade(config, 'heads') File "/usr/local/lib/python3.5/site-packages/alembic/command.py", line 174, in upgrade script.run_env() File "/usr/local/lib/python3.5/site-packages/alembic/script/base.py", line 416, in run_env util.load_python_file(self.dir, 'env.py') File "/usr/local/lib/python3.5/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file module = load_module_py(module_id, path) File "/usr/local/lib/python3.5/site-packages/alembic/util/compat.py", line 68, in load_module_py module_id, path).load_module(module_id) File "<frozen importlib._bootstrap_external>", line 418, in _check_name_wrapper File "<frozen importlib._bootstrap_external>", line 841, in load_module File "<frozen importlib._bootstrap_external>", line 700, in load_module File "<frozen importlib._bootstrap>", line 268, in _load_module_shim File "<frozen importlib._bootstrap>", line 693, in _load File "<frozen importlib._bootstrap>", line 673, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 697, in exec_module File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed File "/usr/local/lib/python3.5/site-packages/airflow/migrations/env.py", line 91, in <module> run_migrations_online() File "/usr/local/lib/python3.5/site-packages/airflow/migrations/env.py", line 86, in run_migrations_online context.run_migrations() File "<string>", line 8, in run_migrations File "/usr/local/lib/python3.5/site-packages/alembic/runtime/environment.py", line 807, in run_migrations self.get_context().run_migrations(**kw) File "/usr/local/lib/python3.5/site-packages/alembic/runtime/migration.py", line 312, in run_migrations for step in self._migrations_fn(heads, self): File "/usr/local/lib/python3.5/site-packages/alembic/command.py", line 163, in upgrade return script._upgrade_revs(revision, rev) File "/usr/local/lib/python3.5/site-packages/alembic/script/base.py", line 329, in _upgrade_revs revs = list(revs) File "/usr/local/lib/python3.5/site-packages/alembic/script/revision.py", line 652, in _iterate_revisions uppers = util.dedupe_tuple(self.get_revisions(upper)) File "/usr/local/lib/python3.5/site-packages/alembic/script/revision.py", line 300, in get_revisions resolved_id, branch_label = self._resolve_revision_number(id_) File "/usr/local/lib/python3.5/site-packages/alembic/script/revision.py", line 433, in _resolve_revision_number self._revision_map File "/usr/local/lib/python3.5/site-packages/alembic/util/langhelpers.py", line 240, in __get__ obj.__dict__[self.__name__] = result = self.fget(obj) File "/usr/local/lib/python3.5/site-packages/alembic/script/revision.py", line 151, in _revision_map down_revision = map_[downrev] KeyError: '856955da8476'
