Actually, I see the error is further down.. File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 469, in do_execute
cursor.execute(statement, parameters) sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) null value in column "dag_id" violates not-null constraint DETAIL: Failing row contains (null, running, 1, f). [SQL: 'INSERT INTO dag_stats (state, count, dirty) VALUES (%(state)s, %(count)s, %(dirty)s)'] [parameters: {'count': 1L, 'state': u'running', 'dirty': False}] It looks like an autoincrement is missing for this table. I'm running `SQLAlchemy==1.1.4` - I see our setup.py specifies any version greater than 0.9.8 -s On Mon, Feb 6, 2017 at 4:11 PM, siddharth anand <san...@apache.org> wrote: > I tried upgrading to 1.8.0rc1 from 1.7.1.3 via pip install > https://dist.apache.org/repos/dist/dev/incubator/airflow/ > airflow-1.8.0rc1+apache.incubating.tar.gz and then running airflow > upgradedb didn't quite work. First, I thought it completed successfully, > then saw errors some tables were indeed missing. I ran it again and > encountered the following exception : > > DB: postgresql://app_coust...@db-cousteau.ep.stage.agari.com:5432/airflow > > [2017-02-07 00:03:20,309] {db.py:284} INFO - Creating tables > > INFO [alembic.runtime.migration] Context impl PostgresqlImpl. > > INFO [alembic.runtime.migration] Will assume transactional DDL. > > INFO [alembic.runtime.migration] Running upgrade 2e82aab8ef20 -> > 211e584da130, add TI state index > > INFO [alembic.runtime.migration] Running upgrade 211e584da130 -> > 64de9cddf6c9, add task fails journal table > > INFO [alembic.runtime.migration] Running upgrade 64de9cddf6c9 -> > f2ca10b85618, add dag_stats table > > INFO [alembic.runtime.migration] Running upgrade f2ca10b85618 -> > 4addfa1236f1, Add fractional seconds to mysql tables > > INFO [alembic.runtime.migration] Running upgrade 4addfa1236f1 -> > 8504051e801b, xcom dag task indices > > INFO [alembic.runtime.migration] Running upgrade 8504051e801b -> > 5e7d17757c7a, add pid field to TaskInstance > > INFO [alembic.runtime.migration] Running upgrade 5e7d17757c7a -> > 127d2bf2dfa7, Add dag_id/state index on dag_run table > > /usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/crud.py:692: > SAWarning: Column 'dag_stats.dag_id' is marked as a member of the primary > key for table 'dag_stats', but has no Python-side or server-side default > generator indicated, nor does it indicate 'autoincrement=True' or > 'nullable=True', and no explicit value is passed. Primary key columns > typically may not store NULL. Note that as of SQLAlchemy 1.1, > 'autoincrement=True' must be indicated explicitly for composite (e.g. > multicolumn) primary keys if AUTO_INCREMENT/SERIAL/IDENTITY behavior is > expected for one of the columns in the primary key. CREATE TABLE statements > are impacted by this change as well on most backends. >