PR created that I think will fix it https://github.com/apache/airflow/pull/8909
-ash On May 19 2020, at 10:08 am, Jarek Potiuk <jarek.pot...@polidea.com> wrote: > Issue created: https://github.com/apache/airflow/issues/8908 > > On Tue, May 19, 2020 at 11:06 AM Asif Saif Uddin <auv...@gmail.com> wrote: > >> value = Exception('<class >> \'sqlalchemy.exc.IntegrityError\'>((\'(psycopg2.errors.UniqueViolation) >> duplicate key value violates..."pg_type_typname_nsp_index"\\nDETAIL: Key >> (typname, typnamespace)=(celery_tasksetmeta, 2200) already >> exists.\\n\',))') >> tb = None >> def reraise(tp, value, tb=None): >> """Reraise exception.""" >> if value.__traceback__ is not tb: >> raise value.with_traceback(tb) >> > raise value >> E Exception: <class >> 'sqlalchemy.exc.IntegrityError'>(('(psycopg2.errors.UniqueViolation) >> duplicate key value violates unique constraint >> "pg_type_typname_nsp_index"\nDETAIL: Key (typname, >> typnamespace)=(celery_tasksetmeta, 2200) already exists.\n',)) >> >> >> seems like a celery sqlalchemy issue. can you try celery from master? feel >> free to open an issue on celery. >> >> On Tue, May 19, 2020 at 2:43 PM Tomasz Urbaszek < >> tomasz.urbas...@polidea.com> wrote: >> >>> Should we create an issue? Maybe Asif (CC) from Celery team will be >>> able to help us? >>> >>> Tomek >>> >>> >>> On Tue, May 19, 2020 at 10:39 AM Ash Berlin-Taylor <a...@apache.org> >>> wrote: >>> > >>> > I'll take a look. I'm surprised by that error though, it looks >>> like two >>> processes are racing to create a table, and not dealing with losing. >>> > >>> > (I suspect we can fix it by pre-creating that tbl,/calling something >>> before we spawn processes, but this really is a bug in celery.) >>> > >>> > On 19 May 2020 07:58:11 BST, Jarek Potiuk <jarek.pot...@polidea.com> >>> wrote: >>> > >Hello Everyone, but mostly I think Kamil, Ash, Kaxil, >>> > > >>> > >I think we need someone who knows a bit more about Celery >>> Executor to >>> > >stabilise one of the tests that is transiently failing. I think this >>> > >started to appear recently (so likely it is one of the optimizations >>> > >implemented recently) >>> > > >>> > >You can see example of such failure here but there are many more >>> > >similar >>> > >ones) >>> > > >>> > > >>> https://github.com/apache/airflow/runs/687537257?check_suite_focus=true#step:6:387 >>> > > >>> > >I think there were recently some optimisations done in this area >>> and I >>> > >believe it might be the root cause of this problem. Can you >>> please help >>> > >to >>> > >solve that problem? I do not want to put that problem to >>> Quarantine, as >>> > >it >>> > >looks like an awfully "real" problem. >>> > > >>> > > >>> > > >>> > >/usr/local/lib/python3.7/site-packages/celery/result.py:333: in >>> > >maybe_throw >>> > > self.throw(value, self._to_remote_traceback(tb)) >>> > >/usr/local/lib/python3.7/site-packages/celery/result.py:326: in throw >>> > > self.on_ready.throw(*args, **kwargs) >>> > >/usr/local/lib/python3.7/site-packages/vine/promises.py:244: in throw >>> > > reraise(type(exc), exc, tb) >>> > >_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ >>> _ _ _ >>> > >_ _ >>> > >_ _ >>> > > >>> > >tp = <class 'Exception'> >>> > >value = Exception('<class >>> > >\'sqlalchemy.exc.IntegrityError\'>((\'(psycopg2.errors.UniqueViolation) >>> > >duplicate key value violates..."pg_type_typname_nsp_index"\\nDETAIL: >>> > >Key >>> > >(typname, typnamespace)=(celery_tasksetmeta, 2200) already >>> > >exists.\\n\',))') >>> > >tb = None >>> > > >>> > > def reraise(tp, value, tb=None): >>> > > """Reraise exception.""" >>> > > if value.__traceback__ is not tb: >>> > > raise value.with_traceback(tb) >>> > >> raise value >>> > >E Exception: <class >>> > >'sqlalchemy.exc.IntegrityError'>(('(psycopg2.errors.UniqueViolation) >>> > >duplicate key value violates unique constraint >>> > >"pg_type_typname_nsp_index"\nDETAIL: Key (typname, >>> > >typnamespace)=(celery_tasksetmeta, 2200) already exists.\n',)) >>> > > >>> > > >>> > > >>> > >J. >>> > > >>> > >-- >>> > > >>> > >Jarek Potiuk >>> > >Polidea <https://www.polidea.com/> | Principal Software Engineer >>> > > >>> > >M: +48 660 796 129 <+48660796129> >>> > >[image: Polidea] <https://www.polidea.com/> >>> >>> >>> >>> -- >>> >>> Tomasz Urbaszek >>> Polidea | Software Engineer >>> >>> M: +48 505 628 493 >>> E: tomasz.urbas...@polidea.com >>> >>> Unique Tech >>> Check out our projects! >>> >> > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> >
