Fantastic! Great job Ash! On Tue, May 19, 2020 at 12:14 PM Ash Berlin-Taylor <a...@apache.org> wrote:
> PR created that I think will fix it > https://github.com/apache/airflow/pull/8909 > > -ash > > On May 19 2020, at 10:08 am, Jarek Potiuk <jarek.pot...@polidea.com> > wrote: > > > Issue created: https://github.com/apache/airflow/issues/8908 > > > > On Tue, May 19, 2020 at 11:06 AM Asif Saif Uddin <auv...@gmail.com> > wrote: > > > >> value = Exception('<class > >> \'sqlalchemy.exc.IntegrityError\'>((\'(psycopg2.errors.UniqueViolation) > >> duplicate key value violates..."pg_type_typname_nsp_index"\\nDETAIL: Key > >> (typname, typnamespace)=(celery_tasksetmeta, 2200) already > >> exists.\\n\',))') > >> tb = None > >> def reraise(tp, value, tb=None): > >> """Reraise exception.""" > >> if value.__traceback__ is not tb: > >> raise value.with_traceback(tb) > >> > raise value > >> E Exception: <class > >> 'sqlalchemy.exc.IntegrityError'>(('(psycopg2.errors.UniqueViolation) > >> duplicate key value violates unique constraint > >> "pg_type_typname_nsp_index"\nDETAIL: Key (typname, > >> typnamespace)=(celery_tasksetmeta, 2200) already exists.\n',)) > >> > >> > >> seems like a celery sqlalchemy issue. can you try celery from master? > feel > >> free to open an issue on celery. > >> > >> On Tue, May 19, 2020 at 2:43 PM Tomasz Urbaszek < > >> tomasz.urbas...@polidea.com> wrote: > >> > >>> Should we create an issue? Maybe Asif (CC) from Celery team will be > >>> able to help us? > >>> > >>> Tomek > >>> > >>> > >>> On Tue, May 19, 2020 at 10:39 AM Ash Berlin-Taylor <a...@apache.org> > >>> wrote: > >>> > > >>> > I'll take a look. I'm surprised by that error though, it looks > >>> like two > >>> processes are racing to create a table, and not dealing with losing. > >>> > > >>> > (I suspect we can fix it by pre-creating that tbl,/calling something > >>> before we spawn processes, but this really is a bug in celery.) > >>> > > >>> > On 19 May 2020 07:58:11 BST, Jarek Potiuk <jarek.pot...@polidea.com> > >>> wrote: > >>> > >Hello Everyone, but mostly I think Kamil, Ash, Kaxil, > >>> > > > >>> > >I think we need someone who knows a bit more about Celery > >>> Executor to > >>> > >stabilise one of the tests that is transiently failing. I think this > >>> > >started to appear recently (so likely it is one of the optimizations > >>> > >implemented recently) > >>> > > > >>> > >You can see example of such failure here but there are many more > >>> > >similar > >>> > >ones) > >>> > > > >>> > > > >>> > https://github.com/apache/airflow/runs/687537257?check_suite_focus=true#step:6:387 > >>> > > > >>> > >I think there were recently some optimisations done in this area > >>> and I > >>> > >believe it might be the root cause of this problem. Can you > >>> please help > >>> > >to > >>> > >solve that problem? I do not want to put that problem to > >>> Quarantine, as > >>> > >it > >>> > >looks like an awfully "real" problem. > >>> > > > >>> > > > >>> > > > >>> > >/usr/local/lib/python3.7/site-packages/celery/result.py:333: in > >>> > >maybe_throw > >>> > > self.throw(value, self._to_remote_traceback(tb)) > >>> > >/usr/local/lib/python3.7/site-packages/celery/result.py:326: in > throw > >>> > > self.on_ready.throw(*args, **kwargs) > >>> > >/usr/local/lib/python3.7/site-packages/vine/promises.py:244: in > throw > >>> > > reraise(type(exc), exc, tb) > >>> > >_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > >>> _ _ _ > >>> > >_ _ > >>> > >_ _ > >>> > > > >>> > >tp = <class 'Exception'> > >>> > >value = Exception('<class > >>> > > >\'sqlalchemy.exc.IntegrityError\'>((\'(psycopg2.errors.UniqueViolation) > >>> > >duplicate key value violates..."pg_type_typname_nsp_index"\\nDETAIL: > >>> > >Key > >>> > >(typname, typnamespace)=(celery_tasksetmeta, 2200) already > >>> > >exists.\\n\',))') > >>> > >tb = None > >>> > > > >>> > > def reraise(tp, value, tb=None): > >>> > > """Reraise exception.""" > >>> > > if value.__traceback__ is not tb: > >>> > > raise value.with_traceback(tb) > >>> > >> raise value > >>> > >E Exception: <class > >>> > >'sqlalchemy.exc.IntegrityError'>(('(psycopg2.errors.UniqueViolation) > >>> > >duplicate key value violates unique constraint > >>> > >"pg_type_typname_nsp_index"\nDETAIL: Key (typname, > >>> > >typnamespace)=(celery_tasksetmeta, 2200) already exists.\n',)) > >>> > > > >>> > > > >>> > > > >>> > >J. > >>> > > > >>> > >-- > >>> > > > >>> > >Jarek Potiuk > >>> > >Polidea <https://www.polidea.com/> | Principal Software Engineer > >>> > > > >>> > >M: +48 660 796 129 <+48660796129> > >>> > >[image: Polidea] <https://www.polidea.com/> > >>> > >>> > >>> > >>> -- > >>> > >>> Tomasz Urbaszek > >>> Polidea | Software Engineer > >>> > >>> M: +48 505 628 493 > >>> E: tomasz.urbas...@polidea.com > >>> > >>> Unique Tech > >>> Check out our projects! > >>> > >> > > > > -- > > > > Jarek Potiuk > > Polidea <https://www.polidea.com/> | Principal Software Engineer > > > > M: +48 660 796 129 <+48660796129> > > [image: Polidea] <https://www.polidea.com/> > > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
