Re: [sage-devel] (cy)PARI in Parallel, Heisenbugs, and Merging Policy

2022-05-19 Thread Vincent Delecroix 




Le 16/05/2022 à 12:35, Dima Pasechnik a écrit :

On Mon, May 16, 2022 at 8:03 AM Vincent Delecroix
<20100.delecr...@gmail.com> wrote:


I would say that code with parallel computations + cypari2 should not
be merged (as cypari2 does not support it).


is it parallel multiprocessing, or parallel multithreading?


True : multiprocessing is not a problem with cypari2. Only
multithreading must be avoided.


If you need parallel + PARI then use the C library directly with the
appropriate threads locks.

If the problem comes from somewhere else, then it would better be sorted
out.

Best
Vincent

Le 16/05/2022 à 06:42, 'Travis Scrimshaw' via sage-devel a écrit :

That sure seems like it. So what should we do about the ticket? Would there
be opposition to merging this piece of code, as there doesn't seem like
there is a fix coming for the likely underlying cypari bug anytime soon?

Best,
Travis


On Sunday, May 15, 2022 at 4:14:31 PM UTC+9 vdelecroix wrote:


Probably related to https://github.com/sagemath/cypari2/issues/107 ?

Le 15/05/2022 à 05:06, 'Travis Scrimshaw' via sage-devel a écrit :

Hi everyone,
On ticket #30423 , Dan, Willie,
and I have been working on a parallel-computation based implementation

for

computing F-matrices that are used in math physics. However, we have been
seeing some doctest failures sporadically that involve segfaults and
linked-list corruption from (cy)PARI. Here are the logs from testing with
the first and the last having full tracebacks.

http://sporadic.stanford.edu/badlog-match
http://sporadic.stanford.edu/badlog
http://sporadic.stanford.edu/badlog1
http://sporadic.stanford.edu/badlog2
http://sporadic.stanford.edu/badlog3

The first question would be if anyone has an idea about what is causing
this. I have this impression that PARI is thread-safe, but I am wondering
if cypari is also thread/parallel-safe or if there are any specific

things

that we should be careful about. (We’ve already had to work around a
pickling issue with polynomials IIRC.)

Second question is that because this is a Heisenbug and I suspect it is
something upstream (and so far, nobody has been able to reproducing it
during an interactive version of Sage), I was wondering what the policy
would be for merging the ticket. I recall in the past that we have merged
tickets with Heisenbugs with followup tickets noting the behavior, but I

am

not 100% sure about that (and I wouldn’t necessarily know how to find any
explicit examples). I was wondering if we could merge the ticket in an
early beta version so that many people/systems can test it to see if it
becomes more reproducible; of course this is assuming that the build bots
are not consistent in reproducing this. Should we just mark any offending
test(s) as “# known bug” and is there some general policy about this?

Thanks,
Travis








--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/fd05e9f5-52b2-f2f8-bc02-c968c0a70e79%40gmail.com.




--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/8a34b501-51b9-595a-b5d3-c81d63c55ced%40gmail.com.

Re: [sage-devel] (cy)PARI in Parallel, Heisenbugs, and Merging Policy

2022-05-16 Thread Dima Pasechnik 
On Mon, May 16, 2022 at 8:03 AM Vincent Delecroix
<20100.delecr...@gmail.com> wrote:
>
> I would say that code with parallel computations + cypari2 should not
> be merged (as cypari2 does not support it).

is it parallel multiprocessing, or parallel multithreading?

>
> If you need parallel + PARI then use the C library directly with the
> appropriate threads locks.
>
> If the problem comes from somewhere else, then it would better be sorted
> out.
>
> Best
> Vincent
>
> Le 16/05/2022 à 06:42, 'Travis Scrimshaw' via sage-devel a écrit :
> > That sure seems like it. So what should we do about the ticket? Would there
> > be opposition to merging this piece of code, as there doesn't seem like
> > there is a fix coming for the likely underlying cypari bug anytime soon?
> >
> > Best,
> > Travis
> >
> >
> > On Sunday, May 15, 2022 at 4:14:31 PM UTC+9 vdelecroix wrote:
> >
> >> Probably related to https://github.com/sagemath/cypari2/issues/107 ?
> >>
> >> Le 15/05/2022 à 05:06, 'Travis Scrimshaw' via sage-devel a écrit :
> >>> Hi everyone,
> >>> On ticket #30423 , Dan, Willie,
> >>> and I have been working on a parallel-computation based implementation
> >> for
> >>> computing F-matrices that are used in math physics. However, we have been
> >>> seeing some doctest failures sporadically that involve segfaults and
> >>> linked-list corruption from (cy)PARI. Here are the logs from testing with
> >>> the first and the last having full tracebacks.
> >>>
> >>> http://sporadic.stanford.edu/badlog-match
> >>> http://sporadic.stanford.edu/badlog
> >>> http://sporadic.stanford.edu/badlog1
> >>> http://sporadic.stanford.edu/badlog2
> >>> http://sporadic.stanford.edu/badlog3
> >>>
> >>> The first question would be if anyone has an idea about what is causing
> >>> this. I have this impression that PARI is thread-safe, but I am wondering
> >>> if cypari is also thread/parallel-safe or if there are any specific
> >> things
> >>> that we should be careful about. (We’ve already had to work around a
> >>> pickling issue with polynomials IIRC.)
> >>>
> >>> Second question is that because this is a Heisenbug and I suspect it is
> >>> something upstream (and so far, nobody has been able to reproducing it
> >>> during an interactive version of Sage), I was wondering what the policy
> >>> would be for merging the ticket. I recall in the past that we have merged
> >>> tickets with Heisenbugs with followup tickets noting the behavior, but I
> >> am
> >>> not 100% sure about that (and I wouldn’t necessarily know how to find any
> >>> explicit examples). I was wondering if we could merge the ticket in an
> >>> early beta version so that many people/systems can test it to see if it
> >>> becomes more reproducible; of course this is assuming that the build bots
> >>> are not consistent in reproducing this. Should we just mark any offending
> >>> test(s) as “# known bug” and is there some general policy about this?
> >>>
> >>> Thanks,
> >>> Travis
> >>>
> >>>
> >>
> >
>
> --
> You received this message because you are subscribed to the Google Groups 
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sage-devel+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sage-devel/fd05e9f5-52b2-f2f8-bc02-c968c0a70e79%40gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/CAAWYfq0B0H2E3JG-yPG%2B%3Dx-ZEutpbbH_NAW_tRD%3DJG9MPqrUiA%40mail.gmail.com.

Re: [sage-devel] (cy)PARI in Parallel, Heisenbugs, and Merging Policy

2022-05-16 Thread Vincent Delecroix 

I would say that code with parallel computations + cypari2 should not
be merged (as cypari2 does not support it).

If you need parallel + PARI then use the C library directly with the
appropriate threads locks.

If the problem comes from somewhere else, then it would better be sorted
out.

Best
Vincent

Le 16/05/2022 à 06:42, 'Travis Scrimshaw' via sage-devel a écrit :

That sure seems like it. So what should we do about the ticket? Would there
be opposition to merging this piece of code, as there doesn't seem like
there is a fix coming for the likely underlying cypari bug anytime soon?

Best,
Travis


On Sunday, May 15, 2022 at 4:14:31 PM UTC+9 vdelecroix wrote:


Probably related to https://github.com/sagemath/cypari2/issues/107 ?

Le 15/05/2022 à 05:06, 'Travis Scrimshaw' via sage-devel a écrit :

Hi everyone,
On ticket #30423 , Dan, Willie,
and I have been working on a parallel-computation based implementation

for

computing F-matrices that are used in math physics. However, we have been
seeing some doctest failures sporadically that involve segfaults and
linked-list corruption from (cy)PARI. Here are the logs from testing with
the first and the last having full tracebacks.

http://sporadic.stanford.edu/badlog-match
http://sporadic.stanford.edu/badlog
http://sporadic.stanford.edu/badlog1
http://sporadic.stanford.edu/badlog2
http://sporadic.stanford.edu/badlog3

The first question would be if anyone has an idea about what is causing
this. I have this impression that PARI is thread-safe, but I am wondering
if cypari is also thread/parallel-safe or if there are any specific

things

that we should be careful about. (We’ve already had to work around a
pickling issue with polynomials IIRC.)

Second question is that because this is a Heisenbug and I suspect it is
something upstream (and so far, nobody has been able to reproducing it
during an interactive version of Sage), I was wondering what the policy
would be for merging the ticket. I recall in the past that we have merged
tickets with Heisenbugs with followup tickets noting the behavior, but I

am

not 100% sure about that (and I wouldn’t necessarily know how to find any
explicit examples). I was wondering if we could merge the ticket in an
early beta version so that many people/systems can test it to see if it
becomes more reproducible; of course this is assuming that the build bots
are not consistent in reproducing this. Should we just mark any offending
test(s) as “# known bug” and is there some general policy about this?

Thanks,
Travis








--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/fd05e9f5-52b2-f2f8-bc02-c968c0a70e79%40gmail.com.

Re: [sage-devel] (cy)PARI in Parallel, Heisenbugs, and Merging Policy

2022-05-15 Thread 'Travis Scrimshaw' via sage-devel 
That sure seems like it. So what should we do about the ticket? Would there 
be opposition to merging this piece of code, as there doesn't seem like 
there is a fix coming for the likely underlying cypari bug anytime soon?

Best,
Travis


On Sunday, May 15, 2022 at 4:14:31 PM UTC+9 vdelecroix wrote:

> Probably related to https://github.com/sagemath/cypari2/issues/107 ?
>
> Le 15/05/2022 à 05:06, 'Travis Scrimshaw' via sage-devel a écrit :
> > Hi everyone,
> > On ticket #30423 , Dan, Willie,
> > and I have been working on a parallel-computation based implementation 
> for
> > computing F-matrices that are used in math physics. However, we have been
> > seeing some doctest failures sporadically that involve segfaults and
> > linked-list corruption from (cy)PARI. Here are the logs from testing with
> > the first and the last having full tracebacks.
> > 
> > http://sporadic.stanford.edu/badlog-match
> > http://sporadic.stanford.edu/badlog
> > http://sporadic.stanford.edu/badlog1
> > http://sporadic.stanford.edu/badlog2
> > http://sporadic.stanford.edu/badlog3
> > 
> > The first question would be if anyone has an idea about what is causing
> > this. I have this impression that PARI is thread-safe, but I am wondering
> > if cypari is also thread/parallel-safe or if there are any specific 
> things
> > that we should be careful about. (We’ve already had to work around a
> > pickling issue with polynomials IIRC.)
> > 
> > Second question is that because this is a Heisenbug and I suspect it is
> > something upstream (and so far, nobody has been able to reproducing it
> > during an interactive version of Sage), I was wondering what the policy
> > would be for merging the ticket. I recall in the past that we have merged
> > tickets with Heisenbugs with followup tickets noting the behavior, but I 
> am
> > not 100% sure about that (and I wouldn’t necessarily know how to find any
> > explicit examples). I was wondering if we could merge the ticket in an
> > early beta version so that many people/systems can test it to see if it
> > becomes more reproducible; of course this is assuming that the build bots
> > are not consistent in reproducing this. Should we just mark any offending
> > test(s) as “# known bug” and is there some general policy about this?
> > 
> > Thanks,
> > Travis
> > 
> > 
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/23dca745-b13f-47b6-9662-dc75f6109784n%40googlegroups.com.

Re: [sage-devel] (cy)PARI in Parallel, Heisenbugs, and Merging Policy

2022-05-15 Thread Vincent Delecroix 

Probably related to https://github.com/sagemath/cypari2/issues/107 ?

Le 15/05/2022 à 05:06, 'Travis Scrimshaw' via sage-devel a écrit :

Hi everyone,
On ticket #30423 , Dan, Willie,
and I have been working on a parallel-computation based implementation for
computing F-matrices that are used in math physics. However, we have been
seeing some doctest failures sporadically that involve segfaults and
linked-list corruption from (cy)PARI. Here are the logs from testing with
the first and the last having full tracebacks.

http://sporadic.stanford.edu/badlog-match
http://sporadic.stanford.edu/badlog
http://sporadic.stanford.edu/badlog1
http://sporadic.stanford.edu/badlog2
http://sporadic.stanford.edu/badlog3

The first question would be if anyone has an idea about what is causing
this. I have this impression that PARI is thread-safe, but I am wondering
if cypari is also thread/parallel-safe or if there are any specific things
that we should be careful about. (We’ve already had to work around a
pickling issue with polynomials IIRC.)

Second question is that because this is a Heisenbug and I suspect it is
something upstream (and so far, nobody has been able to reproducing it
during an interactive version of Sage), I was wondering what the policy
would be for merging the ticket. I recall in the past that we have merged
tickets with Heisenbugs with followup tickets noting the behavior, but I am
not 100% sure about that (and I wouldn’t necessarily know how to find any
explicit examples). I was wondering if we could merge the ticket in an
early beta version so that many people/systems can test it to see if it
becomes more reproducible; of course this is assuming that the build bots
are not consistent in reproducing this. Should we just mark any offending
test(s) as “# known bug” and is there some general policy about this?

Thanks,
Travis




--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/c6f8db11-1f7d-2ce3-5550-1cd3a776a8ce%40gmail.com.

[sage-devel] (cy)PARI in Parallel, Heisenbugs, and Merging Policy

2022-05-14 Thread 'Travis Scrimshaw' via sage-devel 
Hi everyone,
   On ticket #30423 , Dan, Willie, 
and I have been working on a parallel-computation based implementation for 
computing F-matrices that are used in math physics. However, we have been 
seeing some doctest failures sporadically that involve segfaults and 
linked-list corruption from (cy)PARI. Here are the logs from testing with 
the first and the last having full tracebacks.

http://sporadic.stanford.edu/badlog-match
http://sporadic.stanford.edu/badlog
http://sporadic.stanford.edu/badlog1
http://sporadic.stanford.edu/badlog2
http://sporadic.stanford.edu/badlog3

The first question would be if anyone has an idea about what is causing 
this. I have this impression that PARI is thread-safe, but I am wondering 
if cypari is also thread/parallel-safe or if there are any specific things 
that we should be careful about. (We’ve already had to work around a 
pickling issue with polynomials IIRC.)

Second question is that because this is a Heisenbug and I suspect it is 
something upstream (and so far, nobody has been able to reproducing it 
during an interactive version of Sage), I was wondering what the policy 
would be for merging the ticket. I recall in the past that we have merged 
tickets with Heisenbugs with followup tickets noting the behavior, but I am 
not 100% sure about that (and I wouldn’t necessarily know how to find any 
explicit examples). I was wondering if we could merge the ticket in an 
early beta version so that many people/systems can test it to see if it 
becomes more reproducible; of course this is assuming that the build bots 
are not consistent in reproducing this. Should we just mark any offending 
test(s) as “# known bug” and is there some general policy about this?

Thanks,
Travis


-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/ab4daa9c-7353-408a-a5c5-cb14cff1fc03n%40googlegroups.com.