Re: [petsc-users] ASCIIRead error for multiple processors
I have added error checking and clearer documentation on PetscViewerASCIIRead() with https://bitbucket.org/petsc/petsc/pull-requests/1514/add-error-checking-if-petscviewerasciiread/diff Thanks for the report Barry > On Apr 4, 2019, at 3:15 PM, Yuyun Yang via petsc-users > wrote: > > Hello team, > > I’m trying to use PetscViewerASCIIRead() to read in a single interger/scalar > value from an input file. It works for one processor. However, when running > on multiple processors, I’m getting the below error: > > [1]PETSC ERROR: Invalid argument > [1]PETSC ERROR: Insufficient data, read only 0 < 1 items > [1]PETSC ERROR: #1 PetscViewerASCIIRead() line 1054 in > /usr/local/CLAB-2/petsc-3.6/src/sys/classes/viewer/impls/ascii/filev.c > > Is there something wrong with how I’m implementing this, or ASCIIRead does > not work with multiple processors? > > Thanks, > Yuyun
Re: [petsc-users] ASCIIRead error for multiple processors
Ok great, thanks for the explanation!
Best,
Yuyun
From: Matthew Knepley
Sent: Friday, April 5, 2019 7:31 AM
To: Yuyun Yang
Cc: Smith, Barry F. ; petsc-users@mcs.anl.gov
Subject: Re: [petsc-users] ASCIIRead error for multiple processors
On Fri, Apr 5, 2019 at 10:27 AM Yuyun Yang
mailto:yyan...@stanford.edu>> wrote:
Hmm ok. Then should I use this function or not when I'm reading the input? It's
probably still going to give me the same error and unable to proceed?
I'd like to know if I should use something else to work around this problem.
No, what you do is
if (!rank) {
PetscViewerASCIIRead();
MPI_Bcast();
} else {
MPI_Bcast();
}
Thanks,
Matt
Thanks,
Yuyun
Get Outlook for iOS<https://aka.ms/o0ukef>
From: Matthew Knepley mailto:knep...@gmail.com>>
Sent: Friday, April 5, 2019 5:16:12 AM
To: Yuyun Yang
Cc: Smith, Barry F.; petsc-users@mcs.anl.gov<mailto:petsc-users@mcs.anl.gov>
Subject: Re: [petsc-users] ASCIIRead error for multiple processors
On Thu, Apr 4, 2019 at 10:56 PM Yuyun Yang
mailto:yyan...@stanford.edu>> wrote:
So do I call MPI_Bcast right after I call PetscViewerASCIIRead? Is that going
to prevent the other processors from trying to read the same file but were
unable to?
No, all this does is replicate data from process 0 on the other processes.
Matt
Thanks,
Yuyun
Get Outlook for iOS<https://aka.ms/o0ukef>
From: Matthew Knepley mailto:knep...@gmail.com>>
Sent: Thursday, April 4, 2019 7:30:20 PM
To: Yuyun Yang
Cc: Smith, Barry F.; petsc-users@mcs.anl.gov<mailto:petsc-users@mcs.anl.gov>
Subject: Re: [petsc-users] ASCIIRead error for multiple processors
On Thu, Apr 4, 2019 at 9:19 PM Yuyun Yang via petsc-users
mailto:petsc-users@mcs.anl.gov>> wrote:
We are probably not going to use hundreds of processors, but i think it would
be good to just have processor 0 read the input and broadcast that to all the
other processors. Would that be a possible fix? And what would you suggest to
work around this problem for now?
Explicitly call MPI_Bcast().
Matt
Thanks!
Yuyun
Get Outlook for iOS<https://aka.ms/o0ukef>
From: Smith, Barry F. mailto:bsm...@mcs.anl.gov>>
Sent: Thursday, April 4, 2019 3:07:37 PM
To: Yuyun Yang
Cc: petsc-users@mcs.anl.gov<mailto:petsc-users@mcs.anl.gov>
Subject: Re: [petsc-users] ASCIIRead error for multiple processors
Currently PetscViewerFileSetName_ASCII() only opens the file on process 0
(for read or write) thus when you call PetscViewerASCIIRead() from any process
but the first it will be reading from an fd that has not been set and you could
get unpredictable results.
The implementation and documentation for PetscViewerASCIIRead() is buggy.
There are two possible fixes we could make
1) have PetscViewerFileSetName_ASCII() open the file for reading on all
processes or
2) have PetscViewerASCIIRead() generate an error if the process is not rank == 0
Barry
Note that using PetscViewerASCIIRead() from a handful of processes is probably
fine but having hundreds or thousands of processes open the same ASCII file and
reading from it will likely not be scalable.
> On Apr 4, 2019, at 3:15 PM, Yuyun Yang via petsc-users
> mailto:petsc-users@mcs.anl.gov>> wrote:
>
> Hello team,
>
> I’m trying to use PetscViewerASCIIRead() to read in a single interger/scalar
> value from an input file. It works for one processor. However, when running
> on multiple processors, I’m getting the below error:
>
> [1]PETSC ERROR: Invalid argument
> [1]PETSC ERROR: Insufficient data, read only 0 < 1 items
> [1]PETSC ERROR: #1 PetscViewerASCIIRead() line 1054 in
> /usr/local/CLAB-2/petsc-3.6/src/sys/classes/viewer/impls/ascii/filev.c
>
> Is there something wrong with how I’m implementing this, or ASCIIRead does
> not work with multiple processors?
>
> Thanks,
> Yuyun
--
What most experimenters take for granted before they begin their experiments is
infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>
--
What most experimenters take for granted before they begin their experiments is
infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>
--
What most experimenters take for granted before they begin their experiments is
infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>
Re: [petsc-users] ASCIIRead error for multiple processors
On Fri, Apr 5, 2019 at 10:27 AM Yuyun Yang wrote:
> Hmm ok. Then should I use this function or not when I'm reading the input?
> It's probably still going to give me the same error and unable to proceed?
>
> I'd like to know if I should use something else to work around this
> problem.
>
No, what you do is
if (!rank) {
PetscViewerASCIIRead();
MPI_Bcast();
} else {
MPI_Bcast();
}
Thanks,
Matt
> Thanks,
> Yuyun
>
> Get Outlook for iOS <https://aka.ms/o0ukef>
> --
> *From:* Matthew Knepley
> *Sent:* Friday, April 5, 2019 5:16:12 AM
> *To:* Yuyun Yang
> *Cc:* Smith, Barry F.; petsc-users@mcs.anl.gov
> *Subject:* Re: [petsc-users] ASCIIRead error for multiple processors
>
> On Thu, Apr 4, 2019 at 10:56 PM Yuyun Yang wrote:
>
>> So do I call MPI_Bcast right after I call PetscViewerASCIIRead? Is that
>> going to prevent the other processors from trying to read the same file but
>> were unable to?
>>
>
> No, all this does is replicate data from process 0 on the other processes.
>
>Matt
>
>
>> Thanks,
>> Yuyun
>>
>> Get Outlook for iOS <https://aka.ms/o0ukef>
>> --
>> *From:* Matthew Knepley
>> *Sent:* Thursday, April 4, 2019 7:30:20 PM
>> *To:* Yuyun Yang
>> *Cc:* Smith, Barry F.; petsc-users@mcs.anl.gov
>> *Subject:* Re: [petsc-users] ASCIIRead error for multiple processors
>>
>> On Thu, Apr 4, 2019 at 9:19 PM Yuyun Yang via petsc-users <
>> petsc-users@mcs.anl.gov> wrote:
>>
>>> We are probably not going to use hundreds of processors, but i think it
>>> would be good to just have processor 0 read the input and broadcast that to
>>> all the other processors. Would that be a possible fix? And what would you
>>> suggest to work around this problem for now?
>>>
>>
>> Explicitly call MPI_Bcast().
>>
>>Matt
>>
>>
>>> Thanks!
>>> Yuyun
>>>
>>> Get Outlook for iOS <https://aka.ms/o0ukef>
>>> --
>>> *From:* Smith, Barry F.
>>> *Sent:* Thursday, April 4, 2019 3:07:37 PM
>>> *To:* Yuyun Yang
>>> *Cc:* petsc-users@mcs.anl.gov
>>> *Subject:* Re: [petsc-users] ASCIIRead error for multiple processors
>>>
>>>
>>>Currently PetscViewerFileSetName_ASCII() only opens the file on
>>> process 0 (for read or write) thus when you call PetscViewerASCIIRead()
>>> from any process but the first it will be reading from an fd that has not
>>> been set and you could get unpredictable results.
>>>
>>>The implementation and documentation for PetscViewerASCIIRead() is
>>> buggy.
>>>
>>>There are two possible fixes we could make
>>>
>>> 1) have PetscViewerFileSetName_ASCII() open the file for reading on all
>>> processes or
>>> 2) have PetscViewerASCIIRead() generate an error if the process is not
>>> rank == 0
>>>
>>>Barry
>>>
>>> Note that using PetscViewerASCIIRead() from a handful of processes is
>>> probably fine but having hundreds or thousands of processes open the same
>>> ASCII file and reading from it will likely not be scalable.
>>>
>>>
>>>
>>>
>>> > On Apr 4, 2019, at 3:15 PM, Yuyun Yang via petsc-users <
>>> petsc-users@mcs.anl.gov> wrote:
>>> >
>>> > Hello team,
>>> >
>>> > I’m trying to use PetscViewerASCIIRead() to read in a single
>>> interger/scalar value from an input file. It works for one processor.
>>> However, when running on multiple processors, I’m getting the below error:
>>> >
>>> > [1]PETSC ERROR: Invalid argument
>>> > [1]PETSC ERROR: Insufficient data, read only 0 < 1 items
>>> > [1]PETSC ERROR: #1 PetscViewerASCIIRead() line 1054 in
>>> /usr/local/CLAB-2/petsc-3.6/src/sys/classes/viewer/impls/ascii/filev.c
>>> >
>>> > Is there something wrong with how I’m implementing this, or ASCIIRead
>>> does not work with multiple processors?
>>> >
>>> > Thanks,
>>> > Yuyun
>>>
>>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/
>> <http://www.cse.buffalo.edu/~knepley/>
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
Re: [petsc-users] ASCIIRead error for multiple processors
Hmm ok. Then should I use this function or not when I'm reading the input? It's probably still going to give me the same error and unable to proceed? I'd like to know if I should use something else to work around this problem. Thanks, Yuyun Get Outlook for iOS<https://aka.ms/o0ukef> From: Matthew Knepley Sent: Friday, April 5, 2019 5:16:12 AM To: Yuyun Yang Cc: Smith, Barry F.; petsc-users@mcs.anl.gov Subject: Re: [petsc-users] ASCIIRead error for multiple processors On Thu, Apr 4, 2019 at 10:56 PM Yuyun Yang mailto:yyan...@stanford.edu>> wrote: So do I call MPI_Bcast right after I call PetscViewerASCIIRead? Is that going to prevent the other processors from trying to read the same file but were unable to? No, all this does is replicate data from process 0 on the other processes. Matt Thanks, Yuyun Get Outlook for iOS<https://aka.ms/o0ukef> From: Matthew Knepley mailto:knep...@gmail.com>> Sent: Thursday, April 4, 2019 7:30:20 PM To: Yuyun Yang Cc: Smith, Barry F.; petsc-users@mcs.anl.gov<mailto:petsc-users@mcs.anl.gov> Subject: Re: [petsc-users] ASCIIRead error for multiple processors On Thu, Apr 4, 2019 at 9:19 PM Yuyun Yang via petsc-users mailto:petsc-users@mcs.anl.gov>> wrote: We are probably not going to use hundreds of processors, but i think it would be good to just have processor 0 read the input and broadcast that to all the other processors. Would that be a possible fix? And what would you suggest to work around this problem for now? Explicitly call MPI_Bcast(). Matt Thanks! Yuyun Get Outlook for iOS<https://aka.ms/o0ukef> From: Smith, Barry F. mailto:bsm...@mcs.anl.gov>> Sent: Thursday, April 4, 2019 3:07:37 PM To: Yuyun Yang Cc: petsc-users@mcs.anl.gov<mailto:petsc-users@mcs.anl.gov> Subject: Re: [petsc-users] ASCIIRead error for multiple processors Currently PetscViewerFileSetName_ASCII() only opens the file on process 0 (for read or write) thus when you call PetscViewerASCIIRead() from any process but the first it will be reading from an fd that has not been set and you could get unpredictable results. The implementation and documentation for PetscViewerASCIIRead() is buggy. There are two possible fixes we could make 1) have PetscViewerFileSetName_ASCII() open the file for reading on all processes or 2) have PetscViewerASCIIRead() generate an error if the process is not rank == 0 Barry Note that using PetscViewerASCIIRead() from a handful of processes is probably fine but having hundreds or thousands of processes open the same ASCII file and reading from it will likely not be scalable. > On Apr 4, 2019, at 3:15 PM, Yuyun Yang via petsc-users > mailto:petsc-users@mcs.anl.gov>> wrote: > > Hello team, > > I’m trying to use PetscViewerASCIIRead() to read in a single interger/scalar > value from an input file. It works for one processor. However, when running > on multiple processors, I’m getting the below error: > > [1]PETSC ERROR: Invalid argument > [1]PETSC ERROR: Insufficient data, read only 0 < 1 items > [1]PETSC ERROR: #1 PetscViewerASCIIRead() line 1054 in > /usr/local/CLAB-2/petsc-3.6/src/sys/classes/viewer/impls/ascii/filev.c > > Is there something wrong with how I’m implementing this, or ASCIIRead does > not work with multiple processors? > > Thanks, > Yuyun -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/> -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>
Re: [petsc-users] ASCIIRead error for multiple processors
So do I call MPI_Bcast right after I call PetscViewerASCIIRead? Is that going to prevent the other processors from trying to read the same file but were unable to? Thanks, Yuyun Get Outlook for iOS<https://aka.ms/o0ukef> From: Matthew Knepley Sent: Thursday, April 4, 2019 7:30:20 PM To: Yuyun Yang Cc: Smith, Barry F.; petsc-users@mcs.anl.gov Subject: Re: [petsc-users] ASCIIRead error for multiple processors On Thu, Apr 4, 2019 at 9:19 PM Yuyun Yang via petsc-users mailto:petsc-users@mcs.anl.gov>> wrote: We are probably not going to use hundreds of processors, but i think it would be good to just have processor 0 read the input and broadcast that to all the other processors. Would that be a possible fix? And what would you suggest to work around this problem for now? Explicitly call MPI_Bcast(). Matt Thanks! Yuyun Get Outlook for iOS<https://aka.ms/o0ukef> From: Smith, Barry F. mailto:bsm...@mcs.anl.gov>> Sent: Thursday, April 4, 2019 3:07:37 PM To: Yuyun Yang Cc: petsc-users@mcs.anl.gov<mailto:petsc-users@mcs.anl.gov> Subject: Re: [petsc-users] ASCIIRead error for multiple processors Currently PetscViewerFileSetName_ASCII() only opens the file on process 0 (for read or write) thus when you call PetscViewerASCIIRead() from any process but the first it will be reading from an fd that has not been set and you could get unpredictable results. The implementation and documentation for PetscViewerASCIIRead() is buggy. There are two possible fixes we could make 1) have PetscViewerFileSetName_ASCII() open the file for reading on all processes or 2) have PetscViewerASCIIRead() generate an error if the process is not rank == 0 Barry Note that using PetscViewerASCIIRead() from a handful of processes is probably fine but having hundreds or thousands of processes open the same ASCII file and reading from it will likely not be scalable. > On Apr 4, 2019, at 3:15 PM, Yuyun Yang via petsc-users > mailto:petsc-users@mcs.anl.gov>> wrote: > > Hello team, > > I’m trying to use PetscViewerASCIIRead() to read in a single interger/scalar > value from an input file. It works for one processor. However, when running > on multiple processors, I’m getting the below error: > > [1]PETSC ERROR: Invalid argument > [1]PETSC ERROR: Insufficient data, read only 0 < 1 items > [1]PETSC ERROR: #1 PetscViewerASCIIRead() line 1054 in > /usr/local/CLAB-2/petsc-3.6/src/sys/classes/viewer/impls/ascii/filev.c > > Is there something wrong with how I’m implementing this, or ASCIIRead does > not work with multiple processors? > > Thanks, > Yuyun -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>
Re: [petsc-users] ASCIIRead error for multiple processors
We are probably not going to use hundreds of processors, but i think it would be good to just have processor 0 read the input and broadcast that to all the other processors. Would that be a possible fix? And what would you suggest to work around this problem for now? Thanks! Yuyun Get Outlook for iOS<https://aka.ms/o0ukef> From: Smith, Barry F. Sent: Thursday, April 4, 2019 3:07:37 PM To: Yuyun Yang Cc: petsc-users@mcs.anl.gov Subject: Re: [petsc-users] ASCIIRead error for multiple processors Currently PetscViewerFileSetName_ASCII() only opens the file on process 0 (for read or write) thus when you call PetscViewerASCIIRead() from any process but the first it will be reading from an fd that has not been set and you could get unpredictable results. The implementation and documentation for PetscViewerASCIIRead() is buggy. There are two possible fixes we could make 1) have PetscViewerFileSetName_ASCII() open the file for reading on all processes or 2) have PetscViewerASCIIRead() generate an error if the process is not rank == 0 Barry Note that using PetscViewerASCIIRead() from a handful of processes is probably fine but having hundreds or thousands of processes open the same ASCII file and reading from it will likely not be scalable. > On Apr 4, 2019, at 3:15 PM, Yuyun Yang via petsc-users > wrote: > > Hello team, > > I’m trying to use PetscViewerASCIIRead() to read in a single interger/scalar > value from an input file. It works for one processor. However, when running > on multiple processors, I’m getting the below error: > > [1]PETSC ERROR: Invalid argument > [1]PETSC ERROR: Insufficient data, read only 0 < 1 items > [1]PETSC ERROR: #1 PetscViewerASCIIRead() line 1054 in > /usr/local/CLAB-2/petsc-3.6/src/sys/classes/viewer/impls/ascii/filev.c > > Is there something wrong with how I’m implementing this, or ASCIIRead does > not work with multiple processors? > > Thanks, > Yuyun
Re: [petsc-users] ASCIIRead error for multiple processors
Currently PetscViewerFileSetName_ASCII() only opens the file on process 0 (for read or write) thus when you call PetscViewerASCIIRead() from any process but the first it will be reading from an fd that has not been set and you could get unpredictable results. The implementation and documentation for PetscViewerASCIIRead() is buggy. There are two possible fixes we could make 1) have PetscViewerFileSetName_ASCII() open the file for reading on all processes or 2) have PetscViewerASCIIRead() generate an error if the process is not rank == 0 Barry Note that using PetscViewerASCIIRead() from a handful of processes is probably fine but having hundreds or thousands of processes open the same ASCII file and reading from it will likely not be scalable. > On Apr 4, 2019, at 3:15 PM, Yuyun Yang via petsc-users > wrote: > > Hello team, > > I’m trying to use PetscViewerASCIIRead() to read in a single interger/scalar > value from an input file. It works for one processor. However, when running > on multiple processors, I’m getting the below error: > > [1]PETSC ERROR: Invalid argument > [1]PETSC ERROR: Insufficient data, read only 0 < 1 items > [1]PETSC ERROR: #1 PetscViewerASCIIRead() line 1054 in > /usr/local/CLAB-2/petsc-3.6/src/sys/classes/viewer/impls/ascii/filev.c > > Is there something wrong with how I’m implementing this, or ASCIIRead does > not work with multiple processors? > > Thanks, > Yuyun
[petsc-users] ASCIIRead error for multiple processors
Hello team, I'm trying to use PetscViewerASCIIRead() to read in a single interger/scalar value from an input file. It works for one processor. However, when running on multiple processors, I'm getting the below error: [1]PETSC ERROR: Invalid argument [1]PETSC ERROR: Insufficient data, read only 0 < 1 items [1]PETSC ERROR: #1 PetscViewerASCIIRead() line 1054 in /usr/local/CLAB-2/petsc-3.6/src/sys/classes/viewer/impls/ascii/filev.c Is there something wrong with how I'm implementing this, or ASCIIRead does not work with multiple processors? Thanks, Yuyun
