Well I guess it's not a problem in the derived functions if I check to see if MPI is available; if not, call the superclass's version of the virtual function.
On Tue, Aug 28, 2012 at 11:32 AM, Sohail Shafii <sssha...@ucdavis.edu>wrote: > MPI is only used if it's available. It tries to see if there is a > MPIController available. If not, then the number of ranks is 1, which > means that the parallel class calls the super class's requestdata (super > class is the serial wind blade reader). > > However, the parallel class overrides certain functions to make use of > MPI. So even the parallel instantiation will call the superclass's > requestdata, the overriden functions that are called from that point on > assume that MPI is available...this is a problem when the MPI controller is > not available. > > On another note, it appears that the rectilinear grid that I am reading > (VTK file, structured points I think) is read by the vtkFileSeriesReader, > which makes use of the vtkPDataSetReader class. Strange that if I use > vtkPDataSetReader myself in C++ code that uses VTK and not paraview, > bounds/cell information is not generated. So I think that ParaView does > something else besides using that reader...hmmm. > > > On Mon, Aug 27, 2012 at 7:32 PM, Andy Bauer <andy.ba...@kitware.com>wrote: > >> >> >> On Mon, Aug 27, 2012 at 6:53 PM, Sohail Shafii <sssha...@ucdavis.edu>wrote: >> >>> I see...so if MPI is running then it will create a parallel version of >>> the class. Otherwise, it will create the normal reader. >>> >> >> Technically, if ParaView is built with MPI enabled it will create the >> parallel version of the class every time whether or not the client, server, >> or python scripts are actually run with mpiexec, mpirun, or whatever is >> used to start the executable. So you can't rely on MPI being initialized >> and should not initialize it yourself in your class as that can cause >> problems as well. I think that's what you meant but wanted to be as clear >> as possible for the implications for the classes. >> >> >>> >>> As far as the check goes for a multiprocess controller; while it does >>> call the parent request data in the serial case, the inherited parallel >>> class overrides some of the methods. Which is a problem because it refers >>> to its own file pointer in those methods (it uses MPI_File instead of a >>> FILE*). Hmm. >>> >>> Sohail >>> >>> >>> On Thu, Aug 23, 2012 at 3:20 PM, Andy Bauer <andy.ba...@kitware.com>wrote: >>> >>>> Hi Sohail, >>>> >>>> Sorry for the slow reply but I wanted to make sure how things should >>>> work first and that took a bit to figure out. Anyways, this is closer to >>>> what's needed but the way it is supposed to work in VTK is that we use >>>> vtkObjectFactory to determine which one should be created when >>>> vtkWindBladeReader::New() is called. This is done at build time based on >>>> whether or not VTK is built with MPI. So if VTK and ParaView are built in >>>> parallel then there is code added that creates a vtkPWindBladeReader when >>>> vtkWindBladeReader::New() is called. This is regardless of whether or not >>>> the executable is run with mpi initialized or not. Because of this, the >>>> parallel version of the reader is responsible for checking whether mpi is >>>> initialized, ideally by doing >>>> vtkMultiProcessController::GetGlobalController()->IsA("vtkMPIController") >>>> is true, and if it isn't it should just call the parent class's methods for >>>> reading in the files. You may want to look at vtkPNrrdReader and the >>>> CMakeLists.txt file in that directory to see how they did things. Note >>>> though that if it's a single process running that it doesn't do any MPI >>>> calls even in the parallel version so your code will be slightly different >>>> than theirs. >>>> >>>> As for other parts of the email below: >>>> >>>> 1) yes, please keep cc'ing the list as there are some details in here >>>> that will be helpful to other people trying to implement their own parallel >>>> readers or filters >>>> >>>> 2) I'm not sure what's going on with the gradient filter but am >>>> guessing that it has to do with the multiblock data set and that there >>>> probably isn't that same array in each block. But that's just a guess. Did >>>> you try extracting the "air" block and do the vorticity and/or q criterion >>>> computation on that? >>>> >>>> 3) vtkSMReaderFactory may be the class you're looking for to determine >>>> which reader is getting used. ParaViewReaders.xml has a list of readers >>>> that it will try to read in a file. >>>> >>>> Andy >>>> >>>> >>>> On Tue, Aug 21, 2012 at 7:11 PM, Sohail Shafii <sssha...@ucdavis.edu>wrote: >>>> >>>>> <sent via another e-mail address; yahoo prevents me from uploading >>>>> stuff because it's stupid> >>>>> >>>>> Here's the first version. Do you want me to keep ccing the list? Not >>>>> sure if that's necessary at this point because this is related to the >>>>> development of ParaView and not help. >>>>> >>>>> I've run it with an older version of the master, and it seems to read >>>>> in a test .wind file just fine. For now I had to store the >>>>> vtkPWindBladeReader* files into VTK/Parallel/MPI since I don't have >>>>> VTK/IO/MPIParallel module yet. Now when I run the gradient filter after >>>>> the field, the gradient filter thinks that the input array does not have >>>>> the appropriate number of components for Q criterion or vorticity...but it >>>>> should (UVW does have three components per tuple). Might be my outdated >>>>> copy of the git repo. Haven't debugged that gradient code yet. >>>>> >>>>> *I was wondering if this is the type of code that you want? Or >>>>> should the classes be designed differently?* Also, I have only run >>>>> it parallel because if I try to instantiate it inside of the python shell >>>>> in serial it tries to run RequestInformation (which assumes that the file >>>>> path exists but does not). It's a work-in-progress but it's a start. >>>>> >>>>> A random question: if ParaView reads a file, where would I put a >>>>> breakpoint that tells me which reader it ends up picking?? I have some >>>>> separate VTK code that reads in a rectilinear grid, but does not generate >>>>> cell/bounds information. ParaView does generate information about the >>>>> bounds though. >>>>> >>>>> Sohail >>>>> >>>>> ------------------------------ >>>>> *From:* David E DeMarle <dave.dema...@kitware.com> >>>>> *To:* Sohail Shafii <sohailsha...@yahoo.com> >>>>> *Cc:* Andy Bauer <andy.ba...@kitware.com>; "migic...@gmail.com" < >>>>> migic...@gmail.com>; "paraview@paraview.org" <paraview@paraview.org>; >>>>> "Woodring, Jonathan L" <woodr...@lanl.gov> >>>>> *Sent:* Friday, July 20, 2012 10:02 AM >>>>> *Subject:* Re: [Paraview] ParaView thinks that it does not have a >>>>> wind blade reader >>>>> >>>>> Thanks. >>>>> >>>>> In today's git master it now sits in >>>>> VTK/IO/Parallel/vtkWindbladeReader. >>>>> >>>>> The serial version should go in VTK/IO/Geometry. >>>>> >>>>> The MPI version should go in a new module VTK/IO/MPIParallel. >>>>> (VTK/IO/Parallel depends only on vtkSocketController now and should >>>>> not depend on vtkMPIController). >>>>> >>>>> Finally when that is done we can remove the flag in paraview's xml >>>>> that prevents it from being used in serial. >>>>> >>>>> David E DeMarle >>>>> Kitware, Inc. >>>>> R&D Engineer >>>>> 21 Corporate Drive >>>>> Clifton Park, NY 12065-8662 >>>>> Phone: 518-881-4909 >>>>> >>>>> >>>>> On Thu, Jul 19, 2012 at 7:08 PM, Sohail Shafii <sohailsha...@yahoo.com> >>>>> wrote: >>>>> > Well it seems like #1 and #2 can be solved by writing a new parallel >>>>> wind >>>>> > blade reader class. I had similar experience by creating distributed >>>>> > versions of class that I created in other vtk directories. The >>>>> normal wind >>>>> > blade reader would have no parallel calls, which would address #1. >>>>> > >>>>> > What I find interesting is that in your directories, the wind blade >>>>> reader >>>>> > is located in VTK/IO/Parallel. For the ParaView snapshot that I >>>>> downloaded, >>>>> > as well as the 3.14 source, it's still in VTK/Parallel. No Parallel >>>>> folder >>>>> > in VTK/IO. Is this a recent change? >>>>> > >>>>> > So maybe the ideal folder layout would be: >>>>> > >>>>> > VTK/IO/WindBladeReader.* >>>>> > >>>>> > VTK/IO/Parallel/DistributedWindBladeReader.* >>>>> > >>>>> > I can make changes to the WindBladeReader to remove the MPI-IO calls >>>>> and >>>>> > make a distributed wind blade reader class in case you want to add >>>>> them to >>>>> > the repository. The one template I have for the parallel/serial >>>>> split is the >>>>> > vtkStreamTracer and its distributed counterparts so I can use those >>>>> as a >>>>> > guide. If you have better recommendations please let me know. >>>>> > >>>>> > Sohail >>>>> > >>>>> > ________________________________ >>>>> > From: David E DeMarle <dave.dema...@kitware.com> >>>>> > To: Sohail Shafii <sohailsha...@yahoo.com> >>>>> > Cc: Andy Bauer <andy.ba...@kitware.com>; "migic...@gmail.com" >>>>> > <migic...@gmail.com>; "paraview@paraview.org" <paraview@paraview.org >>>>> >; >>>>> > "Woodring, Jonathan L" <woodr...@lanl.gov> >>>>> > Sent: Thursday, July 19, 2012 3:20 PM >>>>> > >>>>> > Subject: Re: [Paraview] ParaView thinks that it does not have a wind >>>>> blade >>>>> > reader >>>>> > >>>>> > Two issues need to be fixed. >>>>> > 1) that behavior in 3.14 is there because of this fix which we had to >>>>> > rush in just before the 3.14 release. >>>>> > >>>>> > Committer: Utkarsh Ayachit <utkarsh.ayac...@kitware.com> 2012-02-08 >>>>> > 17:12:18 >>>>> > Parent: e401d569f6e54c49c3497852463f6b13a01da8b6 (Update version >>>>> after >>>>> > 3.14.0-RC1 tag.) >>>>> > Child: 99fdb1aeb5997c29b43393f45a478c4b53a23109 (Merge topic >>>>> > 'fix_windblade_segfault') >>>>> > Branches: many (31) >>>>> > Follows: v3.14.0-RC1 >>>>> > Precedes: v3.14.0-RC2 >>>>> > >>>>> > Fix segfault with ParaView built with MPI when opening *.vtr >>>>> files. >>>>> > >>>>> > The problem was that vtkWindBlade reader was getting created to >>>>> test >>>>> > "CanReadFile" and that would cause segfault unless MPI was >>>>> initialized. >>>>> > >>>>> > Solved the problem by added support to vtkSMReaderFactory to >>>>> respect the >>>>> > ProcessSupport flag on SourceProxy and change windblade reader XML >>>>> > to say that >>>>> > it only works in parallel. >>>>> > >>>>> > In other words, Windblade reader's Constructor should not take it >>>>> upon >>>>> > itself to initialize MPI. >>>>> > >>>>> > Once that is fixed, we get to issue 2 before paraview 4.0 >>>>> > >>>>> > 2) VTK's new modularization has the rule that no module can behave >>>>> > differently with parallel (or most other cmake config options) turned >>>>> > configured on or off. This is so because it makes it very hard to >>>>> > package VTK when the libs behave differently in different situations. >>>>> > >>>>> > Ie, we can't have any more >>>>> > #ifdef VTK_USE_MPI >>>>> > do something >>>>> > #else >>>>> > do something different >>>>> > #endif >>>>> > in VTK classes. >>>>> > >>>>> > Now if we want a class to behave differently when MPI is turned on or >>>>> > off we have to make a serial only version in a non MPI dependent >>>>> > module and a subclass of that in an MPI dependent module. >>>>> > >>>>> > David E DeMarle >>>>> > Kitware, Inc. >>>>> > R&D Engineer >>>>> > 21 Corporate Drive >>>>> > Clifton Park, NY 12065-8662 >>>>> > Phone: 518-881-4909 >>>>> > >>>>> > >>>>> > On Thu, Jul 19, 2012 at 3:52 PM, Sohail Shafii < >>>>> sohailsha...@yahoo.com> >>>>> > wrote: >>>>> >> When running serially, vtkSMReaderFactory::CanReadFile sees that the >>>>> >> session >>>>> >> has a single process. As such, on line 660 it realizes that the >>>>> source >>>>> >> proxy's process support (in this case the source is a >>>>> vtkWindBladeReader >>>>> >> proxy) is multiple processes, which causes a return of "false" from >>>>> that >>>>> >> function. This makes ParaView think that the wind blade reader >>>>> can't read >>>>> >> a >>>>> >> .wind file in serial. According to Andy, the reader works find in >>>>> >> parallel. >>>>> >> >>>>> >> I assume that a fix for this is pretty trivial? It never did this >>>>> before >>>>> >> in >>>>> >> previous versions of ParaView and I think that the wind blade >>>>> reader code >>>>> >> has remained mostly the same. Andy mentioned some details in person >>>>> when >>>>> >> he >>>>> >> was talking with Jon but at that time I didn't know as much as I do >>>>> now. >>>>> >> There is a mention below about enabling the filter in serial and >>>>> parallel >>>>> >> -- >>>>> >> is this done in a xml file? >>>>> >> >>>>> >> Sohail >>>>> >> >>>>> >> ________________________________ >>>>> >> From: David E DeMarle <dave.dema...@kitware.com> >>>>> >> To: Sohail Shafii <sohailsha...@yahoo.com> >>>>> >> Cc: Andy Bauer <andy.ba...@kitware.com>; "migic...@gmail.com" >>>>> >> <migic...@gmail.com>; "paraview@paraview.org" < >>>>> paraview@paraview.org>; >>>>> >> "Woodring, Jonathan L" <woodr...@lanl.gov> >>>>> >> Sent: Tuesday, July 17, 2012 7:00 PM >>>>> >> >>>>> >> Subject: Re: [Paraview] ParaView thinks that it does not have a >>>>> wind blade >>>>> >> reader >>>>> >> >>>>> >> Each file that has an association with that file extension, made >>>>> via the >>>>> >> XML, has its CanReadFile method called. Put a breakpoint there to >>>>> >> determine >>>>> >> if ParaView even knows about that reader. >>>>> >> >>>>> >> On Tuesday, July 17, 2012, Sohail Shafii wrote: >>>>> >> >>>>> >> Sorry, but I meant to ask a different question. Here's what Andy >>>>> said >>>>> >> before, which relates to my concern: >>>>> >> "> >>>>> >>> I went through the code and it jogged my memory about previous >>>>> problems >>>>> >>> with >>>>> >>> it. It had an explicit MPI call in the constructor that would cause >>>>> >>> problems >>>>> >>> when paraview wasn't getting built with MPI. Any time a user tried >>>>> to >>>>> >>> open >>>>> >>> an unknown file it would try to create a windblade reader to see >>>>> if the >>>>> >>> reader could handle the file and this would cause a crash. If you >>>>> launch >>>>> >>> a >>>>> >>> separate pvserver the wind blade reader will be available in the >>>>> gui." >>>>> >> >>>>> >> What I meant was; which ParaView class "creates" or "tests" a >>>>> reader to >>>>> >> see >>>>> >> if it can handle a file? The test code you specified has a >>>>> predefined data >>>>> >> set that is loaded in, and I think is more of an example to your >>>>> average >>>>> >> user. Correct me if I'm wrong. >>>>> >> >>>>> >> So basically ParaView will recognize the file extension, tries to >>>>> load it, >>>>> >> crashes somewhere, then asks you specify the data format from a >>>>> dialog. >>>>> >> I'm >>>>> >> trying to figure out where it crashes so that I can see how that >>>>> relates >>>>> >> to >>>>> >> the wind blade reader class and any MPI calls that are in there. >>>>> >> >>>>> >> Sohail >>>>> >> >>>>> >> Subject: Re: [Paraview] ParaView thinks that it does not have a >>>>> wind blade >>>>> >> reader >>>>> >> >>>>> >> kargad:ParaView demarle$ find . -name "*WindBlade*" -print >>>>> >> ---> ./VTK/IO/Parallel/Testing/Cxx/TestWindBladeReader.cxx >>>>> >> ./VTK/IO/Parallel/vtkWindBladeReader.cxx >>>>> >> ./VTK/IO/Parallel/vtkWindBladeReader.h >>>>> >> >>>>> >> David E DeMarle >>>>> >> Kitware, Inc. >>>>> >> R&D Engineer >>>>> >> 21 Corporate Drive >>>>> >> Clifton Park, NY 12065-8662 >>>>> >> Phone: 518-881-4909 >>>>> >> >>>>> >> >>>>> >> On Tue, Jul 17, 2012 at 2:09 PM, Sohail Shafii < >>>>> sohailsha...@yahoo.com> >>>>> >> wrote: >>>>> >>> What is the class that is responsible for testing to reader to see >>>>> if it >>>>> >>> can >>>>> >>> open the .wind file? >>>>> >>> >>>>> >>> Sohail >>>>> >>> >>>>> >>> ________________________________ >>>>> >>> From: Andy Bauer <andy.ba...@kitware.com> >>>>> >>> To: Sohail Shafii <sohailsha...@yahoo.com> >>>>> >>> Cc: "Woodring, Jonathan L" <woodr...@lanl.gov>; Sebastien Jourdain >>>>> >>> <sebastien.jourd...@kitware.com>; "migic...@gmail.com" >>>>> >>> <migic...@gmail.com>; >>>>> >>> "paraview@paraview.org" <paraview@paraview.org> >>>>> >>> Sent: Wednesday, July 11, 2012 10:53 AM >>>>> >>> >>>>> >>> Subject: Re: [Paraview] ParaView thinks that it does not have a >>>>> wind >>>>> >>> blade >>>>> >>> reader >>>>> >>> >>>>> >>> I was reluctant to enable the reader in serial (i.e. not using MPI >>>>> IO >>>>> >>> which >>>>> >>> I think is getting done by default with the parallel build) >>>>> because of >>>>> >>> the >>>>> >>> following warnings that I got: >>>>> >>> >>>>> >>> Warning: In /home/acbauer/CODE/ParaView/ >>>>> >>> ParaView/VTK/IO/Parallel/vtkWindBladeReader.cxx, line 942 >>>>> >>> vtkWindBladeReader (0x3a0c800): WindBladeReader error reading file: >>>>> >>> >>>>> /home/acbauer/DATA/VTKLargeData/Data/WindBladeReader/test1_topo.wind >>>>> >>> Premature EOF while reading block of data. Expected 896000 but got >>>>> 0 >>>>> >>> >>>>> >>> I just assumed that it was getting bad results so I didn't check >>>>> the >>>>> >>> serial >>>>> >>> output with the parallel output. In either case the code should be >>>>> >>> changed >>>>> >>> before enabling it in serial so that it reads the files correctly >>>>> and >>>>> >>> doesn't produce any warning messages. >>>>> >>> >>>>> >>> Andy >>>>> >>> >>>>> >>> On Wed, Jul 11, 2012 at 12:04 PM, Sohail Shafii < >>>>> sohailsha...@yahoo.com> >>>>> >>> wrote: >>>>> >>> >>>>> >>> I'm not aware of any option to build/link against mpi-io. This is >>>>> just a >>>>> >>> standard build with mpi support that suffers from this issue. >>>>> >>> >>>>> >>> The ParaView website's builds also have the same bug. >>>>> >>> >>>>> >>> ________________________________ >>>>> >>> From: "Woodring, Jonathan L" <woodr...@lanl.gov> >>>>> >>> To: Sohail Shafii <sohailsha...@yahoo.com>; Andy Bauer >>>>> >>> <andy.ba...@kitware.com>; Sebastien Jourdain >>>>> >>> <sebastien.jourd...@kitware.com> >>>>> >>> Cc: "migic...@gmail.com" <migic...@gmail.com>; " >>>>> paraview@paraview.org" >>>>> >>> <paraview@paraview.org> >>>>> >>> Sent: Tuesday, July 10, 2012 10:06 AM >>>>> >>> >>>>> >>> Subject: Re: [Paraview] ParaView thinks that it does not have a >>>>> wind >>>>> >>> blade >>>>> >>> reader >>>>> >>> >>>>> >>> I added the MPI calls to be in sync with what was being added in >>>>> the >>>>> >>> simulation. >>>>> >>> >>>>> >>> In the SuperComputing demo last year, we updated both the HiGrad >>>>> FireTec >>>>> >>> to >>>>> >>> use MPI-IO and likewise ParaView to do the same thing. >>>>> >>> >>>>> >>> It w >>>>> >> >>>>> >> >>>>> >> >>>>> >> -- >>>>> >> David E DeMarle >>>>> >> Kitware, Inc. >>>>> >> R&D Engineer >>>>> >> 21 Corporate Drive >>>>> >> Clifton Park, NY 12065-8662 >>>>> >> Phone: 518-881-4909 >>>>> >> >>>>> >> >>>>> > >>>>> > >>>>> >>>>> >>>>> >>>> >>> >> >
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview