I see...so if MPI is running then it will create a parallel version of the class. Otherwise, it will create the normal reader.
As far as the check goes for a multiprocess controller; while it does call the parent request data in the serial case, the inherited parallel class overrides some of the methods. Which is a problem because it refers to its own file pointer in those methods (it uses MPI_File instead of a FILE*). Hmm. Sohail On Thu, Aug 23, 2012 at 3:20 PM, Andy Bauer <andy.ba...@kitware.com> wrote: > Hi Sohail, > > Sorry for the slow reply but I wanted to make sure how things should work > first and that took a bit to figure out. Anyways, this is closer to what's > needed but the way it is supposed to work in VTK is that we use > vtkObjectFactory to determine which one should be created when > vtkWindBladeReader::New() is called. This is done at build time based on > whether or not VTK is built with MPI. So if VTK and ParaView are built in > parallel then there is code added that creates a vtkPWindBladeReader when > vtkWindBladeReader::New() is called. This is regardless of whether or not > the executable is run with mpi initialized or not. Because of this, the > parallel version of the reader is responsible for checking whether mpi is > initialized, ideally by doing > vtkMultiProcessController::GetGlobalController()->IsA("vtkMPIController") > is true, and if it isn't it should just call the parent class's methods for > reading in the files. You may want to look at vtkPNrrdReader and the > CMakeLists.txt file in that directory to see how they did things. Note > though that if it's a single process running that it doesn't do any MPI > calls even in the parallel version so your code will be slightly different > than theirs. > > As for other parts of the email below: > > 1) yes, please keep cc'ing the list as there are some details in here that > will be helpful to other people trying to implement their own parallel > readers or filters > > 2) I'm not sure what's going on with the gradient filter but am guessing > that it has to do with the multiblock data set and that there probably > isn't that same array in each block. But that's just a guess. Did you try > extracting the "air" block and do the vorticity and/or q criterion > computation on that? > > 3) vtkSMReaderFactory may be the class you're looking for to determine > which reader is getting used. ParaViewReaders.xml has a list of readers > that it will try to read in a file. > > Andy > > > On Tue, Aug 21, 2012 at 7:11 PM, Sohail Shafii <sssha...@ucdavis.edu>wrote: > >> <sent via another e-mail address; yahoo prevents me from uploading >> stuff because it's stupid> >> >> Here's the first version. Do you want me to keep ccing the list? Not >> sure if that's necessary at this point because this is related to the >> development of ParaView and not help. >> >> I've run it with an older version of the master, and it seems to read in >> a test .wind file just fine. For now I had to store the >> vtkPWindBladeReader* files into VTK/Parallel/MPI since I don't have >> VTK/IO/MPIParallel module yet. Now when I run the gradient filter after >> the field, the gradient filter thinks that the input array does not have >> the appropriate number of components for Q criterion or vorticity...but it >> should (UVW does have three components per tuple). Might be my outdated >> copy of the git repo. Haven't debugged that gradient code yet. >> >> *I was wondering if this is the type of code that you want? Or should >> the classes be designed differently?* Also, I have only run it parallel >> because if I try to instantiate it inside of the python shell in serial it >> tries to run RequestInformation (which assumes that the file path exists >> but does not). It's a work-in-progress but it's a start. >> >> A random question: if ParaView reads a file, where would I put a >> breakpoint that tells me which reader it ends up picking?? I have some >> separate VTK code that reads in a rectilinear grid, but does not generate >> cell/bounds information. ParaView does generate information about the >> bounds though. >> >> Sohail >> >> ------------------------------ >> *From:* David E DeMarle <dave.dema...@kitware.com> >> *To:* Sohail Shafii <sohailsha...@yahoo.com> >> *Cc:* Andy Bauer <andy.ba...@kitware.com>; "migic...@gmail.com" < >> migic...@gmail.com>; "paraview@paraview.org" <paraview@paraview.org>; >> "Woodring, Jonathan L" <woodr...@lanl.gov> >> *Sent:* Friday, July 20, 2012 10:02 AM >> *Subject:* Re: [Paraview] ParaView thinks that it does not have a wind >> blade reader >> >> Thanks. >> >> In today's git master it now sits in VTK/IO/Parallel/vtkWindbladeReader. >> >> The serial version should go in VTK/IO/Geometry. >> >> The MPI version should go in a new module VTK/IO/MPIParallel. >> (VTK/IO/Parallel depends only on vtkSocketController now and should >> not depend on vtkMPIController). >> >> Finally when that is done we can remove the flag in paraview's xml >> that prevents it from being used in serial. >> >> David E DeMarle >> Kitware, Inc. >> R&D Engineer >> 21 Corporate Drive >> Clifton Park, NY 12065-8662 >> Phone: 518-881-4909 >> >> >> On Thu, Jul 19, 2012 at 7:08 PM, Sohail Shafii <sohailsha...@yahoo.com> >> wrote: >> > Well it seems like #1 and #2 can be solved by writing a new parallel >> wind >> > blade reader class. I had similar experience by creating distributed >> > versions of class that I created in other vtk directories. The normal >> wind >> > blade reader would have no parallel calls, which would address #1. >> > >> > What I find interesting is that in your directories, the wind blade >> reader >> > is located in VTK/IO/Parallel. For the ParaView snapshot that I >> downloaded, >> > as well as the 3.14 source, it's still in VTK/Parallel. No Parallel >> folder >> > in VTK/IO. Is this a recent change? >> > >> > So maybe the ideal folder layout would be: >> > >> > VTK/IO/WindBladeReader.* >> > >> > VTK/IO/Parallel/DistributedWindBladeReader.* >> > >> > I can make changes to the WindBladeReader to remove the MPI-IO calls and >> > make a distributed wind blade reader class in case you want to add them >> to >> > the repository. The one template I have for the parallel/serial split >> is the >> > vtkStreamTracer and its distributed counterparts so I can use those as a >> > guide. If you have better recommendations please let me know. >> > >> > Sohail >> > >> > ________________________________ >> > From: David E DeMarle <dave.dema...@kitware.com> >> > To: Sohail Shafii <sohailsha...@yahoo.com> >> > Cc: Andy Bauer <andy.ba...@kitware.com>; "migic...@gmail.com" >> > <migic...@gmail.com>; "paraview@paraview.org" <paraview@paraview.org>; >> > "Woodring, Jonathan L" <woodr...@lanl.gov> >> > Sent: Thursday, July 19, 2012 3:20 PM >> > >> > Subject: Re: [Paraview] ParaView thinks that it does not have a wind >> blade >> > reader >> > >> > Two issues need to be fixed. >> > 1) that behavior in 3.14 is there because of this fix which we had to >> > rush in just before the 3.14 release. >> > >> > Committer: Utkarsh Ayachit <utkarsh.ayac...@kitware.com> 2012-02-08 >> > 17:12:18 >> > Parent: e401d569f6e54c49c3497852463f6b13a01da8b6 (Update version after >> > 3.14.0-RC1 tag.) >> > Child: 99fdb1aeb5997c29b43393f45a478c4b53a23109 (Merge topic >> > 'fix_windblade_segfault') >> > Branches: many (31) >> > Follows: v3.14.0-RC1 >> > Precedes: v3.14.0-RC2 >> > >> > Fix segfault with ParaView built with MPI when opening *.vtr files. >> > >> > The problem was that vtkWindBlade reader was getting created to test >> > "CanReadFile" and that would cause segfault unless MPI was >> initialized. >> > >> > Solved the problem by added support to vtkSMReaderFactory to respect >> the >> > ProcessSupport flag on SourceProxy and change windblade reader XML >> > to say that >> > it only works in parallel. >> > >> > In other words, Windblade reader's Constructor should not take it upon >> > itself to initialize MPI. >> > >> > Once that is fixed, we get to issue 2 before paraview 4.0 >> > >> > 2) VTK's new modularization has the rule that no module can behave >> > differently with parallel (or most other cmake config options) turned >> > configured on or off. This is so because it makes it very hard to >> > package VTK when the libs behave differently in different situations. >> > >> > Ie, we can't have any more >> > #ifdef VTK_USE_MPI >> > do something >> > #else >> > do something different >> > #endif >> > in VTK classes. >> > >> > Now if we want a class to behave differently when MPI is turned on or >> > off we have to make a serial only version in a non MPI dependent >> > module and a subclass of that in an MPI dependent module. >> > >> > David E DeMarle >> > Kitware, Inc. >> > R&D Engineer >> > 21 Corporate Drive >> > Clifton Park, NY 12065-8662 >> > Phone: 518-881-4909 >> > >> > >> > On Thu, Jul 19, 2012 at 3:52 PM, Sohail Shafii <sohailsha...@yahoo.com> >> > wrote: >> >> When running serially, vtkSMReaderFactory::CanReadFile sees that the >> >> session >> >> has a single process. As such, on line 660 it realizes that the source >> >> proxy's process support (in this case the source is a >> vtkWindBladeReader >> >> proxy) is multiple processes, which causes a return of "false" from >> that >> >> function. This makes ParaView think that the wind blade reader can't >> read >> >> a >> >> .wind file in serial. According to Andy, the reader works find in >> >> parallel. >> >> >> >> I assume that a fix for this is pretty trivial? It never did this >> before >> >> in >> >> previous versions of ParaView and I think that the wind blade reader >> code >> >> has remained mostly the same. Andy mentioned some details in person >> when >> >> he >> >> was talking with Jon but at that time I didn't know as much as I do >> now. >> >> There is a mention below about enabling the filter in serial and >> parallel >> >> -- >> >> is this done in a xml file? >> >> >> >> Sohail >> >> >> >> ________________________________ >> >> From: David E DeMarle <dave.dema...@kitware.com> >> >> To: Sohail Shafii <sohailsha...@yahoo.com> >> >> Cc: Andy Bauer <andy.ba...@kitware.com>; "migic...@gmail.com" >> >> <migic...@gmail.com>; "paraview@paraview.org" <paraview@paraview.org>; >> >> "Woodring, Jonathan L" <woodr...@lanl.gov> >> >> Sent: Tuesday, July 17, 2012 7:00 PM >> >> >> >> Subject: Re: [Paraview] ParaView thinks that it does not have a wind >> blade >> >> reader >> >> >> >> Each file that has an association with that file extension, made via >> the >> >> XML, has its CanReadFile method called. Put a breakpoint there to >> >> determine >> >> if ParaView even knows about that reader. >> >> >> >> On Tuesday, July 17, 2012, Sohail Shafii wrote: >> >> >> >> Sorry, but I meant to ask a different question. Here's what Andy said >> >> before, which relates to my concern: >> >> "> >> >>> I went through the code and it jogged my memory about previous >> problems >> >>> with >> >>> it. It had an explicit MPI call in the constructor that would cause >> >>> problems >> >>> when paraview wasn't getting built with MPI. Any time a user tried to >> >>> open >> >>> an unknown file it would try to create a windblade reader to see if >> the >> >>> reader could handle the file and this would cause a crash. If you >> launch >> >>> a >> >>> separate pvserver the wind blade reader will be available in the gui." >> >> >> >> What I meant was; which ParaView class "creates" or "tests" a reader to >> >> see >> >> if it can handle a file? The test code you specified has a predefined >> data >> >> set that is loaded in, and I think is more of an example to your >> average >> >> user. Correct me if I'm wrong. >> >> >> >> So basically ParaView will recognize the file extension, tries to load >> it, >> >> crashes somewhere, then asks you specify the data format from a dialog. >> >> I'm >> >> trying to figure out where it crashes so that I can see how that >> relates >> >> to >> >> the wind blade reader class and any MPI calls that are in there. >> >> >> >> Sohail >> >> >> >> Subject: Re: [Paraview] ParaView thinks that it does not have a wind >> blade >> >> reader >> >> >> >> kargad:ParaView demarle$ find . -name "*WindBlade*" -print >> >> ---> ./VTK/IO/Parallel/Testing/Cxx/TestWindBladeReader.cxx >> >> ./VTK/IO/Parallel/vtkWindBladeReader.cxx >> >> ./VTK/IO/Parallel/vtkWindBladeReader.h >> >> >> >> David E DeMarle >> >> Kitware, Inc. >> >> R&D Engineer >> >> 21 Corporate Drive >> >> Clifton Park, NY 12065-8662 >> >> Phone: 518-881-4909 >> >> >> >> >> >> On Tue, Jul 17, 2012 at 2:09 PM, Sohail Shafii <sohailsha...@yahoo.com >> > >> >> wrote: >> >>> What is the class that is responsible for testing to reader to see if >> it >> >>> can >> >>> open the .wind file? >> >>> >> >>> Sohail >> >>> >> >>> ________________________________ >> >>> From: Andy Bauer <andy.ba...@kitware.com> >> >>> To: Sohail Shafii <sohailsha...@yahoo.com> >> >>> Cc: "Woodring, Jonathan L" <woodr...@lanl.gov>; Sebastien Jourdain >> >>> <sebastien.jourd...@kitware.com>; "migic...@gmail.com" >> >>> <migic...@gmail.com>; >> >>> "paraview@paraview.org" <paraview@paraview.org> >> >>> Sent: Wednesday, July 11, 2012 10:53 AM >> >>> >> >>> Subject: Re: [Paraview] ParaView thinks that it does not have a wind >> >>> blade >> >>> reader >> >>> >> >>> I was reluctant to enable the reader in serial (i.e. not using MPI IO >> >>> which >> >>> I think is getting done by default with the parallel build) because of >> >>> the >> >>> following warnings that I got: >> >>> >> >>> Warning: In /home/acbauer/CODE/ParaView/ >> >>> ParaView/VTK/IO/Parallel/vtkWindBladeReader.cxx, line 942 >> >>> vtkWindBladeReader (0x3a0c800): WindBladeReader error reading file: >> >>> /home/acbauer/DATA/VTKLargeData/Data/WindBladeReader/test1_topo.wind >> >>> Premature EOF while reading block of data. Expected 896000 but got 0 >> >>> >> >>> I just assumed that it was getting bad results so I didn't check the >> >>> serial >> >>> output with the parallel output. In either case the code should be >> >>> changed >> >>> before enabling it in serial so that it reads the files correctly and >> >>> doesn't produce any warning messages. >> >>> >> >>> Andy >> >>> >> >>> On Wed, Jul 11, 2012 at 12:04 PM, Sohail Shafii < >> sohailsha...@yahoo.com> >> >>> wrote: >> >>> >> >>> I'm not aware of any option to build/link against mpi-io. This is >> just a >> >>> standard build with mpi support that suffers from this issue. >> >>> >> >>> The ParaView website's builds also have the same bug. >> >>> >> >>> ________________________________ >> >>> From: "Woodring, Jonathan L" <woodr...@lanl.gov> >> >>> To: Sohail Shafii <sohailsha...@yahoo.com>; Andy Bauer >> >>> <andy.ba...@kitware.com>; Sebastien Jourdain >> >>> <sebastien.jourd...@kitware.com> >> >>> Cc: "migic...@gmail.com" <migic...@gmail.com>; "paraview@paraview.org >> " >> >>> <paraview@paraview.org> >> >>> Sent: Tuesday, July 10, 2012 10:06 AM >> >>> >> >>> Subject: Re: [Paraview] ParaView thinks that it does not have a wind >> >>> blade >> >>> reader >> >>> >> >>> I added the MPI calls to be in sync with what was being added in the >> >>> simulation. >> >>> >> >>> In the SuperComputing demo last year, we updated both the HiGrad >> FireTec >> >>> to >> >>> use MPI-IO and likewise ParaView to do the same thing. >> >>> >> >>> It w >> >> >> >> >> >> >> >> -- >> >> David E DeMarle >> >> Kitware, Inc. >> >> R&D Engineer >> >> 21 Corporate Drive >> >> Clifton Park, NY 12065-8662 >> >> Phone: 518-881-4909 >> >> >> >> >> > >> > >> >> >> >
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview