I took a look at the code. Te problematic fragment (line 114 in vtkEnSightGoldBinaryReader) just checks the file's existance and opens it in standard way. The problem occures in *stat*() function that is fed with the full file name and the reference to structure to receive file params. I tried to read the file from Python shell and the result is the same. I guessed that the reason is an additional slash before the short file name (see error message) but small files are processed OK so the only assumption I can make is that the problem occures in the *stat*() itself on large (> ~2GiB) files due to some strange 32-bit relicts (although itshould work perfectly in 64-bit builds). Can anybody test the opening of large (3...4 GiB or more) EnSight Gold binary files in ParaView 5.0.0 or 5.0.1? I cannot check any assumptions because I only have Linux compilators set up and cannot do the "hello-world"-like stat() test under Windows on a large file...
2016-03-01 8:53 GMT+03:00 Andrew <antech...@gmail.com>: > Hello Samuel. > > Sorry, but I can't undersatand how can the file fragmentation affect it's > readability and parsing. Even if the file has thousands of fragments and > "brain-damaging" :) runlist in many extended FILE records on NTFS > partition, your program, if it uses Win API, will see it like a solid file > (althougt there will be very low performance of sequental access). So there > should be no any faults on reading highly fragmented files. > I have checked out the "direct" reading of EnSight case from the cluster > via samba with ParaView 5.0.0 on Windows machine. The same error appears as > in my first message, although there are *different files and filesystems*. > I think the problem is not a fragmentation but a *file size*. Some > 32-bit-like relicts or so in EnSight reader (it's just a guess, sorry if > I'm wrong). But my hypothesis does not asks the question why does the bug > affect only Windows versions... Maybe there are differences between > versions concerning data types. Do you have any >4 GB binary EnSight 6 or > Gold case to test? I can't (not allowed) to publish my case and it's also > big enough for upload (although theoretically I can do it from my home PC). > > > 2016-02-29 20:27 GMT+03:00 Samuel Key <samuel...@bresnan.net>: > >> Andrew, >> >> Let us focus on MS Windows for a moment. The NTS file system and the use >> of the file system by the O/S is not quite as it should be. >> >> In my experience, the above combination routinely creates an unbelievable >> number of file fragments. Think in terms of confetti. As time goes by, the >> fragmentation builds. Even though you have a respectable amount of RAM, the >> free space itself on the disk driver can get rather disorganized. >> >> You can start by using the MS-supplied defrager. Run this utility five to >> seven times in a row and things might start looking better. However the >> free-space can still remain disjointed. Personally, I prefer a 3rd party >> utility for organizing and defraging the NTS file system (for example, >> PerfectDisk from Raxco). Even this utility has to run a couple of times >> plus once during the O/S re-boot phase to get a decent amount of >> consolidated free space and a somewhat consolidate O/S paging file. All of >> this has to be done on the O/S partition (usually C:) and the partition >> where you read in the 5+ GB Ensight Geometry file. >> >> On my machine, the MS Win7 O/S has its own partition (100GB \C:) plus >> another smaller dedicated partition for a second paging file (28GB \F:). >> All of my simulation work including the EnSight results file-sets reside on >> an entirely separate disk drive (280GB \G: that now needs to be even >> larger). The 3rd party file consolidation software runs "day and night" on >> the \G: partition. >> >> Just to emphasize the point, copying the 5+ GB EnSight geometry file on >> to workstation can result in many scattered fragments even when the >> free-space starts out consolidated. If the free space is not consolidated, >> the file fragmentation will be even more. >> >> The attached screen capture, shows the O/S managed paging-file (brown) in >> the \C: partition is beginning to fragment. >> >> If you go down this path and it helps, please let me know the outcome. >> >> Regards, >> >> Sam Key >> >> On 2/29/2016 1:04 AM, Andrew wrote: >> >> Another "experimental data". I was able to load this big case in ParaView >> viat the CGNS format. It was neccesary to select CGNS 3.0 and HDF5 options >> in CFX Converter. But when I apply the Extract Block filter I see no any >> boundaries, only fluid domain (that is displayed OK). So CGNS format is not >> a wayout... >> >> 2016-02-29 9:00 GMT+03:00 Andrew <antech...@gmail.com>: >> >>> I performed another test with *EnSi.ght 6* (not Gold) format. >>> Conversion was made on Win-7 x64 machine. The same results: PV 4.3 crashed, >>> PV 5.0 "cannot stat" (full error message is in my first post on this >>> topic). Sorry, now I need to work on with my report... >>> >>> >>> >>> >>> >>> 2016-02-29 8:47 GMT+03:00 Andrew < <antech...@gmail.com> >>> antech...@gmail.com>: >>> >>>> Hello. Sorry for silence, I'm usually offline on holydays, although I >>>> have internet connection and read your mail. >>>> >>>> Samuel Key >>>> >>>> Thanks for your attention. >>>> >>>> About transferring files from Linux to Windows. I use binary EnSight >>>> files because CFX converter fails to write so large files in ASCII format. >>>> But I tried to convert from CFX to EnSight (binary) on the same Win-7 x64 >>>> machine with ParaView. Both installed versions (4.3 and 5.0) failed to open >>>> the geometry (I unchecked all fields). ParaView 4.3 crashed, ParaView 5.0 >>>> just cannot open file... >>>> >>>> I think that the root cause may be a geometry file size. It is 4.9 GiB >>>> (5.3 GB), mesh have about 150 millions of cells, not 600 thusands like in >>>> your case. Its possible that ParaView for Windows cannot process so large >>>> EnSight files, although the Linux version reads in this geometry and >>>> displays outline (swapping is so intensive so I cannot displaye something >>>> else). >>>> >>>> 2016-02-26 19:31 GMT+03:00 Samuel Key < <samuel...@bresnan.net> >>>> samuel...@bresnan.net>: >>>> >>>>> Andrew, >>>>> >>>>> For what it is worth, I use ParaView >>>>> (ParaView-5.0.0-Qt4-OpenGL2-Windows-64bit) on Windows-7 64bit. I just now >>>>> tested two medium-sized, EnSight Gold-formatted simulation results files >>>>> (600,000 finite elements) and both worked. >>>>> >>>>> In the past, I have had difficulty with the EnSight ASCII-formatted >>>>> *.case file when moving the simulation results (*.case file and friends) >>>>> from MS Windows to EnSight running on a linux platform. It was necessary >>>>> to >>>>> pass the ASCII *.case file through a 'dos2unix' filter. Transferring files >>>>> the other way might have the same problem??? >>>>> >>>>> Hope this helps. >>>>> >>>>> Samuel Key >>>>> FMA Development, LLC >>>>> 1005 39th Ave NE >>>>> Great Falls, Montana 59404 >>>>> USA >>>>> >>>>> On 2/25/2016 11:26 PM, Andrew wrote: >>>>> >>>>> Hello. We use Ansys CFX in our work but the standard post-processor >>>>> CFD-Post consumes one of Ansys licenses while Paraview is free. So I often >>>>> make pictures for reports with ParaView to preserve Ansys license for >>>>> another users/needs (because other users are not familiar with ParaView >>>>> and >>>>> CFD-Post may consume general "acfd" license feature that may be used for >>>>> calculations). I convert CFX results to EnSight Gold format in CFX Solver >>>>> Manager (Export Results) and then open in ParaView. It was working well on >>>>> Linux (CentOS 6.7 x64, Ubuntu 12.04 x64 with updates) but now I work with >>>>> large files and I need to use another machine that has 128 GB of RAM and >>>>> runs Windows-7 x64. >>>>> I downloaded and installed the following version: >>>>> ParaView-5.0.0-Qt4-OpenGL2-Windows-64bit.exe >>>>> When I open my EnSight-format file I receive the error message that >>>>> geometry file cannot be read. I tried to move EnSight files to a simple >>>>> location (D:\Temp) and renamed them (test.case, test.geom + edited the >>>>> case >>>>> to change geometry file name to "simple" one). The same result, it cannot >>>>> find geometry file... Full error message is as follows: >>>>> >>>>> ERROR: In >>>>> C:\bbd\df0abce0\source-paraview\VTK\IO\EnSight\vtkEnSightGoldBinaryReader.cxx, >>>>> line 114 >>>>> vtkEnSightGoldBinaryReader (0000000009451230): stat failed. >>>>> >>>>> >>>>> ERROR: In >>>>> C:\bbd\df0abce0\source-paraview\VTK\IO\EnSight\vtkEnSightGoldBinaryReader.cxx, >>>>> line 220 >>>>> vtkEnSightGoldBinaryReader (0000000009451230): Unable to open file: >>>>> D:\Storage\Media\Work\Gradirni\CFX\EnSight\C-G002L3S-R020-S000-14.5\/C-G002L3S-R020-S000.geom >>>>> >>>>> >>>>> ERROR: In >>>>> C:\bbd\df0abce0\source-paraview\VTK\IO\EnSight\vtkEnSightReader.cxx, line >>>>> 306 >>>>> vtkEnSightGoldBinaryReader (0000000009451230): error reading geometry >>>>> file >>>>> >>>>> >>>>> ERROR: In >>>>> C:\bbd\df0abce0\source-paraview\VTK\Common\ExecutionModel\vtkExecutive.cxx, >>>>> line 784 >>>>> vtkCompositeDataPipeline (0000000008E1A640): Algorithm >>>>> vtkEnSightGoldBinaryReader(0000000009451230) returned failure for request: >>>>> vtkInformation (00000000093C3920) >>>>> Debug: Off >>>>> Modified Time: 214873 >>>>> Reference Count: 1 >>>>> Registered Events: (none) >>>>> Request: REQUEST_DATA >>>>> ALGORITHM_AFTER_FORWARD: 1 >>>>> FORWARD_DIRECTION: 0 >>>>> FROM_OUTPUT_PORT: 0 >>>>> >>>>> OK, I tried to open it in ParaView 4.3. It says nothing (only border >>>>> of the message window appears) and crashes. At that moment I see in the >>>>> status bar that EnSight reader progress is 100%. >>>>> >>>>> Both ParaView 5.0.0 and 4.3 was tested with EnSight files exported on >>>>> Linux and Windows machine. It didn't help. >>>>> >>>>> OK, I tried to export CFX results to CGNS format. ParaView 5.0.0 >>>>> (without MPI) cannot read it, so I fed it to ParaView 4.3. It says that >>>>> its >>>>> "unable to filed any meshes". >>>>> >>>>> I also performed an experiment on my Linux laptop. It has only 16 GB >>>>> of RAM so its unreal to work with such a big file on this machine but it >>>>> reads the same EnSight case and displays the domain! No arrays selected >>>>> (mesh only), swap file grows up to 8 GB but the file opens without errors >>>>> and crashes. It's only a test because I have not any Linux machine with >>>>> memory large enough (our Linux cluster has 128 GB of RAM on every node >>>>> including master but it has very poor video). >>>>> >>>>> I switched to CFD Post for this case but I want to know are there any >>>>> workarounds to open my EnSight-format results in ParaView for Windows. >>>>> >>>>> Thanks for your attention. >>>>> >>>>> >>>>> _______________________________________________ >>>>> Powered by www.kitware.com >>>>> >>>>> Visit other Kitware open-source projects at >>>>> http://www.kitware.com/opensource/opensource.html >>>>> >>>>> Please keep messages on-topic and check the ParaView Wiki at: >>>>> http://paraview.org/Wiki/ParaView >>>>> >>>>> Search the list archives at: http://markmail.org/search/?q=ParaView >>>>> >>>>> Follow this link to >>>>> subscribe/unsubscribe:http://public.kitware.com/mailman/listinfo/paraview >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Powered by <http://www.kitware.com>www.kitware.com >>>>> >>>>> Visit other Kitware open-source projects at >>>>> <http://www.kitware.com/opensource/opensource.html> >>>>> http://www.kitware.com/opensource/opensource.html >>>>> >>>>> Please keep messages on-topic and check the ParaView Wiki at: >>>>> <http://paraview.org/Wiki/ParaView>http://paraview.org/Wiki/ParaView >>>>> >>>>> Search the list archives at: <http://markmail.org/search/?q=ParaView> >>>>> http://markmail.org/search/?q=ParaView >>>>> >>>>> Follow this link to subscribe/unsubscribe: >>>>> http://public.kitware.com/mailman/listinfo/paraview >>>>> >>>>> >>>> >>> >> >> >> _______________________________________________ >> Powered by www.kitware.com >> >> Visit other Kitware open-source projects at >> http://www.kitware.com/opensource/opensource.html >> >> Please keep messages on-topic and check the ParaView Wiki at: >> http://paraview.org/Wiki/ParaView >> >> Search the list archives at: http://markmail.org/search/?q=ParaView >> >> Follow this link to >> subscribe/unsubscribe:http://public.kitware.com/mailman/listinfo/paraview >> >> >> >> _______________________________________________ >> Powered by www.kitware.com >> >> Visit other Kitware open-source projects at >> http://www.kitware.com/opensource/opensource.html >> >> Please keep messages on-topic and check the ParaView Wiki at: >> http://paraview.org/Wiki/ParaView >> >> Search the list archives at: http://markmail.org/search/?q=ParaView >> >> Follow this link to subscribe/unsubscribe: >> http://public.kitware.com/mailman/listinfo/paraview >> >> >
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Search the list archives at: http://markmail.org/search/?q=ParaView Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/paraview
