Hi Guenter, just for confirmation, did you build both VTK that your converter uses and ParaView with VTK_USE_64BIT_IDS ON?
By the way I'm working with OS X 10.6, if it makes difference... Takuya Takuya OSHIMA, Ph.D. Faculty of Engineering, Niigata University 8050 Ikarashi-Ninocho, Nishi-ku, Niigata, 950-2181, JAPAN From: Guenter Bartsch <guenter.bart...@googlemail.com> Subject: Re: [Paraview] Error when reading large XMLUnstructuredData file Date: Tue, 4 Jan 2011 12:10:27 +0100 > Takuya, > > thanks for your quick reply. however, we do have set > > VTK_USE_64BIT_IDS ON > > and this is a 64bit linux platform (CentOS/RHEL x86_64) in case that > matters - so unfortunately that isn't the solution here. > > thanks anyway and kind regards, > > guenter > > On Tue, Jan 4, 2011 at 12:06 PM, Takuya OSHIMA > <osh...@eng.niigata-u.ac.jp> wrote: >> Hi Guenter and all, >> >> I also had the same issue with .vti (XML vtkImageData) files of >> roughly the same sizes of around 3 GBytes. Currently I'm working >> around the issue by building VTK and ParaView with VTK_USE_64BIT_IDS >> set to ON, which will allow you to load up to 4 GBytes as compared to >> 2 GBytes when VTK_USE_64BIT_IDS is OFF. >> >> However I'd be interested if there's more fundamental way of resolving >> the issue because I'll be handling larger cases soon. Is there a >> format (other than series of legacy VTK format files) that is >> time-aware, parallelizable and allows more than 4 GBytes of >> vtkImageData per partition? >> >> Takuya >> >> Takuya OSHIMA, Ph.D. >> Faculty of Engineering, Niigata University >> 8050 Ikarashi-Ninocho, Nishi-ku, Niigata, 950-2181, JAPAN >> >> From: Guenter Bartsch <guenter.bart...@googlemail.com> >> Subject: [Paraview] Error when reading large XMLUnstructuredData file >> Date: Tue, 4 Jan 2011 11:45:18 +0100 >> >>> hi all, >>> >>> we have run into a problem reading a pretty large (3.6 GB) .vtu file. >>> the file was produced by a fortran program that writes vtk data in the >>> old ascii format which results in a 17G ascii file which takes very >>> long to load in paraview (and results in an error message). so, we >>> have written a converter program that reads the ascii file and outputs >>> xml: >>> >>> >>> int main(int argc, char *argp[]) >>> { >>> if (argc != 2) { >>> cerr << "usage: " << argp[0] << " <vtk filename>\n\n"; >>> return 1; >>> } >>> >>> char *filename = argp[1]; >>> >>> cout << "Input filename: " << filename << "\n"; >>> >>> vtkSmartPointer<vtkUnstructuredGridReader> reader = >>> vtkSmartPointer<vtkUnstructuredGridReader>::New(); >>> reader->SetFileName(filename); >>> >>> char *ofilename = "foo.vtu"; >>> cout << "Output filename: " << ofilename << "\n"; >>> vtkSmartPointer<vtkXMLUnstructuredGridWriter> writer = >>> vtkSmartPointer<vtkXMLUnstructuredGridWriter>::New(); >>> writer->SetFileName(ofilename); >>> writer->SetInputConnection(reader->GetOutputPort()); >>> writer->Write(); >>> >>> return EXIT_SUCCESS; >>> } >>> >>> this program runs fine. now, if i open the resulting file in paraview, >>> i run into this error as soon as i hit apply: >>> >>> ERROR: In >>> /opt/paraview-3.8.1/src/ParaView-3.8.1/VTK/IO/vtkXMLUnstructuredDataReader.cxx, >>> line 652 >>> vtkXMLUnstructuredGridReader (0x143f0c40): Cannot read cell >>> connectivity from Cells in piece 0 because the "connectivity" array is >>> not long enough. >>> >>> the converter did work on a smaller test data set and paraview could >>> open it - only now with the large data set paraview fails. here is the >>> header of the failing file: >>> >>> <?xml version="1.0"?> >>> <VTKFile type="UnstructuredGrid" version="0.1" >>> byte_order="LittleEndian" compressor="vtkZLibDataCompressor"> >>> <UnstructuredGrid> >>> <Piece NumberOfPoints="106496000" >>> NumberOfCells="105122745" > >>> <PointData Vectors="vectors"> >>> <DataArray type="Float32" Name="vectors" >>> NumberOfComponents="3" format="appended" RangeMin="0.0021820231521" >>> RangeMax="24.459394872" offset="0" /> >>> </PointData> >>> <CellData> >>> </CellData> >>> <Points> >>> <DataArray type="Float32" Name="Points" NumberOfComponents="3" >>> format="appended" RangeMin="0" >>> RangeMax="5007.1818403" offset="1580185620" /> >>> </Points> >>> <Cells> >>> <DataArray type="Int64" Name="connectivity" format="appended" >>> RangeMin="" RangeMax="" >>> offset="2045521660" /> >>> <DataArray type="Int64" Name="offsets" format="appended" >>> RangeMin="" RangeMax="" >>> offset="3504937336" /> >>> <DataArray type="UInt8" Name="types" format="appended" >>> RangeMin="" RangeMax="" >>> offset="3671971228" /> >>> </Cells> >>> </Piece> >>> </UnstructuredGrid> >>> <AppendedData encoding="base64"> >>> _WJgAAACAAAAAAAAAvXYAALF2AADJdgAAUHcAA... >>> >>> does anyone know what could be going on here? do we need to configure >>> paraview to handle larger files? is this a bug? could the original >>> ascii data set have been corrupt and the error reflects now in the xml >>> file? >>> >>> thanks in advance and best regards, >>> >>> guenter >>> _______________________________________________ >>> Powered by www.kitware.com >>> >>> Visit other Kitware open-source projects at >>> http://www.kitware.com/opensource/opensource.html >>> >>> Please keep messages on-topic and check the ParaView Wiki at: >>> http://paraview.org/Wiki/ParaView >>> >>> Follow this link to subscribe/unsubscribe: >>> http://www.paraview.org/mailman/listinfo/paraview >> _______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview