[Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-02-16 Thread John Beard
Hi, I was trying to profile the eeschema slow library loads, and I got a bit distracted by RICHIO's FILE_LINE_READER. Internally, it uses a very tight loop of reading single chars at a time from a file descriptor, which looks inefficient. I wrote a benchmarker to compare RICHIO against std::ifstr

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-02-16 Thread Wayne Stambaugh
John, It would have been nice if you would have benchmarked wxFileInputStream as well. There already is an INPUTSTREAM_LINE_READER object which takes a pointer to wxInputStream object. I'm curious how it stacks up against the std::ifstream. There are some interesting wxInputStream objects that

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-02-16 Thread John Beard
Hi Wayne, I added some new profiles for the INPUTSTREAM_LINE_READER. The results are very surprising to me. In debug and release mode, using INPUTSTREAM_LINE_READER with a wxInputFileStream is around 200 times (:-O) slower than a straight std::ifstream, taking over two seconds to read a 6.5MB sho

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-02-17 Thread Wayne Stambaugh
On 2/16/2017 8:59 PM, John Beard wrote: > Hi Wayne, > > I added some new profiles for the INPUTSTREAM_LINE_READER. > > The results are very surprising to me. In debug and release mode, > using INPUTSTREAM_LINE_READER with a wxInputFileStream is around 200 > times (:-O) slower than a straight std:

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-02-17 Thread Nox
What about wxFFileInputStream instead of wxFileInputStream? Am 17.02.2017 um 16:24 schrieb Wayne Stambaugh: On 2/16/2017 8:59 PM, John Beard wrote: Hi Wayne, I added some new profiles for the INPUTSTREAM_LINE_READER. The results are very surprising to me. In debug and release mode, using INP

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-02-17 Thread Wayne Stambaugh
There is also the wrapper wxBufferedInputStream[1] which may improve performance. I'm not sure how all of this fits together. It is not very well documented so some digging around in the wx source may be beneficial. [1] http://docs.wxwidgets.org/3.0/classwx_buffered_input_stream.html On 2/17/20

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-02-19 Thread John Beard
On Sat, Feb 18, 2017 at 1:08 AM, Nox wrote: > What about wxFFileInputStream instead of wxFileInputStream? > wxFFileInputStream appears to be about 5-6 times faster than wxFileInputStream, but that's still much much slower than RICHIO or std::ifstream. $ qa/io_benchmark/io_benchmark /tmp/all.lib

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-02-19 Thread Nox
Really good to know. Maybe these results should be noted down somewhere (code or guidelines?) so this information is not lost over time. Am 19.02.2017 um 10:27 schrieb John Beard: On Sat, Feb 18, 2017 at 1:08 AM, Nox wrote: What about wxFFileInputStream instead of wxFileInputStream? wxFFil

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-03-01 Thread Wayne Stambaugh
On 2/19/2017 4:27 AM, John Beard wrote: > On Sat, Feb 18, 2017 at 1:08 AM, Nox wrote: >> What about wxFFileInputStream instead of wxFileInputStream? >> > > wxFFileInputStream appears to be about 5-6 times faster than > wxFileInputStream, but that's still much much slower than RICHIO or > std::ifs

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-03-01 Thread John Beard
Hi Wayne, Sorry for the delay, I've been a bit distracted by other things. The attached patch adds io_benchmark as a new target under tools. For now, it only includes LINE_READER input benchmarks. It currently supports: * Existing FILE_LINE_READERs * Raw std::ifstream reading (with getline, so n

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-03-02 Thread Wayne Stambaugh
Hey John, Does this patch get applied on top of the previous patch or does it replace the previous patch. Thanks, Wayne On 3/2/2017 12:31 AM, John Beard wrote: > Hi Wayne, > > Sorry for the delay, I've been a bit distracted by other things. The > attached patch adds io_benchmark as a new targe

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-03-02 Thread John Beard
Hi Wayne, On Thu, Mar 2, 2017 at 9:02 PM, Wayne Stambaugh wrote: > > Does this patch get applied on top of the previous patch or does it > replace the previous patch. Sorry, that wasn't very clear! It's a new patch, all rebased into one commit, since I messed with it quite a bit and the intermed

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-03-02 Thread Wayne Stambaugh
On 3/2/2017 8:13 AM, John Beard wrote: > Hi Wayne, > > On Thu, Mar 2, 2017 at 9:02 PM, Wayne Stambaugh wrote: >> >> Does this patch get applied on top of the previous patch or does it >> replace the previous patch. > > Sorry, that wasn't very clear! It's a new patch, all rebased into one > commi

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-03-02 Thread John Beard
On Thu, Mar 2, 2017 at 10:30 PM, Wayne Stambaugh wrote: > > One thing I overlooked is wxWidgets provides wxBufferedInputStream[1] > which takes a reference to a wxInputStream object as an argument. The > reason wxFileInputStream and wxFFileInputStream may be so slow is that > they are not buffere

Re: [Kicad-developers] RICHIO performance - 3 to 30 times slower than std::ifstream

2017-03-02 Thread Wayne Stambaugh
On 3/2/2017 11:35 AM, John Beard wrote: > On Thu, Mar 2, 2017 at 10:30 PM, Wayne Stambaugh wrote: >> >> One thing I overlooked is wxWidgets provides wxBufferedInputStream[1] >> which takes a reference to a wxInputStream object as an argument. The >> reason wxFileInputStream and wxFFileInputStream