On Fri, Nov 18, 2016 at 10:18 AM, James K. Lowden <jklow...@schemamania.org> wrote:
> On Fri, 18 Nov 2016 08:55:11 -0800 > "Kevin O'Gorman" <kevinogorm...@gmail.com> wrote: > > > All of the python code is a single thread. The closest I come > > is a few times where I use subprocess.Popen to create what amounts to > > a pipeline, and one place where I start a number of copies of a C > > program in parallel, but each is a separate process with its own > > input and output files. These C programs have been in use for a > > number of months for earlier stages of this project, and I regard > > them as quite reliable. None of them uses threads, and they are > > mostly very simple filters. > > As you know, a process started with Popen cannot corrupt the Python > process's memory. If you're not doing anything to defeat the GIL, a > segfault inside the Python interpreter would be considered a bug. > > But is it happening in the interpreter, or in SQLite for that matter? > ISTM that's what you need to know. To know that, you're going to need > to run a debug version of the interpreter under gdb. When it faults, a > backtrace will tell you where. That's not definititive proof; memory > corruption is often detected far from where it was caused. But if the > fault is at a consistent place in SQLite code, for example, you can > use a hardware watchpoint to discover what's writing to it. > > I'm game to try that, but unsure how to get such a thing. If I have to build it, it's gonna take a while to assemble all the pieces. because it's gotta have all the parts I use, and be arranged so as not to interfere with normal use of python 3. Seems pretty error-prone itself, but as I said, I'm game. > I don't know what more to suggest. I would be surprised if you find a > fault in Python, in the Python standard library, or in SQLite. I'm > sure it won't be in anything on the other side of a popen call. Are > there non-standard libraries or Python modules in use that you haven't > mentioned? > > The most likely culprit in my mind is RAM. You're exercising new memory > pretty hard, running a bunch of processes at it at full tilt. Any > defect in the chips or DMA could explain what you're seeing. An easy > test, not necessarily cheap, would be to replace the RAM (or, if > possible, run with some removed). > > I have two war stories related to rotten I/O hardware, where the device > appeared to work for all intents and purposes, but was actually a > high-speed bit munger. Those were both over 20 years ago. It will > be interesting to hear if that turns out to be your issue. > > HTH. > > --jkl > > _______________________________________________ > sqlite-users mailing list > sqlite-users@mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > -- #define QUESTION ((bb) || (!bb)) /* Shakespeare */ _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
