Notmuch DB Problems

2018-09-05 Thread mueen
Hi, A few days ago I noticed notmuch new was no longer working (I have it as a cron job so it took a while to figure it out). It just freezes. I do have a Python hook, and it was freezing on the line that opens the database. I tried a notmuch dump. Same problem - freezes Based on some earlier

Re: Notmuch DB Problems

2018-09-05 Thread Jani Nikula
On Tue, 04 Sep 2018, mu...@nawaz.org wrote: > Hi, > > A few days ago I noticed notmuch new was no longer working (I have it > as a cron job so it took a while to figure it out). > > It just freezes. I do have a Python hook, and it was freezing on the > line that opens the database. > > I tried a no

Re: Notmuch DB Problems

2018-09-09 Thread Mueen Nawaz
Jani Nikula writes: > It might be interesting to see an strace log to possibly get an idea > where it gets stuck. > > Is the filesystem writable and working okay? > > If search and show work, I'm guessing it gets stuck in trying to open > the database writable. One hackish idea is to patch notmuc

Re: Notmuch DB Problems

2018-09-10 Thread David Bremner
Mueen Nawaz writes: > After a lot of poking around, I figured out the problem, and this may be > of interest to the developers (although not sure if it is a xapian issue > or a notmuch issue). > > Here's why it would freeze: > > I have a post-new hook that runs a Python script. Depending on whet

Re: Notmuch DB Problems

2018-09-10 Thread Mueen Nawaz
David Bremner writes: Here's why it would freeze: I have a post-new hook that runs a Python script. Depending on whether the new email it is processing matches a rule I have, it will fire off an email to the sender using the SMTP library in Python. I had recently upgraded my MTA (PostFix)

Re: Notmuch DB Problems

2018-09-10 Thread David Bremner
Mueen Nawaz writes: > > DATABASE = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE) OK. So your code is locking the database, and never unlocking it (because of the hang). So that part is at least not mysterious. > I can think of two experiments: I was thinking more along the lines of s

Re: Notmuch DB Problems

2018-09-10 Thread Mueen Nawaz
David Bremner writes: I can think of two experiments: I was thinking more along the lines of something that could be part of the notmuch test suite, i.e. run in a few seconds. Or at worst in 10 minutes or so to be usable to debug. I don't know if this can reliably be done. As I pointed ou

Re: Notmuch DB Problems

2018-09-10 Thread Olly Betts
On Mon, Sep 10, 2018 at 08:01:06AM -0300, David Bremner wrote: > Mueen Nawaz writes: > > Now killing all those jobs did not fix the database. It was still > > broken. And as we saw the second time round, it was /really/ broken - it > > would not even open in read-only mode. > > That seems like so

Re: Notmuch DB Problems

2018-09-10 Thread David Bremner
Olly Betts writes: > > You mean in Xapian? If so, a wishlist bug saying what you're hoping > for might help it happen. > > Cheers, > Olly I filed https://trac.xapian.org/ticket/769#ticket Maybe I'm overthinking it and I should just impliment some kind of loop around a try catch block to

Re: Notmuch DB Problems

2018-09-11 Thread Mueen Nawaz
Olly Betts writes: It doesn't make much sense to me that holding the lock alone could be causing any sort of corruption - that's just an fcntl() lock. I would suggest to make sure you're running Xapian 1.4.7 as that fixed a cursor handling bug which affected notmuch. I didn't find a way t