FIXED Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Gijs Hillenius]

On 11 Jan 2016, Adam Sjøgren wrote:

> Gijs writes:
>
>> Is there some kind of debug flag I can switch on, to see what gnus &
>> bbdb are doing in fancy-split
>
> B t gives you a trace, B q show you where the email you are on will be
> split to.
>
> Otherwise it sounds like you need to go to edebug-defun...

using edebug was great, coz it shows my Gnus configuration works
fine. The last lines al return nil, which points to Gnus copying the
message succesfully and marking the original for deletion . So the
problem lay elsewhere.

I think I fixed it, in Cyrus, by running Squat and recreating an index
of my INBOX. I'm not sure that was it, but now the invisible' messages
are being removed.


___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Adam Sjøgren]

Gijs writes:

> Added them this morning, ran a few tests the past hours, but my messages
> continue to get split correctly, with erroneous copies remaining
> invisible in the INBOX.

Note that I don't use splitting with IMAP, so take what ever I say with
an extra grain of salt...


  O:-),

Adam

-- 
 "Tato zprava byla vytvorena automaticky a proto na niAdam Sjøgren
  prosim neodpovidejte." a...@koldfront.dk


___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Eric Abrahamsen]

Gijs Hillenius  writes:

> On  9 Jan 2016, Adam Sjøgren wrote:
>
>> [...trim your quotes, folks...]
>
> ok!
>
>
> [...]
>
>>> `bbdb/gnus-split-method'. The comments in bbdb-gnus indicate that
>>> you're supposed to do something like this:
>>>
>>> (setq nnmail-split-methods 'bbdb/gnus-split-method)
>>> (setq bbdb/gnus-split-nomatch-function nnimap-split-fancy)
>
> [...]
>
>>
>> I have, however, also these two bbdb-splitting related settings:
>>
>> (setq bbdb/gnus-split-default-group nil
>> bbdb/gnus-split-nomatch-function nil)
>
> I do/did not have latter in my .gnus.
>
> Added them this morning, ran a few tests the past hours, but my messages
> continue to get split correctly, with erroneous copies remaining
> invisible in the INBOX.
>
> I'll try the former, next.

Another option is: bbdb/gnus-nnimap-folder-list-from-bbdb. I used to use
that, and found it simpler and less mysterious. It just dumps a bunch of
splits wherever you use it. I think it's maybe less configurable, but I
found it sufficient, and also less confusing to use.


___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Gijs Hillenius]

On  9 Jan 2016, Adam Sjøgren wrote:

> [...trim your quotes, folks...]

ok!


[...]

>> `bbdb/gnus-split-method'. The comments in bbdb-gnus indicate that
>> you're supposed to do something like this:
>>
>> (setq nnmail-split-methods 'bbdb/gnus-split-method)
>> (setq bbdb/gnus-split-nomatch-function nnimap-split-fancy)

[...]

>
> I have, however, also these two bbdb-splitting related settings:
>
> (setq bbdb/gnus-split-default-group nil
> bbdb/gnus-split-nomatch-function nil)

I do/did not have latter in my .gnus.

Added them this morning, ran a few tests the past hours, but my messages
continue to get split correctly, with erroneous copies remaining
invisible in the INBOX.

I'll try the former, next.
___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Adam Sjøgren]

Eric writes:

> Right, but the basic point remains that bbdb/gnus-split-method will
> hijack the whole process unless you stop it -- though there are multiple
> ways of stopping it. Anyhow, Gijs should have a few options to play with
> now...

You're right, the "put it into fancy-split lambda"-thing can't stand alone.

One of the problems with a fossilised setup: I don't remember how things
are interconnected any more...


  Best regards,

Adam

-- 
 "Jeg kan godt lide den hvor Cobber starter!" Adam Sjøgren
 a...@koldfront.dk


___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Gijs Hillenius]

On 11 Jan 2016, Eric Abrahamsen wrote:


[...]

> Another option is: bbdb/gnus-nnimap-folder-list-from-bbdb. I used to
> use that, and found it simpler and less mysterious. It just dumps a
> bunch of splits wherever you use it. I think it's maybe less
> configurable, but I found it sufficient, and also less confusing to
> use.

Tried this, in vain, too.

FWIW, the section in my .gnus regarding split now looks like this:


(require 'bbdb-gnus) ;;required for bbdb/gnus-nnimap-folder-list-from-bbdb

(setq nnimap-split-rule  'nnimap-split-fancy
  nnimap-split-inbox "INBOX"
  nnimap-split-fancy
 `(|  
("From" "googlealerts.*" "INBOX.google-alerts")
(any lots of STUFF omitted
  "INBOX.tmp")
,@(bbdb/gnus-nnimap-folder-list-from-bbdb)
))

and tested this for 4 incoming emails. The gnus/BBDB splits correctly
but there are still invisible copies in the INBOX


___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Gijs Hillenius]

On 11 Jan 2016, Adam Sjøgren wrote:

> Gijs writes:
>
>> The gnus/BBDB splits correctly but there are still invisible copies
>> in the INBOX
>
> I don't know if it is relevant, but what value does nnmail-crosspost
> have?

setting nnmail-crosspost to nil has zilch effect, unfortunately
___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Gijs Hillenius]

Is there some kind of debug flag I can switch on, to see what gnus &
bbdb are doing in fancy-split
___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Adam Sjøgren]

Gijs writes:

> The gnus/BBDB splits correctly but there are still invisible copies in
> the INBOX

I don't know if it is relevant, but what value does nnmail-crosspost
have?

,[ C-h v nnmail-crosspost RET ]
| nnmail-crosspost is a variable defined in `nnmail.el'.
| Its value is nil
| Original value was t
| 
| Documentation:
| If non-nil, do crossposting if several split methods match the mail.
| If nil, the first match found will be used.
| 
| You can customize this variable.
`


  Best regards,

Adam

-- 
 "Remember the First Rule of Optimization: DON'T."Adam Sjøgren
 a...@koldfront.dk


___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Adam Sjøgren]

Gijs writes:

> Is there some kind of debug flag I can switch on, to see what gnus &
> bbdb are doing in fancy-split

B t gives you a trace, B q show you where the email you are on will be
split to.

Otherwise it sounds like you need to go to edebug-defun...


  Best regards,

Adam

-- 
 "Jeg har den regel, at jeg altid laver fuld install. Adam Sjøgren
  Dvs. _alle_ pakker."   a...@koldfront.dk


___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Eric Abrahamsen]

a...@koldfront.dk (Adam Sjøgren) writes:

> Eric writes:
>
> [...trim your quotes, folks...]
>
>> (setq nnimap-split-fancy
>>   '(| 
>>  ("From" "googlealerts.*" "INBOX.google-alerts")
>>  (any "c[...]" "INBOX.tmp")
>>  ;; Invoke the BBDB
>>  (: (lambda ()
>>   (car (bbdb/gnus-split-method
>>  ;; Default mailbox
>>  ("INBOX" "")))
>>
>> I'm not a splitting expert (I've stopped using it, and wasn't very good
>> at it to begin with), but there are a couple of things odd about this.
>> First, I don't think that's how you're supposed to use
>> `bbdb/gnus-split-method'. The comments in bbdb-gnus indicate that you're
>> supposed to do something like this:
>>
>> (setq nnmail-split-methods 'bbdb/gnus-split-method)
>> (setq bbdb/gnus-split-nomatch-function nnimap-split-fancy)
>
> You don't have to, I have been using the former way since days of yore.
>
> The point is that I want the rules before the call to bbdb/gnus-split-
> method to "overrule" bbdb.
>
> (E.g. I want commit messages to go to a commit-group, regardless of
> whether I have the person who did the commit configured in bbdb to be
> split into another group).
>
> I have, however, also these two bbdb-splitting related settings:
>
>   (setq bbdb/gnus-split-default-group nil
> bbdb/gnus-split-nomatch-function nil)
>
> I don't know if the first could be of relevance here - I guess not,
> because then Gijs' emails would have been going to mail.misc...
>
> (IMAP is complicated.)

Right, but the basic point remains that bbdb/gnus-split-method will
hijack the whole process unless you stop it -- though there are multiple
ways of stopping it. Anyhow, Gijs should have a few options to play with
now...


___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Adam Sjøgren]

Eric writes:

[...trim your quotes, folks...]

> (setq nnimap-split-fancy
>   '(| 
>   ("From" "googlealerts.*" "INBOX.google-alerts")
>   (any "c[...]" "INBOX.tmp")
>   ;; Invoke the BBDB
>   (: (lambda ()
>(car (bbdb/gnus-split-method
>   ;; Default mailbox
>   ("INBOX" "")))
>
> I'm not a splitting expert (I've stopped using it, and wasn't very good
> at it to begin with), but there are a couple of things odd about this.
> First, I don't think that's how you're supposed to use
> `bbdb/gnus-split-method'. The comments in bbdb-gnus indicate that you're
> supposed to do something like this:
>
> (setq nnmail-split-methods 'bbdb/gnus-split-method)
> (setq bbdb/gnus-split-nomatch-function nnimap-split-fancy)

You don't have to, I have been using the former way since days of yore.

The point is that I want the rules before the call to bbdb/gnus-split-
method to "overrule" bbdb.

(E.g. I want commit messages to go to a commit-group, regardless of
whether I have the person who did the commit configured in bbdb to be
split into another group).

I have, however, also these two bbdb-splitting related settings:

  (setq bbdb/gnus-split-default-group nil
bbdb/gnus-split-nomatch-function nil)

I don't know if the first could be of relevance here - I guess not,
because then Gijs' emails would have been going to mail.misc...

(IMAP is complicated.)


  Best regards,

Adam

-- 
 "Some people... some people like cupcakes exclusively,   Adam Sjøgren
  while I myself say there is naught nor ought there be  a...@koldfront.dk
  nothing so exalted on the face of God's grey earth as
  that prince of foods... The muffin!"


___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Gijs Hillenius]

On  8 Jan 2016, Eric Abrahamsen wrote:

> Gijs Hillenius  writes:
>
>> On  7 Jan 2016, Eric Abrahamsen wrote:
>>
>>> Gijs Hillenius  writes:
>>>
 [...] hope snipping this much is ok..

>>
>> If my Gnus (5.13, part of Debian's Emacs) copies messages back
>> into the INBOX, does fancy-split set the delete/expunge IMAP flag
>> to these?
>>
>> Because, if it does not flag them for deletion, would that
>> perhaps explain why the nightly Cyrus pruning does not do
>> anything? Cyrus is set to remove from disk all messages that have
>> been flagged for deletion for 3 days.
>
> No, no flags are set. From IMAP's point of view, splitting is
> simply a MOVE operation. Sorry, I misspoke earlier when I said the
> messages were *copied* back into the same group. They're actually
> MOVEd. That means there's only ever one copy of the message, but
> it gets a new UUID number each time.
>
> No new flags are set at all.

 So, these message come back int the INBOX, but are not shown
 anymore by Gnus. They are also not marked as deletable and that
 would explain why Cyrus does not remove them from disk, yes?
>>>
>>> Right. Once they are given the Seen flag, that flag sticks with them
>>> even if they are moved around (including "moved" into the same group
>>> they were already in). That Seen flag means Gnus won't show them to
>>> you again.
>>>
>>> I think the distinction here is between message deletion, and adding
>>> the "Deleted" flag. Deletion occurs as part of the move process (the
>>> message is in one instant removed from one group and added to
>>> another). If you're MOVEing to the same group, the message is
>>> deleted from that group, then added back to that group. But it keeps
>>> its old flags, and no other flags are put on it (there's nothing in
>>> the splitting process that would add a Deleted flag), so Cyrus
>>> doesn't see any reason why it should garbage-collect the message at
>>> the end of the day.
>>
>> So, we're saying the same thing, correct?
>>
>> What I don't understand is - and apologies for being long-winded:
>>
>> The way I use Gnus: when I start it, or hit g, new messages in the
>> INBOX are fancy-splitted, and moved to various IMAP folders. So I see
>> several INBOX.subfolders with a new, unread messages. These copies,
>> that are kept in the INBOX, as a user I don't know about these, I
>> don't see them in any IMAP mail client. Is that why 'root' sees 8k
>> messages in the INBOX when root happens to look at this users' INBOX
>> folder?
>>
>> How can Cyrus tell these message can safely be trashed? I guess this
>> is a question for a Cryus mailing list, where it not that this
>> "keeping a copy" does not happen for mail that is split on the server
>> by a Sieve script. So, perhaps this is a Gnus fancy-split bug?
>
> Okay hang on, we're not quite talking about the same thing here. If
> messages are split into _other_ groups, then everything should go fine
> -- they should leave the INBOX altogether (what's your value for
> nnimap-inbox on this server?) go to the new group, and Gnus should
> never try to split them again.

Aha! In my case, messages are copied rather than moved to the
INBOX.subfolder. So they end up in two folders: the INBOX (hidden, and
out of reach) and in the INBOX.subfolder.

That will surely be an error on my part. But which one & where..

> The situation I was talking about was when messages _aren't_ matched
> by any split methods, and they stay in the INBOX. In that case, they
> keep getting removed and copied back into the INBOX.

ok, I get this. These messages will always be visible in the mail
client. These messages are not what I'm trying to explain :-)

> If your messages are getting split into other groups, then things
> should work as expected. Remind of the problem again?
>
> Sorry about the confusion...

That is me, probably
___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Eric Abrahamsen]

Gijs Hillenius  writes:

> On  8 Jan 2016, Eric Abrahamsen wrote:
>
>> Gijs Hillenius  writes:
>>
>>> On  7 Jan 2016, Eric Abrahamsen wrote:
>>>
 Gijs Hillenius  writes:

> [...] hope snipping this much is ok..
>
>>>
>>> If my Gnus (5.13, part of Debian's Emacs) copies messages back
>>> into the INBOX, does fancy-split set the delete/expunge IMAP flag
>>> to these?
>>>
>>> Because, if it does not flag them for deletion, would that
>>> perhaps explain why the nightly Cyrus pruning does not do
>>> anything? Cyrus is set to remove from disk all messages that have
>>> been flagged for deletion for 3 days.
>>
>> No, no flags are set. From IMAP's point of view, splitting is
>> simply a MOVE operation. Sorry, I misspoke earlier when I said the
>> messages were *copied* back into the same group. They're actually
>> MOVEd. That means there's only ever one copy of the message, but
>> it gets a new UUID number each time.
>>
>> No new flags are set at all.
>
> So, these message come back int the INBOX, but are not shown
> anymore by Gnus. They are also not marked as deletable and that
> would explain why Cyrus does not remove them from disk, yes?

 Right. Once they are given the Seen flag, that flag sticks with them
 even if they are moved around (including "moved" into the same group
 they were already in). That Seen flag means Gnus won't show them to
 you again.

 I think the distinction here is between message deletion, and adding
 the "Deleted" flag. Deletion occurs as part of the move process (the
 message is in one instant removed from one group and added to
 another). If you're MOVEing to the same group, the message is
 deleted from that group, then added back to that group. But it keeps
 its old flags, and no other flags are put on it (there's nothing in
 the splitting process that would add a Deleted flag), so Cyrus
 doesn't see any reason why it should garbage-collect the message at
 the end of the day.
>>>
>>> So, we're saying the same thing, correct?
>>>
>>> What I don't understand is - and apologies for being long-winded:
>>>
>>> The way I use Gnus: when I start it, or hit g, new messages in the
>>> INBOX are fancy-splitted, and moved to various IMAP folders. So I see
>>> several INBOX.subfolders with a new, unread messages. These copies,
>>> that are kept in the INBOX, as a user I don't know about these, I
>>> don't see them in any IMAP mail client. Is that why 'root' sees 8k
>>> messages in the INBOX when root happens to look at this users' INBOX
>>> folder?
>>>
>>> How can Cyrus tell these message can safely be trashed? I guess this
>>> is a question for a Cryus mailing list, where it not that this
>>> "keeping a copy" does not happen for mail that is split on the server
>>> by a Sieve script. So, perhaps this is a Gnus fancy-split bug?
>>
>> Okay hang on, we're not quite talking about the same thing here. If
>> messages are split into _other_ groups, then everything should go fine
>> -- they should leave the INBOX altogether (what's your value for
>> nnimap-inbox on this server?) go to the new group, and Gnus should
>> never try to split them again.
>
> Aha! In my case, messages are copied rather than moved to the
> INBOX.subfolder. So they end up in two folders: the INBOX (hidden, and
> out of reach) and in the INBOX.subfolder.
>
> That will surely be an error on my part. But which one & where..

Okay, you did describe what was happening earlier in the thread, I just
got distracted. Sorry! So you mentioned your fancy split methods look
something like this:

(setq nnimap-split-fancy
  '(| 
("From" "googlealerts.*" "INBOX.google-alerts")
(any "c[...]" "INBOX.tmp")
;; Invoke the BBDB
(: (lambda ()
 (car (bbdb/gnus-split-method
;; Default mailbox
("INBOX" "")))

I'm not a splitting expert (I've stopped using it, and wasn't very good
at it to begin with), but there are a couple of things odd about this.
First, I don't think that's how you're supposed to use
`bbdb/gnus-split-method'. The comments in bbdb-gnus indicate that you're
supposed to do something like this:

(setq nnmail-split-methods 'bbdb/gnus-split-method)
(setq bbdb/gnus-split-nomatch-function nnimap-split-fancy)

Ie, the bbdb gets a chance to return a split first, and only if it
doesn't come up with anything does nnimap-split-fancy come into play. So
you'd set something like the above in your init, and then remove the
lambda from your fancy splits.

Second, ("INBOX" "") looks weird to me. The splitting examples I've seen
all just end in a plain box name: "INBOX".

Maybe try making those changes, and see if it improves?

Eric


___
info-gnus-english mailing list
info-gnus-english@gnu.org

Re: is this a bug in gnus v5.13? Re: gnus INBOX housekeeping, kept (copies of) mail lying around

[Eric Abrahamsen]

Gijs Hillenius  writes:

> On  7 Jan 2016, Eric Abrahamsen wrote:
>
>> Gijs Hillenius  writes:
>>
>>> [...] hope snipping this much is ok..
>>>
>
> If my Gnus (5.13, part of Debian's Emacs) copies messages back into
> the INBOX, does fancy-split set the delete/expunge IMAP flag to
> these?
>
> Because, if it does not flag them for deletion, would that perhaps
> explain why the nightly Cyrus pruning does not do anything? Cyrus
> is set to remove from disk all messages that have been flagged for
> deletion for 3 days.

 No, no flags are set. From IMAP's point of view, splitting is simply
 a MOVE operation. Sorry, I misspoke earlier when I said the messages
 were *copied* back into the same group. They're actually MOVEd. That
 means there's only ever one copy of the message, but it gets a new
 UUID number each time.

 No new flags are set at all.
>>>
>>> So, these message come back int the INBOX, but are not shown anymore
>>> by Gnus. They are also not marked as deletable and that would explain
>>> why Cyrus does not remove them from disk, yes?
>>
>> Right. Once they are given the Seen flag, that flag sticks with them
>> even if they are moved around (including "moved" into the same group
>> they were already in). That Seen flag means Gnus won't show them to
>> you again.
>>
>> I think the distinction here is between message deletion, and adding
>> the "Deleted" flag. Deletion occurs as part of the move process (the
>> message is in one instant removed from one group and added to
>> another). If you're MOVEing to the same group, the message is deleted
>> from that group, then added back to that group. But it keeps its old
>> flags, and no other flags are put on it (there's nothing in the
>> splitting process that would add a Deleted flag), so Cyrus doesn't see
>> any reason why it should garbage-collect the message at the end of the
>> day.
>
> So, we're saying the same thing, correct?
>
> What I don't understand is - and apologies for being long-winded:
>
> The way I use Gnus: when I start it, or hit g, new messages in the INBOX
> are fancy-splitted, and moved to various IMAP folders. So I see several
> INBOX.subfolders with a new, unread messages. These copies, that are
> kept in the INBOX, as a user I don't know about these, I don't see them
> in any IMAP mail client. Is that why 'root' sees 8k messages in the
> INBOX when root happens to look at this users' INBOX folder?
>
> How can Cyrus tell these message can safely be trashed? I guess this is
> a question for a Cryus mailing list, where it not that this "keeping a
> copy" does not happen for mail that is split on the server by a Sieve
> script. So, perhaps this is a Gnus fancy-split bug?

Okay hang on, we're not quite talking about the same thing here. If
messages are split into _other_ groups, then everything should go fine
-- they should leave the INBOX altogether (what's your value for
nnimap-inbox on this server?) go to the new group, and Gnus should never
try to split them again.

The situation I was talking about was when messages _aren't_ matched by
any split methods, and they stay in the INBOX. In that case, they keep
getting removed and copied back into the INBOX.

If your messages are getting split into other groups, then things should
work as expected. Remind of the problem again?

Sorry about the confusion...

Eric


___
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english