Re: lost track of X messages / socket error

2024-07-31 Thread Oswald Buddenhagen via isync-devel

On Mon, Oct 31, 2022 at 03:43:28PM +, Edd Barrett wrote:

On Fri, Oct 28, 2022 at 05:04:10PM +0200, Oswald Buddenhagen wrote:

i need the full log. in fact, of two consecutive runs, as this is an
inter-run problem. you can limit it on the command line to the affected
mailbox, and send me the log in private.



so, after not even two years after i received the logs (:'-D), i finally
analyzed them. posting for posterity.


Socket error on imap.gmail.com (142.251.5.109:993): timeout.


for unclear reasons, mbsync was pushing out the entire 25 meg message
within a second, while of course it takes a while to actually upload it
and get a response from the server. i suppose the problem was buffer
bloat on the network.
just raising Timeout to something generally considered unreasonable
would be a suitable workaround.


Warning: lost track of 104 pulled message(s)
Warning: lost track of 1 pushed message(s)

Searching the internet, lots of other people have enountered this, and
as far as I can see, the general advice is "don't worry about it".

Problem is, the message doesn't go away when you re-run mbsync.


that's because the sync never finished due to the timeout, and so the
journal was replayed each time and the (non-)problem reported again.
anyway, these messages are gone since commit 254d2be9.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: conflicting changes not going away

2024-07-26 Thread Oswald Buddenhagen via isync-devel

On Fri, Jul 26, 2024 at 09:12:54AM +0200, Alan Schmitt wrote:

Running using --verbose made me realize there is a significant
difference in the number of messages in the local and remote sides:

Opening far side box INBOX...
Opening near side box INBOX...
Loading far side box...
Loading near side box...
near side: 38764 messages, 0 recent
far side: 41361 messages, 0 recent

Should I be worried about this?


at first sight, this seems to be the usual harmless-ish case that
requires resetting MaxPulledUID in .mbsyncstate to zero.
as you're using master, it would be interesting to know whether any of
the messages got lost after the last big slew of commits (gee, that's
been two years ...).
or if you're using davmail. because if so, then .


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Message being deleted locally

2024-07-16 Thread Oswald Buddenhagen via isync-devel

On Tue, Jul 16, 2024 at 04:55:22PM +, Johan Parin wrote:

So are you saying this is an Exchange problem or a DavMail problem?


davmail. that's why i'm strongly suggesting that you try "native"
exchange imap (it's not really native; it's also just a translation
layer, but at least it's directly from ms, and it *did* improve (from
abysmal to just bad)).


I'm guessing I'm not the only user being forced to communicate with
Exchange so this must be a very common problem then..


indeed, i'm getting reports about it being more or less broken at a
disproportional rate.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Message being deleted locally

2024-07-16 Thread Oswald Buddenhagen via isync-devel

On Tue, Jul 16, 2024 at 01:44:50PM +, Johan Parin wrote:

That pulled around 2.5K messages. Now I can see on Outlook web my Inbox
has 19077 items. With notmuch I get after 'notmuch new':

 $ notmuch count folder:work/INBOX
 19074

I guess that is close enough.


well, that probably means that three messages were hidden from view
during that sync ...


So now I hope it can stay synced.


.. which means that this has a very low chance of being the case.


Again I will be satisifed with a model
where I never delete anything anywhere. I hope that if I set Expunge to
None mbsync will never delete anything?


it will still mark the messages as deleted, only physical deletion is
left to something else.
if you don't want to pull deletions, then you need to omit them from the
"Sync" set. but then ...


The only destructive operation I do is sometimes I archive parts of my
Inbox. [...]


.. this won't work.

there is no way to get it working as long as the imap server is
incapable of reliably telling which messages are actually in the
mailbox.

you can observe the network traffic with -Dn, but in all likelihood this
will just tell you that indeed the imap server is omitting some messages
from the FETCH results, without due cause.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Message being deleted locally

2024-07-16 Thread Oswald Buddenhagen via isync-devel

On Tue, Jul 16, 2024 at 05:10:11AM +, Johan Parin wrote:

So what I'm doing is I have copied the settings from Evolution
(including tenantID) from Linux users in the company (as Linux &
Evolution is a supported configuration in our IT policy), so I pretend
I'm using Evolution.


yes, that's par for the course, afaik.


How about

 Sync Pull


yes


 Expunge None


'Expunge Near' would probably fit your setup better, but check the
manual.


Any manipulataion of the .mbsyncstate files?


depending on what actually went wrong, that might be in order.
did you check whether the mails are still on the server? if so, then
resetting MaxPulledUID to zero would cause the messages to magically
re-appear (for the time being ...).


Remove .mbsynctstate?


not recommended in this case.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Message being deleted locally

2024-07-14 Thread Oswald Buddenhagen via isync-devel

On Sun, Jul 14, 2024 at 03:09:58PM +, Johan Parin via isync-devel wrote:

I have a setup at work where the company is using MS Exchange +
OAUTH2. I use DavMail to deal with OAUTH2 and then mbsync to connect to
DavMail. As MUA I use notmuch.

This was working OK for a while I think but then I started noticing
messages missing locally. [...]


there is nothing obviously wrong with your config, except that davmail
is known to be a total pita. even though exchange imap is also a PoS,
you still should have better luck with it if the server is configured to
allow it; how-tos regarding oauth2 can be found in the list archive (a
patch with a high-quality writeup and required scripts that can be
included upstream would be appreciated).

if you want to debug it (primarily for the purpose of making a good bug
report for davmail, i guess), add -D to mbsync's options and try to
catch it in the act of losing a mail.



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: mbsync skips certain messages

2024-07-10 Thread Oswald Buddenhagen via isync-devel

On Wed, Jul 10, 2024 at 11:34:59AM +, Thomas Bergheim wrote:

Everything works great, except for this bug.


isync just thinks it already synced everything it should have. the log
merely reflects a static state, so it doesn't help with figuring out how
this happened.

you can work around the state by hacking MaxPulledUID in the
mbsyncstate files; there are numerous related threads here.


And it happens often enough that I am wondering if there is any long
term solution to this problem?


it might be fixed in master, but that is buggy in other ways.



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: the opposite of MaxMessages

2024-07-09 Thread Oswald Buddenhagen via isync-devel

On Mon, Jul 08, 2024 at 09:48:27AM -0400, Paymon wrote:

On Mon, Jul 08, 2024 at 09:44:27AM GMT, Paymon wrote:

my case is the opposite of that^; i.e. i have a very small quota on
the server
and a fairly big amount incoming messages (mailing lists). i want to archive
the mailing list locally where i have comparatively unlimited storage but
expunge everything on the server so i never hit the quota. is this possible?



wait, ExpireSide, is that what i want here?


yes, it was added with specifically that use case in mind.



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Segmentation fault when syncing for the first time, if compiled with -O2/-O3

2024-07-07 Thread Oswald Buddenhagen via isync-devel

On Sun, Jul 07, 2024 at 02:28:54PM +0800, Runxi Yu wrote:

Could this be made into the next release?


yes, of course. but don't expect this to happen very soon (someone needs
to figure out what exactly is wrong with commit 859b7dd7).


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Segmentation fault when syncing for the first time, if compiled with -O2/-O3

2024-07-04 Thread Oswald Buddenhagen via isync-devel

On Thu, Jul 04, 2024 at 02:00:30PM +0800, runxiyu-ld/isync--- via isync-devel 
wrote:

Note that I only experience this error when ~/.local/share/mail/runxiyu
is empty. If I have a successful previous run, subsequent syncs work.

Running it causes a segfault.

[...]


thanks for the thorough report!

you could make it even better by using -g3 instead of -g (that should
get rid of the "" gunk) and isync's --debug instead of
--verbose. but never mind, it's good enough this time.

does the attached patch help?

From 14e432d890896d996b4c68576303329d66eb6b28 Mon Sep 17 00:00:00 2001
From: Oswald Buddenhagen 
Date: Thu, 4 Jul 2024 09:09:17 +0200
Subject: [PATCH] don't try to qsort() NULL array

while this actually works due to the array size being zero, it's
undefined behavior which makes gcc eliminate a subsequent null check in
the calling function.
---
 src/main_sync.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/main_sync.c b/src/main_sync.c
index 226e3249..fff944c3 100644
--- a/src/main_sync.c
+++ b/src/main_sync.c
@@ -186,7 +186,8 @@ filter_boxes( string_list_t *boxes, const char *prefix, string_list_t *patterns
 			boxarr[num] = NULL;
 		}
 	}
-	qsort( boxarr, num, sizeof(*boxarr), cmp_box_names );
+	if (boxarr)
+		qsort( boxarr, num, sizeof(*boxarr), cmp_box_names );
 	return boxarr;
 }

--
2.44.0.701.g2cf7baacf3.dirty

___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Socket error

2024-06-10 Thread Oswald Buddenhagen via isync-devel

On Mon, Jun 10, 2024 at 09:45:37AM +0200, Henrik Frisk wrote:

So sorry this dragged out.


well, _i_ am in no hurry ...


I would be so happy if you have the time to take
a look at these files attached which are the logs from mbsync and
DavMail.


somewhat guessing here, but this seems to be a genuine timeout due to
davmail just needing that long to load the 60k messages. try configuring
a longer timeout in mbsyncrc.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Issue syncing iCloud INBOX

2024-06-10 Thread Oswald Buddenhagen via isync-devel

On Sun, Jun 09, 2024 at 12:52:26PM -0600, Steve Mayer wrote:

Thanks Oswald! I’ve attached the output of ‘mbsync —debug
iCloud’.


that's not useful, as it is from an obviously stable state. you need to
"catch it in the act". if you can't repro it differently, start from
scratch, logging both the initial sync and the weird second sync.

also, please limit it to a single affected box (chan:INBOX or some such
on the command line).



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Issue syncing iCloud INBOX

2024-06-09 Thread Oswald Buddenhagen via isync-devel

On Sun, Jun 09, 2024 at 09:40:08AM -0600, Steve Mayer via isync-devel wrote:

Any ideas as to what might be going on?


not really. i suppose the server might be doing funny things.
re-run with -D and send me the log - maybe i can spot something.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Socket error

2024-05-16 Thread Oswald Buddenhagen via isync-devel

On Thu, May 16, 2024 at 12:18:36PM +0200, Henrik Frisk wrote:

Despite my best efforts, trying the things suggested here and a bunch
of
other stuff to do with the configuration of davmail and mbsync I can't get
this to work. I guess my main question to this list is if this is a davmail
problem or mbsync?


did you put davmail into verbose/debug mode and inspect the log?

you can try the same on the mbsync side, with -Dn.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Flags not syncing after moving email

2024-04-02 Thread Oswald Buddenhagen via isync-devel

On Mon, Apr 01, 2024 at 09:09:16PM +0200, Oswald Buddenhagen via isync-devel 
wrote:

[...] mail me the logs.


so with the logs at hand it really looks like the imap server is acting
up:

first run:
F: >>> 10 APPEND "Folders/Test2" (\Seen) {6229}
F: + Ready
F: * 1 EXISTS
F: * 1 RECENT
F: 10 OK [APPENDUID 36790870 1] APPEND

second run:
F: >>> 7 UID FETCH 1:1 (UID FLAGS)
F: * 1 FETCH (UID 1 FLAGS ())
F: 7 OK command completed in 243 microsec.

the \seen flag is obviously *just gone*. huh?

as an experiment, see whether the same happens with \flagged.

of course it would be a funny coincidence if both the proton mail bridge
and gmail were affected in the same way. or maybe the underlying
mechanism is actually different - you'd need to repeat the experiments
with gmail to tell.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Flags not syncing after moving email

2024-04-01 Thread Oswald Buddenhagen via isync-devel

On Mon, Apr 01, 2024 at 08:03:22PM +0200, laburnumT wrote:

When I look at my settings in Gmail it is set to Auto-Expunge off -
Wait
for the client to update the server.


that's weird, because the log shows somewhat clearly that a message is
expunged before an expunge or close command is issued.


I can also supply the log for a protonmail account where the same issue
exists.


well, at least that would avoid the confounder of gmail's virtual
folders.

to keep the amount of data manageable, yet useful, proceed like that:
isolate a test mail into a test folder (possibly via a server-side copy
using thunderbird - whatever is necessary to create the actual starting
state), and already create the target folder, but don't put anything
into it yet.
when you sync, specify the two involved folders. like chan:fol1,fol2.
sync it repeatedly, to make sure that the situation is stable. keep the
log of the last such "null run".
then do the move.
then sync repeatedly, until nothing is happening.
mail me the logs.



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Flags not syncing after moving email

2024-04-01 Thread Oswald Buddenhagen via isync-devel

On Mon, Apr 01, 2024 at 01:37:57PM +0200, laburnumT wrote:

Sorry that the file is in a 7z format. It was the first compression
format I had the could get the file below the size limit for sending to
this mailing list.


that's fine, though i guess i should have been more specific - when you
send an entire log, just mail it to me privately.

from a first look, you have auto-expunge activated in gmail's imap
settings. such server behavior is on the fringe of what the spec allows,
and mightily confuses isync.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Flags not syncing after moving email

2024-03-31 Thread Oswald Buddenhagen via isync-devel

On Sat, Mar 30, 2024 at 09:32:02PM +0100, laburnumT wrote:

I have looked at the logs if run with --debug-sync,


--debug-net would be likely more helpful in this case, but usually it's
actually better to just not limit the debug categories at all.


but couldn't figure out what I had done wrong.


possibly nothing. i don't see anything wrong in your config or
description. the server may be acting up, or maybe isync is even buggy.
we'll see.


But I can supply it if that would help.


yes, please.



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: mbsync skips certain messages

2024-03-23 Thread Oswald Buddenhagen via isync-devel

On Fri, Mar 22, 2024 at 01:09:58PM +0100, Henrik Frisk wrote:

Sorry for my lack of knowledge here, but based on what information do
I best identify the message in the debugging output ?


i presume that any message being skipped is a problem, so that's the
only thing to look for in the log. if it turns out that the message
contents matter, using -DN will probably help.

you are not expected to do that yourself. just send me the log along
with the config.



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: mbsync skips certain messages

2024-03-22 Thread Oswald Buddenhagen via isync-devel

On Fri, Mar 22, 2024 at 10:29:28AM +0100, Henrik Frisk wrote:

After a long time of working flawlessly my sync process now skips certain
messages. I have not been able to see how/if the skipped messages relate.


it's pretty hopeless to figure this out unless you can catch it in the
act with debugging enabled (-D option).

you can force it to re-attenpt messages it has already given up on by
hacking MaxPulledUID in the sync state file; several of the most recent
threads on this list had mentions of that.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: duplicates due to X-TUID headers

2024-03-15 Thread Oswald Buddenhagen via isync-devel

On Fri, Mar 15, 2024 at 07:42:32AM +0100, Peter P. wrote:

I re-checked and the duplicated messages are in fact lacking a
message-id
header. I don't know why and how they got removed.

Do you have any recommendataion how I could de-duplicate messages with
missing message-ids?


in that case you can follow that guide you found - eliminate the x-tuids
and then do full-text (?) de-duplication. i didn't investigate the topic
myself, so i have no more specific advice.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: duplicates due to X-TUID headers

2024-03-13 Thread Oswald Buddenhagen via isync-devel

On Wed, Mar 13, 2024 at 12:40:21PM +0100, Peter P. wrote:

These duplicates are not detected by (neo)mutt's ~= filter since some
of these messages contain a X-TUID header, others do not.


that's just wrong. mutt goes by the message-id header and nothing else.


I am interested how/why these headers got introduced, and how I can
avoid this in the future.


these are generated by isync to be able to provide a transactional
guarantee (being able to reliably resume interrupted runs).


Furthermore, some webpages suggest some sed magic[1] to remove these
X-TUID headers to be able to remove duplicates.


that page uses a different de-duplication method which presumably relies
on textual identity.


Will mbsync have any issues when these headers are removed, and
duplicates are removed on the near side?


you can strip them out from your maildir if you feel like it, but it
will just wear down your ssd for no discernable gain.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: hanging after LOGOUT completed

2024-03-04 Thread Oswald Buddenhagen via isync-devel

On Mon, Mar 04, 2024 at 05:03:04AM -0500, agarttha--- via isync-devel wrote:

mbsync 1.4.4 sporadically stopped exiting after mail retrieval. The
debug log (3) is below. Often mbsync will hang after OK LOGOUT.

Perhaps it's a server side change?  Or my mbsync configuration is
wrong after updating to 1.4.4?


that's likely a server bug; it's not closing the connection after the
logout handshake, contrary to the protocol specification. possibly there
is also a bad gateway on the route.

i repeatedly pondered adding a workaround for that, but so far the issue
was always solved by the providers.

however ... you should get a timeout after a while. if that doesn't
happen, we need to dig deeper.


I also tried adding Tunnel "openssl s_client -crlf -connect
imap.purelymail.com:993 -quiet -no_ign_eof" in my config but received
an error (1).


that -crlf is wrong.



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: How to synchronize all messages in a folder from the near to the far side?

2024-02-20 Thread Oswald Buddenhagen via isync-devel

On Tue, Feb 20, 2024 at 10:04:57AM +0100, Peter P. wrote:

* Peter P.  [2024-02-20 10:01]:

What is the best way to completely sync a far side folder to the near
side without keeping any previous near side messages?


depends on what you mean by that.
do you just want to get rid of the near-side copies, so you won't have
dupes when you re-sync?
or are you done with them and don't want to fetch them any more? that
sounds more like a job for fetchmail. or maybe isync's mail expiration
feature.


So either deleting the near side folder completely,


(including the state file.)
yes, starting from scratch always works fine.


or running mbsync --pull without removing the near side state file is
preferable(?).


selective syncing to avoid terminally destructive actions always bears
the risk that you accidentally run a full sync.



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: How to synchronize all messages in a folder from the near to the far side?

2024-02-20 Thread Oswald Buddenhagen via isync-devel

On Mon, Feb 19, 2024 at 07:07:07PM +0100, Peter P. wrote:

Socket error: secure read from -whatever- : error:0A000126:SSL
routines::unexpected eof while reading


that's just generic flakiness of the connection. don't worry.


and on the next run I do get:
Warning: lost track of XXX pushed message(s).


that's ignorable, too. i already removed it in master.


Does this message mean I will end up with duplicates on the far side as
well and will I have to clear the far side every time a full sync does
not go through due to errors?


no.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: How to synchronize all messages in a folder from the near to the far side?

2024-02-19 Thread Oswald Buddenhagen via isync-devel

On Mon, Feb 19, 2024 at 06:07:18PM +0100, Peter P. wrote:

* Oswald Buddenhagen via isync-devel
 [2024-02-19 18:03]:

On Mon, Feb 19, 2024 at 05:09:43PM +0100, Peter P. wrote:
> And no messages get synced to the far side.
>
> In an earlier attempt with the same command, only a few messages were
> copied to the far side:
>
it looks very much like you still have an old state file lying around

Do you mean .mbsyncstate on the near side? So I would try to remove that
file?


yes. or maybe you have one in ~/.mbsync or ~/.local/local/state/isync,
depending on the box' SyncState config.

remember to clear the far side again after removing the state file, or
you'll get duplicates for the few new mails that did get synced.



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: How to synchronize all messages in a folder from the near to the far side?

2024-02-19 Thread Oswald Buddenhagen via isync-devel

On Mon, Feb 19, 2024 at 05:09:43PM +0100, Peter P. wrote:

And no messages get synced to the far side.

In an earlier attempt with the same command, only a few messages were
copied to the far side:


it looks very much like you still have an old state file lying around
(and you got very lucky by using --push, as otherwise you'd have nuked
your local mails by pulling in the deletions).


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: filter duplicates?

2024-02-04 Thread Oswald Buddenhagen via isync-devel

On Sun, Feb 04, 2024 at 04:03:40PM +0100, Peter P. wrote:
So if I am unlucky and mutt on computer1 deletes MessageA as a 
duplicate

of MessageB...
and
mutt on computer2 deletes messageB as a duplicate of messageA
I might end up with both messageA and messageB being synced back up to
the imap server?

if by "synced back up" you mean "all copies are gone", then yes. because 
that's what will happen as a logical consequence of deletion 
propagation.


i strongly recommend to run de-duplication only in one place at a time.
if you can control delivery on the server, you can use formail -D 
(possibly in combination with procmail).
otherwise, a cron job on one of the clients to auto-purge dupes might be 
the best solution.


if you never have unsynced states on two machines at the same time, then 
there is no problem to solve to start with. this is the state you should 
aim for anyway (to avoid seen the same mails as new), but might be a 
tall call when it comes to mails being deleted.



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Mbsync does not sync INBOX emails only

2024-01-23 Thread Oswald Buddenhagen via isync-devel

On Tue, Jan 23, 2024 at 05:05:00PM +, jos...@breatheoutbreathe.in wrote:
I haven't done the nuclear option of deleting INBOX, and I'm not sure 
what this means in practice:



hacking MaxPulledUID in .mbsyncstate would probably do.



that should become clear after some careful reading of the manual.
you'll need a text editor ...

Before I implement any workarounds, may I provide you with any other 
information to help debug the issue?


probably not. the faulty state is fully understood (assuming my 
speculation is correct), and there is little you can reasonably do to 
reproduce the problem. you can only fix up the mess.


this issue has cropped up multiple times in the past, so you'll find 
more explanations of the problem and workaround in the archive.


it might even be that i fixed it in master, but that's not in a 
releasable state (in the archive you'll find information about _that_ as 
well ... :})




___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Mbsync does not sync INBOX emails only

2024-01-23 Thread Oswald Buddenhagen via isync-devel

On Sun, Jan 21, 2024 at 09:30:19AM -0800, Joseph Turner via isync-devel wrote:

I am experiencing the same issue as Oswald.


you mean mark.

anyway, the way i read his mail was that inbox isn't synced at all, 
rather than that just some mails are missing. but in retrospect, that's 
not necessarily the case.


your case looks like what jason reported two weeks ago.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Mbsync does not sync INBOX emails only

2024-01-21 Thread Oswald Buddenhagen via isync-devel

On Sat, Jan 20, 2024 at 05:27:33PM +, Mark Johnno via isync-devel wrote:

$ mbsync -v
isync 1.3.0


that's a tad old. i think i fixed a related bug in 1.4.x.

other than that, the interesting part is what the listing returns for 
the M side. INBOX always exists, but may be hidden by the namespace.

`mbsync -l -a` should give a clue.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Socket Error, Possible protocol violation

2024-01-10 Thread Oswald Buddenhagen via isync-devel

On Wed, Jan 10, 2024 at 09:59:07AM +0100, Bence Ferdinandy wrote:

On Thu Aug 17, 2023 at 14:45, Oswald Buddenhagen  
wrote:

On Thu, Aug 17, 2023 at 07:59:09AM -0400, brittanderson--- via isync-devel 
wrote:
>The error has reoccured. It is:
>
>Socket error: receive buffer full. Probably protocol error.


I also ran into this error with davmail.


i don't think that's necessary. it "just" requires me looking really
hard at the code. the culprit must be one of the last commits to
socket.c (git-bisect'ing it would not hurt). if someone else wants to
take a shot at analyzing it, that would be most welcome.


I started bisecting,

based on what i said, you could pass `src/socket.c` to bisect to 
significantly cut down the iterations.

anyway, as i said in another thread:

you can try reverting 859b7dd7f and later ac3b5186b.

which will probably get you right to the culprit.


but it's a pain due to the config changes between the last release:

it's backwards-compatible; you just get some deprecation notices with 
the newer version. so just use the old config.



I'm getting "Near side box X cannot be opened" on the latest release. I deleted
.local/state/isync/, I cleared ~/.mbsync, I also cleared everything from Path.
What am I not clearing?

this doesn't look like a data incompatibility, so clearing won't do 
anything useful.
try running with -D to see whether it's doing something obviously 
stupid. the next level would be strace'ing it.
anyway, my first guess would be that the config is missing a Create 
statement, which wasn't important until you deleted the local state.



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Why are messages sometimes not pulled from server?

2024-01-05 Thread Oswald Buddenhagen via isync-devel

On Fri, Jan 05, 2024 at 09:04:26AM -0500, Jason Cox wrote:

On Fri Jan 5, 2024 at 5:10 AM EST, Oswald Buddenhagen via isync-devel wrote:

hacking MaxPulledUID in .mbsyncstate would probably do.


I assume I would just decrement it by the number of messages that aren't being 
downloaded?

no. this is an index, not a count. you'd need to look at the synced 
pairs further down, left side.



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


Re: Why are messages sometimes not pulled from server?

2024-01-05 Thread Oswald Buddenhagen via isync-devel

On Thu, Jan 04, 2024 at 08:50:17PM -0500, Jason Cox via isync-devel wrote:
Eventually I just deleted the inbox and did a fresh sync to get the 
missing messages. Is there a better way to resolve this issue?



hacking MaxPulledUID in .mbsyncstate would probably do.
as to why the situation arised in the first place, it's certainly a bug.  
i dimly remember that i fixed some related issues in master (but can't 
release it due to regressions).



___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/gpl-exception' created

2022-09-03 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/gpl-exception' has been created at 28a4636.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: document defaults of referenced XDG env vars

2022-07-09 Thread Oswald Buddenhagen via isync-devel
commit 65cd4429bb04fafadaed702cff342b7a96678fd0
Author: Oswald Buddenhagen 
Date:   Sat Jul 9 08:26:01 2022 +0200

document defaults of referenced XDG env vars

so users don't have to google the spec.

 src/mbsync.1 | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/src/mbsync.1 b/src/mbsync.1
index 7157048..939c8c5 100644
--- a/src/mbsync.1
+++ b/src/mbsync.1
@@ -36,6 +36,7 @@ Multiple replicas of each mailbox can be maintained.
 Read configuration from \fIfile\fR.
 By default, the configuration is read from $XDG_CONFIG_HOME/isyncrc, and
 if that does not exist, ~/.mbsyncrc is tried in turn.
+$XDG_CONFIG_HOME defaults to ~/.config if not set.
 .TP
 \fB-a\fR, \fB--all\fR
 Select all configured Channels. Any Channel/Group specifications on the
@@ -732,7 +733,8 @@ the appended string is made up according to the pattern
 (see also \fBFieldDelimiter\fR below).
 .br
 (Global default: \fI$XDG_STATE_HOME/isync/\fR, with a fallback to
-\fI~/.mbsync/\fR if only that exists)
+\fI~/.mbsync/\fR if only that exists.
+$XDG_STATE_HOME defaults to ~/.local/state if not set.)
 .
 .SS Groups
 .TP
@@ -858,11 +860,11 @@ There is no risk as long as the IMAP mailbox is accessed 
by only one client
 .
 .SH FILES
 .TP
-.B $XDG_CONFIG_HOME/isyncrc
+\fB$XDG_CONFIG_HOME/isyncrc\fR (usually \fB~/.config/isyncrc\fR)
 Default configuration file.
 See also the example file in the documentation directory.
 .TP
-.B $XDG_STATE_HOME/isync/
+\fB$XDG_STATE_HOME/isync/\fR (usually \fB~/.local/state/isync/\fR)
 Directory containing synchronization state files.
 .TP
 .B ~/.mbsyncrc


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: remove duplicate contributor entry

2022-07-05 Thread Oswald Buddenhagen via isync-devel
commit 8648d7a47965f4b9fd185247a3638746cada631e
Author: Oswald Buddenhagen 
Date:   Tue Jul 5 11:40:54 2022 +0200

remove duplicate contributor entry

anton khirnov is already in the main section, no need to have him in the
honorary section as well.

on a completely unrelated note, for posterity: andreas grapentin's
mention is due to commit cf13630a, where i forgot to credit him for the
initial version of the patch.

 AUTHORS | 1 -
 1 file changed, 1 deletion(-)

diff --git a/AUTHORS b/AUTHORS
index 8dc33ae..01af903 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -70,7 +70,6 @@ Honorary Contributors
 
 Alessandro Ghedini 
 Andreas Grapentin 
-Anton Khirnov 
 Aurélien Francillon 
 Ben Kibbey 
 Caspar Schutijser 


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: make DNS lookup asynchronous

2022-07-04 Thread Oswald Buddenhagen via isync-devel
commit ec50c55c36887b86b0143a265acae4b22d117fe9
Author: Oswald Buddenhagen 
Date:   Mon Jul 4 12:50:17 2022 +0200

make DNS lookup asynchronous

true asynchronicity is actually fairly useless, as it's unlikely that
both Stores in a Channel use IMAP, and both host resolutions take
particularly long - the main objective is imposing the Timeout setting.
however, we can't just use setjmp()+alarm(), as longjmp()ing out of
getaddrinfo() is undefined, as it may for example free() just at the
wrong time. so we go for the real thing.

this implementation just fork()s out a process which uses getaddrinfo()
(or gethostbyname()) per lookup. this isn't particularly scalable, but
as we don't expect a lot of lookups, it seems adequate.

 src/socket.c | 281 +++
 src/socket.h |   6 +-
 2 files changed, 195 insertions(+), 92 deletions(-)

diff --git a/src/socket.c b/src/socket.c
index 8f41295..52cd7c2 100644
--- a/src/socket.c
+++ b/src/socket.c
@@ -28,6 +28,7 @@
 #endif
 
 enum {
+   SCK_RESOLVING,
SCK_CONNECTING,
 #ifdef HAVE_LIBSSL
SCK_STARTTLS,
@@ -415,6 +416,7 @@ static void socket_fd_cb( int, void * );
 static void socket_fake_cb( void * );
 static void socket_timeout_cb( void * );
 
+static void socket_resolve( conn_t * );
 static void socket_connect_one( conn_t * );
 static void socket_connect_next( conn_t * );
 static void socket_connect_failed( conn_t * );
@@ -422,15 +424,21 @@ static void socket_connected( conn_t * );
 static void socket_connect_bail( conn_t * );
 
 static void
-socket_open_internal( conn_t *sock, int fd )
+socket_register_internal( conn_t *sock, int fd )
 {
sock->fd = fd;
-   fcntl( fd, F_SETFL, O_NONBLOCK );
init_notifier( >notify, fd, socket_fd_cb, sock );
init_wakeup( >fd_fake, socket_fake_cb, sock );
init_wakeup( >fd_timeout, socket_timeout_cb, sock );
 }
 
+static void
+socket_open_internal( conn_t *sock, int fd )
+{
+   fcntl( fd, F_SETFL, O_NONBLOCK );
+   socket_register_internal( sock, fd );
+}
+
 static void
 socket_close_internal( conn_t *sock )
 {
@@ -441,32 +449,6 @@ socket_close_internal( conn_t *sock )
sock->fd = -1;
 }
 
-#ifndef HAVE_IPV6
-struct addr_info {
-   struct addr_info *ai_next;
-   struct sockaddr_in ai_addr[1];
-};
-
-#define freeaddrinfo(ai) free( ai )
-
-static struct addr_info *
-init_addrinfo( struct hostent *he )
-{
-   uint naddr = 0;
-   for (char **addr = he->h_addr_list; *addr; addr++)
-   naddr++;
-   struct addr_info *caddr = nfzalloc( naddr * sizeof(struct addrinfo) );
-   struct addr_info *ret, **caddrp = 
-   for (char **addr = he->h_addr_list; *addr; addr++, caddr++) {
-   caddr->ai_addr->sin_family = AF_INET;
-   memcpy( >ai_addr->sin_addr.s_addr, *addr, sizeof(struct 
in_addr) );
-   *caddrp = caddr;
-   caddrp = >ai_next;
-   }
-   return ret;
-}
-#endif
-
 void
 socket_connect( conn_t *sock, void (*cb)( int ok, void *aux ) )
 {
@@ -501,77 +483,202 @@ socket_connect( conn_t *sock, void (*cb)( int ok, void 
*aux ) )
info( "\vok\n" );
socket_connected( sock );
} else {
+   socket_resolve( sock );
+   }
+}
+
+static void
+pipe_write( int fd, void *buf, int len )
+{
+   do {
+   int wrote = write( fd, buf, len );
+   if (wrote < 0) {
+   perror( "write" );
+   _exit( 1 );
+   }
+   buf = ((char *)buf) + wrote;
+   len -= wrote;
+   } while (len);
+}
+
+static void
+socket_resolve( conn_t *sock )
+{
+   info( "Resolving %s...\n", sock->conf->host );
+
+   int pfd[2];
+   if (pipe( pfd )) {
+   perror( "pipe" );
+   exit( 1 );
+   }
+   switch (fork()) {
+   case -1:
+   perror( "fork" );
+   exit( 1 );
+   case 0:
+   break;
+   default:
+   close( pfd[1] );
+   socket_register_internal( sock, pfd[0] );
+   sock->state = SCK_RESOLVING;
+   conf_notifier( >notify, 0, POLLIN );
+   socket_expect_activity( sock, 1 );
+   return;
+   }
+
 #ifdef HAVE_IPV6
-   int gaierr;
-   struct addrinfo hints;
-
-   memset( , 0, sizeof(hints) );
-   hints.ai_family = AF_UNSPEC;
-   hints.ai_socktype = SOCK_STREAM;
-   hints.ai_flags = AI_ADDRCONFIG;
-   infon( "Resolving %s... ", conf->host );
-   if ((gaierr = getaddrinfo( conf->host, NULL, , 
>addrs ))) {
-   error( "Error: Cannot resolve server '%s': %s\n", 
conf->host, gai_strerror( gaierr ) );
-   socket_connect_bail( sock );
-   return;
+   struct addrinfo *res, hints = { 

[commit] master: don't refresh progress counters pointlessly

2022-07-04 Thread Oswald Buddenhagen via isync-devel
commit b9a4746b54cd4159941e4bb4650d75cb26a84a25
Author: Oswald Buddenhagen 
Date:   Mon Jul 4 18:03:24 2022 +0200

don't refresh progress counters pointlessly

the mainloop-based refresh timer keeps spinning even if there is no
update. overload stats_steps to signal whether a refresh is needed.

amends 8fbc4323.

 src/main_sync.c | 10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/src/main_sync.c b/src/main_sync.c
index 88baae2..24a88c9 100644
--- a/src/main_sync.c
+++ b/src/main_sync.c
@@ -43,9 +43,11 @@ print_stats( void )
 static void
 stats_timeout( void *aux ATTR_UNUSED )
 {
-   stats_steps = -1;
+   if (stats_steps != -1) {
+   stats_steps = -1;
+   print_stats();
+   }
conf_wakeup( _wakeup, 200 );
-   print_stats();
 }
 
 void
@@ -55,8 +57,10 @@ stats( void )
return;
 
// If the main loop appears to be running, skip the sync path.
-   if (stats_steps < 0)
+   if (stats_steps < 0) {
+   stats_steps = -2;
return;
+   }
 
// Rate-limit the (somewhat) expensive timer queries.
if (++stats_steps < 10)


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: fix Tunnel leaving behind zombies

2022-07-04 Thread Oswald Buddenhagen via isync-devel
commit ced20ad0d962a8809f53cbd214313cad13ad5fab
Author: Oswald Buddenhagen 
Date:   Sun Jul 3 11:52:20 2022 +0200

fix Tunnel leaving behind zombies

this generally went unnoticed, as the tunnel usually terminates right
before we exit anyway. however, if multiple Channels are synced, it may
become visible.

this is a "shotgun" implementation, where the main loop just reaps all
unclaimed children.
arguably, it would be cleaner if each socket actually tracked its own
process, but getting synchronous kills+waits right is tricky, so we
continue to pretend that there is no process as far as the socket layer
is concerned.

poll()/select() are not restartable, so they need EINTR handling now
that SIGCHLD is actually delivered.

 src/main.c  | 40 
 src/main_list.c |  2 +-
 src/main_p.h|  1 +
 src/main_sync.c |  2 +-
 src/util.c  |  4 
 5 files changed, 47 insertions(+), 2 deletions(-)

diff --git a/src/main.c b/src/main.c
index 993a176..8581b0c 100644
--- a/src/main.c
+++ b/src/main.c
@@ -89,6 +89,26 @@ PACKAGE " " VERSION " - mailbox synchronizer\n"
exit( code );
 }
 
+static int child_pipe[2];
+static notifier_t child_notifier;
+
+static void
+childHandler( int n ATTR_UNUSED )
+{
+   // We can't just reap everything here, as we might steal children
+   // from popen(). Let the main loop handle it synchronously instead.
+   char dummy = 0;
+   write( child_pipe[1], , 1 );
+}
+
+static void
+childReaper( int events ATTR_UNUSED, void *aux ATTR_UNUSED )
+{
+   char dummy;
+   while (read( child_pipe[0], , 1 ) == 1) {}
+   while (waitpid( -1, NULL, WNOHANG ) > 0) {}
+}
+
 #ifdef __linux__
 static void ATTR_NORETURN
 crashHandler( int n )
@@ -543,9 +563,29 @@ main( int argc, char **argv )
 
signal( SIGPIPE, SIG_IGN );
 
+   if (pipe( child_pipe )) {
+   perror( "pipe" );
+   return 1;
+   }
+   fcntl( child_pipe[0], F_SETFL, O_NONBLOCK );
+   fcntl( child_pipe[1], F_SETFL, O_NONBLOCK );
+   init_notifier( _notifier, child_pipe[0], childReaper, NULL );
+   conf_notifier( _notifier, 0, POLLIN );
+   struct sigaction sa = { 0 };
+   sa.sa_handler = childHandler;
+   sa.sa_flags = SA_NOCLDSTOP | SA_RESTART;
+   sigaction( SIGCHLD, , NULL );
+
if (mvars->list_stores)
list_stores( mvars, argv + oind );
else
sync_chans( mvars, argv + oind );
return mvars->ret;
 }
+
+void
+cleanup_mainloop( void )
+{
+   cleanup_drivers();
+   wipe_notifier( _notifier );
+}
diff --git a/src/main_list.c b/src/main_list.c
index 48bf4dd..c4aa63e 100644
--- a/src/main_list.c
+++ b/src/main_list.c
@@ -126,7 +126,7 @@ do_list_stores( list_vars_t *lvars )
  next:
advance_store( lvars );
}
-   cleanup_drivers();
+   cleanup_mainloop();
 }
 
 static void
diff --git a/src/main_p.h b/src/main_p.h
index 75f619c..6aa5fe2 100644
--- a/src/main_p.h
+++ b/src/main_p.h
@@ -21,5 +21,6 @@ typedef struct {
 
 void sync_chans( core_vars_t *cvars, char **argv );
 void list_stores( core_vars_t *cvars, char **argv );
+void cleanup_mainloop( void );
 
 #endif
diff --git a/src/main_sync.c b/src/main_sync.c
index 24a88c9..226e324 100644
--- a/src/main_sync.c
+++ b/src/main_sync.c
@@ -496,7 +496,7 @@ do_sync_chans( main_vars_t *mvars )
  next:
advance_chan( mvars );
}
-   cleanup_drivers();
+   cleanup_mainloop();
if (!mvars->cvars->list && (DFlags & PROGRESS))
wipe_wakeup( _wakeup );
 }
diff --git a/src/util.c b/src/util.c
index 3ba7f91..e697cfb 100644
--- a/src/util.c
+++ b/src/util.c
@@ -1110,6 +1110,8 @@ event_wait( void )
case 0:
return;
case -1:
+   if (errno == EINTR)
+   return;
perror( "poll() failed in event loop" );
abort();
default:
@@ -1162,6 +1164,8 @@ event_wait( void )
case 0:
return;
case -1:
+   if (errno == EINTR)
+   return;
perror( "select() failed in event loop" );
abort();
default:


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: fix broken Tunnel potentially causing SIGPIPE

2022-07-04 Thread Oswald Buddenhagen via isync-devel
commit b84137482765c90ce0578b56e9fe322474bf4771
Author: Oswald Buddenhagen 
Date:   Sun Jul 3 11:40:42 2022 +0200

fix broken Tunnel potentially causing SIGPIPE

we need to ignore the signal, so the regular error handling can kick in.

 src/main.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/main.c b/src/main.c
index 82da1ea..993a176 100644
--- a/src/main.c
+++ b/src/main.c
@@ -541,6 +541,8 @@ main( int argc, char **argv )
if (load_config( config ))
return 1;
 
+   signal( SIGPIPE, SIG_IGN );
+
if (mvars->list_stores)
list_stores( mvars, argv + oind );
else


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: abort channel sync when a store is fubar

2022-07-01 Thread Oswald Buddenhagen via isync-devel
commit 460bfbb8ac1eb3cb106bd816566f93ff66d2190f
Author: Oswald Buddenhagen 
Date:   Fri Jul 1 14:54:24 2022 +0200

abort channel sync when a store is fubar

this got lost in d5a5da947.

this also simplifies a nested condition, where the logic has previously
been, but isn't applicable any more.

REFMAIL: 87fsjloz05@wavexx.thregr.org

 src/main_sync.c | 12 +---
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/src/main_sync.c b/src/main_sync.c
index 9c38d20..88baae2 100644
--- a/src/main_sync.c
+++ b/src/main_sync.c
@@ -668,7 +668,7 @@ static void
 do_sync_boxes( main_vars_t *mvars )
 {
mvars->box_cben = 0;
-   for (;;) {
+   while (mvars->state[F] == ST_OPEN && mvars->state[N] == ST_OPEN) {
if (mvars->chanptr->boxlist) {
box_ent_t *mbox = mvars->boxptr;
if (!mbox)
@@ -730,12 +730,10 @@ done_sync( int sts, void *aux )
stats();
if (sts) {
mvars->cvars->ret = 1;
-   if (sts & (SYNC_BAD(F) | SYNC_BAD(N))) {
-   if (sts & SYNC_BAD(F))
-   mvars->state[F] = ST_CLOSED;
-   if (sts & SYNC_BAD(N))
-   mvars->state[N] = ST_CLOSED;
-   }
+   if (sts & SYNC_BAD(F))
+   mvars->state[F] = ST_CLOSED;
+   if (sts & SYNC_BAD(N))
+   mvars->state[N] = ST_CLOSED;
}
mvars->box_done = 1;
if (mvars->box_cben)


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: fix lineup of --ext-exit in help output

2022-07-01 Thread Oswald Buddenhagen via isync-devel
commit f6ccf9c4f5f343e4a1f457d3432134e7c61beb4a
Author: Oswald Buddenhagen 
Date:   Fri Jul 1 12:14:14 2022 +0200

fix lineup of --ext-exit in help output

 src/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/main.c b/src/main.c
index 4607f5d..82da1ea 100644
--- a/src/main.c
+++ b/src/main.c
@@ -47,7 +47,7 @@ PACKAGE " " VERSION " - mailbox synchronizer\n"
 "  -c, --config CONFIG read an alternate config file (default: ~/." EXE "rc)\n"
 "  -D, --debug debugging modes (see manual)\n"
 "  -y, --dry-run   do not actually modify anything\n"
-"  -e, --ext-exit  return extended exit code\n"
+"  -e, --ext-exit  return extended exit code\n"
 "  -V, --verbose   display what is happening\n"
 "  -q, --quiet don't display progress counters\n"
 "  -v, --version   display version\n"


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: improve wording of --ext-exit docu

2022-07-01 Thread Oswald Buddenhagen via isync-devel
commit 92faccc639aa9c8776dde1e2f32466454a2a89b0
Author: Oswald Buddenhagen 
Date:   Fri Jul 1 13:00:01 2022 +0200

improve wording of --ext-exit docu

 src/mbsync.1 | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/src/mbsync.1 b/src/mbsync.1
index 8a8b01f..7157048 100644
--- a/src/mbsync.1
+++ b/src/mbsync.1
@@ -80,8 +80,10 @@ operations are determined, but no modifications are actually 
made
 to either the mailboxes or the state files.
 .TP
 \fB-e\fR, \fB--ext-exit\fR
-Return an extended exit code: Add 32 resp. 64 to the code if any
-modifications were made on the far resp. near side.
+Return an extended exit code: Add 32 or 64 to the code if any
+modifications were made on the far or near side, respectively; these
+are not mutually exclusive, so the code may be 96 if changes were both
+pushed and pulled.
 An error may be reported at the same time, so the code may be for example
 65 if some changes were successfully pulled, while others failed.
 .TP


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: try to make AUTHORS reflect reality

2022-06-26 Thread Oswald Buddenhagen via isync-devel
commit 16ecde504dd1195afcef5af88f5886ad9eceadee
Author: Oswald Buddenhagen 
Date:   Sun Jun 26 12:21:58 2022 +0200

try to make AUTHORS reflect reality

make a complete list of contributors who hold copyright (and those who
don't).

the case of ted is particularly interesting - he recently disclaimed
significant contributions, but it turns out that this isn't true:
the rewrite in 130664b6 incorporated significant portions of his async
IMAP patch for debian.

speaking of debian, i deleted ted's and nicolas' mention as debian
maintainers, as debian/ is basically just a (poorly maintained) mirror,
and has an own copyright file.

 AUTHORS | 105 ++--
 1 file changed, 95 insertions(+), 10 deletions(-)

diff --git a/AUTHORS b/AUTHORS
index 1b973ce..8dc33ae 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -1,17 +1,102 @@
+Contact
+===
+
+Send questions and bug reports to the isync-devel@lists.sourceforge.net
+mailing list.
+
+Do _NOT_ report bugs to Michael, not even in a CC: - he is not actively
+involved in isync development any more.
+
+Lead Developers
+===
+
 Oswald Buddenhagen 
-   * Contributor, current maintainer
+- Current maintainer
+
+Michael Elkins 
+- Original author
+
+Contributors
+
+
+(Some of these people also contributed bugfixes and optimizations.)
+(In chronological order.)
+
+Jeremy Katz 
+- UseNamespace & UseSSL* options
+
+Daniel Resare 
+- Numerous SSL handling improvements
+
+Eivind Eklund 
+- MaxMessages option
 
 Theodore Ts'o 
-   * Contributor, Debian package co-maintainer
+- get-cert script
+- Maildir UID mapping improvements
+- Initial version of partial async IMAP support
 
-Nicolas Boullis 
-   * Debian package maintainer and minor upstream contributions
+Marc Hoersken 
+- CopyArrivalDate option
 
-Michael Elkins 
-   * Original author
+Jack Stone 
+Jan Synacek 
+- SASL support
 
-Send questions and bug reports to the isync-devel@lists.sourceforge.net
-mailing list.
+Jesse Weaver 
+- IMAP stream compression support
 
-_DON'T_ report bugs to Michael, not even in a CC: - he is not actively
-involved in isync development any more.
+Anton Khirnov 
+- ClientKey & ClientCertificate options
+
+Michael J Gruber 
+- Support for the $Forwarded/Passed flag
+
+Patrick Steinhardt 
+- UserCmd option
+
+Oliver Runge 
+- UseKeychain option
+
+Georgy Kibardin 
+- Support for UTF-7 IMAP mailbox names
+
+Honorary Contributors
+=
+
+(These people contributed patches that were too small or obvious
+ to claim copyright, or were rewritten from scratch.)
+(In alphabetical order.)
+
+Alessandro Ghedini 
+Andreas Grapentin 
+Anton Khirnov 
+Aurélien Francillon 
+Ben Kibbey 
+Caspar Schutijser 
+Cedric Ware 
+Dmitrij D. Czarkoff 
+Dmitry Torokhov 
+Felipe Contreras 
+Felix Janda 
+Gergely Risko 
+Sung Pae "guns" 
+Helmut Grohne 
+Hugo Haas 
+Jaroslav Suchanek 
+Jeremie Courreges-Anglas 
+Klemens Nanni 
+Lorenzo Martignoni 
+Magnus Jonsson 
+Marcin Niestroj 
+Martin Stenberg 
+Mike Delaney 
+Nicolas Boullis 
+Nihal Jere 
+Reimar Döffinger 
+Remko Tronçon 
+sb...@users.sf.net
+Thomas Roessler 
+Todd T. Fries 
+Vincent Bernat 
+Yuri D'Elia 


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/notifier-cmd' created

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/notifier-cmd' has been created at b67cd7d.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/debug-various' created

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/debug-various' has been created at 5c6affc.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: autotest: add missing `next` in cmptrash()

2022-06-20 Thread Oswald Buddenhagen via isync-devel
commit c9e57161cc6f605282b084d63cfd19531576de20
Author: Oswald Buddenhagen 
Date:   Mon Jun 20 17:33:03 2022 +0200

autotest: add missing `next` in cmptrash()

this doesn't really matter, as the branches are mutually exclusive
anyway, but still.

amends 01329bdf.

 src/run-tests.pl | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/run-tests.pl b/src/run-tests.pl
index 144ba94..459142a 100755
--- a/src/run-tests.pl
+++ b/src/run-tests.pl
@@ -631,6 +631,7 @@ sub cmptrash($$$)
if (!defined($ph)) {
print STDERR "Missing message $bn:".mn($num)."\n";
$ret = 1;
+   next;
}
if ($ph) {
print STDERR "Message $bn:".mn($num)." is 
placeholder\n";


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch '1.3' rewound

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch '1.3', previously at 9f7f0c9, has been rewound by 1
revision(s) to ed3bfda.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/exchange-workarounds-1.5' created

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/exchange-workarounds-1.5' has been created at 0af9331.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/imap-client-close' created

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/imap-client-close' has been created at 9a70105.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/INBOX-patterns-prefixing' created

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/INBOX-patterns-prefixing' has been created at e35c16c.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/maildir-path-under-inbox' created

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/maildir-path-under-inbox' has been created at b4e8b29.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/simplify-state-naming' created

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/simplify-state-naming' has been created at 77134c3.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/exchange-workarounds-1.2' created

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/exchange-workarounds-1.2' has been created at 0ba4d6e.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/exchange-workarounds-1.4' reset

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/exchange-workarounds-1.4', previously at c225aad, has
been rewound by 1 revision(s) and subsequently fast-forwarded by 1
revision(s) to 547de59.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/maildir-uid-dupes-test' created

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/maildir-uid-dupes-test' has been created at f1f6d33.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/expire-new' created

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/expire-new' has been created at 06bd8a0.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/test-more-paranoid' created

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/test-more-paranoid' has been created at 7574440.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] 1.3: CVE-2021-3657: security fixes

2022-06-20 Thread Oswald Buddenhagen via isync-devel
commit 9f7f0c94b67e9506ebf8ca674dd6cbb6a7989f44
Author: Oswald Buddenhagen 
Date:   Wed Nov 24 16:57:00 2021 +0100

CVE-2021-3657: security fixes

unlike in the 1.4 branch, we use signed ints for offsets and lengths, so
many of the qualifying statements from the 1.4 series don't apply.

 src/drv_imap.c|  9 +
 src/drv_maildir.c |  8 +++-
 src/socket.c  |  8 ++--
 src/sync.c| 15 ++-
 4 files changed, 32 insertions(+), 8 deletions(-)

diff --git a/src/drv_imap.c b/src/drv_imap.c
index 4cc3b2a..0d6c869 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -780,6 +780,11 @@ parse_imap_list( imap_store_t *ctx, char **sp, 
parse_list_state_t *sts )
bytes = cur->len = strtol( s + 1, , 10 );
if (*s != '}' || *++s)
goto bail;
+   if ((uint)bytes >= INT_MAX) {
+   error( "IMAP error: excessively large literal 
from %s "
+  "- THIS MIGHT BE AN ATTEMPT TO HACK 
YOU!\n", ctx->conn.name );
+   goto bail;
+   }
 
s = cur->val = nfmalloc( cur->len + 1 );
s[cur->len] = 0;
@@ -1279,6 +1284,10 @@ parse_list_rsp_p2( imap_store_t *ctx, list_t *list, char 
*cmd ATTR_UNUSED )
}
arg = list->val;
argl = list->len;
+   if (argl > 1000) {
+   warn( "IMAP warning: ignoring unreasonably long mailbox name 
'%.100s[...]'\n", arg );
+   goto skip;
+   }
if ((l = strlen( ctx->prefix ))) {
if (!starts_with( arg, argl, ctx->prefix, l )) {
if (is_inbox( ctx, arg, argl )) {
diff --git a/src/drv_maildir.c b/src/drv_maildir.c
index e1622d3..40a4d2a 100644
--- a/src/drv_maildir.c
+++ b/src/drv_maildir.c
@@ -1147,7 +1147,8 @@ maildir_scan( maildir_store_t *ctx, msg_t_array_alloc_t 
*msglist )
}
goto retry;
}
-   entry->size = st.st_size;
+   // The clipped value is good enough for MaxSize 
comparisons.
+   entry->size = st.st_size > INT_MAX ? INT_MAX : 
(int)st.st_size;
}
if (want_tuid || want_msgid) {
if (!(f = fopen( buf, "r" ))) {
@@ -1534,12 +1535,17 @@ maildir_fetch_msg( store_t *gctx, message_t *gmsg, 
msg_data_t *data,
}
}
fstat( fd,  );
+   if (st.st_size > INT_MAX) {
+   error( "Maildir error: %s is too big", buf );
+   goto mbad;
+   }
data->len = st.st_size;
if (data->date == -1)
data->date = st.st_mtime;
data->data = nfmalloc( data->len );
if (read( fd, data->data, data->len ) != data->len) {
sys_error( "Maildir error: cannot read %s", buf );
+ mbad:
close( fd );
cb( DRV_MSG_BAD, aux );
return;
diff --git a/src/socket.c b/src/socket.c
index 84449e7..d8548a9 100644
--- a/src/socket.c
+++ b/src/socket.c
@@ -898,6 +898,8 @@ do_append( conn_t *conn, buff_chunk_t *bc )
 /* This is big enough to avoid excessive chunking, but is
  * sufficiently small to keep SSL latency low with a slow uplink. */
 #define WRITE_CHUNK_SIZE 1024
+// Huge data blocks (message payloads) are forcibly chunked.
+#define MAX_WRITE_CHUNK_SIZE (1 << 30)
 
 static void
 do_flush( conn_t *conn )
@@ -952,7 +954,8 @@ do_flush( conn_t *conn )
 void
 socket_write( conn_t *conn, conn_iovec_t *iov, int iovcnt )
 {
-   int i, buf_avail, len, offset = 0, total = 0;
+   int i, buf_avail, len, offset = 0;
+   uint total = 0;
buff_chunk_t *bc;
 
for (i = 0; i < iovcnt; i++)
@@ -971,7 +974,8 @@ socket_write( conn_t *conn, conn_iovec_t *iov, int iovcnt )
 * predict a reasonable output buffer size anyway - 
deflatePending() does
 * not account for consumed but not yet compressed 
input, and adding up
 * the deflateBound()s would be a tad *too* 
pessimistic. */
-   buf_avail = total > WRITE_CHUNK_SIZE ? total : 
WRITE_CHUNK_SIZE;
+   buf_avail = total > MAX_WRITE_CHUNK_SIZE ? 
MAX_WRITE_CHUNK_SIZE :
+   total > WRITE_CHUNK_SIZE ? total : 
WRITE_CHUNK_SIZE;
bc = nfmalloc( offsetof(buff_chunk_t, data) + buf_avail 
);
bc->len = 0;
 #ifndef HAVE_LIBZ
diff --git a/src/sync.c b/src/sync.c
index 7d3fe79..46e089c 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -333,7 +333,7 @@ copy_msg_bytes( char **out_ptr, const char *in_buf, int 
*in_idx, int in_len, int
 }
 
 static int
-copy_msg_convert( int in_cr, int 

[commit] branch 'wip/more-autotest-subjects' created

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/more-autotest-subjects' has been created at 1576d52.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/exchange-workarounds' deleted

2022-06-20 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/exchange-workarounds', previously at de220e2, has been
deleted.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] branch 'wip/master-next' deleted

2022-06-19 Thread Oswald Buddenhagen via isync-devel
The branch 'wip/master-next', previously at 596c256, has been deleted.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: bump version

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit a87d6ddaca91adc05cf6f6bf62a26a52ce388e4c
Author: Oswald Buddenhagen 
Date:   Thu Jun 2 12:10:54 2022 +0200

bump version

 configure.ac | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index 3b9b145d..ee74d963 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2,7 +2,7 @@
 # SPDX-FileCopyrightText: 2002-2022 Oswald Buddenhagen 
 dnl SPDX-License-Identifier: GPL-2.0-or-later
 
-AC_INIT([isync], [1.4.4])
+AC_INIT([isync], [1.5.0])
 AC_CONFIG_HEADERS([autodefs.h])
 
 AC_CANONICAL_TARGET


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: whitespace fixes

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 7619705428aabe91573c433195cda30146a89efa
Author: Oswald Buddenhagen 
Date:   Mon Jun 6 12:37:47 2022 +0200

whitespace fixes

consistently surround '|' with spaces.

 src/config.c  |  2 +-
 src/drv_maildir.c | 12 ++--
 src/main.c| 14 +++---
 src/sync.c|  2 +-
 4 files changed, 15 insertions(+), 15 deletions(-)

diff --git a/src/config.c b/src/config.c
index 07201120..456bd477 100644
--- a/src/config.c
+++ b/src/config.c
@@ -403,7 +403,7 @@ merge_ops( int cops, int ops[], const char *chan_name )
if (!(cops & OP_MASK_TYPE))
cops |= OP_DFLT_TYPE;
else if (!(cops & XOP_MASK_DIR))
-   cops |= XOP_PULL|XOP_PUSH;
+   cops |= XOP_PULL | XOP_PUSH;
if (cops & XOP_PULL)
ops[N] |= cops & OP_MASK_TYPE;
if (cops & XOP_PUSH)
diff --git a/src/drv_maildir.c b/src/drv_maildir.c
index 3c528c73..cdffc320 100644
--- a/src/drv_maildir.c
+++ b/src/drv_maildir.c
@@ -1280,7 +1280,7 @@ maildir_open_box( store_t *gctx,
 
nfsnprintf( uvpath, sizeof(uvpath), "%s/.uidvalidity", ctx->path );
 #ifndef USE_DB
-   if ((ctx->uvfd = open( uvpath, O_RDWR|O_CREAT, 0600 )) < 0) {
+   if ((ctx->uvfd = open( uvpath, O_RDWR | O_CREAT, 0600 )) < 0) {
sys_error( "Maildir error: cannot write %s", uvpath );
cb( DRV_BOX_BAD, UIDVAL_BAD, aux );
return;
@@ -1291,11 +1291,11 @@ maildir_open_box( store_t *gctx,
nfsnprintf( uvpath, sizeof(uvpath), "%s/.isyncuidmap.db", 
ctx->path );
if ((ctx->uvfd = open( uvpath, O_RDWR, 0600 )) < 0) {
if (ctx->conf->alt_map) {
-   if ((ctx->uvfd = open( uvpath, O_RDWR|O_CREAT, 
0600 )) >= 0)
+   if ((ctx->uvfd = open( uvpath, O_RDWR | 
O_CREAT, 0600 )) >= 0)
goto dbok;
} else {
nfsnprintf( uvpath, sizeof(uvpath), 
"%s/.uidvalidity", ctx->path );
-   if ((ctx->uvfd = open( uvpath, O_RDWR|O_CREAT, 
0600 )) >= 0)
+   if ((ctx->uvfd = open( uvpath, O_RDWR | 
O_CREAT, 0600 )) >= 0)
goto fnok;
}
sys_error( "Maildir error: cannot write %s", uvpath );
@@ -1484,7 +1484,7 @@ maildir_rescan( maildir_store_t *ctx )
msgapp = >next;
} else {
debug( "  updating message %u\n", msg->uid );
-   msg->status &= ~(M_FLAGS|M_RECENT);
+   msg->status &= ~(M_FLAGS | M_RECENT);
free( msg->base );
free( msg->msgid );
maildir_init_msg( ctx, msg, msglist.array.data + i );
@@ -1605,7 +1605,7 @@ maildir_store_msg( store_t *gctx, msg_data_t *data, int 
to_trash,
 
maildir_make_flags( ctx->conf->info_delimiter, data->flags, fbuf );
nfsnprintf( buf, sizeof(buf), "%s/tmp/%s%s", box, base, fbuf );
-   if ((fd = open( buf, O_WRONLY|O_CREAT|O_EXCL, 0600 )) < 0) {
+   if ((fd = open( buf, O_WRONLY | O_CREAT | O_EXCL, 0600 )) < 0) {
if (errno != ENOENT || !to_trash) {
sys_error( "Maildir error: cannot create %s", buf );
free( data->data );
@@ -1617,7 +1617,7 @@ maildir_store_msg( store_t *gctx, msg_data_t *data, int 
to_trash,
cb( ret, 0, aux );
return;
}
-   if ((fd = open( buf, O_WRONLY|O_CREAT|O_EXCL, 0600 )) < 0) {
+   if ((fd = open( buf, O_WRONLY | O_CREAT | O_EXCL, 0600 )) < 0) {
sys_error( "Maildir error: cannot create %s", buf );
free( data->data );
cb( DRV_BOX_BAD, 0, aux );
diff --git a/src/main.c b/src/main.c
index 7cfea1aa..4607f5dd 100644
--- a/src/main.c
+++ b/src/main.c
@@ -230,7 +230,7 @@ main( int argc, char **argv )
cops |= XOP_PUSH, mvars->ops[F] |= 
XOP_HAVE_TYPE;
} else if (starts_with( opt, -1, "create", 6 )) 
{
opt += 6;
-   op = OP_CREATE|XOP_HAVE_CREATE;
+   op = OP_CREATE | XOP_HAVE_CREATE;
  lcop:
if (!*opt)
cops |= op;
@@ -247,7 +247,7 @@ main( int argc, char **argv )
mvars->ops[F] |= op & (XOP_HAVE_CREATE 
| XOP_HAVE_REMOVE | XOP_HAVE_EXPUNGE | XOP_HAVE_EXPUNGE_SOLO);
} 

[commit] master: update TODO

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 21c85293834d6fa8dea72f17d55a99112bf4c056
Author: Oswald Buddenhagen 
Date:   Fri Dec 24 12:56:02 2021 +0100

update TODO

remove items:
- the "Unidentified socket error" should be gone since 7ba7be111
- imap_commit_cmds() is implemented since cfaa4848d
- we will never use MULTIAPPEND and FETCH BODY with multiple messages,
  as that would significantly complicate matters for no tangible benefit

... and add some new thoughts.

the idea to use strings of colons for quoting patterns doesn't work, as
colons in "regular" patterns could not be quoted.

 TODO | 22 +++---
 1 file changed, 15 insertions(+), 7 deletions(-)

diff --git a/TODO b/TODO
index a04b2536..43f0c0da 100644
--- a/TODO
+++ b/TODO
@@ -1,7 +1,5 @@
 f{,data}sync() usage could be optimized by batching the calls.
 
-make SSL (connect) timeouts produce a bit more than "Unidentified socket 
error".
-
 automatically resume upon transient errors, e.g. "connection reset by peer"
 or timeout after some data was already transmitted.
 possibly also try to handle Exchange's "glitches" somehow.
@@ -10,11 +8,16 @@ uidvalidity lock timeout handling would be a good idea.
 
 should complain when multiple Channels match the same folders.
 
+should complain about nonsensical combinations like Sync Pull + Create Both.
+
 propagate folder deletions even when the folders are non-empty.
 - verify that "most" of the folders in the Channel are still there.
 - refuse to delete unpropagated messages when trashing on the remote side.
 - refuse to delete far side if it has unpropagated messages. symmetry?
 
+add option to suppress complaints about folders that would need creation
+(but not deleted ones).
+
 add message expiration based on arrival date (message date would be too
 unreliable). MaxAge; probably mutually exclusive to MaxMessages.
 
@@ -27,11 +30,12 @@ add support for event notification callbacks.
 make it possible to have different mailbox names for far and near side in
 Patterns.
 - use far:near for the pattern
-  - for quoting, use more colons: the longest sequence of colons is the
-separator
+  - supporting names with colons requires and extension of the parser to
+report which parts of an argument were quoted
 - this makes Groups mostly useless, as they are mostly a workaround for this
   function being missing so far
 - this is needed for move detection, which would work only within one Channel
+- this supersedes MapInbox
 
 add regexp-based mailbox path rewriting to the drivers. user would provide
 expressions for both directions. every transformation would be immediately
@@ -44,18 +48,22 @@ also: idling mode.
 parallel fetching of multiple mailboxes.
 TLS session resumption becomes interesting then as well.
 
-imap_set_flags(): group commands for efficiency, don't call back until
-imap_commit().
+imap_set_msg_flags() & imap_trash_msg(): group commands for efficiency.
 
 add streaming from fetching to storing.
+this is complicated by the IMAP target needing the final size in advance,
+which we can't know in a single pass when newline translation is necessary.
 
 handle custom flags (keywords).
+this is impeded by there being no Maildir-side standard.
 
 make use of IMAP CONDSTORE extension (rfc4551; CHANGEDSINCE FETCH Modifier);
 make use of IMAP QRESYNC extension (rfc5162) to avoid SEARCH to find vanished
 messages.
 
-use MULTIAPPEND and FETCH with multiple messages.
+make use of IMAP CAPABILITY APPENDLIMIT= extension (rfc7889; fastmail & gmail).
+this is really useful only for IMAP <=> IMAP syncs: saves FETCH BODY.
+the message could still become oversized due to conversion.
 
 dummy messages resulting from MaxSize should contain a dump of the original
 message's MIME structure and its (reasonably sized) text parts.


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: drop unused 'flags' parameter of maildir_list_{recurse, inbox, path}()

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 090ba0caa3f52579919f1c43c1e948d5f1853446
Author: Oswald Buddenhagen 
Date:   Tue Jun 7 12:08:50 2022 +0200

drop unused 'flags' parameter of maildir_list_{recurse,inbox,path}()

amends 416ced25.

 src/drv_maildir.c | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/src/drv_maildir.c b/src/drv_maildir.c
index 64e7718a..3c528c73 100644
--- a/src/drv_maildir.c
+++ b/src/drv_maildir.c
@@ -373,7 +373,7 @@ maildir_list_maildirpp( maildir_store_t *ctx, int flags, 
const char *inbox )
 }
 
 static int
-maildir_list_recurse( maildir_store_t *ctx, int isBox, int flags,
+maildir_list_recurse( maildir_store_t *ctx, int isBox,
   const char *inbox, uint inboxLen,
   char *suffix, int suffixLen,
   char *path, int pathLen, char *name, int nameLen )
@@ -449,7 +449,7 @@ maildir_list_recurse( maildir_store_t *ctx, int isBox, int 
flags,
add_string_list( >boxes, name );
path[pl] = 0;
name[nl++] = '/';
-   if (maildir_list_recurse( ctx, isBox + 1, flags, inbox, 
inboxLen, NULL, 0, path, pl, name, nl ) < 0) {
+   if (maildir_list_recurse( ctx, isBox + 1, inbox, 
inboxLen, NULL, 0, path, pl, name, nl ) < 0) {
closedir( dir );
return -1;
}
@@ -460,7 +460,7 @@ maildir_list_recurse( maildir_store_t *ctx, int isBox, int 
flags,
 }
 
 static int
-maildir_list_inbox( maildir_store_t *ctx, int flags )
+maildir_list_inbox( maildir_store_t *ctx )
 {
char path[_POSIX_PATH_MAX], name[_POSIX_PATH_MAX];
 
@@ -470,13 +470,13 @@ maildir_list_inbox( maildir_store_t *ctx, int flags )
 
add_string_list( >boxes, "INBOX" );
return maildir_list_recurse(
-   ctx, 1, flags, NULL, 0, NULL, 0,
+   ctx, 1, NULL, 0, NULL, 0,
path, nfsnprintf( path, _POSIX_PATH_MAX, "%s/", 
ctx->conf->inbox ),
name, nfsnprintf( name, _POSIX_PATH_MAX, "INBOX/" ) );
 }
 
 static int
-maildir_list_path( maildir_store_t *ctx, int flags )
+maildir_list_path( maildir_store_t *ctx )
 {
char path[_POSIX_PATH_MAX], name[_POSIX_PATH_MAX];
 
@@ -488,7 +488,7 @@ maildir_list_path( maildir_store_t *ctx, int flags )
return -1;
const char *inbox = ctx->conf->inbox;
return maildir_list_recurse(
-   ctx, 0, flags, inbox, strlen( inbox ),
+   ctx, 0, inbox, strlen( inbox ),
ctx->conf->path_sfx, strlen( ctx->conf->path_sfx ),
path, nfsnprintf( path, _POSIX_PATH_MAX, "%s/", ctx->conf->path 
),
name, 0 );
@@ -504,9 +504,9 @@ maildir_list_store( store_t *gctx, int flags,
if (conf->sub_style == SUB_MAILDIRPP
? maildir_list_maildirpp( ctx, flags, conf->inbox ) < 0
: flags & LIST_PATH) || ((flags & LIST_PATH_MAYBE) && 
conf->path))
-   && maildir_list_path( ctx, flags ) < 0) ||
+   && maildir_list_path( ctx ) < 0) ||
   ((flags & LIST_INBOX)
-   && maildir_list_inbox( ctx, flags ) < 0))) {
+   && maildir_list_inbox( ctx ) < 0))) {
maildir_invoke_bad_callback( ctx );
cb( DRV_CANCELED, NULL, aux );
} else {


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: add support for Maildir Paths with suffixes

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 46584e5358dbd32b8549b15ce543c33482bc6d60
Author: Oswald Buddenhagen 
Date:   Fri Jun 3 20:58:09 2022 +0200

add support for Maildir Paths with suffixes

that is, Path not ending with a slash.

pedantically, this is a bugfix, as the manual already suggested that
this is possible (and at least one user got the hint, though he was
disappointed).

the IMAP driver already supports this.

 NEWS|  2 ++
 src/drv_maildir.c   | 38 --
 src/mbsyncrc.sample | 26 ++
 3 files changed, 60 insertions(+), 6 deletions(-)

diff --git a/NEWS b/NEWS
index 5472b94c..04389920 100644
--- a/NEWS
+++ b/NEWS
@@ -21,6 +21,8 @@ MaxMessages and MaxSize can be used together now.
 
 Added support for IMAP mailbox names with non-ASCII characters.
 
+Added support for Maildir Paths with suffixes.
+
 The unfiltered list of mailboxes in each Store can be printed now.
 
 A proper summary is now printed prior to exiting.
diff --git a/src/drv_maildir.c b/src/drv_maildir.c
index 8327d233..64e7718a 100644
--- a/src/drv_maildir.c
+++ b/src/drv_maildir.c
@@ -35,6 +35,7 @@ typedef union maildir_store_conf {
struct {
STORE_CONF
char *path;
+   char *path_sfx;
char *inbox;
 #ifdef USE_DB
int alt_map;
@@ -121,16 +122,20 @@ static char *
 maildir_join_path( maildir_store_conf_t *conf, int in_inbox, const char *box )
 {
char *out, *p;
-   const char *prefix;
-   uint pl, bl, n;
+   const char *prefix, *infix;
+   uint pl, il, bl, n;
char c;
 
if (in_inbox || conf->sub_style == SUB_MAILDIRPP) {
prefix = conf->inbox;
+   infix = NULL;
+   il = 0;
} else {
if (maildir_ensure_path( conf ) < 0)
return NULL;
prefix = conf->path;
+   infix = conf->path_sfx;
+   il = strlen( infix ) + 1;
}
pl = strlen( prefix );
for (bl = 0, n = 0; (c = box[bl]); bl++) {
@@ -157,12 +162,16 @@ maildir_join_path( maildir_store_conf_t *conf, int 
in_inbox, const char *box )
default: /* SUB_LEGACY and SUB_UNSET */
break;
}
-   out = nfmalloc( pl + bl + n + 1 );
+   out = nfmalloc( pl + il + bl + n + 1 );
memcpy( out, prefix, pl );
p = out + pl;
if (conf->sub_style == SUB_MAILDIRPP) {
*p++ = '/';
*p++ = '.';
+   } else if (il) {
+   *p++ = '/';
+   memcpy( p, infix, il - 1 );
+   p += il - 1;
}
while ((c = *box++)) {
if (c == '/') {
@@ -366,6 +375,7 @@ maildir_list_maildirpp( maildir_store_t *ctx, int flags, 
const char *inbox )
 static int
 maildir_list_recurse( maildir_store_t *ctx, int isBox, int flags,
   const char *inbox, uint inboxLen,
+  char *suffix, int suffixLen,
   char *path, int pathLen, char *name, int nameLen )
 {
DIR *dir;
@@ -399,6 +409,16 @@ maildir_list_recurse( maildir_store_t *ctx, int isBox, int 
flags,
pl = nfsnprintf( path + pathLen, _POSIX_PATH_MAX - pathLen, 
"%s", ent );
if (pl == 3 && (!memcmp( ent, "cur", 3 ) || !memcmp( ent, 
"new", 3 ) || !memcmp( ent, "tmp", 3 )))
continue;
+   if (suffixLen) {
+   if (!starts_with( ent, pl, suffix, suffixLen ))
+   continue;
+   if (pl == suffixLen) {
+   error( "Maildir error: empty mailbox name under 
%s - did you forget the trailing slash?\n", path );
+   closedir( dir );
+   return -1;
+   }
+   ent += suffixLen;
+   }
pl += pathLen;
if (inbox && equals( path, pl, inbox, inboxLen )) {
// Inbox nested into Path.
@@ -429,7 +449,7 @@ maildir_list_recurse( maildir_store_t *ctx, int isBox, int 
flags,
add_string_list( >boxes, name );
path[pl] = 0;
name[nl++] = '/';
-   if (maildir_list_recurse( ctx, isBox + 1, flags, inbox, 
inboxLen, path, pl, name, nl ) < 0) {
+   if (maildir_list_recurse( ctx, isBox + 1, flags, inbox, 
inboxLen, NULL, 0, path, pl, name, nl ) < 0) {
closedir( dir );
return -1;
}
@@ -450,7 +470,7 @@ maildir_list_inbox( maildir_store_t *ctx, int flags )
 
add_string_list( >boxes, "INBOX" );
return maildir_list_recurse(
-   ctx, 1, flags, NULL, 0,
+   ctx, 1, flags, NULL, 0, NULL, 0,
path, nfsnprintf( 

[commit] master: forbid nesting maildir Path under Inbox again

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 3bfc3c5063b5c5a02d77c97dd43461e7e19ca0f7
Author: Oswald Buddenhagen 
Date:   Fri Jun 3 18:06:50 2022 +0200

forbid nesting maildir Path under Inbox again

it was never documented, and i can't really think of a case where
someone would actually want it, as the MUA side of things would be just
weird. additionally, the case of Path being Inbox+'/' did not work
anyway. will reconsider and fix the latter case if someone complains.

reverts 98bd2b11.

 src/drv_maildir.c | 25 +++--
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/src/drv_maildir.c b/src/drv_maildir.c
index 0f94519c..8327d233 100644
--- a/src/drv_maildir.c
+++ b/src/drv_maildir.c
@@ -365,7 +365,7 @@ maildir_list_maildirpp( maildir_store_t *ctx, int flags, 
const char *inbox )
 
 static int
 maildir_list_recurse( maildir_store_t *ctx, int isBox, int flags,
-  const char *inbox, uint inboxLen, const char *basePath, 
uint basePathLen,
+  const char *inbox, uint inboxLen,
   char *path, int pathLen, char *name, int nameLen )
 {
DIR *dir;
@@ -402,8 +402,6 @@ maildir_list_recurse( maildir_store_t *ctx, int isBox, int 
flags,
pl += pathLen;
if (inbox && equals( path, pl, inbox, inboxLen )) {
// Inbox nested into Path.
-   } else if (basePath && equals( path, pl, basePath, basePathLen 
)) {
-   // Path nested into Inbox.
} else {
if (style == SUB_LEGACY) {
if (*ent == '.') {
@@ -431,7 +429,7 @@ maildir_list_recurse( maildir_store_t *ctx, int isBox, int 
flags,
add_string_list( >boxes, name );
path[pl] = 0;
name[nl++] = '/';
-   if (maildir_list_recurse( ctx, isBox + 1, flags, inbox, 
inboxLen, basePath, basePathLen, path, pl, name, nl ) < 0) {
+   if (maildir_list_recurse( ctx, isBox + 1, flags, inbox, 
inboxLen, path, pl, name, nl ) < 0) {
closedir( dir );
return -1;
}
@@ -451,9 +449,8 @@ maildir_list_inbox( maildir_store_t *ctx, int flags )
ctx->listed |= LIST_INBOX;
 
add_string_list( >boxes, "INBOX" );
-   const char *basePath = ctx->conf->path;
return maildir_list_recurse(
-   ctx, 1, flags, NULL, 0, basePath, basePath ? strlen( basePath ) 
- 1 : 0,
+   ctx, 1, flags, NULL, 0,
path, nfsnprintf( path, _POSIX_PATH_MAX, "%s/", 
ctx->conf->inbox ),
name, nfsnprintf( name, _POSIX_PATH_MAX, "INBOX/" ) );
 }
@@ -471,7 +468,7 @@ maildir_list_path( maildir_store_t *ctx, int flags )
return -1;
const char *inbox = ctx->conf->inbox;
return maildir_list_recurse(
-   ctx, 0, flags, inbox, strlen( inbox ), NULL, 0,
+   ctx, 0, flags, inbox, strlen( inbox ),
path, nfsnprintf( path, _POSIX_PATH_MAX, "%s", ctx->conf->path 
),
name, 0 );
 }
@@ -1908,9 +1905,17 @@ maildir_parse_store( conffile_t *cfg, store_conf_t 
**storep )
}
if (!store->inbox)
store->inbox = expand_strdup( "~/Maildir", NULL );
-   if (store->sub_style == SUB_MAILDIRPP && store->path) {
-   error( "Maildir store '%s': Setting Path is incompatible with 
'SubFolders Maildir++'\n", store->name );
-   cfg->err = 1;
+   if (store->path) {
+   if (store->sub_style == SUB_MAILDIRPP) {
+   error( "Maildir store '%s': Setting Path is 
incompatible with 'SubFolders Maildir++'\n", store->name );
+   cfg->err = 1;
+   } else {
+   uint inboxLen = strlen( store->inbox );
+   if (starts_with( store->path, -1, store->inbox, 
inboxLen ) && store->path[inboxLen] == '/') {
+   error( "Maildir store '%s': Path cannot be 
nested under Inbox\n", store->name );
+   cfg->err = 1;
+   }
+   }
}
nfasprintf( >info_prefix, "%c2,", store->info_delimiter );
nfasprintf( >info_stop, "%c,", store->info_delimiter );


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: add support for UTF-7 mailbox names

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 52c063fd45f327169a08d2eadbb2904678f2bb40
Author: Oswald Buddenhagen 
Date:   Mon May 23 10:12:38 2022 +0200

add support for UTF-7 mailbox names

this finally makes us compliant with IMAP4rev1. how fitting that the
meanwhile released IMAP4rev2 demoted UTF-7 to legacy status ...

based on a patch by Georgy Kibardin .

 TODO|   2 -
 src/.gitignore  |   1 +
 src/Makefile.am |   6 +-
 src/drv_imap.c  |  21 
 src/imap_p.h|   4 +
 src/imap_utf7.c | 288 
 src/tst_imap_utf7.c | 116 ++
 7 files changed, 434 insertions(+), 4 deletions(-)

diff --git a/TODO b/TODO
index dacdeb4e..a04b2536 100644
--- a/TODO
+++ b/TODO
@@ -6,8 +6,6 @@ automatically resume upon transient errors, e.g. "connection 
reset by peer"
 or timeout after some data was already transmitted.
 possibly also try to handle Exchange's "glitches" somehow.
 
-add support for IMAP UTF-7 (for internationalized mailbox names).
-
 uidvalidity lock timeout handling would be a good idea.
 
 should complain when multiple Channels match the same folders.
diff --git a/src/.gitignore b/src/.gitignore
index 5e7fc35d..3139876a 100644
--- a/src/.gitignore
+++ b/src/.gitignore
@@ -3,6 +3,7 @@
 /mbsync
 /mdconvert
 /tst_imap_msgs
+/tst_imap_utf7
 /tst_msg_cvt
 /tst_timers
 /tmp
diff --git a/src/Makefile.am b/src/Makefile.am
index ab564183..69cf29db 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -5,7 +5,7 @@
 mbsync_SOURCES = \
util.c config.c socket.c \
driver.c drv_proxy.c \
-   drv_imap.c imap_msgs.c \
+   drv_imap.c imap_msgs.c imap_utf7.c \
drv_maildir.c \
sync.c sync_state.c sync_msg_cvt.c \
main.c main_sync.c main_list.c
@@ -54,10 +54,12 @@ man_MANS = mbsync.1 $(mdconvert_man)
 
 tst_imap_msgs_SOURCES = tst_imap_msgs.c imap_msgs.c util.c
 
+tst_imap_utf7_SOURCES = tst_imap_utf7.c imap_utf7.c util.c
+
 tst_msg_cvt_SOURCES = tst_msg_cvt.c sync_msg_cvt.c util.c
 tst_msg_cvt_CFLAGS = -DQPRINTF_BUFF=1
 
-check_PROGRAMS = tst_imap_msgs tst_msg_cvt
+check_PROGRAMS = tst_imap_msgs tst_imap_utf7 tst_msg_cvt
 TESTS = $(check_PROGRAMS)
 
 tst_timers_SOURCES = tst_timers.c util.c
diff --git a/src/drv_imap.c b/src/drv_imap.c
index 911df572..ad95e3d2 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -1577,6 +1577,7 @@ list3_rsp_atom( imap_store_t *ctx, char *arg, uint len, 
int type ATTR_UNUSED )
string_list_t *narg;
int argl = (int)len;
uint l;
+   char rarg[1130];  // See imap_utf7_to_utf8() for the origin of that 
number
 
if (!arg)
return LIST_BAD;
@@ -1608,6 +1609,16 @@ list3_rsp_atom( imap_store_t *ctx, char *arg, uint len, 
int type ATTR_UNUSED )
}
if (argl >= 5 && !memcmp( arg + argl - 5, ".lock", 5 )) /* workaround 
broken servers */
return LIST_OK;
+   if (!(CAP(UTF8_ACCEPT) || CAP(UTF8_ONLY))) {
+   int rargl = imap_utf7_to_utf8( arg, argl, rarg );
+   if (rargl < 0) {
+   error( "IMAP error: invalid modified-UTF-7 string 
'%.*s'.\n", argl, arg );
+   return LIST_BAD;
+   }
+   assert( (uint)rargl < sizeof(rarg) );
+   arg = rarg;
+   argl = rargl;
+   }
if (map_name( arg, argl, (char **), offsetof(string_list_t, 
string), ctx->delimiter, "/") < 0) {
warn( "IMAP warning: ignoring mailbox %.*s (reserved character 
'/' in name)\n", argl, arg );
return LIST_OK;
@@ -1665,6 +1676,16 @@ prepare_name( char **buf, const imap_store_t *ctx, const 
char *prefix, const cha
return -1;
default:
memcpy( *buf, prefix, pl );
+   if (!(CAP(UTF8_ACCEPT) || CAP(UTF8_ONLY))) {
+   char *nbuf = imap_utf8_to_utf7( *buf );
+   if (!nbuf) {
+   error( "IMAP error: invalid UTF-8 string 
'%s'\n", *buf );
+   free( *buf );
+   return -1;
+   }
+   free( *buf );
+   *buf = nbuf;
+   }
return 0;
}
 }
diff --git a/src/imap_p.h b/src/imap_p.h
index 76e02e21..1c7933eb 100644
--- a/src/imap_p.h
+++ b/src/imap_p.h
@@ -10,6 +10,7 @@
 #include "driver.h"
 
 //#define DEBUG_IMAP_MSGS
+//#define DEBUG_IMAP_UTF7
 
 typedef union imap_message {
message_t gen;
@@ -45,4 +46,7 @@ void reset_imap_messages( imap_messages_t *msgs );
 void imap_ensure_relative( imap_messages_t *msgs );
 void imap_ensure_absolute( imap_messages_t *msgs );
 
+char *imap_utf8_to_utf7( const char *buf );
+int imap_utf7_to_utf8( const char *buf, int argl, char *outbuf );
+
 #endif
diff --git a/src/imap_utf7.c b/src/imap_utf7.c
new file mode 100644
index ..ac91cdb6
--- /dev/null
+++ b/src/imap_utf7.c
@@ -0,0 +1,288 @@

[commit] master: simplify maildir recursion limitation

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 32d500ed15421628eb29e026b0350d7cd2fcc1f6
Author: Oswald Buddenhagen 
Date:   Fri Jun 3 12:14:05 2022 +0200

simplify maildir recursion limitation

the isDir argument is effectively already a depth counter, so we don't
need a separate one for that.

amends 79797826.

 src/drv_maildir.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/src/drv_maildir.c b/src/drv_maildir.c
index 58f6a77e..9eac0d5d 100644
--- a/src/drv_maildir.c
+++ b/src/drv_maildir.c
@@ -367,7 +367,7 @@ static int maildir_list_inbox( maildir_store_t *ctx, int 
flags, const char *base
 static int maildir_list_path( maildir_store_t *ctx, int flags, const char 
*inbox );
 
 static int
-maildir_list_recurse( maildir_store_t *ctx, int isBox, int flags, int depth,
+maildir_list_recurse( maildir_store_t *ctx, int isBox, int flags,
   const char *inbox, uint inboxLen, const char *basePath, 
uint basePathLen,
   char *path, int pathLen, char *name, int nameLen )
 {
@@ -389,7 +389,7 @@ maildir_list_recurse( maildir_store_t *ctx, int isBox, int 
flags, int depth,
closedir( dir );
return -1;
}
-   if (++depth > 10) {
+   if (isBox > 10) {
// We do the other checks first to avoid confusing error 
messages for files.
error( "Maildir error: path %s is too deeply nested. Symlink 
loop?\n", path );
closedir( dir );
@@ -442,7 +442,7 @@ maildir_list_recurse( maildir_store_t *ctx, int isBox, int 
flags, int depth,
add_string_list( >boxes, name );
path[pl] = 0;
name[nl++] = '/';
-   if (maildir_list_recurse( ctx, isBox + 1, flags, depth, 
inbox, inboxLen, basePath, basePathLen, path, pl, name, nl ) < 0) {
+   if (maildir_list_recurse( ctx, isBox + 1, flags, inbox, 
inboxLen, basePath, basePathLen, path, pl, name, nl ) < 0) {
closedir( dir );
return -1;
}
@@ -463,7 +463,7 @@ maildir_list_inbox( maildir_store_t *ctx, int flags, const 
char *basePath )
 
add_string_list( >boxes, "INBOX" );
return maildir_list_recurse(
-   ctx, 1, flags, 0, NULL, 0, basePath, basePath ? strlen( 
basePath ) - 1 : 0,
+   ctx, 1, flags, NULL, 0, basePath, basePath ? strlen( basePath ) 
- 1 : 0,
path, nfsnprintf( path, _POSIX_PATH_MAX, "%s/", 
ctx->conf->inbox ),
name, nfsnprintf( name, _POSIX_PATH_MAX, "INBOX/" ) );
 }
@@ -480,7 +480,7 @@ maildir_list_path( maildir_store_t *ctx, int flags, const 
char *inbox )
if (maildir_ensure_path( ctx->conf ) < 0)
return -1;
return maildir_list_recurse(
-   ctx, 0, flags, 0, inbox, inbox ? strlen( inbox ) : 0, NULL, 0,
+   ctx, 0, flags, inbox, inbox ? strlen( inbox ) : 0, NULL, 0,
path, nfsnprintf( path, _POSIX_PATH_MAX, "%s", ctx->conf->path 
),
name, 0 );
 }


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: simplify/fix recursive maildir listing

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit acd6b4b0b89d8e706f1463c97582868dec3c190b
Author: Oswald Buddenhagen 
Date:   Fri Jun 3 15:38:35 2022 +0200

simplify/fix recursive maildir listing

there is no need to call maildir_list_{path,inbox}() from
maildir_list_recurse(), as maildir_list_store() will do that anyway -
if it's actually requested, that is. this means that this removes the
implicit listing when Inbox is nested into Path, or vice versa. this is
actually correct, as the Patterns matching would throw away the non-
requested boxes anyway.

 src/drv_maildir.c | 23 +++
 1 file changed, 7 insertions(+), 16 deletions(-)

diff --git a/src/drv_maildir.c b/src/drv_maildir.c
index 9eac0d5d..0f94519c 100644
--- a/src/drv_maildir.c
+++ b/src/drv_maildir.c
@@ -363,9 +363,6 @@ maildir_list_maildirpp( maildir_store_t *ctx, int flags, 
const char *inbox )
return 0;
 }
 
-static int maildir_list_inbox( maildir_store_t *ctx, int flags, const char 
*basePath );
-static int maildir_list_path( maildir_store_t *ctx, int flags, const char 
*inbox );
-
 static int
 maildir_list_recurse( maildir_store_t *ctx, int isBox, int flags,
   const char *inbox, uint inboxLen, const char *basePath, 
uint basePathLen,
@@ -405,16 +402,8 @@ maildir_list_recurse( maildir_store_t *ctx, int isBox, int 
flags,
pl += pathLen;
if (inbox && equals( path, pl, inbox, inboxLen )) {
// Inbox nested into Path.
-   if (maildir_list_inbox( ctx, flags, NULL ) < 0) {
-   closedir( dir );
-   return -1;
-   }
} else if (basePath && equals( path, pl, basePath, basePathLen 
)) {
// Path nested into Inbox.
-   if (maildir_list_path( ctx, flags, NULL ) < 0) {
-   closedir( dir );
-   return -1;
-   }
} else {
if (style == SUB_LEGACY) {
if (*ent == '.') {
@@ -453,7 +442,7 @@ maildir_list_recurse( maildir_store_t *ctx, int isBox, int 
flags,
 }
 
 static int
-maildir_list_inbox( maildir_store_t *ctx, int flags, const char *basePath )
+maildir_list_inbox( maildir_store_t *ctx, int flags )
 {
char path[_POSIX_PATH_MAX], name[_POSIX_PATH_MAX];
 
@@ -462,6 +451,7 @@ maildir_list_inbox( maildir_store_t *ctx, int flags, const 
char *basePath )
ctx->listed |= LIST_INBOX;
 
add_string_list( >boxes, "INBOX" );
+   const char *basePath = ctx->conf->path;
return maildir_list_recurse(
ctx, 1, flags, NULL, 0, basePath, basePath ? strlen( basePath ) 
- 1 : 0,
path, nfsnprintf( path, _POSIX_PATH_MAX, "%s/", 
ctx->conf->inbox ),
@@ -469,7 +459,7 @@ maildir_list_inbox( maildir_store_t *ctx, int flags, const 
char *basePath )
 }
 
 static int
-maildir_list_path( maildir_store_t *ctx, int flags, const char *inbox )
+maildir_list_path( maildir_store_t *ctx, int flags )
 {
char path[_POSIX_PATH_MAX], name[_POSIX_PATH_MAX];
 
@@ -479,8 +469,9 @@ maildir_list_path( maildir_store_t *ctx, int flags, const 
char *inbox )
 
if (maildir_ensure_path( ctx->conf ) < 0)
return -1;
+   const char *inbox = ctx->conf->inbox;
return maildir_list_recurse(
-   ctx, 0, flags, inbox, inbox ? strlen( inbox ) : 0, NULL, 0,
+   ctx, 0, flags, inbox, strlen( inbox ), NULL, 0,
path, nfsnprintf( path, _POSIX_PATH_MAX, "%s", ctx->conf->path 
),
name, 0 );
 }
@@ -495,9 +486,9 @@ maildir_list_store( store_t *gctx, int flags,
if (conf->sub_style == SUB_MAILDIRPP
? maildir_list_maildirpp( ctx, flags, conf->inbox ) < 0
: flags & LIST_PATH) || ((flags & LIST_PATH_MAYBE) && 
conf->path))
-   && maildir_list_path( ctx, flags, conf->inbox ) < 0) ||
+   && maildir_list_path( ctx, flags ) < 0) ||
   ((flags & LIST_INBOX)
-   && maildir_list_inbox( ctx, flags, conf->path ) < 0))) {
+   && maildir_list_inbox( ctx, flags ) < 0))) {
maildir_invoke_bad_callback( ctx );
cb( DRV_CANCELED, NULL, aux );
} else {


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: enable UTF-8 on servers with RFC6855 support

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit efab63fb8e14aaa38c6dede11ea36ab4f46f6572
Author: Oswald Buddenhagen 
Date:   Mon May 23 12:04:57 2022 +0200

enable UTF-8 on servers with RFC6855 support

note that this is a somewhat sloppy implementation, as it simply
assumes that the local system uses UTF-8 - that seems reasonable
nowadays.

 NEWS   |  2 ++
 src/drv_imap.c | 33 ++---
 2 files changed, 32 insertions(+), 3 deletions(-)

diff --git a/NEWS b/NEWS
index 0575d7c1..5472b94c 100644
--- a/NEWS
+++ b/NEWS
@@ -19,6 +19,8 @@ Made the Channel side to expire with MaxMessages configurable.
 
 MaxMessages and MaxSize can be used together now.
 
+Added support for IMAP mailbox names with non-ASCII characters.
+
 The unfiltered list of mailboxes in each Store can be printed now.
 
 A proper summary is now printed prior to exiting.
diff --git a/src/drv_imap.c b/src/drv_imap.c
index f674e21f..911df572 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -286,6 +286,8 @@ enum CAPABILITY {
LITERALMINUS,
MOVE,
NAMESPACE,
+   UTF8_ACCEPT,
+   UTF8_ONLY,
COMPRESS_DEFLATE
 };
 
@@ -306,6 +308,8 @@ static const struct {
{ "LITERAL-", 8 },
{ "MOVE", 4 },
{ "NAMESPACE", 9 },
+   { "UTF8=ACCEPT", 11 },
+   { "UTF8=ONLY", 9 },
{ "COMPRESS=DEFLATE", 16 },
 };
 
@@ -2090,6 +2094,8 @@ static void imap_open_store_compress( imap_store_t * );
 #ifdef HAVE_LIBZ
 static void imap_open_store_compress_p2( imap_store_t *, imap_cmd_t *, int );
 #endif
+static void imap_open_store_enable_utf8( imap_store_t * );
+static void imap_open_store_enable_utf8_p2( imap_store_t *, imap_cmd_t *, int 
);
 static void imap_open_store_namespace( imap_store_t * );
 static void imap_open_store_namespace_p2( imap_store_t *, imap_cmd_t *, int );
 static void imap_open_store_namespace2( imap_store_t * );
@@ -2737,7 +2743,7 @@ imap_open_store_compress( imap_store_t *ctx )
return;
}
 #endif
-   imap_open_store_namespace( ctx );
+   imap_open_store_enable_utf8( ctx );
 }
 
 #ifdef HAVE_LIBZ
@@ -2746,14 +2752,35 @@ imap_open_store_compress_p2( imap_store_t *ctx, 
imap_cmd_t *cmd ATTR_UNUSED, int
 {
if (response == RESP_NO) {
/* We already reported an error, but it's not fatal to us. */
-   imap_open_store_namespace( ctx );
+   imap_open_store_enable_utf8( ctx );
} else if (response == RESP_OK) {
socket_start_deflate( >conn );
-   imap_open_store_namespace( ctx );
+   imap_open_store_enable_utf8( ctx );
}
 }
 #endif
 
+static void
+imap_open_store_enable_utf8( imap_store_t *ctx )
+{
+   if (CAP(UTF8_ACCEPT) || CAP(UTF8_ONLY)) {
+   // We just assume that a server that announces UTF8= also 
supports ENABLE.
+   imap_exec( ctx, NULL, imap_open_store_enable_utf8_p2, "ENABLE 
UTF8=ACCEPT" );
+   } else {
+   imap_open_store_namespace( ctx );
+   }
+}
+
+static void
+imap_open_store_enable_utf8_p2( imap_store_t *ctx, imap_cmd_t *cmd 
ATTR_UNUSED, int response )
+{
+   if (response == RESP_NO) {
+   imap_open_store_bail( ctx, FAIL_FINAL );
+   } else if (response == RESP_OK) {
+   imap_open_store_namespace( ctx );
+   }
+}
+
 static void
 imap_open_store_namespace( imap_store_t *ctx )
 {


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: autotest conversion of even more malformed messages

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 8ba4192b23d3e81e982437c21719480aaabacb2a
Author: Oswald Buddenhagen 
Date:   Tue May 17 13:22:07 2022 +0200

autotest conversion of even more malformed messages

this follows up on 87065c12.

 src/tst_msg_cvt.c | 56 +++
 1 file changed, 52 insertions(+), 4 deletions(-)

diff --git a/src/tst_msg_cvt.c b/src/tst_msg_cvt.c
index 399c5653..d4e10ba9 100644
--- a/src/tst_msg_cvt.c
+++ b/src/tst_msg_cvt.c
@@ -37,6 +37,9 @@ strip_cr( char *buf )
 #define REGULAR 0
 #define FLAGGED 1
 
+#define OK_HEADER 0
+#define PARTIAL_HEADER 1
+
 #define BIG_SIZE 2345687
 #define BIG_SIZE_STR "2.2MiB"
 
@@ -105,7 +108,7 @@ test( const char *name, const char *in, int scr, int rscr, 
const char *out, int
 }
 
 static void
-tests( const char *name, const char *in, const char *out, int add_tuid, int 
minimal, int flagged )
+tests( const char *name, const char *in, const char *out, int add_tuid, int 
minimal, int flagged, int hdr_sts )
 {
test( name, in, NL_UNIX, NL_UNIX, out, NL_ANY, NL_ANY, add_tuid, 
minimal, flagged );
test( name, in, NL_ANY, NL_UNIX, out, NL_UNIX, NL_UNIX, add_tuid, 
minimal, flagged );
@@ -113,7 +116,12 @@ tests( const char *name, const char *in, const char *out, 
int add_tuid, int mini
// Skip if (scr == tcr && !srec), like copy_msg() does.
if (add_tuid) {
test( name, in, NL_UNIX, NL_UNIX, out, NL_UNIX, NL_UNIX, 
ADD_TUID, minimal, flagged );
-   test( name, in, NL_ANY, NL_UNIX, out, NL_ANY, NL_UNIX, 
ADD_TUID, minimal, flagged );
+   if (hdr_sts == OK_HEADER) {
+   test( name, in, NL_ANY, NL_UNIX, out, NL_ANY, NL_UNIX, 
ADD_TUID, minimal, flagged );
+   } else {
+   // If there are no line breaks to detect the style, the 
output defaults to CRLF.
+   test( name, in, NL_ANY, NL_UNIX, out, NL_ANY, NL_ANY, 
ADD_TUID, minimal, flagged );
+   }
test( name, in, NL_ANY, NL_ANY, out, NL_ANY, NL_ANY, ADD_TUID, 
minimal, flagged );
}
 }
@@ -121,13 +129,25 @@ tests( const char *name, const char *in, const char *out, 
int add_tuid, int mini
 static void
 fulltests( const char *name, const char *in, const char *out, int add_tuid )
 {
-   tests( name, in, out, add_tuid, FULL, REGULAR );
+   tests( name, in, out, add_tuid, FULL, REGULAR, OK_HEADER );
+}
+
+static void
+fulltests_ih( const char *name, const char *in, const char *out, int add_tuid )
+{
+   tests( name, in, out, add_tuid, FULL, REGULAR, PARTIAL_HEADER );
 }
 
 static void
 mintests( const char *name, const char *in, const char *out, int flagged )
 {
-   tests( name, in, out, ADD_TUID, MINIMAL, flagged );
+   tests( name, in, out, ADD_TUID, MINIMAL, flagged, OK_HEADER );
+}
+
+static void
+mintests_ih( const char *name, const char *in, const char *out, int flagged )
+{
+   tests( name, in, out, ADD_TUID, MINIMAL, flagged, PARTIAL_HEADER );
 }
 
 #define FROM "From: de\rvil\r\n"
@@ -254,5 +274,33 @@ main( void )
scc in_from_tuid_to_subj_b3[] = FROM IN_TUID TO R_SUBJECT;
mintests( "from / tuid / to / subject w/o crlf", 
in_from_tuid_to_subj_b3, out_from_tuid_to_subj_b1, REGULAR );
 
+   scc in_to_b1[] = R_TO "\r";
+   fulltests_ih( "to w/o lf", in_to_b1, in_to_b1, AS_IS );
+   scc out_to_b1[] = TO OUT_TUID "\r";
+   fulltests_ih( "to w/o lf", in_to_b1, out_to_b1, ADD_TUID );
+   scc out_to_b1_ph[] = TO OUT_TUID NO_SUBJECT PH_BODY;
+   mintests_ih( "to w/o lf", in_to_b1, out_to_b1_ph, REGULAR );
+
+   scc in_to_b2[] = R_TO;
+   fulltests_ih( "to w/o crlf", in_to_b2, in_to_b2, AS_IS );
+   scc out_to_b2[] = TO OUT_TUID;
+   fulltests_ih( "to w/o crlf", in_to_b2, out_to_b2, ADD_TUID );
+   scc out_to_b2_ph[] = TO OUT_TUID NO_SUBJECT PH_BODY;
+   mintests_ih( "to w/o crlf", in_to_b2, out_to_b2_ph, REGULAR );
+
+   scc in_no_hdr[] = BODY;
+   fulltests( "no header", in_no_hdr, in_no_hdr, AS_IS );
+   scc out_no_hdr[] = OUT_TUID BODY;
+   fulltests( "no header", in_no_hdr, out_no_hdr, ADD_TUID );
+   scc out_no_hdr_ph[] = OUT_TUID NO_SUBJECT PH_BODY;
+   mintests( "no header", in_no_hdr, out_no_hdr_ph, REGULAR );
+
+   scc in_empty[] = "";
+   fulltests_ih( "empty", in_empty, in_empty, AS_IS );
+   scc out_empty[] = OUT_TUID;
+   fulltests_ih( "empty", in_empty, out_empty, ADD_TUID );
+   scc out_empty_ph[] = OUT_TUID NO_SUBJECT PH_BODY;
+   mintests_ih( "empty", in_empty, out_empty_ph, REGULAR );
+
return 0;
 }


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: assume Tunnel to be secure more consistently

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 9169ee8fd8eb774642b371b58df81a783f355f8d
Author: Oswald Buddenhagen 
Date:   Wed May 18 17:11:59 2022 +0200

assume Tunnel to be secure more consistently

follows up on 27458133.

 src/drv_imap.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/drv_imap.c b/src/drv_imap.c
index c2a4ac71..f674e21f 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -2577,9 +2577,9 @@ imap_open_store_authenticate2( imap_store_t *ctx )
if (any || !strcasecmp( mech->string, cmech->string )) {
if (!strcasecmp( cmech->string, "LOGIN" )) {
 #ifdef HAVE_LIBSSL
-   if (ctx->conn.ssl || !any)
+   if (ctx->conn.ssl || 
ctx->conn.conf->tunnel || !any)
 #else
-   if (!any)
+   if (ctx->conn.conf->tunnel || !any)
 #endif
auth_login = 1;
else
@@ -2680,7 +2680,7 @@ imap_open_store_authenticate2( imap_store_t *ctx )
if (!ensure_user( srvc ) || !ensure_password( srvc ))
goto bail;
 #ifdef HAVE_LIBSSL
-   if (!ctx->conn.ssl)
+   if (!ctx->conn.ssl && !ctx->conn.conf->tunnel)
 #endif
warn( "*** IMAP Warning *** Password is being sent in 
the clear\n" );
ctx->caps = 0;


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: default to CRLF output in copy_msg_convert()

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit bfcc2d7d19acda0eb524be5037ae57549c8d202f
Author: Oswald Buddenhagen 
Date:   Tue May 17 13:16:34 2022 +0200

default to CRLF output in copy_msg_convert()

if we have nothing to go on, assume canonical line endings.

 src/sync_msg_cvt.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/sync_msg_cvt.c b/src/sync_msg_cvt.c
index 1e49cf7b..dd3b02e5 100644
--- a/src/sync_msg_cvt.c
+++ b/src/sync_msg_cvt.c
@@ -115,7 +115,7 @@ copy_msg_convert( int in_cr, int out_cr, copy_vars_t *vars )
}
break;
}
-   app_cr = out_cr && (!in_cr || hdr_crs);
+   app_cr = out_cr && (!in_cr || hdr_crs || !lines);
if (fix_tuid || fix_subj || fix_hdr)
extra += app_cr + 1;
if (end_hdr)


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: rename SSLType => TLSType

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit faec30abf47b583af841b37c9dede1ae826cc582
Author: Oswald Buddenhagen 
Date:   Mon May 16 15:58:41 2022 +0200

rename SSLType => TLSType

for consistency with TLSVersions.
the variable names are unchanged to avoid churn.

 NEWS   |  2 +-
 src/drv_imap.c | 10 +-
 src/mbsync.1   |  8 
 3 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/NEWS b/NEWS
index 58383b84..0575d7c1 100644
--- a/NEWS
+++ b/NEWS
@@ -13,7 +13,7 @@ Renamed the ReNew/--renew/-N options to Upgrade/--upgrade/-u
 and Delete/--delete/-d to Gone/--gone/-g.
 
 Superseded SSLVersions option with TLSVersions, and disabled TLS v1.0
-and v1.1 by default.
+and v1.1 by default. Renamed SSLType option to TLSType.
 
 Made the Channel side to expire with MaxMessages configurable.
 
diff --git a/src/drv_imap.c b/src/drv_imap.c
index 4a9c669b..0b3a0705 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -3786,7 +3786,15 @@ imap_parse_store( conffile_t *cfg, store_conf_t **storep 
)
}
} else if (!strcasecmp( "CipherString", cfg->cmd )) {
server->sconf.cipher_string = nfstrdup( cfg->val );
+   } else if (!strcasecmp( "TLSType", cfg->cmd )) {
+   goto tlstype;
} else if (!strcasecmp( "SSLType", cfg->cmd )) {
+   static int sslt_warned;
+   if (!sslt_warned) {
+   sslt_warned = 1;
+   warn( "Notice: SSLType is deprecated. Use 
TLSType instead.\n" );
+   }
+ tlstype:
if (!strcasecmp( "None", cfg->val )) {
server->ssl_type = SSL_None;
} else if (!strcasecmp( "STARTTLS", cfg->val )) {
@@ -3794,7 +3802,7 @@ imap_parse_store( conffile_t *cfg, store_conf_t **storep )
} else if (!strcasecmp( "IMAPS", cfg->val )) {
server->ssl_type = SSL_IMAPS;
} else {
-   error( "%s:%d: Invalid SSL type\n", cfg->file, 
cfg->line );
+   error( "%s:%d: Invalid TLS type\n", cfg->file, 
cfg->line );
cfg->err = 1;
}
} else if (!strcasecmp( "TLSVersions", cfg->cmd )) {
diff --git a/src/mbsync.1 b/src/mbsync.1
index 518f1109..8a8b01f9 100644
--- a/src/mbsync.1
+++ b/src/mbsync.1
@@ -313,7 +313,7 @@ Define the IMAP4 Account \fIname\fR, opening a section for 
its parameters.
 \fBHost\fR \fIhost\fR
 Specify the DNS name or IP address of the IMAP server.
 .br
-If \fBTunnel\fR is used, this setting is needed only if \fBSSLType\fR is
+If \fBTunnel\fR is used, this setting is needed only if \fBTLSType\fR is
 not \fBNone\fR and \fBCertificateFile\fR is not used,
 in which case the host name is used for certificate subject verification.
 .
@@ -394,13 +394,13 @@ The list of acceptable authentication mechanisms.
 In addition to the mechanisms listed in the SASL registry (link below),
 the legacy IMAP \fBLOGIN\fR mechanism is known.
 The wildcard \fB*\fR represents all mechanisms that are deemed secure
-enough for the current \fBSSLType\fR setting.
+enough for the current \fBTLSType\fR setting.
 The actually used mechanism is the most secure choice from the intersection
 of this list, the list supplied by the server, and the installed SASL modules.
 (Default: \fB*\fR)
 .
 .TP
-\fBSSLType\fR {\fBNone\fR|\fBSTARTTLS\fR|\fBIMAPS\fR}
+\fBTLSType\fR {\fBNone\fR|\fBSTARTTLS\fR|\fBIMAPS\fR}
 Select the connection security/encryption method:
 .br
 \fBNone\fR - no security.
@@ -410,7 +410,7 @@ This is the default when \fBTunnel\fR is set, as tunnels 
are usually secure.
 after connecting the regular IMAP port 143. Most servers support this,
 so it is the default (unless a tunnel is used).
 .br
-\fBIMAPS\fR - security is established by starting SSL/TLS negotiation
+\fBIMAPS\fR - security is established by starting TLS negotiation
 right after connecting the secure IMAP port 993.
 .
 .TP


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: recognize options even if the build does not support them

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 6dc9312dbcc47643b41c4089931e1528d3e927c9
Author: Oswald Buddenhagen 
Date:   Mon May 16 16:51:54 2022 +0200

recognize options even if the build does not support them

there were several cases of confused users who failed to notice that
they built without OpenSSL and wondered what is wrong with their
config.

 src/drv_imap.c| 65 +--
 src/drv_maildir.c |  7 -
 2 files changed, 52 insertions(+), 20 deletions(-)

diff --git a/src/drv_imap.c b/src/drv_imap.c
index 0b3a0705..c2a4ac71 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -3786,25 +3786,6 @@ imap_parse_store( conffile_t *cfg, store_conf_t **storep 
)
}
} else if (!strcasecmp( "CipherString", cfg->cmd )) {
server->sconf.cipher_string = nfstrdup( cfg->val );
-   } else if (!strcasecmp( "TLSType", cfg->cmd )) {
-   goto tlstype;
-   } else if (!strcasecmp( "SSLType", cfg->cmd )) {
-   static int sslt_warned;
-   if (!sslt_warned) {
-   sslt_warned = 1;
-   warn( "Notice: SSLType is deprecated. Use 
TLSType instead.\n" );
-   }
- tlstype:
-   if (!strcasecmp( "None", cfg->val )) {
-   server->ssl_type = SSL_None;
-   } else if (!strcasecmp( "STARTTLS", cfg->val )) {
-   server->ssl_type = SSL_STARTTLS;
-   } else if (!strcasecmp( "IMAPS", cfg->val )) {
-   server->ssl_type = SSL_IMAPS;
-   } else {
-   error( "%s:%d: Invalid TLS type\n", cfg->file, 
cfg->line );
-   cfg->err = 1;
-   }
} else if (!strcasecmp( "TLSVersions", cfg->cmd )) {
arg = cfg->val;
do {
@@ -3863,7 +3844,44 @@ imap_parse_store( conffile_t *cfg, store_conf_t **storep 
)
cfg->err = 1;
}
} while ((arg = get_arg( cfg, ARG_OPTIONAL, NULL )));
+#else
+   } else if (!strcasecmp( "CertificateFile", cfg->cmd ) ||
+  !strcasecmp( "SystemCertificates", cfg->cmd ) ||
+  !strcasecmp( "ClientCertificate", cfg->cmd ) ||
+  !strcasecmp( "ClientKey", cfg->cmd ) ||
+  !strcasecmp( "CipherString", cfg->cmd ) ||
+  !strcasecmp( "SSLVersion", cfg->cmd ) ||
+  !strcasecmp( "SSLVersions", cfg->cmd ) ||
+  !strcasecmp( "TLSVersions", cfg->cmd )) {
+   error( "Error: " EXE " built without OpenSSL; %s is not 
supported.\n", cfg->cmd );
+   cfg->err = 1;
 #endif
+   } else if (!strcasecmp( "TLSType", cfg->cmd )) {
+   goto tlstype;
+   } else if (!strcasecmp( "SSLType", cfg->cmd )) {
+   static int sslt_warned;
+   if (!sslt_warned) {
+   sslt_warned = 1;
+   warn( "Notice: SSLType is deprecated. Use 
TLSType instead.\n" );
+   }
+ tlstype:
+   if (!strcasecmp( "None", cfg->val )) {
+#ifdef HAVE_LIBSSL
+   server->ssl_type = SSL_None;
+   } else if (!strcasecmp( "STARTTLS", cfg->val )) {
+   server->ssl_type = SSL_STARTTLS;
+   } else if (!strcasecmp( "IMAPS", cfg->val )) {
+   server->ssl_type = SSL_IMAPS;
+#else
+   } else if (!strcasecmp( "STARTTLS", cfg->val ) ||
+  !strcasecmp( "IMAPS", cfg->val )) {
+   error( "Error: " EXE " built without OpenSSL; 
only TLSType None is supported.\n" );
+   cfg->err = 1;
+#endif
+   } else {
+   error( "%s:%d: Invalid TLS type\n", cfg->file, 
cfg->line );
+   cfg->err = 1;
+   }
} else if (!strcasecmp( "AuthMech", cfg->cmd ) ||
 !strcasecmp( "AuthMechs", cfg->cmd )) {
arg = cfg->val;
@@ -3948,6 +3966,15 @@ imap_parse_store( conffile_t *cfg, store_conf_t **storep 
)
if (require_cram)
add_string_list(>auth_mechs, 
"CRAM-MD5");
}
+#ifndef HAVE_LIBSASL
+   for (string_list_t *mech = server->auth_mechs; mech; mech = 
mech->next) {
+   if (strcmp( mech->string, "*" ) && 

[commit] master: disable TLS 1.0 & 1.1 by default

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit a7f1b8647562d3df8b1173c0140c145da9147a73
Author: Oswald Buddenhagen 
Date:   Mon May 16 16:26:15 2022 +0200

disable TLS 1.0 & 1.1 by default

these were not reasonable defaults any more.

 NEWS   | 3 ++-
 src/drv_imap.c | 2 +-
 src/mbsync.1   | 2 +-
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/NEWS b/NEWS
index cbaee113..58383b84 100644
--- a/NEWS
+++ b/NEWS
@@ -12,7 +12,8 @@ they are flagged on the source side.
 Renamed the ReNew/--renew/-N options to Upgrade/--upgrade/-u
 and Delete/--delete/-d to Gone/--gone/-g.
 
-Superseded SSLVersions option with TLSVersions.
+Superseded SSLVersions option with TLSVersions, and disabled TLS v1.0
+and v1.1 by default.
 
 Made the Channel side to expire with MaxMessages configurable.
 
diff --git a/src/drv_imap.c b/src/drv_imap.c
index da988f63..4a9c669b 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -3711,7 +3711,7 @@ imap_parse_store( conffile_t *cfg, store_conf_t **storep )
server->sconf.timeout = 2;
 #ifdef HAVE_LIBSSL
server->ssl_type = -1;
-   server->sconf.ssl_versions = TLSv1 | TLSv1_1 | TLSv1_2 | TLSv1_3;
+   server->sconf.ssl_versions = TLSv1_2 | TLSv1_3;
server->sconf.system_certs = 1;
 #endif
server->max_in_progress = INT_MAX;
diff --git a/src/mbsync.1 b/src/mbsync.1
index 11bda41a..518f1109 100644
--- a/src/mbsync.1
+++ b/src/mbsync.1
@@ -419,7 +419,7 @@ Add/remove the specified TLS versions to/from the set of 
acceptable choices.
 Use old versions only when the server has problems with newer ones.
 Note that new versions are automatically enabled as soon as OpenSSL supports
 them, even if \fBmbsync\fR does not recognize them yet.
-(Default: All starting with 1.0).
+(Default: All starting with 1.2).
 .
 .TP
 \fBSystemCertificates\fR \fByes\fR|\fBno\fR


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: handle copying/trashing failures more carefully

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit f7458a96d30cef20e8b4cb85b622d4eabaee41ae
Author: Oswald Buddenhagen 
Date:   Mon May 16 12:01:18 2022 +0200

handle copying/trashing failures more carefully

report them as errors (not warnings), let them cause a non-zero exit
code, and in the case of trashing, prevent the subsequent expunge. the
exception are messages that just disappeared below our feet.

 src/sync.c | 56 ++
 1 file changed, 44 insertions(+), 12 deletions(-)

diff --git a/src/sync.c b/src/sync.c
index c7cbe11e..598c4337 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -57,6 +57,7 @@ BIT_ENUM(
ST_FIND_NEW,
ST_FOUND_NEW,
ST_SENT_TRASH,
+   ST_TRASH_BAD,
ST_CLOSING,
ST_CLOSED,
ST_SENT_CANCEL,
@@ -82,6 +83,7 @@ sanitize_flags( uchar tflags, sync_vars_t *svars, int t )
 
 enum {
COPY_OK,
+   COPY_GONE,
COPY_NOGOOD,
COPY_CANCELED,
COPY_FAIL,
@@ -144,7 +146,8 @@ msg_fetched( int sts, void *aux )
if (srec || scr != tcr) {
const char *err;
if ((err = copy_msg_convert( scr, tcr, vars ))) {
-   warn( "Warning: message %u from %s %s; 
skipping.\n", vars->msg->uid, str_fn[t^1], err );
+   error( "Error: message %u from %s %s; 
skipping.\n", vars->msg->uid, str_fn[t^1], err );
+   svars->ret |= SYNC_FAIL;
vars->cb( COPY_NOGOOD, 0, vars );
return;
}
@@ -156,7 +159,15 @@ msg_fetched( int sts, void *aux )
vars->cb( COPY_CANCELED, 0, vars );
break;
case DRV_MSG_BAD:
-   vars->cb( COPY_NOGOOD, 0, vars );
+   if (vars->msg->status & M_DEAD) {
+   // The message was expunged under our feet; this is no 
error.
+   vars->cb( COPY_GONE, 0, vars );
+   } else {
+   INIT_SVARS(vars->aux);
+   // Driver already reported error.
+   svars->ret |= SYNC_FAIL;
+   vars->cb( COPY_NOGOOD, 0, vars );
+   }
break;
default:  // DRV_BOX_BAD
vars->cb( COPY_FAIL, 0, vars );
@@ -179,9 +190,10 @@ msg_stored( int sts, uint uid, void *aux )
break;
case DRV_MSG_BAD:
INIT_SVARS(vars->aux);
-   (void)svars;
-   warn( "Warning: %s refuses to store message %u from %s.\n",
- str_fn[t], vars->msg->uid, str_fn[t^1] );
+   // Driver already reported error, but we still need to report 
the source.
+   error( "Error: %s refuses to store message %u from %s.\n",
+  str_fn[t], vars->msg->uid, str_fn[t^1] );
+   svars->ret |= SYNC_FAIL;
vars->cb( COPY_NOGOOD, 0, vars );
break;
default:  // DRV_BOX_BAD
@@ -1404,6 +1416,7 @@ msg_copied( int sts, uint uid, copy_vars_t *vars )
ASSIGN_UID( srec, t, uid, "%sed message", str_hl[t] );
break;
case COPY_NOGOOD:
+   case COPY_GONE:
srec->status = S_DEAD;
JLOG( "- %u %u", (srec->uid[F], srec->uid[N]), "%s failed", 
str_hl[t] );
break;
@@ -1715,10 +1728,22 @@ msgs_flags_set( sync_vars_t *svars, int t )
 static void
 msg_trashed( int sts, void *aux )
 {
-   if (sts == DRV_MSG_BAD)
-   sts = DRV_BOX_BAD;
SVARS_CHECK_RET_VARS(trash_vars_t);
-   JLOG( "T %d %u", (t, vars->msg->uid), "trashed on %s", str_fn[t] );
+   switch (sts) {
+   case DRV_OK:
+   JLOG( "T %d %u", (t, vars->msg->uid), "trashed on %s", 
str_fn[t] );
+   break;
+   case DRV_MSG_BAD:
+   if (vars->msg->status & M_DEAD)
+   break;
+   // Driver already reported error.
+   svars->ret |= SYNC_FAIL;
+   if (svars->opts[t] & OPEN_UID_EXPUNGE)
+   vars->msg->status &= ~M_EXPUNGE;
+   else
+   svars->state[t] |= ST_TRASH_BAD;
+   break;
+   }
free( vars );
trash_done[t]++;
stats();
@@ -1732,9 +1757,18 @@ static void
 msg_rtrashed( int sts, uint uid ATTR_UNUSED, copy_vars_t *vars )
 {
DECL_INIT_SVARS(vars->aux);
+   t ^= 1;
switch (sts) {
case COPY_OK:
-   case COPY_NOGOOD: /* the message is gone or heavily busted */
+   JLOG( "T %d %u", (t, vars->msg->uid), "trashed remotely on %s", 
str_fn[t^1] );
+   break;
+   case COPY_GONE:
+   break;
+   case COPY_NOGOOD:
+   if (svars->opts[t] & OPEN_UID_EXPUNGE)
+   vars->msg->status &= ~M_EXPUNGE;
+   else
+  

[commit] master: prune legacy SSL options

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 7e0e14a686097975aee16d5439400223a287b76c
Author: Oswald Buddenhagen 
Date:   Mon May 16 12:56:46 2022 +0200

prune legacy SSL options

they have been deprecated since v1.2.

 src/drv_imap.c | 76 +++---
 1 file changed, 4 insertions(+), 72 deletions(-)

diff --git a/src/drv_imap.c b/src/drv_imap.c
index e1024e82..8af3e5ac 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -3684,11 +3684,6 @@ imap_parse_store( conffile_t *cfg, store_conf_t **storep 
)
const char *type, *name, *arg;
unsigned u;
int acc_opt = 0;
-#ifdef HAVE_LIBSSL
-   /* Legacy SSL options */
-   int require_ssl = -1, use_imaps = -1;
-   int use_tlsv1 = -1, use_tlsv11 = -1, use_tlsv12 = -1, use_tlsv13 = -1;
-#endif
/* Legacy SASL option */
int require_cram = -1;
 
@@ -3716,30 +3711,14 @@ imap_parse_store( conffile_t *cfg, store_conf_t 
**storep )
server->sconf.timeout = 2;
 #ifdef HAVE_LIBSSL
server->ssl_type = -1;
-   server->sconf.ssl_versions = -1;
+   server->sconf.ssl_versions = TLSv1 | TLSv1_1 | TLSv1_2 | TLSv1_3;
server->sconf.system_certs = 1;
 #endif
server->max_in_progress = INT_MAX;
 
while (getcline( cfg ) && cfg->cmd) {
if (!strcasecmp( "Host", cfg->cmd )) {
-   /* The imap[s]: syntax is just a backwards compat hack. 
*/
-   arg = cfg->val;
-   if (starts_with( arg, -1, "imap:", 5 ))
-   arg += 5;
-#ifdef HAVE_LIBSSL
-   else if (starts_with( arg, -1, "imaps:", 6 )) {
-   arg += 6;
-   server->ssl_type = SSL_IMAPS;
-   if (server->sconf.ssl_versions == -1)
-   server->sconf.ssl_versions = TLSv1 | 
TLSv1_1 | TLSv1_2 | TLSv1_3;
-   }
-#endif
-   if (starts_with( arg, -1, "//", 2 ))
-   arg += 2;
-   if (arg != cfg->val)
-   warn( "%s:%d: Notice: URL notation is 
deprecated; use a plain host name and possibly 'SSLType IMAPS' instead\n", 
cfg->file, cfg->line );
-   server->sconf.host = nfstrdup( arg );
+   server->sconf.host = nfstrdup( cfg->val );
} else if (!strcasecmp( "User", cfg->cmd )) {
server->user = nfstrdup( cfg->val );
} else if (!strcasecmp( "UserCmd", cfg->cmd )) {
@@ -3840,22 +3819,6 @@ imap_parse_store( conffile_t *cfg, store_conf_t **storep 
)
cfg->err = 1;
}
} while ((arg = get_arg( cfg, ARG_OPTIONAL, NULL )));
-   } else if (!strcasecmp( "RequireSSL", cfg->cmd )) {
-   require_ssl = parse_bool( cfg );
-   } else if (!strcasecmp( "UseIMAPS", cfg->cmd )) {
-   use_imaps = parse_bool( cfg );
-   } else if (!strcasecmp( "UseSSLv2", cfg->cmd )) {
-   warn( "Warning: UseSSLv2 is no longer supported\n" );
-   } else if (!strcasecmp( "UseSSLv3", cfg->cmd )) {
-   warn( "Warning: UseSSLv3 is no longer supported\n" );
-   } else if (!strcasecmp( "UseTLSv1", cfg->cmd )) {
-   use_tlsv1 = parse_bool( cfg );
-   } else if (!strcasecmp( "UseTLSv1.1", cfg->cmd )) {
-   use_tlsv11 = parse_bool( cfg );
-   } else if (!strcasecmp( "UseTLSv1.2", cfg->cmd )) {
-   use_tlsv12 = parse_bool( cfg );
-   } else if (!strcasecmp( "UseTLSv1.3", cfg->cmd )) {
-   use_tlsv13 = parse_bool( cfg );
 #endif
} else if (!strcasecmp( "AuthMech", cfg->cmd ) ||
 !strcasecmp( "AuthMechs", cfg->cmd )) {
@@ -3928,39 +3891,8 @@ imap_parse_store( conffile_t *cfg, store_conf_t **storep 
)
}
 #endif
 #ifdef HAVE_LIBSSL
-   if ((use_tlsv1 & use_tlsv11 & use_tlsv12 & use_tlsv13) != -1 || 
use_imaps >= 0 || require_ssl >= 0) {
-   if (server->ssl_type >= 0 || server->sconf.ssl_versions 
>= 0) {
-   error( "%s '%s': The deprecated UseSSL*, 
UseTLS*, UseIMAPS, and RequireSSL options are mutually exclusive with SSLType 
and SSLVersions.\n", type, name );
-   cfg->err = 1;
-   return 1;
-   }
-   warn( "Notice: %s '%s': UseSSL*, UseTLS*, UseIMAPS, and 
RequireSSL are deprecated. Use SSLType and SSLVersions instead.\n", type, name 
);
-   server->sconf.ssl_versions =
-   (use_tlsv1 == 0 ? 0 : TLSv1) |
-   

[commit] master: supersede SSLVersions option with TLSVersions

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit e3056b26e95e3eef7bb3034cedf68453bdce3887
Author: Oswald Buddenhagen 
Date:   Mon May 16 15:57:03 2022 +0200

supersede SSLVersions option with TLSVersions

the new one isn't an exhaustive set anymore, but a delta to the default
(or previous setting). this has the advantage that new versions aren't
automatically disabled anymore as soon as mbsync learns about them if
the user has a setting they fail to adjust.

 NEWS|  2 ++
 src/drv_imap.c  | 36 
 src/mbsync.1|  8 +---
 src/mbsyncrc.sample |  4 ++--
 4 files changed, 45 insertions(+), 5 deletions(-)

diff --git a/NEWS b/NEWS
index a1857c7d..cbaee113 100644
--- a/NEWS
+++ b/NEWS
@@ -12,6 +12,8 @@ they are flagged on the source side.
 Renamed the ReNew/--renew/-N options to Upgrade/--upgrade/-u
 and Delete/--delete/-d to Gone/--gone/-g.
 
+Superseded SSLVersions option with TLSVersions.
+
 Made the Channel side to expire with MaxMessages configurable.
 
 MaxMessages and MaxSize can be used together now.
diff --git a/src/drv_imap.c b/src/drv_imap.c
index 8af3e5ac..da988f63 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -3797,8 +3797,44 @@ imap_parse_store( conffile_t *cfg, store_conf_t **storep 
)
error( "%s:%d: Invalid SSL type\n", cfg->file, 
cfg->line );
cfg->err = 1;
}
+   } else if (!strcasecmp( "TLSVersions", cfg->cmd )) {
+   arg = cfg->val;
+   do {
+   int or_mask = 0, and_mask = 0, val;
+   if (*arg == '+') {
+   or_mask = ~0;
+   } else if (*arg == '-') {
+   and_mask = ~0;
+   } else {
+   error( "%s:%d: TLSVersions arguments 
must start with +/-\n", cfg->file, cfg->line );
+   cfg->err = 1;
+   continue;
+   }
+   arg++;
+   if (!strcmp( "1.0", arg )) {
+   val = TLSv1;
+   } else if (!strcmp( "1.1", arg )) {
+   val = TLSv1_1;
+   } else if (!strcmp( "1.2", arg )) {
+   val = TLSv1_2;
+   } else if (!strcmp( "1.3", arg )) {
+   val = TLSv1_3;
+   } else {
+   error( "%s:%d: Unrecognized TLS version 
'%s'\n", cfg->file, cfg->line, arg );
+   cfg->err = 1;
+   continue;
+   }
+   or_mask &= val;
+   and_mask &= val;
+   server->sconf.ssl_versions = 
(server->sconf.ssl_versions & ~and_mask) | or_mask;
+   } while ((arg = get_arg( cfg, ARG_OPTIONAL, NULL )));
} else if (!strcasecmp( "SSLVersion", cfg->cmd ) ||
   !strcasecmp( "SSLVersions", cfg->cmd )) {
+   static int sslv_warned;
+   if (!sslv_warned) {
+   sslv_warned = 1;
+   warn( "Notice: SSLVersions is deprecated. Use 
TLSVersions instead.\n" );
+   }
server->sconf.ssl_versions = 0;
arg = cfg->val;
do {
diff --git a/src/mbsync.1 b/src/mbsync.1
index b2180114..11bda41a 100644
--- a/src/mbsync.1
+++ b/src/mbsync.1
@@ -414,10 +414,12 @@ so it is the default (unless a tunnel is used).
 right after connecting the secure IMAP port 993.
 .
 .TP
-\fBSSLVersions\fR [\fBSSLv3\fR] [\fBTLSv1\fR] [\fBTLSv1.1\fR] [\fBTLSv1.2\fR] 
[\fBTLSv1.3\fR]
-Select the acceptable SSL/TLS versions.
+\fBTLSVersions\fR {\fB+\fR|\fB-\fR}{\fB1.0\fR|\fB1.1\fR|\fB1.2\fR|\fB1.3\fR} 
...
+Add/remove the specified TLS versions to/from the set of acceptable choices.
 Use old versions only when the server has problems with newer ones.
-(Default: [\fBTLSv1\fR] [\fBTLSv1.1\fR] [\fBTLSv1.2\fR] [\fBTLSv1.3\fR]).
+Note that new versions are automatically enabled as soon as OpenSSL supports
+them, even if \fBmbsync\fR does not recognize them yet.
+(Default: All starting with 1.0).
 .
 .TP
 \fBSystemCertificates\fR \fByes\fR|\fBno\fR
diff --git a/src/mbsyncrc.sample b/src/mbsyncrc.sample
index 1bc9e384..50e4981a 100644
--- a/src/mbsyncrc.sample
+++ b/src/mbsyncrc.sample
@@ -36,7 +36,7 @@ Sync PullNew Push
 IMAPStore personal
 Host host.play.com
 Port 6789
-SSLType None
+TLSType None
 
 Channel personal
 Far :personal:

[commit] master: postpone check for cancellation in some driver callbacks

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 4c141231447cba66c38f1a47768c153753fe7811
Author: Oswald Buddenhagen 
Date:   Fri May 13 13:02:35 2022 +0200

postpone check for cancellation in some driver callbacks

the purpose of these checks is preventing triggering more driver calls,
while immediate followups to already completed calls should be still
made.

note that some of the checks are strictly speaking redundant, as
chaining into a next phase will return immediately anyway when the
other side isn't ready yet. but we keep them for clarity.

 src/sync.c | 27 ++-
 1 file changed, 22 insertions(+), 5 deletions(-)

diff --git a/src/sync.c b/src/sync.c
index b6336f72..c7cbe11e 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -255,7 +255,7 @@ check_ret( int sts, void *aux )
cancel_sync( svars );
return 1;
}
-   return check_cancel( svars );
+   return 0;
 }
 
 #define SVARS_CHECK_RET \
@@ -263,6 +263,13 @@ check_ret( int sts, void *aux )
return; \
DECL_INIT_SVARS(aux)
 
+// After drv->cancel_cmds() on our side, commands may still complete
+// successfully, while the other side is already dead.
+#define SVARS_CHECK_RET_CANCEL \
+   SVARS_CHECK_RET; \
+   if (check_cancel( svars )) \
+   return
+
 #define SVARS_CHECK_RET_VARS(type) \
type *vars = (type *)aux; \
if (check_ret( sts, vars->aux )) { \
@@ -444,7 +451,7 @@ box_confirmed2( sync_vars_t *svars, int t )
 static void
 box_deleted( int sts, void *aux )
 {
-   SVARS_CHECK_RET;
+   SVARS_CHECK_RET_CANCEL;
delete_state( svars );
svars->drv[t]->finish_delete_box( svars->ctx[t] );
sync_bail( svars );
@@ -453,7 +460,7 @@ box_deleted( int sts, void *aux )
 static void
 box_created( int sts, void *aux )
 {
-   SVARS_CHECK_RET;
+   SVARS_CHECK_RET_CANCEL;
svars->drv[t]->open_box( svars->ctx[t], box_opened, AUX );
 }
 
@@ -767,7 +774,7 @@ box_loaded( int sts, message_t *msgs, int total_msgs, int 
recent_msgs, void *aux
uchar sflags, nflags, aflags, dflags;
uint hashsz, idx;
 
-   SVARS_CHECK_RET;
+   SVARS_CHECK_RET_CANCEL;
svars->state[t] |= ST_LOADED;
svars->msgs[t] = msgs;
info( "%s: %d messages, %d recent\n", str_fn[t], total_msgs, 
recent_msgs );
@@ -1411,6 +1418,8 @@ msg_copied( int sts, uint uid, copy_vars_t *vars )
new_done[t]++;
stats();
svars->new_pending[t]--;
+   if (check_cancel( svars ))
+   return;
msgs_copied( svars, t );
 }
 
@@ -1484,6 +1493,8 @@ msgs_found_new( int sts, message_t *msgs, void *aux )
int num_lost = match_tuids( svars, t, msgs );
if (num_lost)
warn( "Warning: lost track of %d %sed message(s)\n", num_lost, 
str_hl[t] );
+   if (check_cancel( svars ))
+   return;
msgs_new_done( svars, t );
 }
 
@@ -1512,6 +1523,8 @@ flags_set( int sts, void *aux )
flags_done[t]++;
stats();
svars->flags_pending[t]--;
+   if (check_cancel( svars ))
+   return;
msgs_flags_set( svars, t );
 }
 
@@ -1710,6 +1723,8 @@ msg_trashed( int sts, void *aux )
trash_done[t]++;
stats();
svars->trash_pending[t]--;
+   if (check_cancel( svars ))
+   return;
sync_close( svars, t );
 }
 
@@ -1734,6 +1749,8 @@ msg_rtrashed( int sts, uint uid ATTR_UNUSED, copy_vars_t 
*vars )
trash_done[t]++;
stats();
svars->trash_pending[t]--;
+   if (check_cancel( svars ))
+   return;
sync_close( svars, t );
 }
 
@@ -1782,7 +1799,7 @@ sync_close( sync_vars_t *svars, int t )
 static void
 box_closed( int sts, int reported, void *aux )
 {
-   SVARS_CHECK_RET;
+   SVARS_CHECK_RET_CANCEL;
if (!reported) {
for (sync_rec_t *srec = svars->srecs; srec; srec = srec->next) {
if (srec->status & S_DEAD)


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: stop abusing SYNC_* for what really should be COPY_*

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit ffb290084a41ca017305109f2464951db6ccf070
Author: Oswald Buddenhagen 
Date:   Fri May 13 11:44:59 2022 +0200

stop abusing SYNC_* for what really should be COPY_*

 src/sync.c | 39 +++
 src/sync.h |  2 --
 2 files changed, 23 insertions(+), 18 deletions(-)

diff --git a/src/sync.c b/src/sync.c
index 527d972d..4a8f8175 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -80,6 +80,13 @@ sanitize_flags( uchar tflags, sync_vars_t *svars, int t )
 }
 
 
+enum {
+   COPY_OK,
+   COPY_NOGOOD,
+   COPY_CANCELED,
+   COPY_FAIL,
+};
+
 static void msg_fetched( int sts, void *aux );
 
 static void
@@ -108,7 +115,7 @@ msg_fetched( int sts, void *aux )
INIT_SVARS(vars->aux);
if (check_cancel( svars )) {
free( vars->data.data );
-   vars->cb( SYNC_CANCELED, 0, vars );
+   vars->cb( COPY_CANCELED, 0, vars );
return;
}
 
@@ -138,7 +145,7 @@ msg_fetched( int sts, void *aux )
const char *err;
if ((err = copy_msg_convert( scr, tcr, vars ))) {
warn( "Warning: message %u from %s %s; 
skipping.\n", vars->msg->uid, str_fn[t^1], err );
-   vars->cb( SYNC_NOGOOD, 0, vars );
+   vars->cb( COPY_NOGOOD, 0, vars );
return;
}
}
@@ -146,13 +153,13 @@ msg_fetched( int sts, void *aux )
svars->drv[t]->store_msg( svars->ctx[t], >data, !srec, 
msg_stored, vars );
break;
case DRV_CANCELED:
-   vars->cb( SYNC_CANCELED, 0, vars );
+   vars->cb( COPY_CANCELED, 0, vars );
break;
case DRV_MSG_BAD:
-   vars->cb( SYNC_NOGOOD, 0, vars );
+   vars->cb( COPY_NOGOOD, 0, vars );
break;
default:  // DRV_BOX_BAD
-   vars->cb( SYNC_FAIL, 0, vars );
+   vars->cb( COPY_FAIL, 0, vars );
break;
}
 }
@@ -165,20 +172,20 @@ msg_stored( int sts, uint uid, void *aux )
 
switch (sts) {
case DRV_OK:
-   vars->cb( SYNC_OK, uid, vars );
+   vars->cb( COPY_OK, uid, vars );
break;
case DRV_CANCELED:
-   vars->cb( SYNC_CANCELED, 0, vars );
+   vars->cb( COPY_CANCELED, 0, vars );
break;
case DRV_MSG_BAD:
INIT_SVARS(vars->aux);
(void)svars;
warn( "Warning: %s refuses to store message %u from %s.\n",
  str_fn[t], vars->msg->uid, str_fn[t^1] );
-   vars->cb( SYNC_NOGOOD, 0, vars );
+   vars->cb( COPY_NOGOOD, 0, vars );
break;
default:  // DRV_BOX_BAD
-   vars->cb( SYNC_FAIL, 0, vars );
+   vars->cb( COPY_FAIL, 0, vars );
break;
}
 }
@@ -265,7 +272,7 @@ check_ret( int sts, void *aux )
DECL_INIT_SVARS(vars->aux)
 
 #define SVARS_CHECK_CANCEL_RET \
-   if (sts == SYNC_CANCELED) { \
+   if (sts == COPY_CANCELED) { \
free( vars ); \
return; \
} \
@@ -1390,17 +1397,17 @@ msg_copied( int sts, uint uid, copy_vars_t *vars )
SVARS_CHECK_CANCEL_RET;
sync_rec_t *srec = vars->srec;
switch (sts) {
-   case SYNC_OK:
+   case COPY_OK:
if (!uid)  // Stored to a non-UIDPLUS mailbox
svars->state[t] |= ST_FIND_NEW;
else
ASSIGN_UID( srec, t, uid, "%sed message", str_hl[t] );
break;
-   case SYNC_NOGOOD:
+   case COPY_NOGOOD:
srec->status = S_DEAD;
JLOG( "- %u %u", (srec->uid[F], srec->uid[N]), "%s failed", 
str_hl[t] );
break;
-   default:
+   default:  // COPY_FAIL
cancel_sync( svars );
free( vars );
return;
@@ -1716,10 +1723,10 @@ msg_rtrashed( int sts, uint uid ATTR_UNUSED, 
copy_vars_t *vars )
 {
SVARS_CHECK_CANCEL_RET;
switch (sts) {
-   case SYNC_OK:
-   case SYNC_NOGOOD: /* the message is gone or heavily busted */
+   case COPY_OK:
+   case COPY_NOGOOD: /* the message is gone or heavily busted */
break;
-   default:
+   default:  // COPY_FAIL
cancel_sync( svars );
free( vars );
return;
diff --git a/src/sync.h b/src/sync.h
index 6bca0761..ced07e7c 100644
--- a/src/sync.h
+++ b/src/sync.h
@@ -84,8 +84,6 @@ extern const char *str_fn[2], *str_hl[2];
 #define SYNC_OK   0 /* assumed to be 0 */
 #define SYNC_FAIL 1
 #define SYNC_BAD(fn)  (4<<(fn))
-#define SYNC_NOGOOD   16 /* internal */
-#define SYNC_CANCELED 32 /* internal */
 
 #define BOX_POSSIBLE -1
 

[commit] master: dissolve SVARS_CHECK_CANCEL_RET

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit f29dbb18f1a9469e4fc38bec081a6e70ded078c1
Author: Oswald Buddenhagen 
Date:   Fri May 13 12:14:13 2022 +0200

dissolve SVARS_CHECK_CANCEL_RET

it was small and used only twice, so didn't really pull its weight.

 src/sync.c | 15 ++-
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/src/sync.c b/src/sync.c
index 4a8f8175..b6336f72 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -271,13 +271,6 @@ check_ret( int sts, void *aux )
} \
DECL_INIT_SVARS(vars->aux)
 
-#define SVARS_CHECK_CANCEL_RET \
-   if (sts == COPY_CANCELED) { \
-   free( vars ); \
-   return; \
-   } \
-   DECL_INIT_SVARS(vars->aux)
-
 static void
 message_expunged( message_t *msg, void *aux )
 {
@@ -1394,7 +1387,7 @@ box_loaded( int sts, message_t *msgs, int total_msgs, int 
recent_msgs, void *aux
 static void
 msg_copied( int sts, uint uid, copy_vars_t *vars )
 {
-   SVARS_CHECK_CANCEL_RET;
+   DECL_INIT_SVARS(vars->aux);
sync_rec_t *srec = vars->srec;
switch (sts) {
case COPY_OK:
@@ -1409,6 +1402,8 @@ msg_copied( int sts, uint uid, copy_vars_t *vars )
break;
default:  // COPY_FAIL
cancel_sync( svars );
+   FALLTHROUGH
+   case COPY_CANCELED:
free( vars );
return;
}
@@ -1721,13 +1716,15 @@ msg_trashed( int sts, void *aux )
 static void
 msg_rtrashed( int sts, uint uid ATTR_UNUSED, copy_vars_t *vars )
 {
-   SVARS_CHECK_CANCEL_RET;
+   DECL_INIT_SVARS(vars->aux);
switch (sts) {
case COPY_OK:
case COPY_NOGOOD: /* the message is gone or heavily busted */
break;
default:  // COPY_FAIL
cancel_sync( svars );
+   FALLTHROUGH
+   case COPY_CANCELED:
free( vars );
return;
}


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: add --ext-exit option

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 4d75c45507b302d301fee5d6ae413b559f3a
Author: Oswald Buddenhagen 
Date:   Tue May 10 14:11:36 2022 +0200

add --ext-exit option

this is only a partial solution for reporting changes, but it's
adequate if the goal is merely triggering a bulk action, like
re-indexing the local store.

inspired by patches posted by Yuri D'Elia .

 NEWS| 2 ++
 TODO| 2 --
 src/common.h| 2 ++
 src/main.c  | 6 ++
 src/main_sync.c | 5 +
 src/mbsync.1| 6 ++
 src/sync.c  | 2 +-
 7 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/NEWS b/NEWS
index 0331feea..a1857c7d 100644
--- a/NEWS
+++ b/NEWS
@@ -27,6 +27,8 @@ Added support for mirroring deletions more accurately.
 
 Added --dry-run option.
 
+Added --ext-exit option.
+
 [1.4.0]
 
 The 'isync' compatibility wrapper was removed.
diff --git a/TODO b/TODO
index 10bd61f2..dacdeb4e 100644
--- a/TODO
+++ b/TODO
@@ -25,8 +25,6 @@ add alternative treatments of expired messages. 
ExpiredMessageMode: Prune
 separate folder - ArchiveSuffix, default .archive).
 
 add support for event notification callbacks.
-it would be also possible to report more differentiated exit codes, but
-that seems too limiting in the general case.
 
 make it possible to have different mailbox names for far and near side in
 Patterns.
diff --git a/src/common.h b/src/common.h
index 8e613f9d..940e74d5 100644
--- a/src/common.h
+++ b/src/common.h
@@ -115,6 +115,8 @@ BIT_ENUM(
 
DRYRUN,
 
+   EXT_EXIT,
+
ZERODELAY,
KEEPJOURNAL,
FORCEJOURNAL,
diff --git a/src/main.c b/src/main.c
index 0efd7b8e..7cfea1aa 100644
--- a/src/main.c
+++ b/src/main.c
@@ -47,6 +47,7 @@ PACKAGE " " VERSION " - mailbox synchronizer\n"
 "  -c, --config CONFIG read an alternate config file (default: ~/." EXE "rc)\n"
 "  -D, --debug debugging modes (see manual)\n"
 "  -y, --dry-run   do not actually modify anything\n"
+"  -e, --ext-exit  return extended exit code\n"
 "  -V, --verbose   display what is happening\n"
 "  -q, --quiet don't display progress counters\n"
 "  -v, --version   display version\n"
@@ -221,6 +222,8 @@ main( int argc, char **argv )
DFlags |= op;
} else if (!strcmp( opt, "dry-run" )) {
DFlags |= DRYRUN;
+   } else if (!strcmp( opt, "ext-exit" )) {
+   DFlags |= EXT_EXIT;
} else if (!strcmp( opt, "pull" )) {
cops |= XOP_PULL, mvars->ops[F] |= 
XOP_HAVE_TYPE;
} else if (!strcmp( opt, "push" )) {
@@ -461,6 +464,9 @@ main( int argc, char **argv )
case 'y':
DFlags |= DRYRUN;
break;
+   case 'e':
+   DFlags |= EXT_EXIT;
+   break;
case 'T':
for (; *ochar; ) {
switch (*ochar++) {
diff --git a/src/main_sync.c b/src/main_sync.c
index 07b28838..9c38d208 100644
--- a/src/main_sync.c
+++ b/src/main_sync.c
@@ -388,6 +388,11 @@ sync_chans( core_vars_t *cvars, char **argv )
if (!cvars->list) {
flushn();
summary();
+   if (DFlags & EXT_EXIT) {
+   for (int t = 0; t < 2; t++)
+   if (new_done[t] || flags_done[t] || 
trash_done[t] || expunge_done[t])
+   cvars->ret |= 32 << t;
+   }
}
 }
 
diff --git a/src/mbsync.1 b/src/mbsync.1
index 60141346..b2180114 100644
--- a/src/mbsync.1
+++ b/src/mbsync.1
@@ -79,6 +79,12 @@ Enter simulation mode: the Channel status is queried and all 
required
 operations are determined, but no modifications are actually made
 to either the mailboxes or the state files.
 .TP
+\fB-e\fR, \fB--ext-exit\fR
+Return an extended exit code: Add 32 resp. 64 to the code if any
+modifications were made on the far resp. near side.
+An error may be reported at the same time, so the code may be for example
+65 if some changes were successfully pulled, while others failed.
+.TP
 \fB-V\fR, \fB--verbose\fR
 Enable \fIverbose\fR mode, which displays what is currently happening.
 .TP
diff --git a/src/sync.c b/src/sync.c
index 0dd6c9e7..a588ae38 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -1776,7 +1776,7 @@ sync_close( sync_vars_t *svars, int t )
 
if ((svars->chan->ops[t] & (OP_EXPUNGE | OP_EXPUNGE_SOLO)) && !(DFlags 
& FAKEEXPUNGE)
/*&& !(svars->state[t] & ST_TRASH_BAD)*/) {
-   if (Verbosity >= TERSE) {
+   if (Verbosity >= TERSE || (DFlags & EXT_EXIT)) {
if (svars->opts[t] & OPEN_UID_EXPUNGE) {
for (message_t *tmsg = 

[commit] master: prefer DECL_INIT_SVARS over DECL_SVARS+INIT_SVARS

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 9e0efd409a3cc9c0ce45e55702234f04bec3bef1
Author: Oswald Buddenhagen 
Date:   Wed May 11 12:56:16 2022 +0200

prefer DECL_INIT_SVARS over DECL_SVARS+INIT_SVARS

... as a result of relying on C99+.

 src/sync.c | 21 ++---
 1 file changed, 6 insertions(+), 15 deletions(-)

diff --git a/src/sync.c b/src/sync.c
index 286564ae..527d972d 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -240,11 +240,9 @@ check_cancel( sync_vars_t *svars )
 static int
 check_ret( int sts, void *aux )
 {
-   DECL_SVARS;
-
if (sts == DRV_CANCELED)
return 1;
-   INIT_SVARS(aux);
+   DECL_INIT_SVARS(aux);
if (sts == DRV_BOX_BAD) {
svars->ret |= SYNC_FAIL;
cancel_sync( svars );
@@ -254,27 +252,24 @@ check_ret( int sts, void *aux )
 }
 
 #define SVARS_CHECK_RET \
-   DECL_SVARS; \
if (check_ret( sts, aux )) \
return; \
-   INIT_SVARS(aux)
+   DECL_INIT_SVARS(aux)
 
 #define SVARS_CHECK_RET_VARS(type) \
type *vars = (type *)aux; \
-   DECL_SVARS; \
if (check_ret( sts, vars->aux )) { \
free( vars ); \
return; \
} \
-   INIT_SVARS(vars->aux)
+   DECL_INIT_SVARS(vars->aux)
 
 #define SVARS_CHECK_CANCEL_RET \
-   DECL_SVARS; \
if (sts == SYNC_CANCELED) { \
free( vars ); \
return; \
} \
-   INIT_SVARS(vars->aux)
+   DECL_INIT_SVARS(vars->aux)
 
 static void
 message_expunged( message_t *msg, void *aux )
@@ -376,11 +371,9 @@ sync_boxes( store_t *ctx[], const char * const names[], 
int present[], channel_c
 static void
 box_confirmed( int sts, uint uidvalidity, void *aux )
 {
-   DECL_SVARS;
-
if (sts == DRV_CANCELED)
return;
-   INIT_SVARS(aux);
+   DECL_INIT_SVARS(aux);
if (check_cancel( svars ))
return;
 
@@ -467,11 +460,9 @@ box_created( int sts, void *aux )
 static void
 box_opened( int sts, uint uidvalidity, void *aux )
 {
-   DECL_SVARS;
-
if (sts == DRV_CANCELED)
return;
-   INIT_SVARS(aux);
+   DECL_INIT_SVARS(aux);
if (check_cancel( svars ))
return;
 


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: make expiration target side configurable

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 8566283c59a25871b42e15fa6ddbce676afb8f12
Author: Oswald Buddenhagen 
Date:   Thu May 5 20:31:43 2022 +0200

make expiration target side configurable

REFMAIL: 87k0fauw7q@wavexx.thregr.org

 NEWS |   2 +
 src/config.c |  13 ++
 src/mbsync.1 |   9 +++-
 src/run-tests.pl |  95 +-
 src/sync.c   | 118 +++
 src/sync.h   |   1 +
 src/sync_p.h |   6 +--
 src/sync_state.c |  22 +++--
 8 files changed, 199 insertions(+), 67 deletions(-)

diff --git a/NEWS b/NEWS
index bbee6112..7b4d557d 100644
--- a/NEWS
+++ b/NEWS
@@ -12,6 +12,8 @@ they are flagged on the source side.
 Renamed the ReNew/--renew/-N options to Upgrade/--upgrade/-u
 and Delete/--delete/-d to Gone/--gone/-g.
 
+Made the Channel side to expire with MaxMessages configurable.
+
 MaxMessages and MaxSize can be used together now.
 
 The unfiltered list of mailboxes in each Store can be printed now.
diff --git a/src/config.c b/src/config.c
index d7b4ff6b..07201120 100644
--- a/src/config.c
+++ b/src/config.c
@@ -261,6 +261,17 @@ getopt_helper( conffile_t *cfile, int *cops, 
channel_conf_t *conf )
conf->use_internal_date = parse_bool( cfile );
} else if (!strcasecmp( "MaxMessages", cfile->cmd )) {
conf->max_messages = parse_int( cfile );
+   } else if (!strcasecmp( "ExpireSide", cfile->cmd )) {
+   arg = cfile->val;
+   if (!strcasecmp( "Far", arg )) {
+   conf->expire_side = F;
+   } else if (!strcasecmp( "Near", arg )) {
+   conf->expire_side = N;
+   } else {
+   error( "%s:%d: invalid ExpireSide argument '%s'\n",
+  cfile->file, cfile->line, arg );
+   cfile->err = 1;
+   }
} else if (!strcasecmp( "ExpireUnread", cfile->cmd )) {
conf->expire_unread = parse_bool( cfile );
} else {
@@ -481,6 +492,7 @@ load_config( const char *where )
 
gcops = 0;
glob_ok = 1;
+   global_conf.expire_side = N;
global_conf.expire_unread = -1;
   reloop:
while (getcline(  )) {
@@ -505,6 +517,7 @@ load_config( const char *where )
channel = nfzalloc( sizeof(*channel) );
channel->name = nfstrdup( cfile.val );
channel->max_messages = global_conf.max_messages;
+   channel->expire_side = global_conf.expire_side;
channel->expire_unread = global_conf.expire_unread;
channel->use_internal_date = 
global_conf.use_internal_date;
cops = 0;
diff --git a/src/mbsync.1 b/src/mbsync.1
index 79a6afba..87672c66 100644
--- a/src/mbsync.1
+++ b/src/mbsync.1
@@ -579,6 +579,12 @@ case you need to enable this option.
 (Global default: \fBno\fR).
 .
 .TP
+\fBExpireSide\fR \fBFar\fR|\fBNear\fR
+Selects on which side messages should be expired when \fBMaxMessages\fR is
+configured.
+(Global default: \fBNear\fR).
+.
+.TP
 \fBSync\fR {\fBNone\fR|[\fBPull\fR] [\fBPush\fR] [\fBNew\fR] [\fBOld\fR] 
[\fBUpgrade\fR] [\fBGone\fR] [\fBFlags\fR] [\fBFull\fR]}
 Select the synchronization operation(s) to perform:
 .br
@@ -693,7 +699,8 @@ date\fR) is actually the arrival time, but it is usually 
close enough.
 .
 .P
 \fBSync\fR, \fBCreate\fR, \fBRemove\fR, \fBExpunge\fR, \fBExpungeSolo\fR,
-\fBMaxMessages\fR, \fBExpireUnread\fR, and \fBCopyArrivalDate\fR
+\fBMaxMessages\fR, \fBExpireUnread\fR, \fBExpireSide\fR,
+and \fBCopyArrivalDate\fR
 can be used before any section for a global effect.
 The global settings are overridden by Channel-specific options,
 which in turn are overridden by command line switches.
diff --git a/src/run-tests.pl b/src/run-tests.pl
index 9786e037..144ba946 100755
--- a/src/run-tests.pl
+++ b/src/run-tests.pl
@@ -266,12 +266,28 @@ sub parse_chan($;$)
}
 
$$ss{max_pulled} = resolv_msg($$ics[0], $cs, "far");
-   $$ss{max_expired} = resolv_msg($$ics[1], $cs, "far");
+   $$ss{max_expired_far} = resolv_msg($$ics[1], $cs, "far");
$$ss{max_pushed} = resolv_msg($$ics[2], $cs, "near");
+   $$ss{max_expired_near} = 0;
 
return $cs;
 }
 
+sub flip_chan($)
+{
+   my ($cs) = @_;
+
+   ($$cs{far}, $$cs{near}) = ($$cs{near}, $$cs{far});
+   ($$cs{far_trash}, $$cs{near_trash}) = ($$cs{near_trash}, 
$$cs{far_trash});
+   my $ss = $$cs{state};
+   ($$ss{max_pulled}, $$ss{max_pushed}) = ($$ss{max_pushed}, 
$$ss{max_pulled});
+   ($$ss{max_expired_far}, $$ss{max_expired_near}) = 
($$ss{max_expired_near}, $$ss{max_expired_far});
+   for my $ent (@{$$ss{entries}}) {
+   ($$ent[0], $$ent[1]) = ($$ent[1], $$ent[0]);
+   $$ent[2] =~ tr/<>/> 0,
-   max_expired => 0,
+   max_expired_far => 0,
max_pushed => 0,
+ 

[commit] master: make more use of SVARS_CHECK_RET{,_VARS} macro

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 6bfffa177ac72b876fd2114b83d03f1cd55a60a6
Author: Oswald Buddenhagen 
Date:   Wed May 11 12:37:14 2022 +0200

make more use of SVARS_CHECK_RET{,_VARS} macro

 src/sync.c | 26 --
 1 file changed, 4 insertions(+), 22 deletions(-)

diff --git a/src/sync.c b/src/sync.c
index a588ae38..286564ae 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -451,12 +451,7 @@ box_confirmed2( sync_vars_t *svars, int t )
 static void
 box_deleted( int sts, void *aux )
 {
-   DECL_SVARS;
-
-   if (check_ret( sts, aux ))
-   return;
-   INIT_SVARS(aux);
-
+   SVARS_CHECK_RET;
delete_state( svars );
svars->drv[t]->finish_delete_box( svars->ctx[t] );
sync_bail( svars );
@@ -465,12 +460,7 @@ box_deleted( int sts, void *aux )
 static void
 box_created( int sts, void *aux )
 {
-   DECL_SVARS;
-
-   if (check_ret( sts, aux ))
-   return;
-   INIT_SVARS(aux);
-
+   SVARS_CHECK_RET;
svars->drv[t]->open_box( svars->ctx[t], box_opened, AUX );
 }
 
@@ -779,7 +769,6 @@ static void msgs_copied( sync_vars_t *svars, int t );
 static void
 box_loaded( int sts, message_t *msgs, int total_msgs, int recent_msgs, void 
*aux )
 {
-   DECL_SVARS;
sync_rec_t *srec, **srecmap;
message_t *tmsg;
flag_vars_t *fv;
@@ -787,9 +776,7 @@ box_loaded( int sts, message_t *msgs, int total_msgs, int 
recent_msgs, void *aux
uchar sflags, nflags, aflags, dflags;
uint hashsz, idx;
 
-   if (check_ret( sts, aux ))
-   return;
-   INIT_SVARS(aux);
+   SVARS_CHECK_RET;
svars->state[t] |= ST_LOADED;
svars->msgs[t] = msgs;
info( "%s: %d messages, %d recent\n", str_fn[t], total_msgs, 
recent_msgs );
@@ -1722,14 +1709,9 @@ msgs_flags_set( sync_vars_t *svars, int t )
 static void
 msg_trashed( int sts, void *aux )
 {
-   trash_vars_t *vars = (trash_vars_t *)aux;
-   DECL_SVARS;
-
if (sts == DRV_MSG_BAD)
sts = DRV_BOX_BAD;
-   if (check_ret( sts, vars->aux ))
-   return;
-   INIT_SVARS(vars->aux);
+   SVARS_CHECK_RET_VARS(trash_vars_t);
JLOG( "T %d %u", (t, vars->msg->uid), "trashed on %s", str_fn[t] );
free( vars );
trash_done[t]++;


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: add --dry-run mode

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit abb596709b6fd4f1d020370f5f546db0d932f324
Author: Oswald Buddenhagen 
Date:   Thu May 5 18:33:59 2022 +0200

add --dry-run mode

REFMAIL: 20211130142121.xon5oygrpdfj5...@fastmail.com

 NEWS |  2 ++
 src/common.h |  2 ++
 src/drv_proxy.c  | 82 +++-
 src/drv_proxy_gen.pl | 34 ++
 src/main.c   |  6 
 src/main_sync.c  |  8 +++--
 src/mbsync.1 |  5 +++
 src/sync_state.c | 13 +++
 8 files changed, 142 insertions(+), 10 deletions(-)

diff --git a/NEWS b/NEWS
index 9faa4825..bbee6112 100644
--- a/NEWS
+++ b/NEWS
@@ -22,6 +22,8 @@ Added new sync operation 'Old'.
 
 Added support for mirroring deletions more accurately.
 
+Added --dry-run option.
+
 [1.4.0]
 
 The 'isync' compatibility wrapper was removed.
diff --git a/src/common.h b/src/common.h
index c7ad08c5..92d1e89e 100644
--- a/src/common.h
+++ b/src/common.h
@@ -113,6 +113,8 @@ BIT_ENUM(
 
PROGRESS,
 
+   DRYRUN,
+
ZERODELAY,
KEEPJOURNAL,
FORCEJOURNAL,
diff --git a/src/drv_proxy.c b/src/drv_proxy.c
index 0cca3a52..37dbc34f 100644
--- a/src/drv_proxy.c
+++ b/src/drv_proxy.c
@@ -23,6 +23,8 @@ typedef union proxy_store {
gen_cmd_t *pending_cmds, **pending_cmds_append;
gen_cmd_t *check_cmds, **check_cmds_append;
wakeup_t wakeup;
+   uint fake_nextuid;
+   char is_fake;  // Was "created" by dry-run
char force_async;
 
void (*expunge_callback)( message_t *msg, void *aux );
@@ -139,8 +141,11 @@ static @type@proxy_@name@( store_t *gctx )
 {
proxy_store_t *ctx = (proxy_store_t *)gctx;
 
-   @type@rv = ctx->real_driver->@name@( ctx->real_store );
-   debug( "%sCalled @name@, ret=@fmt@\n", ctx->label, rv );
+   @type@rv;
+   @pre_invoke@
+   @indent_invoke@rv = ctx->real_driver->@name@( ctx->real_store );
+   @post_invoke@
+   debug( "%sCalled @name@@print_fmt_dry@, ret=@fmt@\n", 
ctx->label@print_pass_dry@, rv );
return rv;
 }
 //# END
@@ -151,9 +156,12 @@ static @type@proxy_@name@( store_t *gctx@decl_args@ )
proxy_store_t *ctx = (proxy_store_t *)gctx;
 
@pre_print_args@
-   debug( "%sEnter @name@@print_fmt_args@\n", ctx->label@print_pass_args@ 
);
+   debug( "%sEnter @name@@print_fmt_dry@@print_fmt_args@\n", 
ctx->label@print_pass_dry@@print_pass_args@ );
@print_args@
-   @type@rv = ctx->real_driver->@name@( ctx->real_store@pass_args@ );
+   @type@rv;
+   @pre_invoke@
+   @indent_invoke@rv = ctx->real_driver->@name@( 
ctx->real_store@pass_args@ );
+   @post_invoke@
debug( "%sLeave @name@, ret=@print_fmt_ret@\n", ctx->label, 
@print_pass_ret@ );
return rv;
 }
@@ -165,9 +173,11 @@ static @type@proxy_@name@( store_t *gctx@decl_args@ )
proxy_store_t *ctx = (proxy_store_t *)gctx;
 
@pre_print_args@
-   debug( "%sEnter @name@@print_fmt_args@\n", ctx->label@print_pass_args@ 
);
+   debug( "%sEnter @name@@print_fmt_dry@@print_fmt_args@\n", 
ctx->label@print_pass_dry@@print_pass_args@ );
@print_args@
-   ctx->real_driver->@name@( ctx->real_store@pass_args@ );
+   @pre_invoke@
+   @indent_invoke@ctx->real_driver->@name@( ctx->real_store@pass_args@ );
+   @post_invoke@
debug( "%sLeave @name@\n", ctx->label );
@action@
 }
@@ -226,9 +236,11 @@ proxy_do_@name@( gen_cmd_t *gcmd )
proxy_store_t *ctx = cmd->ctx;
 
@pre_print_args@
-   debug( "%s[% 2d] Enter @name@@print_fmt_args@\n", ctx->label, 
cmd->tag@print_pass_args@ );
+   debug( "%s[% 2d] Enter @name@@print_fmt_dry@@print_fmt_args@\n", 
ctx->label, cmd->tag@print_pass_dry@@print_pass_args@ );
@print_args@
-   ctx->real_driver->@name@( ctx->real_store@pass_args@, proxy_@name@_cb, 
cmd );
+   @pre_invoke@
+   @indent_invoke@ctx->real_driver->@name@( ctx->real_store@pass_args@, 
proxy_@name@_cb, cmd );
+   @post_invoke@
debug( "%s[% 2d] Leave @name@\n", ctx->label, cmd->tag );
 }
 
@@ -252,10 +264,51 @@ static @type@proxy_@name@( store_t *gctx@decl_args@, void 
(*cb)( @decl_cb_args@v
debug( "  %s\n", box->string );
 //# END
 
+//# DEFINE select_box_pre_invoke
+   ctx->is_fake = 0;
+//# END
+
+//# DEFINE create_box_driable 1
+//# DEFINE create_box_fake_invoke
+   ctx->is_fake = 1;
+//# END
 //# DEFINE create_box_counted 1
 
+//# DEFINE open_box_fakeable 1
+//# DEFINE open_box_fake_invoke
+   ctx->fake_nextuid = 1;
+//# END
+//# DEFINE open_box_fake_cb_args , 1
+
+//# DEFINE get_uidnext_fakeable 1
+//# DEFINE get_uidnext_fake_invoke
+   rv = ctx->fake_nextuid;
+//# END
+//# DEFINE get_uidnext_post_real_invoke
+   ctx->fake_nextuid = rv;
+//# END
+
+//# DEFINE get_supported_flags_fakeable 1
+//# DEFINE get_supported_flags_fake_invoke
+   rv = 255;
+//# END
+
+//# DEFINE confirm_box_empty_fakeable 

[commit] master: include expunges in progress counters

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit edc901b7afc55d5ec55c4d642fbe6419e02b791c
Author: Oswald Buddenhagen 
Date:   Tue May 10 13:30:17 2022 +0200

include expunges in progress counters

 NEWS|  1 +
 src/main_sync.c | 24 
 src/mbsync.1|  4 ++--
 src/sync.c  | 29 -
 src/sync.h  |  1 +
 5 files changed, 48 insertions(+), 11 deletions(-)

diff --git a/NEWS b/NEWS
index 7b4d557d..0331feea 100644
--- a/NEWS
+++ b/NEWS
@@ -19,6 +19,7 @@ MaxMessages and MaxSize can be used together now.
 The unfiltered list of mailboxes in each Store can be printed now.
 
 A proper summary is now printed prior to exiting.
+This includes expunges, which are now included in the progress as well.
 
 Added new sync operation 'Old'.
 
diff --git a/src/main_sync.c b/src/main_sync.c
index 18fbc872..07b28838 100644
--- a/src/main_sync.c
+++ b/src/main_sync.c
@@ -9,7 +9,7 @@
 
 #define nz(a, b) ((a) ? (a) : (b))
 
-static int ops_any[2], trash_any[2];
+static int ops_any[2], trash_any[2], expunge_any[2];
 static int chans_total, chans_done;
 static int boxes_total, boxes_done;
 
@@ -20,7 +20,7 @@ static wakeup_t stats_wakeup;
 static void
 print_stats( void )
 {
-   char buf[3][64];
+   char buf[3][80];
char *cs;
static int cols = -1;
 
@@ -29,10 +29,11 @@ print_stats( void )
int ll = sprintf( buf[2], "C: %d/%d  B: %d/%d", chans_done, 
chans_total, boxes_done, boxes_total );
int cls = (cols - ll - 10) / 2;
for (int t = 0; t < 2; t++) {
-   int l = sprintf( buf[t], "+%d/%d *%d/%d #%d/%d",
+   int l = sprintf( buf[t], "+%d/%d *%d/%d #%d/%d -%d/%d",
 new_done[t], new_total[t],
 flags_done[t], flags_total[t],
-trash_done[t], trash_total[t] );
+trash_done[t], trash_total[t],
+expunge_done[t], expunge_total[t] );
if (l > cls)
buf[t][cls - 1] = '~';
}
@@ -91,6 +92,11 @@ summary( void )
",\nwould move %d %s message(s) to trash" :
",\nmoved %d %s message(s) to trash",
trash_done[t], str_fn[t] );
+   if (expunge_any[t])
+   printf( (DFlags & DRYRUN) ?
+   ",\nwould expunge %d message(s) from %s" :
+   ",\nexpunged %d message(s) from %s",
+   expunge_done[t], str_fn[t] );
}
puts( "." );
 }
@@ -235,10 +241,12 @@ add_channel( chan_ent_t ***chanapp, channel_conf_t *chan, 
int ops[] )
}
if (chan->ops[t] & OP_MASK_TYPE)
ops_any[t] = 1;
-   if ((chan->ops[t] & (OP_EXPUNGE | OP_EXPUNGE_SOLO)) &&
-   (chan->stores[t]->trash ||
-(chan->stores[t^1]->trash && 
chan->stores[t^1]->trash_remote_new)))
-   trash_any[t] = 1;
+   if (chan->ops[t] & (OP_EXPUNGE | OP_EXPUNGE_SOLO)) {
+   expunge_any[t] = 1;
+   if (chan->stores[t]->trash ||
+   (chan->stores[t^1]->trash && 
chan->stores[t^1]->trash_remote_new))
+   trash_any[t] = 1;
+   }
}
 
**chanapp = ce;
diff --git a/src/mbsync.1 b/src/mbsync.1
index 87672c66..60141346 100644
--- a/src/mbsync.1
+++ b/src/mbsync.1
@@ -779,13 +779,13 @@ If \fBmbsync\fR's output is connected to a console, it 
will print progress
 counters by default. The output will look like this:
 .P
 .in +4
-C: 1/2  B: 3/4  F: +13/13 *23/42 #0/0  N: +0/7 *0/0 #0/0
+C: 1/2  B: 3/4  F: +13/13 *23/42 #0/0 -0/0  N: +0/7 *0/0 #0/0 -0/0
 .in -4
 .P
 This represents the cumulative progress over Channels, boxes, and messages
 affected on the far and near side, respectively.
 The message counts represent added messages, messages with updated flags,
-and trashed messages, respectively.
+trashed messages, and expunged messages, respectively.
 No attempt is made to calculate the totals in advance, so they grow over
 time as more information is gathered.
 .P
diff --git a/src/sync.c b/src/sync.c
index 83e63385..0dd6c9e7 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -17,6 +17,7 @@ uint BufferLimit = 10 * 1024 * 1024;
 int new_total[2], new_done[2];
 int flags_total[2], flags_done[2];
 int trash_total[2], trash_done[2];
+int expunge_total[2], expunge_done[2];
 
 static void sync_ref( sync_vars_t *svars ) { ++svars->ref_count; }
 static void sync_deref( sync_vars_t *svars );
@@ -286,6 +287,10 @@ message_expunged( message_t *msg, void *aux )
msg->srec->msg[t] = NULL;
msg->srec = NULL;
}
+   if (msg->status & M_EXPUNGE) {
+   expunge_done[t]++;
+   stats();
+   }
 }
 
 static void 

[commit] master: increase timer resolution to milliseconds

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 1867a7c5ea290f7832a680c42e070cbe433faf34
Author: Oswald Buddenhagen 
Date:   Tue May 10 10:58:57 2022 +0200

increase timer resolution to milliseconds

seconds are too coarse for what we intend. technically, centisecs would
be sufficient (and more honest, given that we prefer coarse timers,
which have only tick precision), but that's a somewhat untypical unit.

 src/common.h  |  4 +++-
 src/drv_imap.c|  4 ++--
 src/drv_maildir.c |  4 ++--
 src/main.c|  1 +
 src/tst_timers.c  |  9 +
 src/util.c| 48 +++
 6 files changed, 53 insertions(+), 17 deletions(-)

diff --git a/src/common.h b/src/common.h
index 92d1e89e..8e613f9d 100644
--- a/src/common.h
+++ b/src/common.h
@@ -323,9 +323,11 @@ typedef struct {
list_head_t links;
void (*cb)( void *aux );
void *aux;
-   time_t timeout;
+   int64_t timeout;
 } wakeup_t;
 
+void init_timers( void );
+int64_t get_now( void );
 void init_wakeup( wakeup_t *tmr, void (*cb)( void * ), void *aux );
 void conf_wakeup( wakeup_t *tmr, int timeout );
 void wipe_wakeup( wakeup_t *tmr );
diff --git a/src/drv_imap.c b/src/drv_imap.c
index 7074e437..e1024e82 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -3713,7 +3713,7 @@ imap_parse_store( conffile_t *cfg, store_conf_t **storep )
return 0;
}
 
-   server->sconf.timeout = 20;
+   server->sconf.timeout = 2;
 #ifdef HAVE_LIBSSL
server->ssl_type = -1;
server->sconf.ssl_versions = -1;
@@ -3761,7 +3761,7 @@ imap_parse_store( conffile_t *cfg, store_conf_t **storep )
server->sconf.port = (ushort)port;
}
} else if (!strcasecmp( "Timeout", cfg->cmd )) {
-   server->sconf.timeout = parse_int( cfg );
+   server->sconf.timeout = parse_int( cfg ) * 1000;
} else if (!strcasecmp( "PipelineDepth", cfg->cmd )) {
if ((server->max_in_progress = parse_int( cfg )) < 1) {
error( "%s:%d: PipelineDepth must be at least 
1\n", cfg->file, cfg->line );
diff --git a/src/drv_maildir.c b/src/drv_maildir.c
index b904913f..60f85f90 100644
--- a/src/drv_maildir.c
+++ b/src/drv_maildir.c
@@ -665,7 +665,7 @@ maildir_store_uidval( maildir_store_t *ctx )
return DRV_BOX_BAD;
}
}
-   conf_wakeup( >lcktmr, 2 );
+   conf_wakeup( >lcktmr, 2000 );
return DRV_OK;
 }
 
@@ -761,7 +761,7 @@ maildir_uidval_lock( maildir_store_t *ctx )
}
}
ctx->uvok = 1;
-   conf_wakeup( >lcktmr, 2 );
+   conf_wakeup( >lcktmr, 2000 );
return DRV_OK;
 }
 
diff --git a/src/main.c b/src/main.c
index ba467ac9..0efd7b8e 100644
--- a/src/main.c
+++ b/src/main.c
@@ -150,6 +150,7 @@ main( int argc, char **argv )
int oind, cops = 0, op, ms_warn = 0, renew_warn = 0, delete_warn = 0;
 
tzset();
+   init_timers();
gethostname( Hostname, sizeof(Hostname) );
if ((ochar = strchr( Hostname, '.' )))
*ochar = 0;
diff --git a/src/tst_timers.c b/src/tst_timers.c
index 51563149..4e1d1ca3 100644
--- a/src/tst_timers.c
+++ b/src/tst_timers.c
@@ -9,7 +9,7 @@
 typedef struct {
int id;
int first, other, morph_at, morph_to;
-   time_t start;
+   int64_t start;
wakeup_t timer;
wakeup_t morph_timer;
 } tst_t;
@@ -18,7 +18,7 @@ static void
 timer_start( tst_t *timer, int to )
 {
printf( "starting timer %d, should expire after %d\n", timer->id, to );
-   time( >start );
+   timer->start  = get_now();
conf_wakeup( >timer, to );
 }
 
@@ -28,7 +28,7 @@ timed_out( void *aux )
tst_t *timer = (tst_t *)aux;
 
printf( "timer %d expired after %d, repeat %d\n",
-   timer->id, (int)(time( 0 ) - timer->start), timer->other );
+   timer->id, (int)(get_now() - timer->start), timer->other );
if (timer->other >= 0) {
timer_start( timer, timer->other );
} else {
@@ -44,7 +44,7 @@ morph_timed_out( void *aux )
tst_t *timer = (tst_t *)aux;
 
printf( "morphing timer %d after %d\n",
-   timer->id, (int)(time( 0 ) - timer->start) );
+   timer->id, (int)(get_now() - timer->start) );
timer_start( timer, timer->morph_to );
 }
 
@@ -55,6 +55,7 @@ main( int argc, char **argv )
 {
int i;
 
+   init_timers();
for (i = 1; i < argc; i++) {
char *val = argv[i];
tst_t *timer = nfmalloc( sizeof(*timer) );
diff --git a/src/util.c b/src/util.c
index 767694d7..3ba7f910 100644
--- a/src/util.c
+++ b/src/util.c
@@ -12,6 +12,7 @@
 #include 
 #include 
 #include 
+#include 
 
 int Verbosity = TERSE;
 int DFlags;
@@ -1000,10 +1001,41 @@ wipe_notifier( notifier_t *sn )
 #endif
 }
 
-static time_t
+#if 

[commit] master: rate-limit progress counters

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 8fbc4323f4bd005b40a370c7de695cda981a5d24
Author: Oswald Buddenhagen 
Date:   Thu May 19 21:24:52 2022 +0200

rate-limit progress counters

for simple local operations, they could easily dominate the cpu load.
also, over a slow remote tty, they could slow down things noticeably.

 src/main_sync.c | 51 ++---
 1 file changed, 44 insertions(+), 7 deletions(-)

diff --git a/src/main_sync.c b/src/main_sync.c
index 3d3d90f6..18fbc872 100644
--- a/src/main_sync.c
+++ b/src/main_sync.c
@@ -13,16 +13,17 @@ static int ops_any[2], trash_any[2];
 static int chans_total, chans_done;
 static int boxes_total, boxes_done;
 
-void
-stats( void )
+static int stats_steps;
+static int64_t stats_stamp;
+static wakeup_t stats_wakeup;
+
+static void
+print_stats( void )
 {
char buf[3][64];
char *cs;
static int cols = -1;
 
-   if (!(DFlags & PROGRESS))
-   return;
-
if (cols < 0 && (!(cs = getenv( "COLUMNS" )) || !(cols = atoi( cs 
cols = 80;
int ll = sprintf( buf[2], "C: %d/%d  B: %d/%d", chans_done, 
chans_total, boxes_done, boxes_total );
@@ -38,6 +39,37 @@ stats( void )
progress( "\r%s  F: %.*s  N: %.*s", buf[2], cls, buf[0], cls, buf[1] );
 }
 
+static void
+stats_timeout( void *aux ATTR_UNUSED )
+{
+   stats_steps = -1;
+   conf_wakeup( _wakeup, 200 );
+   print_stats();
+}
+
+void
+stats( void )
+{
+   if (!(DFlags & PROGRESS))
+   return;
+
+   // If the main loop appears to be running, skip the sync path.
+   if (stats_steps < 0)
+   return;
+
+   // Rate-limit the (somewhat) expensive timer queries.
+   if (++stats_steps < 10)
+   return;
+   stats_steps = 0;
+
+   int64_t now = get_now();
+   if (now < stats_stamp + 300)
+   return;
+   stats_stamp = now;
+
+   print_stats();
+}
+
 static void
 summary( void )
 {
@@ -339,8 +371,10 @@ sync_chans( core_vars_t *cvars, char **argv )
}
mvars->chanptr = chans;
 
-   if (!cvars->list)
-   stats();
+   if (!cvars->list && (DFlags & PROGRESS)) {
+   init_wakeup( _wakeup, stats_timeout, NULL );
+   stats_timeout( NULL );
+   }
do_sync_chans( mvars );
main_loop();
if (!cvars->list) {
@@ -398,6 +432,7 @@ static void
 do_sync_chans( main_vars_t *mvars )
 {
while (mvars->chanptr) {
+   stats_steps = 0;  // Determine main loop use afresh
mvars->chan = mvars->chanptr->conf;
info( "Channel %s\n", mvars->chan->name );
for (int t = 0; t < 2; t++) {
@@ -445,6 +480,8 @@ do_sync_chans( main_vars_t *mvars )
advance_chan( mvars );
}
cleanup_drivers();
+   if (!mvars->cvars->list && (DFlags & PROGRESS))
+   wipe_wakeup( _wakeup );
 }
 
 static void


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: add support for incomplete headers to copy_msg_convert()

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 5b9256f5dcd32e217637938d904d582d5f81c0fd
Author: Oswald Buddenhagen 
Date:   Tue May 3 20:01:29 2022 +0200

add support for incomplete headers to copy_msg_convert()

in the spirit of being liberal about what we accept.

this completely re-structures the header processing loop, with the nice
side effect of eliminating the gotos.

REFMAIL: 87bkyzhoov@curie.anarc.at

 src/sync_msg_cvt.c | 118 ++---
 src/tst_msg_cvt.c  |  83 +--
 2 files changed, 159 insertions(+), 42 deletions(-)

diff --git a/src/sync_msg_cvt.c b/src/sync_msg_cvt.c
index 472fcf6b..1e49cf7b 100644
--- a/src/sync_msg_cvt.c
+++ b/src/sync_msg_cvt.c
@@ -44,52 +44,82 @@ copy_msg_convert( int in_cr, int out_cr, copy_vars_t *vars )
uint in_len = vars->data.len;
uint idx = 0, sbreak = 0, ebreak = 0, break2 = UINT_MAX;
uint lines = 0, hdr_crs = 0, bdy_crs = 0, app_cr = 0, extra = 0;
-   uint add_subj = 0;
+   uint add_subj = 0, fix_tuid = 0, fix_subj = 0, fix_hdr = 0, end_hdr = 0;
 
if (vars->srec) {
- nloop: ;
-   uint start = idx;
-   uint line_cr = 0;
-   char pc = 0;
-   while (idx < in_len) {
-   char c = in_buf[idx++];
-   if (c == '\n') {
+   for (;;) {
+   uint start = idx;
+   uint line_cr = 0;
+   uint got_line = 0;
+   char pc = 0;
+   while (idx < in_len) {
+   char c = in_buf[idx++];
+   if (c == '\n') {
+   if (pc == '\r')
+   line_cr = 1;
+   got_line = 1;
+   break;
+   }
+   pc = c;
+   }
+   if (!ebreak && starts_with_upper( in_buf + start, 
(int)(in_len - start), "X-TUID: ", 8 )) {
+   extra = (sbreak = start) - (ebreak = idx);
+   if (!vars->minimal)
+   break;
+   continue;
+   }
+   if (break2 == UINT_MAX && vars->minimal &&
+   starts_with_upper( in_buf + start, (int)(in_len - 
start), "SUBJECT:", 8 )) {
+   break2 = start + 8;
+   if (break2 < in_len && in_buf[break2] == ' ')
+   break2++;
+   }
+   hdr_crs += line_cr;
+   if (got_line) {
+   lines++;
+   if (idx - line_cr - 1 != start)
+   continue;
+   // Empty line => end of headers
+   } else {
+   // The line is incomplete.
if (pc == '\r')
-   line_cr = 1;
-   if (!ebreak && starts_with_upper( in_buf + 
start, (int)(in_len - start), "X-TUID: ", 8 )) {
-   extra = (sbreak = start) - (ebreak = 
idx);
-   if (!vars->minimal)
-   goto oke;
-   } else {
-   if (break2 == UINT_MAX && vars->minimal 
&&
-   starts_with_upper( in_buf + start, 
(int)(in_len - start), "SUBJECT:", 8 )) {
-   break2 = start + 8;
-   if (break2 < in_len && 
in_buf[break2] == ' ')
-   break2++;
-   }
-   lines++;
-   hdr_crs += line_cr;
+   idx--;  // For simplicity, move back 
before trailing CR
+   if (idx != start) {
+   // The line is non-empty, so schedule 
completing it
+   fix_hdr = 1;
+   // ... and put our headers after it. 
(It would seem easier
+   // to prepend them, as then we could 
avoid the fixing - but
+   // the line might be a continuation. We 
could also prepend
+   // it to _all_ pre-exiting headers, but 
then we would risk
+   // masking an (incorrectly present) 
leading 

[commit] master: factor copy_msg_convert() out to own source file

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit a32964c34eef2a3f11dcbd7eb37a4b545ebce1d5
Author: Oswald Buddenhagen 
Date:   Sun May 1 14:16:28 2022 +0200

factor copy_msg_convert() out to own source file

 src/Makefile.am|   2 +-
 src/sync.c | 180 -
 src/sync_msg_cvt.c | 179 
 src/sync_p.h   |  11 +++
 4 files changed, 191 insertions(+), 181 deletions(-)

diff --git a/src/Makefile.am b/src/Makefile.am
index ad8552b7..f1eb9961 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -7,7 +7,7 @@ mbsync_SOURCES = \
driver.c drv_proxy.c \
drv_imap.c imap_msgs.c \
drv_maildir.c \
-   sync.c sync_state.c \
+   sync.c sync_state.c sync_msg_cvt.c \
main.c main_sync.c main_list.c
 noinst_HEADERS = \
common.h config.h socket.h \
diff --git a/src/sync.c b/src/sync.c
index dd85d7e7..f0762415 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -79,15 +79,6 @@ sanitize_flags( uchar tflags, sync_vars_t *svars, int t )
 }
 
 
-typedef struct copy_vars {
-   void (*cb)( int sts, uint uid, struct copy_vars *vars );
-   void *aux;
-   sync_rec_t *srec; /* also ->tuid */
-   message_t *msg;
-   msg_data_t data;
-   int minimal;
-} copy_vars_t;
-
 static void msg_fetched( int sts, void *aux );
 
 static void
@@ -103,177 +94,6 @@ copy_msg( copy_vars_t *vars )
 
 static void msg_stored( int sts, uint uid, void *aux );
 
-static void
-copy_msg_bytes( char **out_ptr, const char *in_buf, uint *in_idx, uint in_len, 
int in_cr, int out_cr )
-{
-   char *out = *out_ptr;
-   uint idx = *in_idx;
-   if (out_cr != in_cr) {
-   char c;
-   if (out_cr) {
-   for (; idx < in_len; idx++) {
-   if ((c = in_buf[idx]) != '\r') {
-   if (c == '\n')
-   *out++ = '\r';
-   *out++ = c;
-   }
-   }
-   } else {
-   for (; idx < in_len; idx++) {
-   if ((c = in_buf[idx]) != '\r')
-   *out++ = c;
-   }
-   }
-   } else {
-   memcpy( out, in_buf + idx, in_len - idx );
-   out += in_len - idx;
-   idx = in_len;
-   }
-   *out_ptr = out;
-   *in_idx = idx;
-}
-
-static char *
-copy_msg_convert( int in_cr, int out_cr, copy_vars_t *vars )
-{
-   char *in_buf = vars->data.data;
-   uint in_len = vars->data.len;
-   uint idx = 0, sbreak = 0, ebreak = 0, break2 = UINT_MAX;
-   uint lines = 0, hdr_crs = 0, bdy_crs = 0, app_cr = 0, extra = 0;
-   uint add_subj = 0;
-
-   if (vars->srec) {
- nloop: ;
-   uint start = idx;
-   uint line_crs = 0;
-   while (idx < in_len) {
-   char c = in_buf[idx++];
-   if (c == '\r') {
-   line_crs++;
-   } else if (c == '\n') {
-   if (!ebreak && starts_with_upper( in_buf + 
start, (int)(in_len - start), "X-TUID: ", 8 )) {
-   extra = (sbreak = start) - (ebreak = 
idx);
-   if (!vars->minimal)
-   goto oke;
-   } else {
-   if (break2 == UINT_MAX && vars->minimal 
&&
-   starts_with_upper( in_buf + start, 
(int)(in_len - start), "SUBJECT:", 8 )) {
-   break2 = start + 8;
-   if (break2 < in_len && 
in_buf[break2] == ' ')
-   break2++;
-   }
-   lines++;
-   hdr_crs += line_crs;
-   }
-   if (idx - line_crs - 1 == start) {
-   if (!ebreak)
-   sbreak = ebreak = start;
-   if (vars->minimal) {
-   in_len = idx;
-   if (break2 == UINT_MAX) {
-   break2 = start;
-   add_subj = 1;
-   }
-   }
-   goto oke;
-   }
-   goto nloop;
-   }
-   }
-   free( in_buf );
-   

[commit] master: preserve stray CRs in copy_msg_convert()

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit ed92816fdb6cb43b3c65bb051404000d524ea958
Author: Oswald Buddenhagen 
Date:   Tue May 3 19:22:24 2022 +0200

preserve stray CRs in copy_msg_convert()

strip only CRs followed by LFs. this has mostly theoretical value (but
will make a subsequent change simpler).

 src/sync_msg_cvt.c | 44 
 src/tst_msg_cvt.c  |  2 +-
 2 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/src/sync_msg_cvt.c b/src/sync_msg_cvt.c
index 93b43676..472fcf6b 100644
--- a/src/sync_msg_cvt.c
+++ b/src/sync_msg_cvt.c
@@ -13,19 +13,19 @@ copy_msg_bytes( char **out_ptr, const char *in_buf, uint 
*in_idx, uint in_len, i
char *out = *out_ptr;
uint idx = *in_idx;
if (out_cr != in_cr) {
-   char c;
if (out_cr) {
-   for (; idx < in_len; idx++) {
-   if ((c = in_buf[idx]) != '\r') {
-   if (c == '\n')
-   *out++ = '\r';
-   *out++ = c;
-   }
+   for (char c, pc = 0; idx < in_len; idx++) {
+   if (((c = in_buf[idx]) == '\n') && (pc != '\r'))
+   *out++ = '\r';
+   *out++ = c;
+   pc = c;
}
} else {
-   for (; idx < in_len; idx++) {
-   if ((c = in_buf[idx]) != '\r')
-   *out++ = c;
+   for (char c, pc = 0; idx < in_len; idx++) {
+   if (((c = in_buf[idx]) == '\n') && (pc == '\r'))
+   out--;
+   *out++ = c;
+   pc = c;
}
}
} else {
@@ -49,12 +49,13 @@ copy_msg_convert( int in_cr, int out_cr, copy_vars_t *vars )
if (vars->srec) {
  nloop: ;
uint start = idx;
-   uint line_crs = 0;
+   uint line_cr = 0;
+   char pc = 0;
while (idx < in_len) {
char c = in_buf[idx++];
-   if (c == '\r') {
-   line_crs++;
-   } else if (c == '\n') {
+   if (c == '\n') {
+   if (pc == '\r')
+   line_cr = 1;
if (!ebreak && starts_with_upper( in_buf + 
start, (int)(in_len - start), "X-TUID: ", 8 )) {
extra = (sbreak = start) - (ebreak = 
idx);
if (!vars->minimal)
@@ -67,9 +68,9 @@ copy_msg_convert( int in_cr, int out_cr, copy_vars_t *vars )
break2++;
}
lines++;
-   hdr_crs += line_crs;
+   hdr_crs += line_cr;
}
-   if (idx - line_crs - 1 == start) {
+   if (idx - line_cr - 1 == start) {
if (!ebreak)
sbreak = ebreak = start;
if (vars->minimal) {
@@ -83,6 +84,7 @@ copy_msg_convert( int in_cr, int out_cr, copy_vars_t *vars )
}
goto nloop;
}
+   pc = c;
}
free( in_buf );
return "has incomplete header";
@@ -91,12 +93,14 @@ copy_msg_convert( int in_cr, int out_cr, copy_vars_t *vars )
extra += 8 + TUIDL + app_cr + 1;
}
if (out_cr != in_cr) {
-   for (; idx < in_len; idx++) {
+   for (char pc = 0; idx < in_len; idx++) {
char c = in_buf[idx];
-   if (c == '\r')
-   bdy_crs++;
-   else if (c == '\n')
+   if (c == '\n') {
lines++;
+   if (pc == '\r')
+   bdy_crs++;
+   }
+   pc = c;
}
extra -= hdr_crs + bdy_crs;
if (out_cr)
diff --git a/src/tst_msg_cvt.c b/src/tst_msg_cvt.c
index afd437a4..399c5c2b 100644
--- a/src/tst_msg_cvt.c
+++ b/src/tst_msg_cvt.c
@@ -130,7 +130,7 @@ mintests( const char *name, const char *in, const char 
*out, int flagged )
tests( name, in, out, ADD_TUID, MINIMAL, flagged );
 }
 
-#define FROM "From: 

[commit] master: add test for copy_msg_convert()

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 669f7dbd8fa520fe0cb8580c090ba70e63abb7d8
Author: Oswald Buddenhagen 
Date:   Sat Jun 18 16:43:16 2022 +0200

add test for copy_msg_convert()

 src/.gitignore|   1 +
 src/Makefile.am   |   5 +-
 src/tst_msg_cvt.c | 181 ++
 src/util.c|   4 +-
 4 files changed, 189 insertions(+), 2 deletions(-)

diff --git a/src/.gitignore b/src/.gitignore
index 4c388832..5e7fc35d 100644
--- a/src/.gitignore
+++ b/src/.gitignore
@@ -3,6 +3,7 @@
 /mbsync
 /mdconvert
 /tst_imap_msgs
+/tst_msg_cvt
 /tst_timers
 /tmp
 
diff --git a/src/Makefile.am b/src/Makefile.am
index f1eb9961..ab564183 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -54,7 +54,10 @@ man_MANS = mbsync.1 $(mdconvert_man)
 
 tst_imap_msgs_SOURCES = tst_imap_msgs.c imap_msgs.c util.c
 
-check_PROGRAMS = tst_imap_msgs
+tst_msg_cvt_SOURCES = tst_msg_cvt.c sync_msg_cvt.c util.c
+tst_msg_cvt_CFLAGS = -DQPRINTF_BUFF=1
+
+check_PROGRAMS = tst_imap_msgs tst_msg_cvt
 TESTS = $(check_PROGRAMS)
 
 tst_timers_SOURCES = tst_timers.c util.c
diff --git a/src/tst_msg_cvt.c b/src/tst_msg_cvt.c
new file mode 100644
index ..afd437a4
--- /dev/null
+++ b/src/tst_msg_cvt.c
@@ -0,0 +1,181 @@
+// SPDX-FileCopyrightText: 2022 Oswald Buddenhagen 
+// SPDX-License-Identifier: GPL-2.0-or-later
+//
+// isync test suite
+//
+
+#include "sync_p.h"
+
+#define TUID "one two tuid"
+
+static_assert( sizeof(TUID) - 1 == TUIDL, "TUID size mismatch" );
+
+static size_t
+strip_cr( char *buf )
+{
+   size_t i, j = 0;
+   char c, pc = 0;
+   for (i = 0; (c = buf[i]); i++) {
+   if (c == '\n' && pc == '\r')
+   j--;
+   buf[j++] = c;
+   pc = c;
+   }
+   buf[j] = 0;
+   return j;
+}
+
+#define NL_UNIX 0
+#define NL_ANY 1
+
+#define AS_IS 0
+#define ADD_TUID 1
+
+#define FULL 0
+#define MINIMAL 1
+
+#define REGULAR 0
+#define FLAGGED 1
+
+#define BIG_SIZE 2345687
+#define BIG_SIZE_STR "2.2MiB"
+
+#define SEP "="
+
+static void
+test( const char *name, const char *in, int scr, int rscr, const char *out, 
int tcr, int rtcr, int add_tuid, int minimal, int flagged )
+{
+   assert( !rscr || scr );
+   assert( !rtcr || tcr );
+   assert( !minimal || add_tuid );
+   assert( !flagged || minimal );
+
+   printf( "Testing %s, %s (%s) => %s (%s)%s%s%s ...\n", name,
+   rscr ? "CRLF" : "LF", scr ? "Any" : "Unix", rtcr ? "CRLF" : 
"LF", tcr ? "Any" : "Unix",
+   add_tuid ? ", add TUID" : "", minimal ? ", minimal" : "", 
flagged ? ", flagged" : "" );
+
+   sync_rec_t srec;
+   message_t msg;
+   copy_vars_t vars;
+   vars.minimal = minimal;
+   if (add_tuid) {
+   vars.srec = 
+   memcpy( vars.srec->tuid, TUID, TUIDL );
+   if (minimal) {
+   vars.msg = 
+   vars.msg->size = BIG_SIZE;
+   vars.data.flags = flagged ? F_FLAGGED : 0;
+   }
+   } else {
+   vars.srec = 0;
+   }
+   vars.data.data = strdup( in );
+   vars.data.len = rscr ? strlen( in ) : strip_cr( vars.data.data );
+   char *orig = strdup( vars.data.data );
+   const char *err = copy_msg_convert( scr, tcr,  );
+   if (err) {
+   printf( "FAIL: %s!\n", err );
+   exit( 1 );
+   }
+   if (!rtcr) {
+   char *tout = strdup( out );
+   size_t toutl = strip_cr( tout );
+   if (toutl != vars.data.len || memcmp( vars.data.data, tout, 
toutl )) {
+   xprintf( "FAIL!\n"
+SEP " Input " SEP "\n%!\n"
+SEP " Expected output " SEP "\n%!\n"
+SEP " Output " SEP "\n%.*!\n" SEP "\n",
+orig, tout, vars.data.len, vars.data.data );
+   exit( 1 );
+   }
+   free( tout );
+   } else {
+   size_t outl = strlen( out );
+   if (outl != vars.data.len || memcmp( vars.data.data, out, outl 
)) {
+   xprintf( "FAIL!\n"
+SEP " Input " SEP "\n%!\n"
+SEP " Expected output (%u bytes) " SEP 
"\n%!\n"
+SEP " Actual output (%u bytes) " SEP 
"\n%.*!\n" SEP "\n",
+orig, outl, out, vars.data.len, vars.data.len, 
vars.data.data );
+   exit( 1 );
+   }
+   }
+   free( orig );
+   free( vars.data.data );
+}
+
+static void
+tests( const char *name, const char *in, const char *out, int add_tuid, int 
minimal, int flagged )
+{
+   test( name, in, NL_UNIX, NL_UNIX, out, NL_ANY, NL_ANY, add_tuid, 
minimal, flagged );
+   test( name, in, NL_ANY, NL_UNIX, out, NL_UNIX, NL_UNIX, add_tuid, 
minimal, flagged );
+   test( name, in, NL_ANY, 

[commit] master: factor out ADD_NL() inside copy_msg_convert()

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit dbffebf560a8418703ad9e1d987adbfa08fd3822
Author: Oswald Buddenhagen 
Date:   Tue May 3 13:38:57 2022 +0200

factor out ADD_NL() inside copy_msg_convert()

 src/sync_msg_cvt.c | 15 +--
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/src/sync_msg_cvt.c b/src/sync_msg_cvt.c
index 4f8403ed..93b43676 100644
--- a/src/sync_msg_cvt.c
+++ b/src/sync_msg_cvt.c
@@ -131,6 +131,13 @@ copy_msg_convert( int in_cr, int out_cr, copy_vars_t *vars 
)
extra += add_subj ? strlen(dummy_subj) + app_cr + 1 : 
strlen(dummy_pfx);
}
 
+#define ADD_NL() \
+   do { \
+   if (app_cr) \
+   *out_buf++ = '\r'; \
+   *out_buf++ = '\n'; \
+   } while (0)
+
vars->data.len = in_len + extra;
if (vars->data.len > INT_MAX) {
free( in_buf );
@@ -150,9 +157,7 @@ copy_msg_convert( int in_cr, int out_cr, copy_vars_t *vars )
out_buf += 8;
memcpy( out_buf, vars->srec->tuid, TUIDL );
out_buf += TUIDL;
-   if (app_cr)
-   *out_buf++ = '\r';
-   *out_buf++ = '\n';
+   ADD_NL();
idx = ebreak;
 
if (break2 != UINT_MAX && break2 >= sbreak) {
@@ -163,9 +168,7 @@ copy_msg_convert( int in_cr, int out_cr, copy_vars_t *vars )
} else {
memcpy( out_buf, dummy_subj, strlen(dummy_subj) 
);
out_buf += strlen(dummy_subj);
-   if (app_cr)
-   *out_buf++ = '\r';
-   *out_buf++ = '\n';
+   ADD_NL();
}
}
}


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: return error strings from copy_msg_convert()

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 46d244533e13af878301d0dfd4ed8fb5e911696d
Author: Oswald Buddenhagen 
Date:   Sun May 1 11:55:58 2022 +0200

return error strings from copy_msg_convert()

... instead of reporting conversion errors directly in copy_msg_convert().
this makes it easier to autotest properly.

this reverts bc15e571 in spirit, sidestepping the problem it tried to
solve instead.

 src/sync.c | 18 --
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/src/sync.c b/src/sync.c
index 48d5947a..dd85d7e7 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -133,8 +133,8 @@ copy_msg_bytes( char **out_ptr, const char *in_buf, uint 
*in_idx, uint in_len, i
*in_idx = idx;
 }
 
-static int
-copy_msg_convert( int in_cr, int out_cr, copy_vars_t *vars, int t )
+static char *
+copy_msg_convert( int in_cr, int out_cr, copy_vars_t *vars )
 {
char *in_buf = vars->data.data;
uint in_len = vars->data.len;
@@ -180,10 +180,8 @@ copy_msg_convert( int in_cr, int out_cr, copy_vars_t 
*vars, int t )
goto nloop;
}
}
-   warn( "Warning: message %u from %s has incomplete header; 
skipping.\n",
- vars->msg->uid, str_fn[t^1] );
free( in_buf );
-   return 0;
+   return "has incomplete header";
  oke:
app_cr = out_cr && (!in_cr || hdr_crs);
extra += 8 + TUIDL + app_cr + 1;
@@ -231,10 +229,8 @@ copy_msg_convert( int in_cr, int out_cr, copy_vars_t 
*vars, int t )
 
vars->data.len = in_len + extra;
if (vars->data.len > INT_MAX) {
-   warn( "Warning: message %u from %s is too big after conversion; 
skipping.\n",
- vars->msg->uid, str_fn[t^1] );
free( in_buf );
-   return 0;
+   return "is too big after conversion";
}
char *out_buf = vars->data.data = nfmalloc( vars->data.len );
idx = 0;
@@ -275,7 +271,7 @@ copy_msg_convert( int in_cr, int out_cr, copy_vars_t *vars, 
int t )
memcpy( out_buf, dummy_msg_buf, dummy_msg_len );
 
free( in_buf );
-   return 1;
+   return NULL;
 }
 
 static void
@@ -318,7 +314,9 @@ msg_fetched( int sts, void *aux )
scr = svars->can_crlf[t^1];
tcr = svars->can_crlf[t];
if (srec || scr != tcr) {
-   if (!copy_msg_convert( scr, tcr, vars, t )) {
+   const char *err;
+   if ((err = copy_msg_convert( scr, tcr, vars ))) {
+   warn( "Warning: message %u from %s %s; 
skipping.\n", vars->msg->uid, str_fn[t^1], err );
vars->cb( SYNC_NOGOOD, 0, vars );
return;
}


___
isync-devel mailing list
isync-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/isync-devel


[commit] master: rework IMAP list parsing

2022-06-19 Thread Oswald Buddenhagen via isync-devel
commit 3aead3300865b33b086f12961621d655a16bd6c2
Author: Oswald Buddenhagen 
Date:   Fri Jun 10 18:29:55 2022 +0200

rework IMAP list parsing

replace the "DOM-like" model with a "streaming" model, i.e., the tokens
are now immediately processed by stateful callbacks as they are found.
this avoids plenty of allocations and copies (though not of the message
contents, yet).

 src/drv_imap.c | 839 ++---
 1 file changed, 524 insertions(+), 315 deletions(-)

diff --git a/src/drv_imap.c b/src/drv_imap.c
index 120bcdab..dd5977a4 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -58,26 +58,61 @@ typedef union imap_store_conf {
};
 } imap_store_conf_t;
 
-#define NIL(void*)0x1
-#define LIST   (void*)0x2
-
-typedef struct _list {
-   struct _list *next, *child;
-   char *val;
-   uint len;
-} list_t;
+typedef union imap_store imap_store_t;
 
-#define MAX_LIST_DEPTH 5
+enum { AtomNil, AtomString, AtomLiteral, AtomChunkedLiteral };
 
-typedef union imap_store imap_store_t;
+typedef struct {
+   int (*enter)( imap_store_t *ctx );
+   int (*leave)( imap_store_t *ctx );
+   int (*atom)( imap_store_t *ctx, char *val, uint len, int type );
+   int (*done)( imap_store_t *ctx, int resp );
+} parse_list_cb_t;
 
 typedef struct {
-   list_t *head, **stack[MAX_LIST_DEPTH];
-   int (*callback)( imap_store_t *ctx, list_t *list );
-   int level, need_bytes;
+   parse_list_cb_t *callback;
+   uint need_bytes;
+   int level, chunked, big_literal;
+   enum { NotLiteral, AtomicLiteral, ChunkedLiteral } in_literal;
const char *err;
 } parse_list_state_t;
 
+typedef enum {
+   NsInit,
+   Ns1st,
+   Ns1stNs,
+   Ns1stDelim,
+   NsDone
+} ParseNsState;
+
+typedef enum {
+   FetchInit,
+   FetchAttrib,
+   FetchUid,
+   FetchFlags,
+   FetchFlagVal,
+   FetchDate,
+   FetchSize,
+   FetchBody,
+   FetchBodyChunk,
+   FetchHeaders,
+   FetchHeaderFields,
+   FetchHeaderBracket,
+   FetchHeaderCont
+} ParseFetchState;
+
+typedef enum {
+   ListInit,
+   ListAttrib,
+   ListSkip
+} ParseListState;
+
+typedef enum {
+   PermflagsInit,
+   PermflagsFlags,
+   PermflagsHasFwd
+} ParsePermflagsState;
+
 typedef struct imap_cmd imap_cmd_t;
 
 union imap_store {
@@ -106,12 +141,25 @@ union imap_store {
int total_msgs, recent_msgs;
uint uidvalidity, uidnext;
imap_messages_t msgs;
-   uint fetch_seq;  // FETCH results
+   struct {  // pending FETCH result
+   char *body, *msgid;
+   time_t date;
+   uint seq, uid, size, body_len;
+   uchar flags, status;
+   char tuid[TUIDL];
+   } fetch;
uint caps;  // CAPABILITY results
string_list_t *auth_mechs;
parse_list_state_t parse_list_sts;
const char *parse_list_what;
char *parse_list_cmd;
+   union {
+   ParseNsState ns_state;
+   ParseFetchState fetch_state;
+   ParseListState list_state;
+   ParsePermflagsState permflags_state;
+   int any_state;  // We always init this one to zero
+   };
// Command queue
imap_cmd_t *pending, **pending_append;
imap_cmd_t *in_progress, **in_progress_append;
@@ -682,39 +730,6 @@ next_arg( char **ps, uint *len )
return ret;
 }
 
-static int
-is_opt_atom( list_t *list )
-{
-   return list && list->val && list->val != LIST;
-}
-
-static int
-is_atom( list_t *list )
-{
-   return list && list->val && list->val != NIL && list->val != LIST;
-}
-
-static int
-is_list( list_t *list )
-{
-   return list && list->val == LIST;
-}
-
-static void
-free_list( list_t *list )
-{
-   list_t *tmp;
-
-   for (; list; list = tmp) {
-   tmp = list->next;
-   if (is_list( list ))
-   free_list( list->child );
-   else if (is_atom( list ))
-   free( list->val );
-   free( list );
-   }
-}
-
 enum {
LIST_OK,
LIST_PARTIAL,
@@ -724,23 +739,20 @@ enum {
 static int
 parse_imap_list( imap_store_t *ctx, char **sp, parse_list_state_t *sts )
 {
-   list_t *cur, **curp;
char *s = *sp, *d, *p;
-   int bytes;
+   uint bytes;
uint n;
char c;
 
+   assert( ctx );
assert( sts );
-   assert( sts->level > 0 );
-   curp = sts->stack[--sts->level];
-   bytes = sts->need_bytes;
-   if (bytes >= 0) {
-   sts->need_bytes = -1;
+   if (sts->in_literal != NotLiteral) {
+   bytes = sts->need_bytes;
if (!bytes)
 

  1   2   3   4   5   6   7   >