Re: [Mutt] #3638: Compilation errors for 1.6
On Sat, Apr 20, 2013 at 12:09:49PM -, Mutt wrote: > mk[sd]temp() exist as handy temp creators that avoid foot shooting. > But if you care about the resultant filename, you can make your > own. That's mentioned in the open(.. O_CREAT|O_EXCL ..) in the spec > below. Sure, you can make your own custom temp file creator, but the whole point of having a library function is that getting this right is much harder than it seems at first glance, and the OS should make it easy for you. But besides, that IS what Mutt has... its own custom temp file creator. Which uses mktemp() to generate file names. Which is totally fine. > # mkstemp() - secure family > http://pubs.opengroup.org/onlinepubs/9699919799/functions/mkstemp.html > # mktemp() - insecure, removed from spec mktemp() IS NOT insecure. The way it has been used over the years is. The warning is, in fact, a lie; it's an overreaction to a related but DIFFERENT problem. Writing your own random filename generator would essentially amount to re-writing mktemp(), except that it would likely be less secure (potentially due to insufficient randomness, etc.), unless you knew what you were doing. So replacing mktemp() is pointless. So long as the weakness is documented (which it is), and the way to use it correctly is understood (which it is), there's no problem with mktemp(). It's unfortunate if there are redundant code blocks, but that is not a bug, per se. It might be worth opening a bug to get them cleaned up. If there are calls to fopen() which are for writing, these should be looked at, and if they're correct, a comment to that effect is probably worthwhile, in similar fashion to the sprintf() comments... Regardless, concerning the mktemp compiler warning, the easiest and best solution is to ignore the warning. It is patently wrong. > It's the O_EXCL that protects you, not the stats. If you mean stat(), I don't think I ever said anything about stat(), unless you're talking about the stat() which mktemp() does to try to make sure it does not give you a file name that already exists. You're correct that this does not protect you... No one here ever said it did. If you're talking about the file modes, then you're mistaken. It's the combination of O_EXCL AND the file mode that protects you. YOU NEED BOTH. An attacker could otherwise simply wait until you've created your temp file, open it for write, and write their exploit into it, right over whatever you wrote. Is this easy? No (the writing is if your file modes allow it; the guessing of the correct file name is not). But it has been demonstrated that brute-forcing it IS possible. > 'When you go to create it' implies beforehand, which is wrong. > Only the O_EXCL can save you from the race. No, it doesn't imply that at all. When you go to create it: open(..., O_CREAT|O_EXCL, 0600); This is what Mutt does. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpMpmiohXWjz.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
> Yep, it does what's on the tin and people plain used it wrong, > decades of wrong, bad wrong. So it got deprecated and turned into a > warn to wake people up :) The problem with deprecating mktemp() it is that it serves a perfectly valid and useful function, which in and of itself is not insecure. Mutt is using it in exactly that way. I'm not saying there are NO weaknesses in safe_open()... or in code that calls it; but this ISN'T one of them. > Given O_EXCL (if it works, and checking the result) is what actually > neuters the race, then the X's exist not for security but give: That's false. O_EXCL is not the end of the story. Using a randomly generated name IS IMPORTANT, because it makes it harder for an attacker to effect a collision. If the attacker can either predict the filename, or can effectively create files that exhaust the set of possible file names, then the attacker can DoS your application. It's important that the universe of possible names be large, to make it as hard as possible for such an attack to be effected. Using a generator with poor randomness, or with a small universe, is BAD. Now, at a quick glance, it doesn't appear that Mutt tries to deal with this, i.e. safe_open() does not detect failures and retry, but it probably should make an attempt. But I didn't look at the calling code, it may be handled there (though I think it shouldn't be). One possible way to address this would be, at start-up, create a randomly generated temporary directory which persists for the life of Mutt, which has only write perms for the owner, and write all your temp files there. Mut does something like this... but not quite. The advantage here is you can try to do this before Mutt tries to do anything "important" with temp files, in a loop until TMP_MAX, and then if you fail, you can be absolutely certain that something bad is happening that requires your investigation. Under such conditions, mutt should complain loudly and exit. But, once the directory is created, you never need to worry about the collision problem again (except within your own program), for the life of your program. Ideally, the library function would register a call with atexit() to make sure this directory gets cleaned up upon exit of the program, though the user (the programmer) should explicitly do this when it makes sense to do so. You should absolutely not roll your own random temp file generator, unless you're prepared to have a bona fide security expert review it. Doing this purely to avoid a bogus warning is a horrible, horrible idea. You should just put this idea out of your mind. > I think I saw a stat() coded, then later on, the O_EXCL, which There is an lstat(), which exists to establish that the target is not a symlink. That is also an important part of the process, and serves to foil a different class of attacks than a simple race. It is remarkably, astonishingly hard to create temp files safely. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpC1fLOoj0vL.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Sat, Apr 20, 2013 at 10:14:44PM -, Mutt wrote: > > The point is, even if your system has mkstemps(), you're probably > > better off using mktemp() the way Mutt uses it instead. You just have > > to make sure that the file does not already exist when you go to > > create it, and that you create the file 0600 so that evildoers can't > > write their exploit over your data... and Mutt does that. > > That is not sufficient with pre-NFSv3. No, but it's generally a bad idea to put temp files on NFS. By default, if you do nothing, they'll never end up there (unless your sysadmin mounts /tmp from an nfs export, which would be dumb). And that's as it should be. Mutt currently doesn't deal well with this case, but I'm not convinced it should try. I think it would be better to document that it's not safe to put temp files on NFS. Another problem with NFS is locking, esp. pre-v3. Different vendors got it wrong in different ways. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp9xzLtxAa7d.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Tue, Apr 23, 2013 at 12:39:25AM +0200, Vincent Lefevre wrote: > It seems that the problem is not the function (API) itself, > but broken implementations (due to the original 4.3BSD one): > the mktemp(3) man page under Linux says: > > BUGS > Never use mktemp(). Some implementations follow 4.3BSD and replace > XX by the current process ID and a single letter, so that at > most 26 different names can be returned. [...] > The existence of broken, insecure implementations due to history > is a good reason to avoid this function. Maybe. The man page is I think outdated; I'm not convinced this is still an issue. The issues with mktemp() are long well understood and I'm pretty sure every vendor has fixed this in anything resembling a currently supported OS. Regardless, I did outline a way to use mkstemp() to do this, years ago. There's a reference to it in an earlier comment in this bug: http://marc.info/?l=mutt-dev&m=128900518711082 I'd be reasonably happy to take some time to rewrite safe_open() and company, EXCEPT that I already have a patch for a 1.6 bug (#3298) that's been completely ignored for four years. In fact, Michael very recently went out of his way to provide a completely independent implementation (without ever commenting on the one I already provided), which I pointed out completely failed to address the problem, resulting in its reversion... I then provided an improved patch which, like the first, has been completely ignored. This, despite the lengthy conversation we just had about that very problem. That experience, combined with a much earlier (2004-ish?) one where I had to pull some teeth to get committed a couple of patches related to usability of the PGP UI, leaves me extremely disinclined to spend time writing code for Mutt. I have no inclination to spend my own free time on a solution that's just going to be unceremoniously dismissed. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpG2iy3jn64W.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Tue, Apr 23, 2013 at 10:48:43AM +0200, Vincent Lefevre wrote: > > ago. There's a reference to it in an earlier comment in this bug: > > > > http://marc.info/?l=mutt-dev&m=128900518711082 > > But there's a race condition, and I think that the name with the > suffix can easily be guessed from the file that has just been > created. This is solved by using the randomized temporary (sub)directory I mentioned in another post. But even if you don't go that route (it basically requires a global variable to track the tempdir, which some may find distasteful), at least you can use the O_EXCL/link methods to detect the race, and perform a reasonable number of retries. If you try, say, ten times, and they all fail, it's a reasonable assumption that someone is messing with you, and you should investiage that. Worst case, mutt should try the link X times, and fail, indicating that something bad is happening, and then EXIT. > FYI, I've tested your new patch and I'm using it, and it seems to > work well. Good to know, thanks Vincent. It would be nice if someone would actually take the time to review the code... But whatever, I've done my piece, and I think it's quite good. > And yes, it's a pity patches are ignored. Indeed. On Tue, Apr 23, 2013 at 07:47:56AM -0400, grarpamp wrote: > Oops, I tend to see DoS as separate and latter to security :) They're one in the same. Availability is one of the three attributes of information security (the other two being confidentiality and integrity). > > you can add random characters before the suffix, and in case of > > Random chars from where? Right, and now you see why I suggested you should not try to implement your own. Some systems have usable /dev/(u)random, some do not. Those that do not may not have an adequate source of randomness, or finding one may be dependent upon system usage. > Even functions like rand() or random() can be sufficient with a seed > difficult to guess (whether Mutt is interactive or not), This is surprisingly hard, unless your system has a known good way of generating randomness: http://docstore.mik.ua/orelly/networking/puis/ch23_08.htm > Not sure whether this is up to date: > > http://www.gnu.org/software/gnulib/manual/html_node/mkstemps.html That's a lot of systems. This is an annoyingly hard problem. Ideally you want to rely on standard library functions, so you can make it the problem of the operating system. Then, at least, you can tell users to put pressure on the OS vendor to provide reasonable implementations that don't put them at risk. It's kind of a cop-out, but in some ways it may be the best that you can hope for. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpZuszxNq5nr.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Tue, Apr 23, 2013 at 01:28:34PM -0500, Derek Martin wrote: > This is an annoyingly hard problem. Incidentally, once you have identified a good source of random data, the problem becomes easier. For example, take some number of random bits which is likely to be adequate (current implementations appear to use 6 * ([A-Z][a-z][0-9]) = 62^6 ~= 2^36 possible combinations, which seems like plenty), and base-64 encode that into a string. Use that as the random part of your file name. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpIponQe5g6y.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Tue, Apr 23, 2013 at 03:20:22PM -0400, grarpamp wrote: > > This is solved by using the randomized temporary (sub)directory I > > Well I guess if you're carrying around this dir for anything longer than > n-time, and the attacker is your UID, Then you lose. Period. Trying to protect yourself from an attacker who has your credentials is automatic FAIL. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpVjK5RW1Nuu.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Tue, Apr 23, 2013 at 11:58:05PM +0200, Vincent Lefevre wrote: > If the goal is to create a temporary file to view an attachment, the > contents of the attachment (and/or the mail itself) can be used as a > source of random data. I suppose that the attacker isn't the one who > sent the mail in question and the mailbox isn't public. You can't suppose that. :) The message may very well be one that was sent by the attacker, specifically to get the user to fall into his trap. > More generally, if a mailbox is open and non-empty, this is a source > of random data too... If the user is not careful to protect the mail store with restrictive permissions, an attacker may very well already have the contents of the file. This presupposes that the user is ignorant or unconcerned about security issues; but many of them are. Additionally, in either case, if only plain text is involved, then the resulting randomness is quite poor; as natural language tends to fall into very recognizable patterns, there's not enough entropy. There's also the question of how you will use the data once you read it from the file; for instance simply using what you read may expose the contents to anyone who has access to the directory where the temp file will be written. Again, the subdirectory approach eliminates this particular issue. Bottom line, if you don't know for sure that it's a good source of randomness, you must assume it isn't. I told you this was hard. ;-) One problem to solve here is to identify what platforms are considered supported. If there's such a list then it's not so hard to identify the randomness sources on each of its members. Of course, someone needs to DO it... -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpUw2yoBK3Tx.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Wed, Apr 24, 2013 at 02:45:23AM +0200, Vincent Lefevre wrote: > On 2013-04-23 19:13:54 -0500, Derek Martin wrote: > > On Tue, Apr 23, 2013 at 11:58:05PM +0200, Vincent Lefevre wrote: > > > If the goal is to create a temporary file to view an attachment, the > > > contents of the attachment (and/or the mail itself) can be used as a > > > source of random data. I suppose that the attacker isn't the one who > > > sent the mail in question and the mailbox isn't public. > > > > You can't suppose that. :) The message may very well be one that > > was sent by the attacker, specifically to get the user to fall into > > his trap. > > If the attacker doesn't want the user to read his attachments, I don't > see the point of sending him a mail in the first place. :) Cute, but DoS is not the only vector as you well know. Using the message store or any part of the message store is not a workable solution. It's (in general) data from an untrusted source and must be treated as such. In particular, untrusted data must not be used as a source of input of any sort for functions used to secure the system. Doing so creates a potential attack vector. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpsGEVE533C7.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Wed, Apr 24, 2013 at 11:16:39AM -0600, Kyle Wheeler wrote: > So, what is the purpose of the entropy here? If the point is to find > a likely-to-be-unique name for temporary files associated with the > message, wouldn't something like an MD5 or SHA1 sum of the email in > question be sufficient? NO! I've already explained why. > An attacker might be able to predict the sum with a certain accuracy > if he already has access to the message, but... what exactly are we > protecting against here? There are AT LEAST three relevant classes of attack: - race condition allowing an attacker to take control of the CONTENT of the temporary file, possibly thereby effecting an attack via some other program (e.g. feeding bad data to an attachment MIME handler which allows arbitrary code execution). - DoS by (repeatedly) creating the temporary file that the application is trying to create, preventing it from ever being able to create a safe temporary file - Symlink attack causing the file to point to some other file than the user intended (possibly causing him to overwrite the contents of something important, e.g. /etc/password if it's writable by that user, or $HOME/.ssh/authorized_keys, etc.) There are probably more; I'm pretty knowledgable about this topic, but NOT an expert. To foil all of these, you need to ensure the file name is not predictable, that the range of file names is large enough that the attacker can't create a large portion of the possible values in the time frame of a potential race, and the file needs to be opened securely. You MUST do ALL of these things. There are really two aspects to getting this right: opening the file securely, and providing an adequately random name generator to avoid a collision. We already know how to do the former; what we're discussing now is the latter. If you use anything in the messge store as the source of your randomness, then potentially--and most especially if you use the current message being read, or any part of it, you give the attacker COMPLETE CONTROL over the randomness. Therefore, you GUARANTEE his success. > If the goal is to keep the attacker from DoS'ing the user... That is only PART of the goal. There are much worse outcomes. > let's think about this a little. Very smart people have already thought about this A LOT. There are numerous articles and papers on the topic. If you really want to implement a solution for this that doesn't use the system libraries, you should go read some of them. The conclusion (or one of them) is using untrusted data--e.g. a message supplied by anyone other than the user--as a source of randomness is FAIL. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpu1vs8jcumG.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Wed, Apr 24, 2013 at 03:37:37AM +0200, Vincent Lefevre wrote: > On 2013-04-23 20:06:10 -0500, Derek Martin wrote: > > Using the message store or any part of the message store is not a > > workable solution. > > I disagree. The security experts all agree on this... if you try to implement something like that, it will not withstand a rigorous security audit. > The entropy is typically created from untrusted data. Only indirectly, in a way that an attacker can not control without direct access to the target's hardware. On anything resembling a modern system, entropy is generally generated by measuring hardware timings (disk access, key presses, etc.), reading electrical noise from sound hardware, and other similar things that an attacker has no control over--unless they have physical access to the hardware, in which case you already pretty much lose. > Note that message headers generally contain random data from different > machines; It's NOT random. It's very predictable, if you're familiar with the receiver's site. And if you (the attacker) are AT the receiver's site (i.e. you're another user on the public server they're using to get their mail), all you need to do to get an EXACT COPY of the message is to CC yourself. You MUST NOT use the message as a source of randomness. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpot3GYAdpeF.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Wed, Apr 24, 2013 at 10:43:06PM +0200, Vincent Lefevre wrote: > It is random and not predictable. For instance: > > Received: from ioooi.vinc17.net (ioooi.vinc17.net [92.243.22.117]) > by xvii.vinc17.org (Postfix) with ESMTP id 66D0D40C037 > ^^^ WRONG. $ mail -s "testing" test1 test2 hi. . Cc: $ sudo md5sum /var/spool/mail/test1 f7b6d3ca015ad2f0b3f39e0dc6335763 /var/spool/mail/test1 $ sudo md5sum /var/spool/mail/test2 f7b6d3ca015ad2f0b3f39e0dc6335763 /var/spool/mail/test2 End of discussion, as far as I'm concerned. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpF3mpq6bCbI.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Thu, Apr 25, 2013 at 02:36:03AM +0200, Vincent Lefevre wrote: > On 2013-04-24 19:04:12 -0500, Derek Martin wrote: > > On Wed, Apr 24, 2013 at 10:43:06PM +0200, Vincent Lefevre wrote: > > $ sudo md5sum /var/spool/mail/test1 > > f7b6d3ca015ad2f0b3f39e0dc6335763 /var/spool/mail/test1 > > $ sudo md5sum /var/spool/mail/test2 > > f7b6d3ca015ad2f0b3f39e0dc6335763 /var/spool/mail/test2 > > > > End of discussion, as far as I'm concerned. > > This happens when mail is sent locally on the machine. You're wrong again--it happens whenever the recipient's mail server is the SMTP machine which recieves the mail, such as at many sites which use Windows clients with SMTP outgoing and IMAP incoming--but even if you're right, that's more than enough. Many people who use Mutt are college students who are using mutt on their local university server. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp1AhVTUrokU.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Thu, Apr 25, 2013 at 09:47:09AM -0500, Derek Martin wrote: > On Thu, Apr 25, 2013 at 02:36:03AM +0200, Vincent Lefevre wrote: > > On 2013-04-24 19:04:12 -0500, Derek Martin wrote: > > This happens when mail is sent locally on the machine. > > You're wrong again--it happens whenever the recipient's mail server > is the SMTP machine which recieves the mail, such as at many sites > which use Windows clients with SMTP outgoing and IMAP incoming--but > even if you're right, that's more than enough. Many people who use > Mutt are college students who are using mutt on their local university > server. In "normal" e-mail operations, this may be dependent on the behavior of the mail client, i.e. if it sends separate messages for each recipient. In an attack, this is not interesting. The attacker is at your site (he must be, in order to effect a temp file attack). So, he can either send the message to the target and himself from your machine, or he can just telnet to your SMTP server and write the message by hand. The point is, in exactly the circumstances where a temporary file attack would be possible, it is TRIVIAL for an attacker to get an exact copy of a message sent to the target. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp1sA4FgPYMt.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
Kyle, On Wed, Apr 24, 2013 at 01:48:58PM -0600, Kyle Wheeler wrote: > Assuming mkstemps() is sufficient for safely creating temporary > files, is there a reason not to simply borrow the FreeBSD > implementation? Their implementation seems relatively > straightforward and only relies on arc4random.c, which is entirely > self-contained. This comes back to another point I made: identifying what systems are considered supported. The implementation uses /dev/urandom, which both Linux and FreeBSD have, but not all systems do. Most modern POSIX-like systems do, but again... what's supported? There is (I think) one little problem with that implementation though: there's contingency code in the implementation for when opening /dev/urandom fails, but it just takes whatever was on the stack at the address of the data structure it allocates there. This is a very bad source of randomness. On systems that implement a secure stack, that value will be pre-initialized to a known value (probably 0 or some special implementation-specific value--there's a term for this but it escapes me at the moment). On systems that don't implement a secure stack, the value will depend on runtime-specific usage patterns, which may or may not be random, and which an attacker may be able to manipulate in some cases. If you can't open /dev/urandom on a system that has it, your system is broken, and in security-sensitive contexts, you should probably fail hard. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpUrZXkncHQS.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Fri, Apr 26, 2013 at 01:48:36AM +0200, Vincent Lefevre wrote: > On 2013-04-25 10:13:30 -0500, Derek Martin wrote: > > In "normal" e-mail operations, this may be dependent on the behavior > > of the mail client, i.e. if it sends separate messages for each > > recipient. In an attack, this is not interesting. The attacker is at > > your site (he must be, in order to effect a temp file attack). So, he > > can either send the message to the target and himself from your > > machine, or he can just telnet to your SMTP server and write the > > message by hand. > The server normally adds a "Received:" header with some information > unknown to the attacker. I'm sorry Vincent but you're wrong. The header gets added, it's the same in both the target and attacker's copies. All that's required to guarantee this is to name both recipients in the same SMTP transaction on the target's SMTP server. The copy delivered to both the target and the attacker will be 100% identical. I already showed you it was. > Anyway, I've seen that you haven't proposed anything else. > Is your point to make Mutt uninstallable on machines without > a /dev/random? I have in fact. Two and a half years ago even. Use mkstemp() and use the link() method to create the full name. The whole point of this subthread is that choosing not to rely on the system-provided library routines is folly. You can't provide anything better portably--your system libraries will already use the best source of randomness available to them. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpnOZ8uwyghE.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Fri, Apr 26, 2013 at 01:49:10PM +0200, Vincent Lefevre wrote: > On 2013-04-25 23:57:24 -0500, Derek Martin wrote: > > I'm sorry Vincent but you're wrong. The header gets added, it's the > > same in both the target and attacker's copies. > > Wrong. This is not what I can see. Well then you don't have a clue. Let me spell it out for you: Script started on Fri 26 Apr 2013 10:08:08 AM CDT $ ls -l /var/spool/mail total 0 -rw--- 1 ddm mail 0 Apr 26 10:04 ddm $ sudo useradd test1 $ sudo useradd test2 $ sudo useradd attacker $ telnet mail.mydomain.SANITIZED 25 Trying SANITIZED_IP... Connected to mail.mydomain.SANITIZED. Escape character is '^]'. 220 SANITIZED ESMTP Sendmail 8.13.8/8.13.8; Fri, 26 Apr 2013 10:09:04 -0500 ehlo pizzashack.org 250-SANITIZED Hello SANITIZED [SANITIZED_IP], pleased to meet you 250-ENHANCEDSTATUSCODES 250-PIPELINING 250-8BITMIME 250-SIZE 250-DSN 250-ETRN 250-DELIVERBY 250 HELP mail from: attacker 553 5.5.4 attacker... Domain name required for sender address attacker mail from: attac...@somewhere.com 250 2.1.0 attac...@somewhere.com... Sender ok rcpt to: test1 250 2.1.5 test1... Recipient ok rcpt to: test2 250 2.1.5 test2... Recipient ok rcpt to: attacker 250 2.1.5 attacker... Recipient ok data 354 Enter mail, end with "." on a line by itself From: m...@mydomain.com To: test1, test2, attacker Subject: hi! test . 250 2.0.0 r3QF946I031914 Message accepted for delivery quit 221 2.0.0 SANITIZED closing connection 250 2.0.0 r3QF946I031914 Message accepted for delivery quit 221 2.0.0 SANITIZED closing connection Connection closed by foreign host. $ sudo md5sum /var/spool/mail/* aa3c292a25e61fdb736a3507f8b75bc9 /var/spool/mail/attacker d41d8cd98f00b204e9800998ecf8427e /var/spool/mail/ddm aa3c292a25e61fdb736a3507f8b75bc9 /var/spool/mail/test1 aa3c292a25e61fdb736a3507f8b75bc9 /var/spool/mail/test2 $ sudo cat /var/spool/mail/attacker [sudo] password for ddm: From attac...@somewhere.com Fri Apr 26 10:10:49 2013 Return-Path: Received: from pizzashack.org (SANITIZED [SANITIZED_IP]) by SANITIZED (8.13.8/8.13.8) with ESMTP id r3QF946I031914; Fri, 26 Apr 2013 10:09:50 -0500 Date: Fri, 26 Apr 2013 10:09:04 -0500 Message-Id: <201304261509.r3QF946I031914@SANITIZED> From: m...@mydomain.com To: test1@SANITIZED, test2@SANITIZED, attacker@SANITIZED Subject: hi! test $ exit Script done on Fri 26 Apr 2013 10:11:25 AM CDT YOU ARE WRONG. This is a trivial attack. If you're unable to see that at this point, no one can possibly help you. > > The whole point of this subthread is that choosing not to rely on > > the system-provided library routines is folly. You can't provide > > anything better portably--your system libraries will already use the > > best source of randomness available to them. > > Of course one can: just add randomness to what the system libraries > provided. You can't. You can't be trusted to get it right. You've just provent that. > In particular, mkstemp() will just provide a unique > filename, without a suffix. The unique filename may be predictable > (the spec doesn't require that it shouldn't be). But the security community does. No one does this anymore, unless their system is incapable of providing adequate randomness. The system libraries already implement the best possible hope you have of getting this right. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpDveyjcoC4Y.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Fri, Apr 26, 2013 at 06:52:55PM +0100, Ian Collier wrote: > Opening the file with O_CREAT|O_EXCL is a solid defense against the > first two attacks in almost all common cases except when the temp > filesystem is mounted over NFSv2. That's right, but this is why your TEMP_DIR shouldn't ever be on NFS. Or at least, when NFS is NFS < v3. > The open questions as far as I'm concerned are: > > - what does the OS's mkstemp guarantee about randomness; The spec guarantees nothing; but in practice implementations are better than that. The reality is, if you care about security, you're not using an older system, you're not using NFSv2, and none of these issues affect you. If that's not the case, it's really your fault (and your problem). Again, it comes down to what platforms are (or should be) considered supported for Mutt. > - what does it guarantee about race conditions on NFSv2, and Nothing. > - is mkstemp followed by link any safer than mktemp followed by O_EXCL? No, but you can't link() a file that doesn't already exist, so you need to have opened the file securely in the first place. The discussion regarding the O_EXCL race in the man page describes how link() can be used safely to create a *LOCK FILE*; it is not adequate for opening a secure temporary file. The assumption is that in your lock file construct, the contents of the lock file are not interesting. With a temporary file, they are. > I therefore conclude that there is a large class of systems on which > mktemp et al do not use the best source of randomness available to them > unless you think the CPU's time stamp counter is difficult enough to > predict. The hardware clock is sufficient... when it is. In other words, on systems which have a clock resolution that is actually microseconds, it's fine. This should cover most modern systems. On systems where the resolution is actually only 1/60s, it's not. Though, even that is better than the suggestion to use the current e-mail as a source of randomness... but just barely. Using /dev/urandom on systems that have it isn't without its own problems... if your system doesn't have enough entropy, reading from /dev/urandom will block until it does. An attacker can drain all the entropy and issue a very large read, which will cause it to block essentially indefinitely. I'm telling you, this is a very hard problem. Solving this problem in a manner that's both good enough and portable is extremely difficult; it may even turn out to be impossible. You basically have to trust that the people who implemented your system did the best they could on your particular platform. The system should do its best to find a unique filename, to avoid the DoS to the extent that it's possible. BUT, it should expect to fail; if it fails repeatedly, the application should complain very loudly and warn you that you may be under attack, so you can go investigate, and find/fire/beat/arrest the attacker. As long as you do that, you're OK. And again, if you create a locked down, randomly named directory at application start time, and use that for all of your temporary files, you greatly minimize the risk of any of this being an issue, even on systems where it might otherwise be... -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgplYGxIzBsJL.pgp Description: PGP signature
Re: [Mutt] #3638: Compilation errors for 1.6
On Fri, Apr 26, 2013 at 08:57:45PM +0100, Ian Collier wrote: > On Fri, Apr 26, 2013 at 02:17:49PM -0500, Derek Martin wrote: > > Using /dev/urandom on systems that have it isn't without its own > > problems... if your system doesn't have enough entropy, reading from > > /dev/urandom will block until it does. > > On systems with a Linux kernel, /dev/urandom does not block but produces > lower entropy pseudorandom numbers instead. The /dev/random device > does block, and is used when full entropy is essential. Sorry, yes, you're correct. It's not just Linux... it's basically every major Unix variant in production today, though implementations vary. Also: http://en.wikipedia.org/wiki//dev/random In 2004, Landon Curt Noll, Simon Cooper, and Mel Pleasant tested a variety of random number generators, including the /dev/random implementations in FreeBSD 5.2.1, Linux 2.4.21-20, Solaris 8 patch 108528-18, and Mac OS X 10.3.5.[7] They indicated that none of these /dev/random implementations were cryptographically secure because their outputs had uniformity flaws. So, using the microsecond-resolution system clock is probably as good, if not better. Like I said, this is an annoyingly hard problem. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp6s_yDHJD31.pgp Description: PGP signature
Re: [PATCH] Fix segfault when sourcing file with error
On Wed, Aug 21, 2013 at 05:49:24PM -0400, Aaron Schrab wrote:
> Clear newly allocated error buffer to avoid attempt to dereference an
> invalid pointer when reporting an error while sourcing a file. Without
> this change I was seeing segfaults when attempting to source a file
> containing a send2-hook with an invalid regexp in the pattern.
> ---
> commands.c |1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/commands.c b/commands.c
> index 13b12dd..fea9408 100644
> --- a/commands.c
> +++ b/commands.c
> @@ -618,6 +618,7 @@ void mutt_enter_command (void)
> buffer[0] = 0;
> if (mutt_get_field (":", buffer, sizeof (buffer), M_COMMAND) != 0 ||
> !buffer[0])
> return;
> + memset (&err, 0, sizeof (err));
> err.dsize = STRING;
> err.data = safe_malloc(err.dsize);
> memset (&token, 0, sizeof (token));
Given this code and the fix, it seems likely that the problem you ran
into is that err.dptr is used uninitialized. Since err is half
initialized in the code that follows, I personally think it would be
slightly preferable to add:
err.dptr = NULL;
/* not sure if this is used, but that's not a reason to skip initializing it
*/
err.destroy = 0;
It's slighly more code, but it's clearer and avoids assigning
err.dsize and err.data twice.
But that said, your fix seems fine.
THAT said, I'm betting there are other places in the code where there
are BUFFER structs which aren't initialized properly. Might be worth
creating a function/macro to initialize these things properly... e.g.:
initialize_buffer(BUFFER *b, size_t size, int destroy)
{
b->dsize = size;
b->data = SOME_MALLOC(size);
b->dptr = NULL;
b->destroy = destroy;
}
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgp09QTZm5c_4.pgp
Description: PGP signature
Re: [PATCH] Fix segfault when sourcing file with error
On Thu, Aug 22, 2013 at 10:51:12AM -0500, Derek Martin wrote: > THAT said, I'm betting there are other places in the code where there > are BUFFER structs which aren't initialized properly. Might be worth > creating a function/macro to initialize these things properly... e.g.: In fact, such functions already exist, though I think they could be improved a bit or additional ones provided, so, for example, the user doesn't need to allocate the data buffer (as my suggested implementation in my previous e-mail does). Though, there are several ways to handle the memory allocation, which might beg for a few different versions of the constructor... which I won't elaborate on as I suppose it's obvious enough and probably not very interesting. In any event, having found the mutt_buffer_* functions (in protos.h and muttlib.c), I think the initializations should use those functions instead of calling memset(). -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp9U5X6fI_ty.pgp Description: PGP signature
Re: Fix segfault when sourcing file with error
On Thu, Aug 22, 2013 at 03:06:11PM -0400, Aaron Schrab wrote:
> There are a number of places where the data for the buffer is
> allocated in a different place than the buffer itself, or where the
> buffer data is a stack variable. That type of API wouldn't work in
> those cases.
Indeed... this is what I was alluding to when I mentioned there were
different ways to handle the memory allocation, and in general I would
prefer using stack-allocated structures to avoid possible memory
corruption issues.
You could go crazy, and have all of these (with whatever names):
/* allocate both the struct and the data member */
BUFFER *mutt_buffer_create (BUFFER *b, size_t s, int destroy);
/* allocate the data, but not the buffer */
void *mutt_buffer_alloc (BUFFER * const b, size_t s, int destroy);
/* allocate neither, use stack-allocated stuff */
void *mutt_buffer_construct (BUFFER * const b, char *data, size_t s)
{
/* data has to be (at least cast to) char * (not const) since it's defined
that way */
b->data = data;
b->dsize = s;
b->dptr = NULL;
/* nothing to destroy */
b->destroy = 0;
}
But this may be overkill. :) I'm not sure how/if "destroy" is used in
existing code, but if it isn't already you can use this such that all
three types of buffers can be passed to buffer_free(). Though I think
you still want two destructors: buffer_free which never free()s the
struct itself, and buffer_destroy which frees both the data and the
struct itself, if destroy is non-zero.
Anyway... I must be very bored today. =8^)
> --- 8< ---
>
> Subject: [PATCH] Initialize BUFFER variables
>
> Ensure that BUFFER variables are initialized to prevent later attempts
> to traverse an uninitialized pointer.
> ---
> commands.c |1 +
> hook.c |7 +--
> imap/command.c |1 +
> imap/imap.c|1 +
> 4 files changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/commands.c b/commands.c
> index 13b12dd..357b354 100644
> --- a/commands.c
> +++ b/commands.c
> @@ -618,6 +618,7 @@ void mutt_enter_command (void)
> buffer[0] = 0;
> if (mutt_get_field (":", buffer, sizeof (buffer), M_COMMAND) != 0 ||
> !buffer[0])
> return;
> + mutt_buffer_init (&err);
> err.dsize = STRING;
> err.data = safe_malloc(err.dsize);
> memset (&token, 0, sizeof (token));
> diff --git a/hook.c b/hook.c
> index 3fdcfb2..f9f8588 100644
> --- a/hook.c
> +++ b/hook.c
> @@ -281,7 +281,8 @@ void mutt_folder_hook (char *path)
> BUFFER err, token;
>
> current_hook_type = M_FOLDERHOOK;
> - +
> + mutt_buffer_init (&err);
> err.dsize = STRING;
> err.data = safe_malloc (err.dsize);
> memset (&token, 0, sizeof (token));
> @@ -332,7 +333,8 @@ void mutt_message_hook (CONTEXT *ctx, HEADER *hdr, int
> type)
> HOOK *hook;
>
> current_hook_type = type;
> - +
> + mutt_buffer_init (&err);
> err.dsize = STRING;
> err.data = safe_malloc (err.dsize);
> memset (&token, 0, sizeof (token));
> @@ -476,6 +478,7 @@ void mutt_account_hook (const char* url)
> if (inhook)
> return;
>
> + mutt_buffer_init (&err);
> err.dsize = STRING;
> err.data = safe_malloc (err.dsize);
> memset (&token, 0, sizeof (token));
> diff --git a/imap/command.c b/imap/command.c
> index 4b47de2..6dfeb62 100644
> --- a/imap/command.c
> +++ b/imap/command.c
> @@ -778,6 +778,7 @@ static void cmd_parse_lsub (IMAP_DATA* idata, char* s)
> url_ciss_tostring (&url, buf + 11, sizeof (buf) - 10, 0);
> safe_strcat (buf, sizeof (buf), "\"");
> memset (&token, 0, sizeof (token));
> + mutt_buffer_init (&err);
> err.data = errstr;
> err.dsize = sizeof (errstr);
> if (mutt_parse_rc_line (buf, &token, &err))
> diff --git a/imap/imap.c b/imap/imap.c
> index 83b05d6..b263abf 100644
> --- a/imap/imap.c
> +++ b/imap/imap.c
> @@ -1828,6 +1828,7 @@ int imap_subscribe (char *path, int subscribe)
> if (option (OPTIMAPCHECKSUBSCRIBED))
> {
> memset (&token, 0, sizeof (token));
> +mutt_buffer_init (&err);
> err.data = errstr;
> err.dsize = sizeof (errstr);
> snprintf (mbox, sizeof (mbox), "%smailboxes \"%s\"",
> --
> 1.7.10.4
>
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgpP2vWlVztIo.pgp
Description: PGP signature
Re: Fix segfault when sourcing file with error
On Thu, Aug 22, 2013 at 03:34:33PM -0500, Derek Martin wrote: > You could go crazy, and have all of these (with whatever names): Or you could pick one and always use it... Makes the patch more work tho. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpFtoaU1zyPQ.pgp Description: PGP signature
Re: The future of mutt...
On Mon, Sep 30, 2013 at 09:01:55AM -0400, Patrick Shanahan wrote: > * jpac...@redhat.com [09-30-13 07:29]: > > by this email I'd like to open discussion about the future of the mutt > > project. From year to year we can witness a small, but certain decline > > in the overall mutt project vitality. This is in direct contrast with > > the user-base which is (according to tickets and mailing lists) both > > quite active and interested. > > Considering mutt's maturity in it's *intended* design, improvements and/or > changes in direction greatly diminish one's expections of major changes > are not and cannot be seen which would give the impression of a loss of > "vitality". This is nonsense. There have been many discussions on this list about possible improvements, covering a wide range of functional and UI areas. The 1.6 release is supposedly waiting on some of those... and has been for a very long time now. And THAT completely ignores the idea that Mutt's design is over 15 years old, and its design philosophy is much, much older. Mutt, and its user base (or at least a substantial segment of it), could certainly benefit from being brought into this century. > The "user-base" activity you cite would give one the opposite impression > of *vitality*. No it doesn't, not in the slightest. The OP was talking about the vitality of development, not the vitality of the user base. The project is all but dead. > You apparently are dissatisfied with some function or lack thereof. Why > not suggest changes or improvements you feel necessary rather than > possibly creating a schism. Because such discussions invariably go nowhere. This is precisely the problem. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpUUzrjb8moE.pgp Description: PGP signature
Re: The future of mutt...
Eric, On Wed, Oct 02, 2013 at 12:29:23AM -0400, Eric S. Raymond wrote: > Derek Martin : > > This is nonsense. There have been many discussions on this list > > about possible improvements, covering a wide range of functional and > > UI areas. The 1.6 release is supposedly waiting on some of those... > > and has been for a very long time now. And THAT completely ignores > > the idea that Mutt's design is over 15 years old, and its design > > philosophy is much, much older. Mutt, and its user base (or at least > > a substantial segment of it), could certainly benefit from being > > brought into this century. > > I'm just a user, not a dev. But you might have heard my name before. I'm well aware of who you are, and this is not the first time we've corresponded... You've been around a long time, much as Mutt has. :) But not everyone is a 55-year-old hacker who wrote fetchmail. And not everyone wants to be. Please keep that in mind as you read the rest of my response. [And no offense is meant here; I'm simply quantifying, in some sense, where your perspective comes from... My perspective is not *so* dissimilar to yours.] > I'm open to improvements in the UI. There are some seriously annoying > misfeatures near PGP/GPG key management that could stand to be fixed. > One in particular gets me every time - if you try to PGP-encrypt > outgoing mail, but no key in your list matches, it is pure hell trying > to abort the key-selection mode. This is just the tip of the iceberg, really. Much more interesting would be overall improvements in new mail handling (one of the biggest things 1.6 is supposedly waiting on), better search capabilities and/or virtual folders, etc.--things that virtually all modern e-mail clients do; things that are nearly considered fundamental to modern e-mail users. These are features that people--including a lot of Mutt users--really want, but which Mutt does poorly or not at all. And--like it or not--HTML mail has become an important part of business communication for a lot of us, even in the technical world. However Mutt's ability to /handle/ that defacto standard is woeful, and its ability to /generate/ it is essentially nil. If you're willing to spend time hacking a solution together yourself, it's theoretically possible to make it work, but such a solution is far from polished or even what I would call usable. The one point I'll make about UI improvements is that it's extremely difficult to provide a polished, consistent UI, when the functionality to which you are interfacing is not your own--unless perhaps you've developed a fully modular framework that provides hooks for such things. But Mutt is not that. It's more like the Colonial Fleet in Battlestar Galactica: a mishmash of disparate ships (programs) that work together only begrudgingly, out of fundamental necessity and threat of extinction. =8^) This is why I think fundamentally, e-mail clients MUST move along the spectrum to a more (not necessarily entirely) monolithic design. Anyway, I was going to write yet another extremely long response to this until I realized I'm not saying anything I haven't said probably a dozen times on this list, every time the subject is broached by someone new, roughly every 6 months or so on average. So having said this much, I'm going to bow out of this discussion, and suggest that rather than rehashing it yet again, people go read the archives, looking for any of the many threads about mutt development stagnation. To briefly summarize though: There is a host of functionality that real users want, which it would be difficult to argue would not improve the usability, power and utility of Mutt. Ultimately, the maintainers themselves agree that development has stagnated, and consider it problematic, but seem to be unsure what to do about it, yet are also reticent to relinquish control of the project--or even to seek out fresh talent with more resources (time and motivation) to work on the various problems. Whenever this discussion occurs there's some flurry of activity around and concerning solving it, of varying duration and intensity, with progressively diminishing efficacy. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpCKU6n0Bkzq.pgp Description: PGP signature
Re: The future of mutt...
On Mon, Sep 30, 2013 at 01:28:03PM +0200, jpac...@redhat.com wrote: > by this email I'd like to open discussion about the future of the mutt > project. One thing I would like to point out here is that there IS some semblance of a roadmap... http://dev.mutt.org/trac/ I first would like to comment that I think development and fixing bugs are two different activities. To my best recollection all of the most recent work we've seen has been the latter; we've seen very little actual development for quite a while, AFAIK. At any rate... at the bottom of the page, you'll find links that point to the list of bugs for the 1.6 release to be considered ready, as well as an outdated list of bugs for the next point release (it still points to 1.5.21, which has been released--three years ago). There is also a list of topics under "Brainstorming" which describe less formally the roadmap beyond 1.6. Though, by way of describing the problems intended to be addressed, the pages so linked provide links to mailing list archives which are no longer accessible. To be useful again they should summarize the problem locally, or at least update the links to archives which are still accessible. The larger problem is no one is working on these items... Brendan started working on them, but clearly lacks the time and/or motivation to continue. As David said (and as I and others have pointed out many times in the past), volunteer developers have been repeatedly discouraged from contributing, if not by intent, by the practice of ignoring contributions due to lack of time, or, worse... Is there anyone around who has time and skill who wants to work on any of these roadmap items? I'm somewhat inclined to think that for progress to happen at this point, what will be required is for the maintainers to actively solicit volunteer developers to work on *specific* issues on which they actually want work to progress, and for the maintainers to commit to working with those volunteers to steer the work into something that can ultimately be accepted into Mutt. Without this, I don't see any path to progress... development isn't going to just spontaneously materialize. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpD4cw1EBlxk.pgp Description: PGP signature
Re: The future of mutt...
On Wed, Oct 02, 2013 at 12:04:46PM +0300, Alexander Gattin wrote: > On Tue, Oct 01, 2013 at 10:12:14PM -0500, Derek > Martin wrote: > > and has been for a very long time now. And THAT > > completely ignores the idea that Mutt's design > > is over 15 years old, and its design philosophy > > is much, much older. Mutt, and its user base > > (or at least a substantial segment of it), could > > certainly benefit from being brought into this > > century. > > The "old" design you talk about comes from UNIX > concepts which power all the iThings, Androids and > Kindles you most probably use and adore yourself. This isn't an unreasonable point, so I'll address it. I've been part of the Unix and Mutt communities for a very long time. I'm well aware of the philosophy behind its design, and the benefits thereof. But as I said, it has its limitations, and Mutt has, IMO, reached them. In Mutt's context, the Unix philosophy works very well for things like handling e-mail attachments, but it works much less well for things that are inherent to handling mail folders, like searching for mail. Users want, and rightfully expect, a powerful, uniform interface for doing that; but what Mutt offers is behind the times, largely due to its design, but also largely due to general stagnation. Mutt's design does make it possible for highly motivated people to solve *most* of these inherent deficiencies using a combination of complicated macros and a variety of external tools... but this is a BAD solution. It's bad because the interface is not uniform or consistent; it's bad because it requires the configuration and maintenance of maybe a couple dozen applications, instead of one or three; it's bad because it drastically increases ramp-up time and learning curve; it's bad because the amount of work involved to solve all such problems is prohibitive; it's bad because it requires every user to solve the problem again for himself. Mutt has become less an e-mail client, than it is the thread that holds all the pieces of your self-hacked e-mail client together. This made sense 20 years ago; it makes much less sense now, now that e-mail is both mature and a part of nearly everyone's daily life. > This "wanted functionality" thing are just input > data for developer that can be accepted, discarded > or reworked. Until we have a developer, this > discussion doesn't have any practical sense -- > there's no party that ultimately takes decisions. On this we agree; Mutt needs developers to avoid rotting. The question is, how to get them? > If you think mutt development is dead and miss > something badly, you are free to fork mutt and > implement missing features yourself. You should really go read the archives, as I suggested; this has been addressed many times. Forks are bad, and so is maintaining patches indefinitely. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgprSIm7NkBpb.pgp Description: PGP signature
Re: The future of mutt...
On Thu, Oct 03, 2013 at 10:22:00AM +0300, Alexander Gattin wrote: > On Wed, Oct 02, 2013 at 11:03:44AM -0500, Derek Martin wrote: > > In Mutt's context, the Unix philosophy works very well for things > > like handling e-mail attachments, but it works much less well for > > things that are inherent to handling mail folders, like searching > > for mail. > > I use / ~b for searching in IMAP folder. Yeah, but what if you have 50 folders, and you don't know which one the message you're looking for is in? I won't respond to the rest of your points except to say that this has all been argued out before. I'm far from alone in my opinions, but yes I must admit that they are just opinions. Again, I refer you to the archives. > I know about forking and maintaining patches > first hand (I had to maintain some against Linux > kernel - usually at some point I get tired and try > to slip at least most important parts upstream). Exactly. The problem here is that even if you had prepared good patches, the maintainers are too busy and/or too disinterested in your patches to take the time to even look at them (or, at least, they are very good at leaving people with that impression, hence this thread being rehashed repeatedly). > P.S. > My point was that users' forum is a bad place to > discuss development. Then your point is invalid; is isn't a users' forum, it's the developers' mailing list. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpw1TbedsVHa.pgp Description: PGP signature
Re: The future of mutt...
On Thu, Oct 03, 2013 at 11:21:40AM +0300, Alexander Gattin wrote:
> On Thu, Oct 03, 2013 at 02:46:07AM -0500, Derek Martin wrote:
> > On Thu, Oct 03, 2013 at 10:22:00AM +0300, Alexander Gattin wrote:
> > > I use / ~b for searching in IMAP folder.
> >
> > Yeah, but what if you have 50 folders, and you don't know which
> > one the message you're looking for is in?
>
> Usually I know approximately where the message is so I don't have to
> search 50 folders or dosens of archives
Yeah but that's you... Others do. This is a feature that's been
requested several times, and it's one I myself would love to have
though I've never said so--as I'm sure many other users who would like
it never have said so.
> > I won't respond to the rest of your points except to say that this
> > has all been argued out before.
>
> Yes, but you spoke in general words ("behind the times", "largely
> due to its design", "inherent deficiencies", "the interface is not
> uniform or consistent" and so on) and I hate general discussions
Well then go read the archives where the specifics have been
delineated, as I have already told you three times now, instead of
nagging me to waste my time by repeating them for the tenth time.
> > Exactly. The problem here is that even if you had prepared good
> > patches, the maintainers are too busy and/or too disinterested in
> > your patches to take the time to even look at them
>
> Well, let's look at my last patch for mutt:
> http://dev.mutt.org/trac/ticket/3566
[...]
> Mind you, the bug was only relevant for mutt+yandex users and for
> limited time, so not the top priority definitely.
What, you want a counter example?
http://dev.mutt.org/trac/ticket/3298
- Had a working patch 4 years ago.
- Dev apparently didn't like the patch, wrote his own without a
single word of comment on the original patch until he submitted his
own
- *dev got it wrong* -- did not address the issue at all.
- A new patch was supplied that worked more like what the dev had in
mind, and it has sat in that state ever since, untouched,
uncommented on.
- The person who actually submitted the bug has commit privs (now).
There have been lots of cases like this over the years, causing many
people who were interested in contributing to start threads like this
one, or to just simply walk away disgusted. And this was a small
bug/patch! Imagine if someone wanted to submit a large feature...
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgprxsAg62ydl.pgp
Description: PGP signature
Re: The future of mutt...
On Fri, Oct 04, 2013 at 12:06:29PM +0300, Alexander Gattin wrote: > On Thu, Oct 03, 2013 at 11:16:19PM -0500, Derek > Martin wrote: > > What, you want a counter example? > > Yes, > > > http://dev.mutt.org/trac/ticket/3298 > > This one is a miss. Sorry, you're wrong. Even if the patch was complete garbage, the point is there was never any discussion from the devs as to WHY. It was completely ignored for three and a half years. That's exactly the problem we're discussing. So no, it's not a miss. > > - Had a working patch 4 years ago. > > I don't like some parts of your original patch too, by the way. I don't care if you like it. The first patch was technically correct with the trivial exception of having removed the string.h header. Since strchr() returns an int, and the function was called correctly, this error has ZERO impact on the patch. But it's also not interesting, since there was a new patch within 24 hours. The only version of the patch which had any technical errors was the third... I tried to clean up some of the code which calls this function, and made a mistake in doing that. The final version fixes that. > MUA may choose to operate offline, so some hacks around libresolv > (like reading /etc/resolv.conf) are OK instead of just returning -1: NO, THEY ARE NOT. If your system is not using DNS for local name resolution, then using anything in /etc/resolv.conf IS WRONG. PERIOD. There are explanations of this in the bug, as well as in the comments of the last patch. > Also, the gethostname() method is just plain wrong in many > situations. gethostname() is NEVER wrong; it always gives you the configured host name of your host. If that ever returns an answer that's wrong, then it's because your system is misconfigured. Go fix it. Regardless, you clearly fail to understand how the call to gethostname is being used (in all versions of the patch). Even if your hostname is configured wrong, the method I used will still work, so long as your system is consistently configured wrong. In other words, it will use the configured hostname to look up its hostname using your configured name resolution service. Internally it looks up the IP associated with your hostname (which if you are misconfigured, it will most likely get by looking for a match in /etc/hosts, which is hopefully likewise misconfigured), and then does a name resolution on that IP address. It only uses what gethostname() returns as your domain name if that name resolution process fails, and the node name contains a domain. But if the name resolution process fails, your system is badly misconfigured. On hosts which frequently move between networks (and, I would argue, even on hosts which are fixed to a given network) the configured hostname should be UNQUALIFIED. The system should determine its qualified name via whatever host name resolution mechanism it is configured to use. If it does *ANYTHING ELSE* IT IS WRONG. If you want Mutt to use something OTHER than what the configured name resolution system gives you, or the system has no qualified domain name, you MUST configure Mutt to tell it what to use. > IMHO it's obvious your patch was wronfg and original implementation > is correct. Then you clearly don't understand how hostname resolution works on Unix systems as well as you think you do. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpzXH_DvAQuI.pgp Description: PGP signature
Re: The future of mutt...
On Sun, Oct 06, 2013 at 11:01:20AM -0700, Alexander Gattin wrote: > On Fri, Oct 04, 2013 at 01:06:35PM -0500, Derek > Martin wrote: > On the other hand, your patch breaks some of > setups that do use DNS. It doesn't break anything, because if you don't get the domain you expect, you simply set it in your muttrc. BUT it won't be wrong if you configure file resolution, and make sure you have appropriate entries in your /etc/hosts file, which you MUST do if your system is going to work correctly, in the cases you described. As long as you do that correctly, this patch WILL work. If you go read the update to the bug I just submitted, you'll see that parsing /etc/resolv.conf directly can be wrong, even when your system is using DNS, and is configured 100% correctly. That's not true with the patch. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpbHqxyEbtx9.pgp Description: PGP signature
Re: The future of mutt...
On Sun, Oct 06, 2013 at 11:01:20AM -0700, Alexander Gattin wrote: > On Fri, Oct 04, 2013 at 01:06:35PM -0500, Derek > Martin wrote: > > Even if the patch was complete garbage, the > > point is there was never any discussion from the > > devs as to WHY. It was completely ignored for > > three and a half years. > > Yes, ignoring your patch because of > technical/stylistic problems is wrong. Probably > there was another reason, like the whole concept > of gethostname() being flawed and unclear, so devs > decided they had other things to do. > > > with the trivial exception of having removed the > > string.h header. Since strchr() returns an int, > > Well: > >char *strchr(const char *s, int c); > ... > > > > MUA may choose to operate offline, so some > > > hacks around libresolv (like reading > > > /etc/resolv.conf) are OK instead of just > > > returning -1: > > > > NO, THEY ARE NOT. If your system is not using > > DNS for local name resolution, then using anything > > in /etc/resolv.conf IS WRONG. PERIOD. > > If my system is not using DNS for local name > resolution, then I can still use > /etc/network/interfaces (or whatever) scripts to > edit /etc/resolv.conf just for mutt if so desired. > > On the other hand, your patch breaks some of > setups that do use DNS. > > > gethostname() is NEVER wrong; it always gives > > you the configured host name of your host. > > gethostname() is wrong when you use your own > preferred `hostname` on corporate network, not the > one the corporate DNS server assigns to you. > > Or when you connect via GPRS/PPP or any other > dialup? > > > On hosts which frequently move between networks > > (and, I would argue, even on hosts which are > > fixed to a given network) the configured > > hostname should be UNQUALIFIED. > > The example I've shown you was run on a system > with unqualified hostname. > > If I remove "127.0.1.1 ux280p.ckee" record from > /etc/hosts (it's put there by Debian's installer), > then /tmp/domainname/gethostname returns nothing > at all. You'd prefer empty answer? > > > The system should determine its qualified name > > via whatever host name resolution mechanism it > > is configured to use. > > My system has several IP addresses and several > hostnames (depending on interface/network). Its > `hostname` never changes though. > The one in example is called ux280p. > The one I'm currently writing this email from is > called x505. > > > If you want Mutt to use something OTHER than > > what the configured name resolution system gives > > you, or the system has no qualified domain name, > > you MUST configure Mutt to tell it what to use. > > I could use /etc/resolv.conf just fine. > > -- > With best regards, > xrgtn > -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp_bqjRMaYFz.pgp Description: PGP signature
Re: The future of mutt...
On Sun, Oct 06, 2013 at 11:01:20AM -0700, Alexander Gattin wrote: > On Fri, Oct 04, 2013 at 01:06:35PM -0500, Derek > Martin wrote: > If my system is not using DNS for local name > resolution, then I can still use > /etc/network/interfaces (or whatever) scripts to > edit /etc/resolv.conf just for mutt if so desired. You could, but that is a brain-dead solution: you can just set it in muttrc. And you MUST do so, if that is the only way mutt can reliably determine what to use. You can even do this programmatically, if you need to, by sourcing a different config file containing correct configuration settings for each domain your machine might be in, based on any arbitrary command you care to run in your muttrc. > On the other hand, your patch breaks some of > setups that do use DNS. It doesn't, unless those setups are misconfigured. > The example I've shown you was run on a system > with unqualified hostname. > > If I remove "127.0.1.1 ux280p.ckee" record from This is your misconfiguration. That host should contain the domain you want to use. Removing it is equally misconfigured (actually worse, as you likely need that entry for your system to work at all). > > The system should determine its qualified name > > via whatever host name resolution mechanism it > > is configured to use. > > My system has several IP addresses and several > hostnames (depending on interface/network). Wrong, it has exactly one hostname, as does every TCP/IP-networked host. It may have several domain names corresponding to the multiple IPs on your machine... those are not hostnames, they are domain names. > I could use /etc/resolv.conf just fine. YOU could, but Mutt can not; not reliably. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpHdw9n9MBIU.pgp Description: PGP signature
Re: The future of mutt...
On Sun, Oct 06, 2013 at 11:20:15AM -0700, Alexander Gattin wrote: > > It doesn't break anything, because if you don't > > get the domain you expect, > > Yes, it doesn't break, it just returns wrong > domain. If you set it in your muttrc, it will ALWAYS be right. If you fix your /etc/hosts misconfiguration, it will always be right. It's not my patch's fault you aren't configured properly. > > parsing /etc/resolv.conf directly can be wrong, > > Yes, it can. Yes, it can. Under those same circumstances, my patch CAN NOT be wrong. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpFBrksrWevF.pgp Description: PGP signature
Re: The future of mutt...
On Sun, Oct 06, 2013 at 11:20:15AM -0700, Alexander Gattin wrote: > On Sun, Oct 06, 2013 at 01:15:26PM -0500, Derek > Martin wrote: > Yes, it doesn't break, it just returns wrong > domain. Let me be very clear here: This patch CAN NOT produce a domain which is wrong, in the sense that any domain it obtains is a valid, configured, resolvable domain for that host. It MAY produce a domain which is not the one you want to use; in that case it is up to you, the user, to configure it properly in Mutt. The resolv.conf solution CAN, IN FACT, PRODUCE A WRONG DOMAIN. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgptvY2V2ttBT.pgp Description: PGP signature
Re: The future of mutt...
On Sun, Oct 06, 2013 at 12:02:24PM -0700, Alexander Gattin wrote: > On Sun, Oct 06, 2013 at 11:20:15AM -0700, > Alexander Gattin wrote: > > On Sun, Oct 06, 2013 at 01:15:26PM -0500, Derek > > Martin wrote: > > > parsing /etc/resolv.conf directly can be wrong, > > No solution is ideal here. I don't agree, though I may be using a different standard of "ideal" than you are. The patch is ideal, where the ideal which it conforms to must be that it programmatically determines the domain of the machine correctly in all circumstances where that is possible, and NEVER produces a result which is invalid for that machine. If we factor in what is possible, this is ideal. We can not expect code to read the domain from the user's mind (not yet anyway)... When it fails, because determining the domain which the user wants to use is impossible, the user must supply it. That is as ideal as configuration gets. The existing solution fails to do this. Even if you don't consider the patch to be ideal, by now it must be clear that it is technically superior and preferable to the existing solution. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpa6gqndswBA.pgp Description: PGP signature
Re: The future of mutt...
On Sun, Oct 06, 2013 at 12:01:05PM -0700, Alexander Gattin wrote: > You have several hostnames or A records or domain names or whatever. > Then you have `hostname`, which is configured in kernel, at least in > Linux (cat /proc/sys/kernel/hostname), which may match some A > record, or not. Or match partially. Your method does not even do > that. You're still making assumptions here that the host is using DNS, which is by no means the only way to configure host name resolution, and in fact almost all Unix systems use a combination of multiple methods simultaneously. The hostname need not match any A records. For my patch to work, all that matters is that the configured host resolution mechanism can turn the hostname into an IP, and that resolving that IP gets you a domain name. That's guaranteed to work, because if it doesn't your machine is broken. > Just imagine that there was no such thing as `hostname` in kernel. I'm not going to imagine that because it's a requirement of TCP/IP. All machines have a hostname. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp0yriSIxyGp.pgp Description: PGP signature
Re: The future of mutt...
On Sun, Oct 06, 2013 at 01:30:09PM -0500, Derek Martin wrote: > On Sun, Oct 06, 2013 at 11:01:20AM -0700, Alexander Gattin wrote: > > If my system is not using DNS for local name resolution, then I > > can still use /etc/network/interfaces (or whatever) scripts to > > edit /etc/resolv.conf just for mutt if so desired. > > You could, but that is a brain-dead solution Well, maybe not brain-dead, but certainly brain-damaged. It was a nice idea, it's just wrong. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpCoCzj8ndoR.pgp Description: PGP signature
Re: The future of mutt...
On Mon, Oct 07, 2013 at 10:56:13AM +0300, Alexander Gattin wrote: > On Sun, Oct 06, 2013 at 02:19:11PM -0500, Derek Martin wrote: > > I'm not going to imagine that because it's a requirement of > > TCP/IP. All machines have a hostname. > > In my opinion `hostname` in the kernel is just a hint. Well, I mean seriously, if you're going to tell me that a required part of configuring networking is just a "hint" then I thank you for giving me what I need to excuse myself from this conversation. It is non-negotiable that all machines configured for TCP/IP networking require a hostname, and that that hostname must resolve to a valid IP on that machine. A machine not configured thusly does not have a valid TCP/IP configuration, and as such it is outside the scope of what Mutt should consider. Your position is essentially that it's better for Mutt to get it wrong sometimes when the config is 100% correct, in order to be able to sometimes deal with a configuration that's absolutely broken, than it is to always get it right when the configuration is correct. That's insane. We're done here. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpotse6OcNSX.pgp Description: PGP signature
Re: The future of mutt... - intermediate aggregation
On Thu, Oct 17, 2013 at 09:54:06PM +0200, Oswald Buddenhagen wrote: > On Thu, Oct 17, 2013 at 10:29:49AM +0200, jpac...@redhat.com wrote: > > On 10/07/2013 10:29 AM, Oswald Buddenhagen wrote: > chasing behind a quick-moving branch with much lower quality standards is > anything between deeply demotivating and unrealistic - that's why you > would need paid people to accomplish that feat. Or at least, someone who has vision and focus, AND the time to dedicate to the project. Those people exist, though they are rare (the most prominent example being Linus T., during the time period when he was not being paid for his work driving Linux). Mutt might not *any longer* be able to garner that kind of support. The number of people I know who use Mutt today has become A LOT smaller than the number of people I know who previously used Mutt. It's a small project which fills a particular niche that is becoming less and less interesting, based on my conversations with many of the people I know who use or have used Mutt. Specifically: For a lot of people, its lack of modern features provided by the likes of gmail, Evolution, Thunderbird, etc. has come to outweigh the benefits of having a curses-based client with a small footprint. There clearly are some die-hard users who don't feel that way, but in my experience even many of those have spent a significant amount of time investigating whether something else would meet their needs better. Myself included. And for what it's worth, for me at least, it mostly boils down to a tie of tradeoffs, where sticking with the thing I already know breaks the tie. Some of the existing projects are pretty close to convincing me to move away from Mutt, though. And if there ever was a decent client that could work in BOTH curses mode AND GUI mode, I would switch in a heartbeat. > > ...] So, it would be a bit ill-judged if someone really wanted to make > > a huge refactorization or alike of mutt only to prepare it for major > > functionality improvements. > > > what you are saying is "mutt is from the last millenium, and that's > where it should stay". that's an understandable attitude for somebody > who merely wants to get rid of a backlog of 3rd party patches, but isn't > exactly a perspective that would motivate anybody to revitalize the > project (it's oxymoronic to start with). Indeed. And up to this point, the maintainers have more or less indicated an unwillingness to go down that path. Which in my mind puts Mutt basically in maintenance mode; i.e. development is dead. Sooner or later I will find something else that "sucks less" enough to move on... -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpWItp49Xh5E.pgp Description: PGP signature
Re: The future of mutt... - intermediate aggregation
On Thu, Oct 24, 2013 at 02:05:07PM +0200, Holger Weiß wrote: > > Of course, but they build only a minority and therefore if the others > > don't like their work, why not to revert the commit or rewrite the patch > > with prompting the original author that the patch was really bad? > > This sounds so awesome! No need for maintainers. The community will > just magically take over all their work. > > Of course, in practice, it doesn't work this way. Occasional > contributors add their favourite feature or fix a bug they stumbled > over. That's it. They provide patches, they don't do patch review. This hasn't been true for Mutt, at least historically. Some of the people who submit patches infrequently have taken the time to review other patches (myself included)... though many of those people now do actually have commit rights. I'm one of those people, who still does not have commit rights (and frankly, don't want them). I don't entirely disagree with what you're saying; I'm just saying it isn't a universal truth. And there are still at least a few in the Mutt community who are not maintainers but who still are willing (even eager) to take the time to do some of the things you are talking about. Though, it may not be enough... -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp9M53DdADRg.pgp Description: PGP signature
Re: The future of mutt... - intermediate aggregation
On Thu, Oct 24, 2013 at 08:50:51PM +0200, Holger Weiß wrote: > * Derek Martin [2013-10-24 10:46]: > > This hasn't been true for Mutt, at least historically. Some of the > > people who submit patches infrequently have taken the time to review > > other patches (myself included)... > > However, this misses the point I was trying to make. The Mutt project > always had dedicated maintainers (who've actually been criticised for > being too conservative more than once). I didn't miss the point, I was simply making a different one: There are no absolutes; different projects can succeed under different conditions. It just depends upon the personalities involved. Dedicated maintainers generally improve your chances at success; though not so much when they actively block progress by their inaction, whether due to design or indifference or some other factor. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpqHgetUZgNk.pgp Description: PGP signature
Re: Mutt crashes in batch mode.
Alastair, This unfortunately isn't a very useful bug report: First, you should upgrade to the latest stock Mutt, to see if the problem exists there. There's no sense spending time on bugs that are already fixed. Then, at a minimum, use muttbug to have mut generate a bug report with your configure options etc.. More usefully, see this link: http://dev.mutt.org/trac/wiki/DebugTips And send along a backtrace, and maybe even a full debug trace showing what Mutt is doing leading up to the crash. HTH -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. On Tue, Dec 10, 2013 at 10:46:57AM +, Alastair Growcott wrote: > There is an old (2008/2009) thread on an identical sounding bug. But that is > for an older version of Mutt than I have. > > I am using: Mutt 1.5.20 (2009-12-10) > I am trying to connect to a Microsoft exchange server and have the following > muttrc: > > set smtp_url=smtp://myn...@server.mydomain.com > set smtp_authenticators=login > set smtp_pass=PASSWD > set from=myn...@mydomain.com > > When I use mutt in interactive mode it works fine and sends messages without > prompting for authentication. > > When I use mutt with the -i option: > > mutt -i ~/blah.txt -s "Test message" myn...@mydomain.com > > It works in a semi-interactive mode and there are no errors, but it doesn't > send anything. > > When I use mutt in batch mode: > > cat ~/blah.txt | mutt -s "Test message" myn...@mydomain.com > > or: > > mutt -s "Test message" myn...@mydomain.com < ~/blah.txt > > Then it segfaults. > > > pgpFqzpbzRGoq.pgp Description: PGP signature
Re: Fix buffer overrun ...
On Wed, Mar 12, 2014 at 07:36:24PM -0700, Claus Assmann wrote:
> On Thu, Mar 13, 2014, Moritz Barsnick wrote:
>
> > "Release early, release often." ;-)
> > (Less than six months this time, instead of three years.)
>
> Thanks to a buffer overflow...
>
> It would have been much better if that didn't happen.
> Hmm, maybe it's finally time to get rid of strcat(), strcpy(), etc?
Consider the following short program:
#include
#include
int main(int argc, char **argv)
{
char a[3] = "abc";
char b[] = "fooliciouslylongstringamabob";
char c[] = "foo";
int s,t,max;
/* Compare the full string, i.e. whichever is larger */
s = strlen(a);
t = strlen(b);
max = s >= t ? s : t;
if (!strncmp(a, b, max)) printf("a (%s) and b (%s) are equal\n", a, b);
else printf("a (%s) and b (%s) are not equal\n", a, b);
/* Compare the minimum, so we don't overrun */
s = strlen(c);
t = strlen(b);
max = s <= t ? s : t;
if (!strncmp(c, b, max)) printf("c (%s) and b (%s) are equal\n", c, b);
else printf("c (%s) and b (%s) are not equal\n", c, b);
return 0;
}
This program makes use of only "safe" string functions, yet it has two
problems, including a buffer overrun, due to a programming error in
the initialization of a, and an incorrect result due to an incorrect
attempt to skirt a fundamental problem with the way that strings work
in C. It may or may not crash, depending on your architecture and how
the bits of memory happen to be aligned on your machine.
Any function that deals with memory manipulation (virtually all of
them) can be dangerous if you use it wrong. If you look at the code
where the "non-safe" functions are used, you'll see that in general
care is taken to make sure there is an accounting of bounds.
Unfortunately, sometimes when old code is updated, the maintainer
forgets to re-check that everything is copacetic. This can still
happen with the "safe" versions of all these functions too.
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgpJER7hwTEEd.pgp
Description: PGP signature
Re: Fix buffer overrun ...
On Fri, Mar 14, 2014 at 12:49:09PM -0700, Claus Assmann wrote: > On Fri, Mar 14, 2014, Derek Martin wrote: > > > Unfortunately, sometimes when old code is updated, the maintainer > > forgets to re-check that everything is copacetic. This can still > > happen with the "safe" versions of all these functions too. > > Sure, but those functions significantly reduce the risk. In general, sure... but in any particular program, not necessarily. It depends on how they're used. It depends on what errors the programmer is inclined to make. And it depends on how the data they are passed gets initialized. Also, as you saw if you looked into Moritz's message, it's not always possible to replace the "unsafe" function with the "safe" one, so it depends on that as well. There's also the practical to consider: If you have a moderately large base of old code that's known to work, that gets updated only infrequently, it's probably not worth spending the time to change the code in every conceivable place to update it to current coding standards. Such a change itself creates many opportunities to introduce new bugs, which will of course need to be discovered, debugged, and fixed. If the code is pretty stable (as Mutt has proven to be) it's very likely far less work to leave the code alone and fix the few bugs that come up in the course of making needed changes, than to make sweeping changes that don't affect the functionality at all. Though, perhaps you can find someone who wants to do that work and is committed to finding and testing every possible code path that may change as a result. Coding is easy/fun/interesting; testing sucks. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpIeimjurwvV.pgp Description: PGP signature
Re: Fix buffer overrun ...
On Fri, Mar 14, 2014 at 10:35:11PM +0100, Matthias Andree wrote: > I figured that the practical result depends on the compiler. clang on > FreeBSD 9.2 amd64 silently fixes up the b0rked initialization, gcc does > not. Neither complains, though, because the initialization is legal, > and the strncmp() inconsistency shown might be intentional, too. This > is an artificial example anyways and not so useful. I disagree; the example is, clearly, just an example, but it illustrates real mistakes that real programmers make in real code. A more likely method to make the initialization mistake is to dynamically allocate a buffer, and make an off-by-X error filling it. Or in sizing it in the first place. Happens all the time. I've found a few of those by compiling some code on a different architecture... in several cases it ran fine on 32-bit but crashed immediately on 64. And the second case is pretty obviously dumb, but it also happens to be a bug I fixed in production code in the past... ;-) When you're in a rush, or being careless (or just plain bad at writing code) it's not so hard to make bonehead mistakes. And "safe" functions don't universally safeguard you from that. The point is, using a "safe" function is a poor substitute for paying attention. A bug is a bug, and if you have one, you need to fix it. I dislike those warnings because any half-competent programmer is aware of the *n* versions of functions, and if you chose not to use one there was probably a reason. Please, Mr. Compiler, please stop complaining about something I know is perfectly valid and not broken. More specifically, -Wall should not enable these warnings, you should have to explicitly turn them on separately, IMO. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpe9X3as4E14.pgp Description: PGP signature
Re: Fix buffer overrun ...
On Sat, Mar 15, 2014 at 12:21:40AM +0100, Matthias Andree wrote:
> Am 15.03.2014 00:36, schrieb David Laight:
> > IIRC the warnings come from a property of the symbol in the linker
> > not the compiler...
That doesn't make it better. ;)
> There are strlcpy and strlcat, which take the output buffer capacity,
> and permit checking if truncation happened or no.
$ cat strjunk.c
#include
#include
int main(int argc, char **argv)
{
char *a = "foo";
char *b = "bar";
char c[7];
strlcpy(a, c, 7);
strlcat(b, c, 7);
printf("%s\n", c);
return 0;
}
$ gcc -o sj strjunk.c
[...]/ccW4BTCh.o: In function `main':
strjunk.c:(.text+0x38): undefined reference to `strlcpy'
strjunk.c:(.text+0x55): undefined reference to `strlcat'
collect2: ld returned 1 exit status
So... not so helpful.
> > OTOH generating a C++ exception is likely to be even nastier.
Indeed.
> > Some system's header files have started forcing programs to check
> > the error returns from some library functions.
> > That gets to be a PITA - is some cases you really don't care.
>
> Cast to void.
That's not any less annoying than checking a return value you don't
care about. It's 6 useless characters, * many occurences in your
program.
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgphi4z0_J77m.pgp
Description: PGP signature
Re: Fix buffer overrun ...
On Fri, Mar 14, 2014 at 04:52:51PM -0700, Claus Assmann wrote: > On Fri, Mar 14, 2014, Derek Martin wrote: > > > char *a = "foo"; > > char *b = "bar"; > > char c[7]; > > strlcpy(a, c, 7); > > strlcat(b, c, 7); > > I guess you didn't read the man page? Gee, I wonder why I never thought of that?!?! $ man strlcpy No manual entry for strlcpy $ man strlcat No manual entry for strlcat > Oh, you use Linux. So does probably the vast majority of the Mutt audience. > Of course they don't have useful man pages and miss important > functions. Geez, even SunOS 5.x has these. The fact is these functions are non-standard BSD-isms. You can't write portable code that uses them. > Ok, this isn't going anywhere useful. Indeed. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpwHZvJc2HAz.pgp Description: PGP signature
Re: Fix buffer overrun ...
On Sat, Mar 15, 2014 at 01:11:03AM +0100, Andreas Krennmair wrote: > * Claus Assmann [2014-03-15 01:00]: > >Oh, you use Linux. Of course they don't have useful man pages and > >miss important functions. Geez, even SunOS 5.x has these. > > Except strlcat() on Solaris has slightly different semantics than > the OpenBSD implementation. And here's a nice thread that puts forth very strong arguments that using these functions is dumb. https://www.sourceware.org/ml/libc-alpha/2000-08/msg00052.html The ultimate point of the thread is one I more or less already made in my first post in the thread: If you're going to do string manipulation without potentially botching it, you need to know how big your buffers are, AND you need to know that you don't have too much data to fit in them. If you're going to have to figure that out anyway, then using str[nl]cpy is not any better than just using strcpy. Or memcpy, for that matter. Especially if you do a lot of that... it's more efficient. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpXDAgL9gRMz.pgp Description: PGP signature
Re: experimental DAG sorting patch
On Mon, Mar 17, 2014 at 11:16:34PM +0100, Anders Helmersson wrote: > Several years ago I wrote a patch for allowing sorting mails that reply > to several messages. It is attached as an mq patch, see README.DAG for > details. The mails can be sorted using directed acyclic graphs (DAGs), > which can be seen as an extension of the tree structure. I might be interested in this patch (whether or not it gets applied), but this seems like it could get messy fast. What does it look like if you reply to 4 messages, where: - The first message is the original post - The second is a reply to the first - the third is a reply to both the first and the second - the fourth is a reply to only the first Can you show an example with that structure? Thanks > Since multi-reply messages occur very seldom (once a year on mutt-dev) I > don't find it worth implementing, but it could be of interest to someone > on this list. I actually reply to multiple messages very frequently. Not so much on this list (though, sometimes), but more often on work-related mailing lists and certain other Linux-related mailing lists. I'm inclined to think that they tend to happen infrequently because a) people don't realize the feature exists, and/or b) they are lazy about trimming replies anyway, so (as happens at work a lot) they just include the entire thread in every reply. [Sigh] -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpMdnwa6CoBl.pgp Description: PGP signature
Re: experimental DAG sorting patch
On Tue, Mar 18, 2014 at 06:57:40PM +0100, Anders Helmersson wrote: > On Tue, 2014-03-18 at 10:57:48 -0500, Derek Martin wrote: > > - The first message is the original post > > - The second is a reply to the first > > - the third is a reply to both the first and the second > > - the fourth is a reply to only the first > > > > Can you show an example with that structure? > > This will be the same as for the tree sorting. OK, I see how you're doing the DAG stuff. I guess to see what I wanted to see, I would have needed to specify a more complicated graph, something like this: 1 -> 2 2 -> 3 1 -> 4 4 -> 5 5 -> 6 6 -> 7 1 -> 8 3,7,8 -> 9 =8^) -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpYNy13ZJx54.pgp Description: PGP signature
group reply behavior
All, I wanted to take a moment to point out that I think the group-reply behavior, in particular when Reply-To is set, is not intuitive, and I dare say also not sane. g reply to all recipients Great! Exactly what I want. Except, if reply-to is set, it doesn't do that. What I want it to do is reply to the sender (obviously via reply-to if that is set and not being ignored) AND all of the recipients. But when Reply-To is set, it ONLY replies to the address in Reply-To. What is the rationale for this? It is counter to the user's expectation, and provides nothing that the simple reply does not. So what's the point? -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp6t7v2RhlrJ.pgp Description: PGP signature
while I'm at it, list-reply behavior tweak
Simple question: Is there any reason why list-reply should not use any List-Post headers to get the address to reply to? It seems like this would obviate the need for users to manually configure lists they were subscribed to, as long as they had appropriate headers... -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp16X4oC07OV.pgp Description: PGP signature
The impact of DMARC (Re: group reply behavior)
On Wed, May 21, 2014 at 03:04:29PM -0500, Derek Martin wrote: > All, > > I wanted to take a moment to point out that I think the group-reply > behavior, in particular when Reply-To is set, is not intuitive, and I > dare say also not sane. [...] > What I want it to do is reply to the sender (obviously via reply-to > if that is set and not being ignored) AND all of the recipients. But > when Reply-To is set, it ONLY replies to the address in Reply-To. It's probably worth mentioning... I'm asking about this due to ongoing DMARC discussions on various other mailing lists; inserting a reply-to header makes mail clients almost universally do the wrong thing with group-reply functionality, it seems. It seems likely that doing this is about to become a lot more popular, and mail clients that don't deal with this well are going to make a lot of people frustrated. Granted, Mutt's list-reply functionality mostly makes this uninteresting; however there are often times when I want to reply to BOTH the sender AND the list (e.g. if the mailing list is known to be kind of slow, and the poster is looking for a fast answer; or if the sender requests direct replies, etc.), and therefore don't use it. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpjRSoWFNCtN.pgp Description: PGP signature
Re: Mutt code indentation style?
On Thu, Sep 11, 2014 at 10:08:05AM +0100, Óscar Pereira wrote: > On Wed, Sep 10, 2014 at 12:32:04PM -0500, David Champion wrote: > >>>So I guess the question is... does someone still care about code > >>>indentation, or is it just the page [1] that is hopelessly outdated? > > > >I don't personally like the historical mutt coding style ]...] > > I should mention in passing that from what I've seen so far, I'm not > fond of mutt's coding style either: to me at least it makes code > harder to read. Same here. For what it's worth (which is very little, I should think) it's one of the two factors that's kept me from contributing more to mutt than the meager submissions I've managed to get included, the other being the difficulty in getting a maintainer to even comment on your patch submission, for much of anything other than straight-up segfault fixes... -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpMiXRTSlRt9.pgp Description: PGP signature
Re: Mutt code indentation style?
On Thu, Sep 11, 2014 at 10:12:43AM -0700, Gary Johnson wrote:
> I think that's just a matter of taste and what your used to.
I don't. To some extent, yes... definitely. But some coding styles
really do make code harder to read, and code that's harder to read is
harder to understand.
The GNU standard:
- makes it harder to line up indented blocks visually, because the
indent is too small to be easily distinguishable given many nested
blocks (especially if they are not neatly balanced)
- makes it harder to distinguish conditionals from function calls,
both for the programmer, and for coding tools that try to care
- empty parens separated from their function name by a space look
like an operator (and indeed in some languages, they are)
- wastes lines with nothing but a '{'
The latter point may seem trivial, but if you're trying to READ a
function written this way, it can result in as much as 50% of your
logic being postponed to the next screen--depending on the contents of
the function--making it harder for you to refer to earlier parts of
the code without paging, losing sight of the part of the code you're
trying to decipher. This is bad.
Lastly, some editors / IDE's may rely on function calls being
formatted without a space [e.g. foo() vs. foo ()] to properly colorize
function calls. For instance, in VIM, this is the only way to
colorize C function names in your program (there's a keyword to tell
vim the names of functions, but it's really only useful for things
that are standard, or at least common library code). You can tell vim
to do this with something like:
sy match cFunction "[A-Za-z0-9_]\+("me=e-1
But! If you configure this:
- if(foo) will be colorized like a function call
- foo (bar) will not be colorized, since foo is not a keyword, and it
does not match the pattern
If you don't configure it, only function names that vim knows about
(mostly standard library / POSIX functions) will be colorized.
Functions in your code won't be colorized, producing inconsistent
colorization.
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgptcwaTFcjBM.pgp
Description: PGP signature
Re: Mutt code indentation style?
On Tue, Sep 16, 2014 at 11:09:04AM +0200, Vincent Lefevre wrote:
> On 2014-09-11 19:42:43 -0500, Derek Martin wrote:
> > On Thu, Sep 11, 2014 at 10:12:43AM -0700, Gary Johnson wrote:
> > > I think that's just a matter of taste and what your used to.
>
> I agree, but I just think that tab characters should never be used
> for indenting. They can break alignment in diff output, and break
> copy-paste under some conditions too.
In principle it doesn't matter, as long as they are used (or not used)
consistently. I used to favor tab for indent, space for alignment,
BUT...
In practice it's unworkable. Humans aren't meticulous enough to get
it right consistently, and you can't even get people to configure this
consistently in their editors, so your source code becomes a mess.
Pretty much guaranteed.
> With the GNU coding style, one generally has a 4-column indent,
> due to the "{" / "}" lines, which is enough (and sometimes too
You don't. The '{' lines count--they are not invisible. So you get a
bunch of blocks that look like someone is drawing mountain ranges,
instead of writing blocks. It also wastes lines, as I said.
for (i = 0; i < n; i++)
{
if (i == 2)
{
do_some_stuff();
do_other_stuff();
}
something_else();
}
blech();
Horrible. The braces are nothing but noise, and reduce the blockiness
of your blocks.
for (i = 0; i < n; i++){
if (i == 2){
do_some_stuff();
do_other_stuff();
}
something_else();
}
yay();
So much more concise (20% fewer lines), clearer, and the brackets line
up with the flow control statements which start their respective
blocks so you can find them at a glance. They also line up with other
statements at the same logical indent level, which they don't with
GNU. Much, much easier to visually parse.
It matters little in these trivial examples, but in longer blocks
with more complicated logic, it can matter very much. Especially if
someone updating the code makes a few mistakes here and there... it's
generally a lot easier to pick them out because they stand out more as
being out of place.
> much in case of many nested blocks). I've seen code with 8-column
> indenting, and this is completely unreadable in a 80-column
> terminal because most of the lines are too long.
This can happen, but I've often found it happens due to bad design.
Well structured code can avoid it most of the time... it's very often
a sign your functions are doing too much. I think 4 is a good
compromise, but 8 is better than 2. When it can't be avoided,
splitting the line and lining up your function arguments keeps it most
readable. You may also just have to try harder to keep element names
short but still descriptive.
> Now, perhaps editors should provide block coloring to distinguish
> them better (or side bars in different styles).
>
> > - makes it harder to distinguish conditionals from function calls,
> > both for the programmer, and for coding tools that try to care
>
> If you mean "if (...)" vs "foo(...)", I think it should be better
> done via coloring of *keywords*.
No, that's a giant fail.
int my_func(void);
[...]
int main(...)
{
int a = my_func();
...
}
Keyword coloring can not color my_func, EVER, because it is not a
keyword. Syntax coloring can. Some editors, like vim, can be told
that my_func is a function, and colorize it accordingly... however if
two different programmers use that name, and one uses it as a function
and the other uses it as data, you lose. If you stick to foo() as
function call and foo () as conditional, it always gets colorized
properly (at least in all of the currently most popular languages).
Sure, it dictates some elements of style, but it only serves to
eliminate BAD style. ;-) (Yes, that's somewhat tongue in cheek.)
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgp9CVZCz0YRW.pgp
Description: PGP signature
Re: Solutions for CVE-2014-9116 (ticket #3716)
On Mon, Dec 01, 2014 at 11:00:11AM -0800, Kevin J. McCarthy wrote: > In general, mutt checks NULLs pretty well, but returning NULL from > mutt_substrdup() isn't without risk of just generating a segfault in > another place. So personally I would vote for the second or third > choice. Returning NULL from something that allocates memory is always a good (and I dare say the best) option. It's the standard behavior of all C standard memory allocators, and if the caller crashes on NULL dereference, in general you will know EXACTLY where the problem is. Your function could also make use of errno (setting it for instance to ENOMEM or EINVAL) to further clarify what actually went wrong. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp6_sWct6Vzx.pgp Description: PGP signature
Re: Solutions for CVE-2014-9116 (ticket #3716)
On Mon, Dec 01, 2014 at 06:13:04PM -0800, Kevin J. McCarthy wrote:
> > Returning NULL from something that allocates memory is always a good
> > (and I dare say the best) option.
[...]
> Note that mutt's safe_malloc() exits the program when an allocation
> error occurs though.
...excepting the case where the requested size of the memory being
allocated is zero. In that case, it returns NULL (well, zero, but
close enough--I'd prefer it actually returned NULL so the intention is
more clear, but that's mostly a style point).
Note that this behavior is probably the Wrong Thing™, since it appears
that the point of safe_malloc is to make sure that it never returns
NULL, so that the caller need never check the return value. Certainly
that assumption was made in the code in mutt_substrdup()... Thus that
function should probably also exit in that case, though with a
different error message to distinguish the case. I would argue that
such a case is clearly a bug, so it should probably abort() to produce
a core dump.
Likewise, mutt_substrdup() should check its arguments to make sure
they can not produce a string of size less than 0, and abort if they
would.
Regarding SKIPWS() / skip_email_wsp():
It seems to me that there are only two cases here: one where mutt
should only skip ' ' and '\t', and another where it should skip all
whitespace in the current locale. So seems to me this could be solved
by defining skip_email_wsp() as:
#define EMAIL_WSP " \t"
/* s must be returned non-const; skip const and skip the casts */
static inline char *skip_email_wsp(char *s, int strict_space)
{
if (!s) return s;
if (strict_space) return s + strspan(s, EMAIL_WSP);
while (*s && isspace((unsigned char)*s)) s++;
return s;
}
Then just call skip_email_wsp with the right flag value for the case
you're dealing with, i.e. 1 for rfc822.c and sendlib.c, and 0
otherwise.
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgpZCoe_rtG7o.pgp
Description: PGP signature
Re: Solutions for CVE-2014-9116 (ticket #3716)
On Sat, Dec 06, 2014 at 11:28:37AM -0600, Derek Martin wrote:
> Likewise, mutt_substrdup() should check its arguments to make sure
> they can not produce a string of size less than 0, and abort if they
> would.
Note that in the case of size == 0, mutt_substrdup() need not call
safe_malloc() at all, it can simply return strdup(""). It needs to do
at least that though, as the caller will expect that it needs to free
its return value. That is, unless analysis can prove that no caller
ever expects the possibility of a zero-length substring, in which case
that case also can trigger an abort().
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgpclVnFgr18O.pgp
Description: PGP signature
Re: Solutions for CVE-2014-9116 (ticket #3716)
Lastly note the definition of safe_strdup():
char *safe_strdup (const char *s)
{
char *p;
size_t l;
if (!s || !*s)
return 0;
l = strlen (s) + 1;
p = (char *)safe_malloc (l);
memcpy (p, s, l);
return (p);
}
In particular, note that this returns NULL when (!*s). However
strdup() will actually return a perfectly valid string in this case: a
malloc'd single-character string containing '\0' (i.e. ""). Which is
useful, unless you KNOW that is a bug in your program.
If it's intended that it is a bug, then the line should be changed to:
if (!s || !*s){
mutt_error("safe_strdup passed NULL or a NULL string");
abort();
}
Otherwise, just check s (but still abort).
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgpw3lQdWPEm3.pgp
Description: PGP signature
Re: [PATCH] Add abort to mutt_substrdup
Just noticed one other thing I didn't notice before: On Sun, Dec 07, 2014 at 02:27:13PM -0800, Kevin J. McCarthy wrote: > Add parameter checking and abort to mutt_substrdup. (references #3716) [...] >if (end) > len = end - begin; This strikes me as wrong. It depends on how the caller passes end... but this seems likely a fence post error to me (obviously I did not look--I would do so now but I need to step out, so I just have time to type this quick note). If the string is size 4, and starts at address 0, then it occupies addresses 0-3. 3-0 - 3, not 4. Obviously if end is the next character after the end of the string, this works as expected. If that is the case it should be called out as such with a comment. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpM9fNGAxUW4.pgp Description: PGP signature
Re: [Mutt] #3723: Message fetched if `color index blue red "~h 'Importance: high'"` is in place
On Tue, Dec 30, 2014 at 11:10:09AM -, Mutt wrote: > If color index … … "~h –" is set up, mutt goes on fetching every message > on the current page, after all headers were fetched. Is it fetching the messages so that it can match on the MIME headers? -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgphzO_eSRkx8.pgp Description: PGP signature
Re: mutt: Add option to encrypt postponed messages. (closes #3665)
On Tue, Jan 13, 2015 at 08:52:41AM -0800, Kevin J. McCarthy wrote: > Just to make sure the feature is clear, this patch only encrypts the > postponed message to $postpone_encrypt_as. Even if the message were > somehow obtained, signed or unsigned, it would still only be readable by > you. I was going to suggest this anyway, but I think this drives home the point: postpone_encrypt_as should really be called postpone_encrypt_to--it specifies the encryption key TO which you will encrypt the data. Clearly this need not be your own key, though very little else would normally make sense. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp9PoPyp4gQD.pgp Description: PGP signature
Re: remote instances
On Sat, Feb 28, 2015 at 07:41:13PM -0500, Patrick Shanahan wrote: > > of course multiple instances are possible. i want mailto: to be handled by > > a single instance of mutt; the one running on tty1 in my case. > > But why jump thru hoops when there is no necessity. What reasons are > there? Because the user wants a different user experience than the one provided by running multiple instances of Mutt, obviously. What is so hard to understand about that? Here's an example of why: I run my desktop with all my "applications" in windows which are tiled in essentially fixed positions on my screen (with some overlap, but as little as I can manage). There is not a shred of free screen real estate on my main desktop; I need to switch to alternate desktops when I want to do something out of the ordinary, in order to not mess with those other windows overmuch. When applications pop up new windows, it breaks how I work in some fashion or other--not that I can't handle the occasional pop-up, but in order to keep my workflow streamlined I want that to happen as infrequently as possible. Adding something where new windows pop up with new mutt instances is generally unwelcome in my workflow, no matter how well it may "work" in the context of Mutt. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpGZrCPJ9q6Q.pgp Description: PGP signature
Re: mutt leaves temporary files in $TMPDIR
I see this also. Actually I reported a bug about this (or something
very similar) many years ago (13 in fact), which Thomas provided a fix
for. I wonder if it just got regressed somwehre along the line, or if
there was a separate issue (or perhaps the original fix was
incomplete). I've been too lazy to report it again, but I can confirm
that it still happens for me with more recent mutt versions.
http://marc.info/?l=mutt-dev&m=103964554117728&w=2
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
On Sat, Feb 28, 2015 at 08:45:34PM +0100, Dennis Preiser wrote:
> Hello,
>
> I've observed the following issue with mutt:
>
> When multiple
>
> -BEGIN PGP xyz——
> -END PGP xyz——
>
> blocks are present and (Esc P) was invoked, mutt
> leaves temporary files in $TMPDIR.
>
> Root cause seems to be pgp.c:pgp_application_pgp_handler(). If multiple
> blocks are present (a signature and a public key; more then one public
> key etc.) mutt_mktemp() and safe_fopen() are called multiple times for
> tmpfname and outfile without closing and unlinking before.
>
> Attached is a sample mbox file with two public key blocks. To reproduce
> this issue open the mbox, invoke (Esc P), quit
> mutt and inspect $TMPDIR.
>
> The attached patch fixes this for me.
>
> Dennis
> diff -urp mutt-1.5.23hg_orig/pgp.c mutt-1.5.23hg/pgp.c
> --- mutt-1.5.23hg_orig/pgp.c 2015-01-28 09:00:03.0 +0100
> +++ mutt-1.5.23hg/pgp.c 2015-02-26 12:39:21.0 +0100
> @@ -444,6 +444,16 @@ int pgp_application_pgp_handler (BODY *m
>mutt_error _("Could not decrypt PGP message");
> mutt_sleep (1);
> rc = -1;
> + if (tmpfp)
> + {
> + safe_fclose (&tmpfp);
> + mutt_unlink (tmpfname);
> + }
> + if (pgpout)
> + {
> + safe_fclose (&pgpout);
> + mutt_unlink (outfile);
> + }
> goto out;
> }
>}
> @@ -501,6 +511,16 @@ int pgp_application_pgp_handler (BODY *m
> else
> state_attach_puts (_("[-- END PGP SIGNED MESSAGE --]\n"), s);
>}
> + if (tmpfp)
> + {
> +safe_fclose (&tmpfp);
> +mutt_unlink (tmpfname);
> + }
> + if (pgpout)
> + {
> +safe_fclose (&pgpout);
> +mutt_unlink (outfile);
> + }
> }
> else
> {
> @@ -517,17 +537,6 @@ int pgp_application_pgp_handler (BODY *m
> out:
>m->goodsig = (maybe_goodsig && have_any_sigs);
>
> - if (tmpfp)
> - {
> -safe_fclose (&tmpfp);
> -mutt_unlink (tmpfname);
> - }
> - if (pgpout)
> - {
> -safe_fclose (&pgpout);
> -mutt_unlink (outfile);
> - }
> -
>FREE(&gpgcharset);
>
>if (needpass == -1)
> From dp-m...@d--p.de Wed Dec 26 18:45:17 2012
> Date: Wed, 26 Dec 2012 18:45:16 +0100
> From: Dennis Preiser
> To: Dennis Preiser
> Subject: message with multiple PGP public keys
> Message-ID: <20121226174515.ga17...@coredump.d--p.de>
> MIME-Version: 1.0
> Content-Type: text/plain; charset=utf-8
> Content-Transfer-Encoding: 8bit
> Status: RO
>
> Lorem ipsum dolor sit amet, consectetur, adipisci velit …
>
> pub 2048R/F9C68CE7 2011-10-05 Dennis Preiser
> uidDennis Preiser
> sub 2048R/49C64B1D 2011-10-05
>
> pub 2048R/F9C68CE7 2011-10-05 Dennis Preiser
> uidDennis Preiser
> sub 2048R/49C64B1D 2011-10-05
>
pgp0VD4zd4L8t.pgp
Description: PGP signature
Re: remote instances
On Mon, Mar 02, 2015 at 02:18:49PM -0500, Ben Boeckel wrote: > On Mon, Mar 02, 2015 at 12:23:54 -0600, Derek Martin wrote: > > Here's an example of why: [example omitted for brevity...] > Some alternative solutions I can think of (since this problem seems > hardly restricted to mutt's behavior): Note that what I wrote was an example relevant to me... it may have nothing to do with what the OP wants or his reasons for wanting it. > - use a tiling window manager (since this seems to match how you are > working now); or > - use tmux to spawn terminal commands inside of existing windows; or > - use KWin which can enforce your window positions using rules and > then nothing can interfere with them. That said, if your screen is already full (as mine is) and your apps are already the size you want them (as mine are), I'm not sure how any of these really help... other than maybe the tmux solution. But as I say, the OP's reasons may be unrelated, and any of these may or may not address his issue. The functionality the OP is describing is not exactly ground-breaking; it has existed for decades elsewhere, and is kind of nice. If someone were motivated, it doesn't seem like it would be all that hard to add... -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpXkFV_cz4xM.pgp Description: PGP signature
Re: CVE-2014-9116 and mutt developer version
On Mon, Mar 02, 2015 at 02:29:34PM -0800, Kevin J. McCarthy wrote: > Christian Rebischke wrote: > > Hello Guys, > > Sorry for interrupting your development process. I have one question to the > > CVE-2014-9116. Its fixed in stable yes. But is it fixed in the development > > version 1.5.23 too? I can't find any information about this. > > It was fixed in our hg repos at commit 0aebf1df4359, but we have not > had a release with the fix in it yet. It does seem that making one would be prudent... Most people run a dev build, and many people build from source. They would not have the fix unless they were paying close attention and build from the repo. Was there at least notification of this on the announce list? -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpmD1fHNlSLN.pgp Description: PGP signature
Re: choices in crypt-gpgme.c
On Sat, May 09, 2015 at 10:39:39AM -0700, Kevin J. McCarthy wrote: > When I rewrote those functions, "f" seemed to be an undocumented choice > doing the same thing as (c)lear and corresponding to "forget it" in the > comments. I wrote the patch that led to these menus being in their current state, arguing that the original menus were too ambiguous and informal. The original choice was "(f)orget it"--it took some convincing for Thomas (Rosselar) to take the patch, and when he did he insisted on keeping the (f) choice as an undocumented choice because his brain was hard-wired to use it. There's a discussion about this in the archives... if anyone cares to dig it up. Probably about 10 or 11 years ago I'm guessing, but I could be wrong. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpPRq79sn8mD.pgp Description: PGP signature
Re: choices in crypt-gpgme.c
On Mon, May 11, 2015 at 09:48:05AM +0200, Moritz Barsnick wrote: > On Sun, May 10, 2015 at 21:47:07 -0500, Derek Martin wrote: > > There's a discussion about this in the archives... if anyone cares to > > dig it up. Probably about 10 or 11 years ago I'm guessing, but I > > could be wrong. > > Wow, that was (slightly) before I subscribed to the list. ;-) > > It must be this thread (yes, between 10 and 11 years ago, bingo), > though I can't get gmane to show me the whole thread: > http://thread.gmane.org/gmane.mail.mutt.devel/6144 Yep that's the thread I meant. I had actually brought this up first about a year earlier, and the original version of the patch I wrote (still up on pizzashack.org) is dated Sept. 10, 2003. So yeah, long time ago. :-) -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpMMKvQxs6lM.pgp Description: PGP signature
Re: choices in crypt-gpgme.c
On Sun, May 17, 2015 at 01:04:41PM -0400, David J. Weller-Fahy wrote: > Also, here's the original bug thread: > > http://marc.info/?t=10837990403&r=1 Yes and no... IIRC in that thread I actually pointed out several older related bugs. The actual orignal bug was: http://dev.mutt.org/trac/ticket/1022 That bug is hard to find in the archives though, and the bug itself in trac is somehow a mishmash of the original plus the one you pointed out which I opened. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpssPWFhvCyR.pgp Description: PGP signature
Re: [patch] Implement GMail-style muting of threads
On Wed, Sep 09, 2015 at 12:12:19AM +0200, Andreas Schäfer wrote: > I've prepared a patch for mutt 1.5.24 which allows users to mute > threads like Google Mail does. Does collapsing the thread not suffice? -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp1V5gJluYrV.pgp Description: PGP signature
Re: recovering trac account
On Fri, Sep 11, 2015 at 10:40:41AM +0200, Olaf Hering wrote: > How does one get into trac? The "Forgot password" link requires username > and email, I know just the latter. Username is not obvious from Ticket #2421. For what it's worth, I've tried recovering my trac account several times, and even after (seemingly successfully) doing so, I was still not able to get in with it. I'm pretty sure I've seen other list members say the same thing. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpoNBAHamYpR.pgp Description: PGP signature
Re: Feature Request: Pattern Modifier Corresponding to Format String %F
On Tue, Oct 27, 2015 at 09:53:58PM -0400, Grady Martin wrote: > Hello, all. When index_format contains %F, it is confusing when > simple_search is unable to match what is being displayed. How does > everyone else feel? I feel like if this is a problem for you, you should change index_format and/or simple_search such that they match, in whichever way you find preferable. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpP9wk5rIMrq.pgp Description: PGP signature
Re: Feature Request: Pattern Modifier Corresponding to Format String %F
On Thu, Oct 29, 2015 at 02:21:59PM +0100, Michael Tatge wrote: > * On Thu, Oct 29, 2015 05:28AM -0400 Grady Martin (sunnycemet...@gmail.com) > muttered: > > Searching for "Derek," for example, does not return messages from you. > > Why would you search for "Derek[comma]"? Standard English quoting rules... > Searching for "Derek" with the default $simple_search "~f %s | ~s %s" > returns lots and lots of messages from "Derek Martin". He means, when he himself is the sender (i.e. the "from" address). My first message was meant to be a hint, rather than to spell out the answer. Here's another hint: simple_search, like index_format, is a variable. It can be changed... You probably can't get precisely what you want, but you can approximate it. Search != display--they do different things, and have different behaviors. I happen to think the default behavior is sensible. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpdWQYgJh6i1.pgp Description: PGP signature
Re: [PATCH RFC 0 of 2] Complete RFC 2369 support
On Sat, Jan 02, 2016 at 01:13:08PM -0600, David Champion wrote: > > 5. Feel free to add a copyright line in listmenu.c (and also put > > yourself in the COPYRIGHT file!) > > Eh, I've never been too concerned about overtly claiming copyright, but > I probaby should after all this time. :) And many, many worthwhile contributions... -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpT3APfQvNip.pgp Description: PGP signature
Re: decoding of base64 fails
On Fri, Feb 19, 2016 at 03:23:15PM +0100, Olaf Hering wrote: > Its Friday, so I ran it in gdb. The bug is in mutt_decode_base64. After > the base64 stream ends, a newline comes. Then the '-- ' and the > remaining msg. There's a reason all three mail clients got this wrong... AFAICT, the bug is in the data; I believe the message is malformed, probably due to the action of the mailing list management software, by the looks: > # X-MIME-Notice: attachments may have been removed from this message It's been a long while since I was very familiar with the relevant RFCs, but I'm fairly certain that the base64 data should not be mixed with any plain text (the mailing list unsubscribe info) in the same message part; if you're going to mix base64 with non-base64 data, the message should separate the two with MIME part headers. Mutt probably could do a better job of handling this, but all it really need do is complain that your message is b0rked. Mailing list management software should never mangle messages. When it does so, at best they thwart the intentions of the sender, and at worst it produces broken messages. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpbyIILPut4N.pgp Description: PGP signature
Re: 1.6.0 proposed schedule
On Mon, Mar 07, 2016 at 01:12:06PM +0100, Vincent Lefevre wrote: > patch-1.5.23hg.dm.domain.vl.1 is from Derek Martin, written in 2013, > and I've modified the patch to avoid a memory leak. It fixes: > https://dev.mutt.org//trac/ticket/3298 Nice catch, looks good. :-) -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp7Lz_qpRskJ.pgp Description: PGP signature
Re: Ticket 3298 patch feedback
On Mon, Mar 07, 2016 at 04:32:42PM -0800, Kevin J. McCarthy wrote: > Overall the patch looks fine. There are three small changes I'd like to > make (attached as a patch file): > > 1. Since uname sets errno, change the fputs to perror. > > 2. safe_strdup() checks for NULL params, so remove the redundant > NONULL(). > > 3. buffer is allocated on the stack, and can't be NULL, so remove the NONULL. > > If they look okay to you, I'll just test a bit more and push this out. That's all fine, though one concern with perror is the error message is occasionally unintelligible or confusing in context. But it should really be impossible for uname() to fail, and the man page even indicates there are no defined errors. So it probably doesn't matter what you do there, in practice. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpGLHWOqXqqb.pgp Description: PGP signature
Re: NeoMutt[2/4] - Bug Fixes
On Mon, Mar 07, 2016 at 04:30:29PM +, Richard Russon wrote:
> This "feature" is small collection of bug fixes.
> Some are from Karel Zak's Notmuch repository; the others are mine.
Are the changes implemented in the patches documented anywhere? I'm
interested to know more about some of these, as they sound potentially
unsafe or otherwise problematic. For instance:
[...]
> * Use unlocked libc IO everywhere.
How do you ensure data integrity of, say, an mbox mailstore if you're
not locking?
> * Remove TLS version notification
Is this a violation of the license?
> * Bye srandom() and random()
Why? Does Mutt use them in a way that is particularly sensitive to
very long repeating sequences? What do you do instead?
[...]
> * add strndup.c strnlen.c
Why? The standard "n" functions are almost always misused, and
usually introduce more problems than they solve, and should generally
be avoided. Though, if you really need them, the C standard has
provided these two since 2008, and GLIBC, and I'd imagine BSD libc,
for probably considerably longer (though I'm too lazy to actually
look). If your system is running an OS that's older than that, you
should probably set it on fire. =8^)
Why should they be avoided? Here:
char fu[] = { 'a', 'b', 'c' };
char *inp = "somerville";
char buf[11];
...
if (strncmp("somestuff", fu, 9) == 0) { // b0rk: overrun unterminated string! }
if (strncmp("somestuff", inp, 3) == 0) { // b0rk: unintended match! }
if (strncpy(buf, 11, inp)) { //b0rk: buf is NOT null-terminated! }
if (strncpy(buf, 10, inp)) { //b0rk: buf is STILL NOT null-terminated, AND
silently truncated }
if (strncpy(buf, 3, inp)) { //b0rk: getting the picture? :) }
If you're trying to use these functions to guard against buffer
overruns/unterminated strings, most probably You're Doing It Wrong.™
And you're providing a false sense of security in the process.
These cases are particularly bad because when they happen, the program
appears to work fine but produces wrong behavior *sometimes*, making
it hard to track down. Using strcpy() instead of strncpy() or
strcmp() instead of strncmp() will just crash if the input hasn't been
properly sanitized, which will tell you EXACTLY what's wrong.
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgpn1GgH6LKTY.pgp
Description: PGP signature
Re: NeoMutt[2/4] - Bug Fixes
On Tue, Mar 08, 2016 at 04:39:57PM +, Richard Russon wrote: > > How do you ensure data integrity of, say, an mbox mailstore if you're > > not locking? > > Don't know. > > https://github.com/neomutt/neomutt/commit/5adde051f0ae67bfc8aeea0a26060c4e6b0685e4 Yeah, looked into this for a minute, and determined it's atrocious. Should lead to mail store corruption unless you're only using maildir. The fact that Mutt isn't multithreaded is not interesting; the fact that multiple processes (e.g. your MDA) may read and write the mail store files simultaneously IS. Locking is important, and this patch is misguided. > This effectively hides the message, but also hides messages you may want > to see. > > https://github.com/neomutt/neomutt/commit/d53c0bb24d98635f3264fc1a99d38ec10c071584 Yeah, this is much less harmful, but I personally would want it dumped due to that "hides messages you may want to see" part. But I also don't use TLS since I read mail locally. But I don't remember seeing it when I used IMAPS--is it new (for some definition of "new" relevant for Mutt)? :) > Bye srandom() and random() This is fine--just not clear that it's worth departing from upstream. There's been discussion about doing something like this for temporary file names in the past but what the OS provides by default has been deemed "good enough" by the experts... > > Why? > > https://github.com/neomutt/neomutt/commit/6fa23faa08b19e1748ca20505abddfb47293a906 > > OK, the patch doesn't change Mutt. Notmuch uses strndup and the patch > is a build workaround for systems that don't support it. Yeah, again I personally would dump this. I've been meaning to have a look at how mutt uses the n functions for quite a while; in principle I'd like to see them removed entirely, though they do have their uses. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpGu709UcpyO.pgp Description: PGP signature
Re: NeoMutt[2/4] - Bug Fixes
On Tue, Mar 08, 2016 at 01:20:51PM -0500, Ben Boeckel wrote: > > Yeah, looked into this for a minute, and determined it's atrocious. > > Should lead to mail store corruption unless you're only using maildir. > > The fact that Mutt isn't multithreaded is not interesting; the fact > > that multiple processes (e.g. your MDA) may read and write the mail > > store files simultaneously IS. Locking is important, and this patch > > is misguided. > > Reading the manpages, it seems that these are about not using a > process-level mutex lock, not file-level locking mechanisms. Hmm... Seems you are right. My bad, I did not read carefully enough. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpaBdk21sUoC.pgp Description: PGP signature
Re: Neomutt - Release 20160404 (Mutt-1.6.0)
On Tue, Apr 05, 2016 at 02:43:15PM +0100, Richard Russon wrote: > The usual pattern is: > developer writes some code > it doesn't meet the mutt-dev standards (for whatever reason) > developer gives up > time passes > variants of the patch turn up (because people like the feature) > distros start bundling different versions the patch > > Mutt should be doing everything it can to bring developers together. > NeoMutt's lower bar for entry should help. Yes, with a caveat. I myself have started several "Mutt is stagnating" threads over the years, so I'm sympathetic to this. But it's also still important that Mutt not lose sight of its raison d'etre: to be the mail client that sucks less. [Sorry Vincent, I can't type the French accents. ;-)] That means, in part, that when new code is written, it be evaluated with an eye toward quality, and that code that isn't up to par be rejected. Mutt is also already extremely featureful; part of the job of the maintainer(s) is to evaluate how any given proposed feature fits in with the current code base, whether any additional complexity is worth what the feature provides, and whether new features or changes in functionality (or their implementations) compromise the integrity or security of Mutt. Whatever criticisms of the Mutt dev process you might have, it's hard to argue that any of the previous maintainers failed to take quality into consideration; and this focus does, necessarily I think, lead to a more conservative development process. Kevin has newly taken over that role, and has been doing a fine job, but let's also try to help him acheive those goals by not asking to have every kitchen sink patch committed by end-of-business tomrrow. =8^) [I know you're not--I'm being fecetious.] While we may strive to move forward, at the same time we all need to be dilligent to make sure Mutt stays the powerful, reliable, managable (and maintainable) mail client it has always been. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpsJM9DVBYzV.pgp Description: PGP signature
Re: Neomutt - Release 20160404 (Mutt-1.6.0)
On Tue, Apr 05, 2016 at 11:04:59AM -0700, David Champion wrote: > OK - that's a good track to have. The thing that made me think otherwise > was the removal of hg-related components. Kevin has the final say now > but we've never discussed moving to git, and I don't see what doing > so would accomplish particularly other than allow/make people use one > hosting system instead of another. The one thing I see is that git is far more popular than Mercurial, which means that many more developers know it. Personally, I use neither, but I'd be a lot more inclined to learn git as I can see doing so would have far greater value to me. I say this to provide an example, not because I think that what I think matters more than what others do (though, clearly, it does). =8^) > > However, sidebar has been waiting on the sidelines for ten years, > > already. [...] > > The issues were pretty strong at the time. I'm not completely opposed > to this but I would like to feel comfortable that it's on a maintainable > track and that we can converge it toward code that we like, not just > put up with. I'd be interested in any perspective past developers/ > maintainers might recall about it (tlr, me, brendan?) I think you're on to the biggest issue: Neither Mutt nor the patch promote maintainability or extensibility of UI enhancements (I'm not really in a position to evaluate this, only stating what I think past objections were). Both Mutt and the patch need to be taught to do things the right way, both to make maintaining that feature easier, and to make future extensibility more managable. Perhaps this should be a 2.0 goal? :) FWIW related to this: One thing I've always wanted to see was to create a uniform API for the UI, and also for mailboxes, so that it was easier to basically completely replace the UI, and to implement new mail store types. If we used console-based widgets for the UI it'd probably make adding the sidebar a lot cleaner and maintainable. I'm certain such libraries already exist... -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpYCbUAxhD8d.pgp Description: PGP signature
Re: Neomutt - Release 20160404 (Mutt-1.6.0)
On Thu, Apr 07, 2016 at 11:37:28AM -0700, David Champion wrote: > > I think you're on to the biggest issue: Neither Mutt nor the patch > > promote maintainability or extensibility of UI enhancements (I'm not > > really in a position to evaluate this, only stating what I think past > > objections were). Both Mutt and the patch need to be taught to do > > things the right way, both to make maintaining that feature easier, > > and to make future extensibility more managable. > > I'm a big fan of gradual core enhancements that promote extensibility > (including patching). I don't know that this should be a primary focus > for mutt right now, but I think any community efforts toward this should > be taken seriously. I mostly agree, but there are points where you have to simply acknowledge that what you have does not lend itself to gradual improvement, and a rewrite really is necessary. I'm not saying that's the case here, but I think it might be. Where I work, some teams have had a "make the smallest useful change" policy, justified by a desire to minimize risk while still getting things done. Over time, this has led to software that has become extremely fragile and nearly impossible to change. > > FWIW related to this: One thing I've always wanted to see was to > > create a uniform API for the UI, and also for mailboxes, so that it > > was easier to basically completely replace the UI, and to implement > > new mail store types. If we used console-based widgets for the UI > > I believe there was a discussion a while back -- maybe 10 years or so, > even -- about building on the mx.c framework to make mailbox drivers > more of a thing. Anyone remember this? I do, but not in more specific terms than you describe. But I think I actually started that thread. Twice. :) -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpi1jMdcf1TO.pgp Description: PGP signature
Re: Neomutt - Release 20160404 (Mutt-1.6.0)
On Sat, Apr 09, 2016 at 10:35:14AM +0200, Oswald Buddenhagen wrote: > almost every single sizeable project that chose the route of a (near) > rewrite from scratch died, or the rewrite never took off. Firstly, I was speaking somewhat generally. I'm not suggesting that mutt needs to be rewritten in its entirety. Some of it is quite modular and most of it is reusable. Though, I haven't really made an effort to read over the code to evaluate that, as TBH I really don't have the time or the inclination right now. > but then, "it's so bad, we have to rewrite it from scratch" just attests > a lack of imagination Not necessarily. When a code base gets into this state, it's not necessarily the case that refactoring is impossible; it's that doing a good job requires that the person(s) doing the refactor necessarily need to be intimately familiar with the code in its entirety, in order to understand what changes need to be made to acheive the desired outcome without breaking anything. In such cases, it's often much less effort for someone or several someones to lay out a design and implement it. The important details are already in their head, and the rest can be made up as you go, to a large extent. Admittedly, it's more likely to be successful in a commercial environment, where there's significant (and monetary) incentive to do the work. But there are at least some examples even in OSS that have been successful. It just depends on finding people with the motivation to do the work. > > Where I work, some teams have had a "make the smallest useful change" > > policy, justified by a desire to minimize risk while still getting > > things done. > > > well, that's just stupid. It is, and it's not. No company has infinite resources, and you have to do the best with what you have. This is mostly a case of, "keep the old stuff alive while we focus our efforts on replacing it with newer, shinier stuff," at least where I'm at. > > Over time, this has led to software that has become extremely fragile > > and nearly impossible to change. > > > of course it has. > > kind of reminds me of mutt, actually. ;) Well, that's sort of my point. ;-) -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgppOgQPaxoWz.pgp Description: PGP signature
Re: [PATCH] Create a wrapper sys_socket.h to work around Solaris namespace issues. (closes #3833)
On Mon, Apr 25, 2016 at 03:38:46PM -0700, Kevin J. McCarthy wrote: > Here's a work around I'd like to include in the upcoming 1.6.1 (which > will be based on the stable branch). > > The conflicts currently only affect getdomain.c, but it's a problem > waiting to happen in the other three files, so I think it's better to > just wrap the include. > > Please comment if you think there is a better way to do this. I'm > reluctant to rename the macros since Vincent pointed out this would > break patches. Well, to be fair, M_ is a pretty crappy prefix to use in a project's macros. Not that using them in system header files is much better... But maybe it wouldn't be such a bad idea to bite the bullet: Do a find/replace of all such macros and change the prefix to MUTT_ instead. With 1.6 freshly out it may not be a bad time to do that, and FWIW if you're ever going to do this, there's no better time than the present. As time goes on, it'll only get harder. As for breaking patches... so? Sure it's a pain, but it's still just a find/replace, with a look over the diff to make sure nothing weird happened. Progress is painful. I would've suggested rewriting in C++ and using namespaces... =8^) I could probably be pursuaded to write a script to do that find and replace, and maybe even run it to generate a patch... -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp3gNVBMUVrY.pgp Description: PGP signature
Re: [PATCH] Create a wrapper sys_socket.h to work around Solaris namespace issues. (closes #3833)
On Tue, Apr 26, 2016 at 09:54:48AM -0700, Kevin J. McCarthy wrote:
> On Mon, Apr 25, 2016 at 08:19:37PM -0500, Derek Martin wrote:
> > Well, to be fair, M_ is a pretty crappy prefix to use in a project's
> > macros. Not that using them in system header files is much better...
>
> I think you have a point. What I'd like to do, though, is include this
> patch in the stable branch so I can push it with 1.6.1 (hopefully next
> week).
I can probably make some time for it this weekend. FWIW, here's what
I've used to find the list of symbols which would need to change (run
in the root of the mutt-1.6 source tree):
$ find . -name "*.[ch]" -exec grep -Ro '\WM_\w\+' {} \; |sed 's/^\W*//' |sort
-u > symbols
$ wc -l symbols
245 symbols
The mutt source needs to be checked for the modified versions of each
of those symbols (which I will do). It should be only slightly harder
to write a script that does that, and then only slightly harder still
to do the actual find and replace. But I'll wait for further
discussion before I do that, and if I find conflicts I won't proceed
without discussion.
And the list, should anyone like to look it over:
$ cat symbols
M_ACCOUNTHOOK
M_ACCT_LOGIN
M_ACCT_PASS
M_ACCT_PORT
M_ACCT_SSL
M_ACCT_TYPE_IMAP
M_ACCT_TYPE_NONE
M_ACCT_TYPE_POP
M_ACCT_TYPE_SMTP
M_ACCT_USER
M_ACL_ADMIN
M_ACL_CREATE
M_ACL_DELETE
M_ACL_DELMX
M_ACL_EXPUNGE
M_ACL_INSERT
M_ACL_LOOKUP
M_ACL_POST
M_ACL_READ
M_ACL_SEEN
M_ACL_WRITE
M_ADDRESS
M_ADD_FROM
M_ALIAS
M_ALL
M_AND
M_APPEND
M_ASKNO
M_ASKYES
M_AS_TEXT
M_AUTOVIEW
M_BODY
M_BUFFY
M_CC
M_CHARCONV
M_CHARSETHOOK
M_CLEAR
M_CMD
M_CM_CHARCONV
M_CM_DECODE
M_CM_DECODE_CRYPT
M_CM_DECODE_PGP
M_CM_DECODE_SMIME
M_CM_DISPLAY
M_CM_NOHEADER
M_CM_PREFIX
M_CM_PRINTING
M_CM_REPLYING
M_CM_UPDATE
M_CM_VERIFY
M_CM_WEED
M_COLLAPSED
M_COMMAND
M_COMPOSE
M_COMPOSE_NOFREEHEADER
M_CONT
M_CRYPTHOOK
M_CRYPT_ENCRYPT
M_CRYPT_SIGN
M_CRYPT_VERIFIED
M_DATE
M_DATE_RECEIVED
M_DELETE
M_DELETED
M_DETACH_PROCESS
M_DISPLAY
M_DISPLAYFLAGS
M_DUPLICATED
M_EDIT
M_EFILE
M_ENTER_C
M_ENTER_S
M_EOL
M_EXPIRED
M_FCCHOOK
M_FILE
M_FIRSTDONE
M_FLAG
M_FLAGS
M_FOLDERHOOK
M_FORMAT_ARROWCURSOR
M_FORMAT_FORCESUBJ
M_FORMAT_INDEX
M_FORMAT_MAKEPRINT
M_FORMAT_NOFILTER
M_FORMAT_OPTIONAL
M_FORMAT_STAT_FILE
M_FORMAT_TREE
M_FROM
M_FULL_MSG
M_GENERATE_UIDVALIDITY
M_GROUP
M_HEADER
M_HIDE
M_HORMEL
M_ICONVHOOK
M_ICONV_HOOK_FROM
M_ICONV_HOOK_TO
M_ID
M_IGNORE
M_IMAP
M_IMAP_CONN_NONEW
M_IMAP_CONN_NOSELECT
M_LIMIT
M_LIST
M_LOCKED
M_MAILBOXES
M_MAILCAP
M_MAILDIR
M_MATCH_FULL_ADDRESS
M_MAXRANGE
M_MBOX
M_MBOXHOOK
M_MESSAGE
M_MESSAGEHOOK
M_MH
M_MIMEATTACH
M_MMDF
M_MODEFMT
M_NEW
M_NEWFOLDER
M_NEW_MAIL
M_NEW_SOCKET
M_NEW_SSL_SOCKET
M_NO
M_NONE
M_NOSHOW
M_NOSORT
M_NOSPAM
M_NOSYSRC
M_OLD
M_OR
M_OS
M_PAGER_ATTACHMENT
M_PAGER_MARKER
M_PAGER_MESSAGE
M_PAGER_NOWRAP
M_PAGER_NSKIP
M_PAGER_RETWINCH
M_PARTS_TOPLEVEL
M_PASS
M_PATTERN
M_PDR_ABSOLUTE
M_PDR_DONE
M_PDR_ERROR
M_PDR_ERRORDONE
M_PDR_MINUS
M_PDR_NONE
M_PDR_PLUS
M_PDR_WINDOW
M_PENDINGPREFIX
M_PERSONAL_FROM
M_PERSONAL_RECIP
M_PGP_KEY
M_POP
M_PRINT
M_PRINTING
M_PROGRESS_MSG
M_PROGRESS_SIZE
M_QUIET
M_READ
M_READONLY
M_RECIPIENT
M_REDRAW_INIT
M_REDRAW_LINE
M_REFERENCE
M_REGULAR
M_REOPENED
M_REPLIED
M_REPLYHOOK
M_REPLYING
M_RO
M_SASL_MAXBUF
M_SAVEHOOK
M_SAVE_APPEND
M_SAVE_OVERWRITE
M_SCORE
M_SEARCH
M_SEARCH_DOWN
M_SEARCH_UP
M_SELECT
M_SEL_BUFFY
M_SEL_FOLDER
M_SEL_MULTI
M_SEND2HOOK
M_SENDER
M_SENDHOOK
M_SET_DRAFT
M_SET_INV
M_SET_RESET
M_SET_UNSET
M_SHOW
M_SHOWCOLOR
M_SHOWFLAT
M_SIZE
M_SOCK_LOG_CMD
M_SOCK_LOG_FULL
M_SOCK_LOG_HDR
M_SPAM
M_SUBJECT
M_SUBSCRIBED_LIST
M_SUPERSEDED
M_TAG
M_THREAD
M_THREAD_COLLAPSE
M_THREAD_GET_HIDDEN
M_THREAD_NEXT_UNREAD
M_THREAD_UNCOLLAPSE
M_THREAD_UNREAD
M_TO
M_TOKEN_COMMENT
M_TOKEN_CONDENSE
M_TOKEN_EQUAL
M_TOKEN_PATTERN
M_TOKEN_QUOTE
M_TOKEN_SEMICOLON
M_TOKEN_SPACE
M_TREE
M_TREE_
M_TREE_BTEE
M_TREE_EQUALS
M_TREE_HIDDEN
M_TREE_HLINE
M_TREE_LLCORNER
M_TREE_LTEE
M_TREE_MAX
M_TREE_MISSING
M_TREE_RARROW
M_TREE_SPACE
M_TREE_STAR
M_TREE_TTEE
M_TREE_ULCORNER
M_TREE_VLINE
M_TYPES
M_UNDELETE
M_UNGROUP
M_UNMAILBOXES
M_UNREAD
M_UNREFERENCED
M_UNTAG
M_VERIFY
M_WEED
M_WHOLE_MSG
M_XLABEL
M_YES
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgpQX2qjPj1Wa.pgp
Description: PGP signature
Re: [PATCH] Create a wrapper sys_socket.h to work around Solaris namespace issues. (closes #3833)
On Sun, May 01, 2016 at 07:26:53PM -0700, Kevin J. McCarthy wrote: > On Sun, May 01, 2016 at 07:02:38PM +0200, Vincent Lefevre wrote: > > On 2016-04-26 09:54:48 -0700, Kevin J. McCarthy wrote: > > > David, Brendan, Vincent (and anyone else), does the prefix 'MUTT_' sound > > > reasonable, or is there another prefix that would be better? > > > > It is reasonable, but as reasonable as M_ (for a prefix used > > internally). If systems can break M_, they could break almost any > > prefix (if particular if it corresponds to something readable). The difference is the OS is using M_, and such short prefixes are common for OS-level libraries. It's not so surprising that there's a conflict here. Whereas using MUTT_ instead--I'm certainly unaware of any other software that uses it, and for a new project to start Mutt clearly has been around a lot longer... What is the likelihood that you'd find a conflict in a library that's meant to compile with Mutt, or which Mutt itself requires to compile? Methinks pretty slim... :) > The patch now in 1.6.1 "fixes" this issue for mutt, but it's an ugly > band-aid. While I'm in no hurry to make such a large change to our > code, Derek is right that now's as good a time as there is. I've > already made some pretty large changes with the windows patch-set, so > the next default-branch release is already going to cause pain to > external patches. FWIW I was busy this weekend and totally forgot about this... I'll try to find some time this week. I think the perl command that Andras gave is adequate to produce the required change; I'd just want to do something slightly more complicated to identify any conflicts first. > > Now, if it is OK to break patches, how about doing some other clean-up > > at the same time? I mean, document code styling and make it uniform, > > and fix typos in comments. > > Yes, that sounds okay too. I'll start a wiki page this week with what I > perceive as the code styles. If we want to make large-scale changes, I > would ideally like to designate a tool to do those (so we can easily > verify the patches). I've always wanted software projects to generate a configuration for indent, and then run it on every submission... or something of the sort. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp4ov4kazWZ5.pgp Description: PGP signature
Re: [PATCH] Create a wrapper sys_socket.h to work around Solaris namespace issues. (closes #3833)
On Tue, May 03, 2016 at 11:11:44AM +0200, Vincent Lefevre wrote: > On 2016-05-02 15:02:44 -0500, Derek Martin wrote: > > The difference is the OS is using M_, and such short prefixes are > > common for OS-level libraries. > > That's really the worst thing that they could do. The reason is that > public non-OS library headers should not use reserved names (because > they are reserved), so that the best thing they can do is to use long > prefixes to avoid clashes between them. Thus, applications should use > only short prefixes (for private use) in order to reduce the risk of > clashes with these library headers (due do chains of inclusions, they > don't always know what headers are used). And this is what they often > do in practice. I don't think this makes any sense, Vincent. POSIX is littered with such short prefixes, such as S_ for stat, O_ for open flags, etc.. > > What is the likelihood that you'd find a conflict in a library > > that's meant to compile with Mutt, or which Mutt itself requires > > to compile? Methinks pretty slim... :) > > But that's not impossible. This wouldn't be the first time a new > library takes a name already used by an application, with a possible > future dependency. Nothing is impossible, in the world of software. But I think this is so unlikely that I highly doubt it's worth worrying about. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgp2fTyiU8_qs.pgp Description: PGP signature
[PATCH] change M_* symbols to MUTT_*
Finally got around to this. The patch is obviously quite long; I suppose there's nothing to be done about that. But it's too long for the mailing list, so I posted it here: http://www.pizzashack.org/mutt/mutt_symbol.patch I did confirm that there were no conflicts, and the patch applies cleanly to 1.6.1. I'm composing this e-mail from the patched version now. =8^) -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. signature.asc Description: PGP signature
Re: [PATCH 10 of 23] pop: populate probe callback
On Thu, May 05, 2016 at 11:44:52PM -0400, Damien Riegel wrote:
> struct mx_ops mx_pop_ops = {
> + .probe = pop_probe,
>.close = pop_close_mailbox,
> };
Isn't this initialization style a GNUism? I think we should avoid it.
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
signature.asc
Description: PGP signature
Re: [PATCH 00 of 23] decouple mailboxes from the rest of the application
On Sat, May 07, 2016 at 04:14:36AM -0400, Damien Riegel wrote: > Because that means mx.c would have some kind of knowledge of all the > mailboxes it supports. It is yet another switch case or if statement > that would need to be modified when adding a new kind of mailbox. I > think multiple calls to stat have a very minimal perfomance impact, so > it doesn't really matter for now. I didn't read all the patches, so I'm not really sure what the use case for the "multiple calls to stat" is, so take this comment if it's relevant and ignore it otherwise. :) Calling stat() multiple times is generally fine, when you know that the target is a local filesystem. It starts to get more "interesting" if your filesystem is networked (network latency vastly outweighs the actual stat time), and/or if your filesystem is simulated via FUSE or some other abstraction library, where the stat operation is expensive (like a database lookup). It may not be relevant to the average user, but it can matter. This is the world I live in. =8^) -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. signature.asc Description: PGP signature
Re: [PATCH] change M_* symbols to MUTT_*
On Sun, May 08, 2016 at 12:33:27PM -0700, Kevin J. McCarthy wrote:
> > I did confirm that there were no conflicts, and the patch applies
> > cleanly to 1.6.1. I'm composing this e-mail from the patched version
> > now. =8^)
>
> Thanks a lot for doing this, Derek. Do you have the script you used for
> this? I wonder if we should put it somewhere, so external packagers
> and use it on patches if needed.
I did not really use a script, per se. For the safety check, I used
the following sequence of commands:
$ find . -name "*.[ch]" -exec grep -Ro '\WM_\w\+' {} \; |sed 's/^\W*M_/MUTT_/'
|sort -u > symbols
$ mv symbols ..
$ cat ../symbols | while read pattern; do echo "$pattern:"; if grep -Rlw
"$pattern" .; then echo "Symbol \"$pattern\" is unsafe"; fi; done
Then, to make the changes, I used the perl command posted by Andras:
$ perl -wpi -e 's/\bM_(\w+)\b/MUTT_$1/g' `find . -name '*.[ch]' -print`
Then to generate the diff I used the rather obvious:
$ diff -ur mutt-1.6.1.orig mutt-1.6.1 > patch
I did of course test that the patch applied cleanly to a pristine
third copy of the source.
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgpXJcn2jBZZR.pgp
Description: PGP signature
Re: [PATCH 10 of 23] pop: populate probe callback
On Sun, May 08, 2016 at 12:40:07PM -0700, Kevin J. McCarthy wrote:
> On Sun, May 08, 2016 at 01:08:47PM -0500, Derek Martin wrote:
> > On Thu, May 05, 2016 at 11:44:52PM -0400, Damien Riegel wrote:
> > > struct mx_ops mx_pop_ops = {
> > > + .probe = pop_probe,
> > >.close = pop_close_mailbox,
> > > };
> >
> > Isn't this initialization style a GNUism? I think we should avoid it.
>
> I looked at this too, but was surprised to find out this is in C99
> (section 6.7.8). <http://c0x.coding-guidelines.com/6.7.8.html>
No kidding... Learn something new every day. ;)
> We have AC_PROG_CC_C99 in our configure.ac, so I think it should be okay
> to start allowing some of the C99 changes in. Do you agree?
What, adopting a standard that was only ratified 17 years ago?!?! That's
crazy talk! =8^)
Yeah I think it's fine, and good.
--
Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address. Replying to it will result in
undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgp1wYVUyCOoR.pgp
Description: PGP signature
Re: [PATCH] change M_* symbols to MUTT_*
On Mon, May 09, 2016 at 10:53:06AM -0400, Damien Riegel wrote: > Alternatively, you could change change each symbol one at a time. Well, > maybe each enum ou group of defines could be done in one commit, but it > doesn't make sense to change all M_* to MUTT_* in one giant commit. I don't entirely disagree; but on the other hand, I think doing what you suggest is crazy. ;-) I can't imagine doing a change like this any way other than full-auto would be practicable. I was glad to see there were no conflicts, as that would have made things a lot more complicated. > As a sidenote, you linked a diff, not a patch, so there is no commit > message attached to the changes. Hmm? patch == diff, not sure what distinction you're making. There's no commit message because I don't commit things. :) -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpTYc6yRXlST.pgp Description: PGP signature
Re: [PATCH 11 of 23] provide a facility to register mailboxes
On Thu, May 05, 2016 at 11:44:53PM -0400, Damien Riegel wrote: > +#ifdef USE_IMAP > + MX_REGISTER_MAILBOX(imap); > +#endif > + MX_REGISTER_MAILBOX(maildir); > + MX_REGISTER_MAILBOX(mbox); > + MX_REGISTER_MAILBOX(mh); > + MX_REGISTER_MAILBOX(mmdf); > +#ifdef USE_POP > + MX_REGISTER_MAILBOX(pop); > +#endif I'm slightly nervous about using the MX_ prefix here, as this seems likely to conflict with macros in libraries related to sending mail or even DNS. OTOH MX_REGISTER_MAILBOX seems unlikely to conflict... I'm really beginning to think that no one should use C anymore ;-) -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpYjSXmo0m2R.pgp Description: PGP signature
Re: [PATCH 00 of 23] decouple mailboxes from the rest of the application
On Sat, May 07, 2016 at 04:14:36AM -0400, Damien Riegel wrote: > > * The "probe" design: > > In theory this sounds good, but I don't know that the trade-off is worth > > it. You still have a single "mx_register_all()" function with #ifdefs > > in it to put together the linked list. But now, you have to loop > > through one-by-one, performing multiple stats and such until you find > > the right one. > > Because that means mx.c would have some kind of knowledge of all the > mailboxes it supports. It is yet another switch case or if statement > that would need to be modified when adding a new kind of mailbox. I > think multiple calls to stat have a very minimal perfomance impact, so > it doesn't really matter for now. At some later point, mx_get_magic > should be split, and its content dispatched in the different probe > functions. But I didn't want to make the patchset larger with such > cleanup commits. OK, I get it now. I think this could work OK if the parent did a single stat, and perhaps did an opendir if the object was a directory, and then passed copies/pointers/whatever of those objects into the registered probe functions (pointers for sure). There's no need to do multiple stats. Then at least the only efficiency loss is the extra function calls. If dir, the probe function should assume it needs to do rewinddir(). This of course only makes sense if the mail store is a file on a local file system (or a networked file system that looks local to the user). I still didn't get through all the patches, not sure how this is meant to work with non-file mail stores. But that's my $.02! -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpZKlFyqActX.pgp Description: PGP signature
Re: [PATCH 00 of 23] decouple mailboxes from the rest of the application
On Mon, May 09, 2016 at 08:57:55PM -0400, Damien Riegel wrote: > On Mon, May 09, 2016 at 06:42:33PM -0500, Derek Martin wrote: > > On Sat, May 07, 2016 at 04:14:36AM -0400, Damien Riegel wrote: > > OK, I get it now. I think this could work OK if the parent did a > > single stat, and perhaps did an opendir if the object was a directory, > > and then passed copies/pointers/whatever of those objects into the > > registered probe functions (pointers for sure). There's no need to do > > multiple stats. Then at least the only efficiency loss is the extra > > function calls. If dir, the probe function should assume it needs to > > do rewinddir(). > > Thanks for your input Derek. My code was based on the assumption that > mx_get_magic is idempotent, but maybe it is not true. Now that I've actually looked at the code... ;-) I think it is, in the sense that you mean. That's not the issue though, if I understand correctly. I think your probe design, keeping the knowledge about a mailbox implementation separate from the rest of mutt, is a good idea, and I didn't mean to discourage that. I'm just pointing out that as implemented by your current set of patches, it has a performance concern, but it's one that can be overcome. Maybe I'm not making myself clear, so I'll try again. The issue is that Mutt is going to call your registered probe functions one at a time, until one of them returns that it matches the mailbox type. Currently in all cases this is a function that wraps the mx_get_magic() function, which always checks every possible path, until it finds one that matches. Kevin was (I think) pointing out that this is inefficient, which it is. You'll stat each relevant path up to n times, where n is the number of mail box types you've registered. Currently the mx_get_magic() function and friends stat roughly 10 paths, so you'll do up to roughly n*10 stat() calls. As you say, a normal stat takes negligible time and the user will never notice; and probably the data will end up served from some cache somewhere in the filesystem layer an/or hardware. However, for example, if the mail store is on an NFS server with a slow link, in a worst-case scenario may add up to being very noticable to the user (NFS caching may or may not help hide some of this). If there were 10 mailbox types and your latency to the NFS server was 100ms, this might end up taking roughly 2 * 100ms * 10 mailboxes * 10 paths = 20,000ms, or 20s. Yuck. What I'm suggesting is that if the function that runs through the probe functions does the work in advance, and caches as much of it as makes sense, you can get most of that back. I just took a look at the mx_get_magic() function and those that it calls... The lot of that basically just stats a bunch of paths--there aren't even any opendir() calls. SO: - register a TYPE of folder, where the type is either local file stor or not (or something of the sort) - register a list of paths that your mailbox driver needs to stat to identify mail stores of its type (can be NULL) - FIRST the driver function should check for non-local file stores (like IMAP and POP), since those require no stat() calls (and I believe should mostly just be a string compare, specifying the protocol). - THEN have the driver function pre-fetch the stat structs and cache them somehow (linked list, hash table, whatever), and pass that structure to the probe functions Now you can break up mx_get_magic() into its relative peices, and make each piece the probe function for that specific mailbox type. All must take a pointer to your cache structure, from which they'll get the struct stat structures they'll use to make their decision. Then they'll just pull out the stat structs they need. You could do this by having two different probe functions in your struct--one for local file store, and another for non-local. This would be sufficient to distinguish the type, though for neatness you may want to have a field that specifies it explicitly. For each mail store type you would register the correct one and leave the other NULL. Then your driver function would call all of the probe functions for non-local, which need not take your stat cache structure. This way you only need to generate the stats if at least one of them is needed. Alternatively, have just one, use the type field, and leave the stat struct cache argument NULL for the non-local types (and still wait to generate the cache until they all fail). Does this make sense? Note also that you need not necessarily implement something like this for the first pass... It's of course up to Kevin how and when (and if) he wants to take the various patches. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an inva
Re: anybody up for coding conversation view?
On Tue, May 10, 2016 at 01:10:00PM -0700, Yoshiki Vazquez-Baeza wrote: > As you mention, this depends on the settings you have. However if I > understand correctly, the structure of a thread is determined by the > in-reply-to header attribute. GitHub does not include this attrribute in > the emails they send you, so all your emails are piled in a single > thread but with no thread structure on their own, as far as I know, > there's no way to fix this. That's not quite true. Look at strict_threads and sort_re. Though, if I'm being honest, I never could make heads or tails of what the difference between sort_re being set or unset are. You can, of course, also connect threads manually, thought that's obviously a lose. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpPjRt7Zxp_L.pgp Description: PGP signature
