Re: Why fighting spam with whitelists doesn't work [was Re: Contributing a mailet]
On 4 Feb 2004, at 11:23, [EMAIL PROTECTED] wrote: Stefano, Thanks, you make a great point against reject emails. It was not my intent to create a new reject email but rather to reject it at the incoming SMTP message level. But, as Serge mentions, I might not be able to do include the URL to apply for whitelist at the SMTP reject level, and anyway the mailet API does not support such functionality. I was counting on such capabilities to do rejects without the annoying side effects you mentioned. I obviously need to do more research into how to properly reject without causing extra emails. As to a whitelisted sender being infected by a worm and sending spam, I do not see that as a big flaw, [sound of stefano banging his head on the wall] especially if you already have an anti-virus filter on your inbound mail filter chain (a normal precaution). A clever worm spreads much more quickly than any anti-virus update. If you think that worms are not a problem in today's internet, think again. Then again. I can't imagine a huge number of spams coming that way, don't know what huge is for you but 400 a day is enough for me. and it would be easy to contact the sender and warn him of his infection. Are you reading what I write? there is no way for me to know *who* is infected. Ok, enough. I already spent too much time on this. -- Stefano. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Contributing a mailet
On 2 Feb 2004, at 11:44, [EMAIL PROTECTED] wrote: Noel, Could you please tell me whether you are replying as a representative of the James project, or as a fellow contributor that knows the answers? Also, I did not see anywhere in the James site a list of contributed mailets... kind of strange considering that mailets are supposed to be the main purpose of James. I probably missed it. Can you point me to the right URL? I don't think there is a repository for mailets just yet. In reply to your questions: a) I'll preface that the mailet I am implementing is not for everyone. I expect that 10% of administrators will find it too restrictive, while the rest will probably find it ideal. The mailet will reject all email from sources that were not preapproved in a whitelist. Associated tools will handle managing of the list and allow new HUMAN senders to request addition to the list in a manner that is not annoying to the receiving user. Whitelist are evil! I receive tons of those are you really you? emails because those bastard warms forge the email address. Do you realize that by creating a whitelist you are, in fact, becoming a spammer yourself? c) I have read the ASF license and related info (or as much as I could understand), and it seems acceptable since my primary goal is to end spam once and for all, with whitelists? please but I'd like you to clarify a couple of points: 1. Will I be able to keep credit as the author of the mailet? If so, how? with the @author tag in javadocs. 2. If this anti-spam measure is as successful as I expect it to be, I fear that the spammers of the world will try anything to kill it and the Microsofts of the world will try to steal it. And you think that a software license would stop a corporation from taking one of their programmers and reimplement the software you wrote? What you are looking for is called software patent. You can't get one in europe (yet)... or you can apply for one in the US. It's very expensive and takes years. By that time, the spam problem will have been solved by somebody else anyway. Because of it I have been sitting on this design for 3 years. [paint puzzled look on my face] Is it true that the Apache Software Foundation provides free legal defense for its contributors if they are sued as a consequence of a contribution? This never happened in the past, but yes, you cannot be sued if you contribute the code to the foundation because the foundation is the owner and not you. But keep in mind that this only works if you did nothing illegal. If the foundation finds out that you did something illegal, you are on your own. If it is not, I will not be able to contribute it and will have to go make it commercial just to make enough money to defend myself. Whatever. -- Stefano. smime.p7s Description: S/MIME cryptographic signature
bogus attribution in license
the james license states that: Portions of this software are based upon public domain software originally written at the National Center for Supercomputing Applications, University of Illinois, Urbana-Champaign. but this is not true. I suggest you to fix this. -- Stefano. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [proposal] Doco
On Tuesday, Oct 28, 2003, at 10:11 Europe/Rome, Steven Noels wrote: Stefano Mazzocchi wrote: 2) minimal, efficient and secure workflow (should satisfy board@ security concerns) FWIW: Doco came to my mind when finding http://kokochi.com:8080/sapience/test.jsp in Hippo's blog. Can I be anal for a sec? cool, but what do we do with it? -- Stefano. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [proposal] Doco
On Tuesday, Oct 28, 2003, at 13:10 Europe/Rome, Steven Noels wrote: Stefano Mazzocchi wrote: Doco came to my mind when finding http://kokochi.com:8080/sapience/test.jsp in Hippo's blog. Can I be anal for a sec? LOL. Sure, as long as you wipe properly afterwards, no problem here. :-) cool, but what do we do with it? Well, since people had to come up with anti-spam measures for weblog comments, because of some nitwits which abuse the comment facility of modern weblog engines using automated tools, to raise Google rankings of spam-linked websites, I figure the same will start happening with any light-weight content contribution tool in the upcoming months. We have seen similar abuse on the Cocoon wiki as well. Rather than completely offload the burden of moderation to a bunch of people, we should come up with an upfront facility for blocking automated abuse. Similar to the facilities found in MSN Hotmail, Paypal and others who use some generated image to validate sign up for their system, having something like this Sapience thing might reduce the moderation burden in the long run. It's cool for sure, but it might be some good way to make sure only humans edit content, and not a bunch of malevolent bots. That is a *very good point* and I think you are perfectly right: we should put it into Doco. Any idea on how to do it? -- Stefano. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [proposal] Doco
On Sunday, Oct 26, 2003, at 23:33 Europe/Rome, Noel J. Bergman wrote: He's not questioning whether it's encrypted. His point is, doco sends an email to an address, and you respond. It gives very little control, even if there is a compromise. AIUI, the proposed solution would allow anyone to edit content, and contribute it as a patch. Content could include defacements, changes to .htaccess, and CGI scripts. nah, dude, look: doco has a very precise editing access point. You can *ONLY* modify xml content. So, changes to .htaccess, CGI scripts, servlet upload, sql injection, cross-site-scripting, and you next favorite attack will NOT work because the system prevents it by design [not saying it cannot happen, but if it does it's a bug, not a faulty design] So long as we can maintain oversight, this is OK. But if someone can spoof the oversight, then I would be concerned. I would *NEVER* propose a system where the users can inject an .htaccess file, even thru approuval. One single oversight mistake and infrastructure@ shuts us down in half a second (if they ever allow the system to run, and I personally wouldn't). The unique and only security concern here is defacement. I am not trying to shoot this idea down. I think it is a good thing to do, and will really help. I really appreciate your concerns and, please, keep in mind that I read and send my email via SSH tunnels, so I *KNOW* what goes on my tcp/ip stack [also, IMAP over compressed SSH is orders of magnitude faster when you sit in italy and your imap server is in london! ;-)] Anyway, here is a potential attack: Doco sends email to james (localhost). James multiplexes the email to the moderators (vanilla SMTP) the attacker sniffs the continuation-ID replies with the sniffed continuation-ID James calls Doco and restarts the workflow Doco sends back email logging the approuval Potential sniffing places are: 1) right after the doco box (FYI, moof is located at Apple, nagoya at Sun) 2) right before the mail server of the moderator or by people who have enough priviledges on the moderator mail server. The above attack is plausible and it would allow the attacker to get stuff in the repository, but *NOT* to prevent other moderators to see that this content has been approuved. [that would require substantial forging of the output stream... possible as well, but much harder to achieve] The other moderators would see it as a mistake and get there fixing it before the next site update. Keep in mind that there are *two* 12 hour stages, so, even if the attacker attacks right before the first staging (the forrestbot execution), the moderators have 12 hours to understand that something was wrong and login to fix things before the other stage (rsync from minotaur) takes place. I think you proposed to use SMTP over SSL for mail relay, that would reduce the ability of somebody to prevent sniffing the continuation-ID and provide that attack. I do agree it would help reducing the risk, but would all moderator's SMTP server support that? [keep in mind that a sniffer could understand the list of moderators just by watching email outgoing traffic, even if encrypted, by comparing with the list of users on the mail lists] Another solution is to force moderators to digitally sign their email, but this would require a much more intrusive setup. At the end, I don't think anybody would do such an attack since: 1) it can't be proved that it was an attack (we can always say it was a mistake of the moderators), so you can't go around feeling proud of it or impressing friends of the cracker scene [ego is probably 99% of the reasons to deface a web site] 2) the attacker cannot stop others from keeping oversight on what was approuved (doco will send email *after* the moderation to keep oversight of everthing that happened) 3) if we use the lazy consensus moderation method (require 3 accept and no reject), the attack is just as easy, but the chance that the moderator that the attacker would *fake* is offline for 24 hours and cannot yell WTF, I didn't do it at the moderator list is much less So, adding SSL relay wouldn't hurt, but wouldn't help much either since we can't force moderators to have a mail server that accepts that kind of relay (don't even know if mine does!) at least, this is MHO. -- Stefano. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [proposal] Doco
On Monday, Oct 27, 2003, at 15:35 Europe/Rome, Robert Koberg wrote: nah, dude, look: doco has a very precise editing access point. You can *ONLY* modify xml content. So, changes to .htaccess, CGI scripts, servlet upload, sql injection, cross-site-scripting, and you next favorite attack will NOT work because the system prevents it by design [not saying it cannot happen, but if it does it's a bug, not a faulty design] FWIW, I agree. Perhaps the submit goes to a well-formedness check (or even better?, schema/dtd validation). If it fails, it doesn't even enter the approval process. Absolutely. This wasn't mentioned, but planned. I will do relaxng validation before allowing any xml data into the system. This should be enough for documentation. Perhaps a notification email is sent describing that an invalid submittal was sent. Nah, it would just fail and log the failure. No need to spam further since it might well be a bug in the editing software ;-) [I have experienced a few of them as well] The user is returned an error page saying the post was rejected, in case it was just a mistake. On another note, can images/PDFs/other-binaries be uploaded? Damn, forgot about this! My suggestion would be to process the binary file and determine if it's an image or not. If not, reject it right away. [there should be *NO* need to upload any other binary file ] -- Stefano. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: What the EU OSS migration guidelines say about JAMES
On Friday, Oct 24, 2003, at 19:53 Europe/Rome, Jens A. Jensen wrote: Please do not forget that we need some easy-to-use, GUI-based as well as command line based administration tools. The present Remote Manager telnet thing is archaic, unsecure, and next to impossible to use if you have more than just a handful of users to administer. Also, we need a GUI-based tool to handle config.xml in an intuitive and yet structured way. Keep in mind that sysadms configure stuff over the line, GUIs are useless in those cases. Even more: if you tell a sysadm that he/she has to use a GUI to configure something, he/she would not even consider using it. The administration tool is not unsecure if you do it over an ssh tunnel, yet, I wish there was a way to configure the system without using a tool at all, just good old text editor over the wire. 'usability' means easy to use for the pool of users that you are targetting, you improve usability by writing tons of comments into the configuration file (httpd.conf style, for example) not by making a GUI on top of it. These guys don't know what to do with GUIs anyway and an administration tool makes it impossible to automate the configuration (say for backup, replication in clustering environment, etc). low-tech as it seems, simple flat files with a lot of comments are probably the most usable technology for system administrators: the real users of any server. not a suggestion, just food for thought. -- Stefano. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: What the EU OSS migration guidelines say about JAMES
Danny Angus wrote: Holy F*** thats nice attention ... I wonder what they think it lacks? reading it, it looks like the author wants IMAP. This is what I think james lacks: 1) solid fully functional IMAP 2) server side filtering (procmail/sieve like) 3) native OS integration stuff (to avoid running it as root) of course, #2 will need the ability to call external processes via stdin/stdout (so that stuff like spam filtering or virus checking still works). Without those things, it just looks as a proof of concept. -- Stefano. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [proposal] Doco
Danny Angus wrote: Kenny, This pattern looks like the one currently used by the mail moderators for apache lists, and as such it's familiarity is probably a benefit. Exactly. I would exactly mimic the behavior that moderators are used to. While James is easily capable of supporting much richer functionality, the problem occurs when you try to figure out how you can use the extremely minimal information that is guaranteed to be passed back in the headers of a reply. TBH disentangling conflicting approve/reject messages is always going to require a people-centric solution as it is a symptom of a people problem (conflicting opinions in the team). As with revision control the software can only go so far and it needs to be supported by having a communicative team with agreed and shared goals. very well said. The workflow system should *NOT* turn into a communication media (like some people do over issue tracking software, for example), but as a more efficient (think asynchronous) way to do approuvals. So, instead of having to log into a web site, see the list of changes and approuve each one of them, you just have to go thru your email and decide to reply or not. One useful addition might be to accept the first command and reject all subsequent ones with a message to the effect that the change has already been approved/rejected, it would then fall to the commiters to discuss and resolve the conflict through a more collaborative channel. Seems like a good idea to me. -- Stefano. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [proposal] Doco
Joerg Heinicke wrote: Stefano wrote c) the email will have reply-to set to the allow action and a continuation ID. and will have the CC: header set to reject with the continuation ID. So, by replying to the email d) by hitting reply, the workflow will approuve the changes e) by hitting reply to all, the workflow will reject them with no further action Kenny Smith wrote: Hi Stefano, I think the overall concept for this project is pretty cool and very well thought out. The one part that stands out to me as a weak link is the reply vs. reply to all mechanism. It seems prone to human errors and to things like vacation messages. How would conflicts be handled (one person rejects and one approves)? At the very least, I would provide a mechanism that allows changes to be just as easily revoked as they were approved so that late that one night when someone accidentally hits reply to all instead of reply that they can easily fix it. I can only join. Email workflow is bad in general, but additionally the above system is to much error prone. I has worked for the apache group since 1995. Yes, there are occasional mistakes and some message gets moderated thru even if it shouldn't, but compared to the traffic we get this is really nothing. We should at least use the body with an explicite accept and reject in it. This can't be done by accident, while it can happen for sending a mail. But even this I don't like much. What about authorization? only doc committers are subscribed. Are the committers registered with specific mail addresses? of course, how else? What happens if the preferred address changes? they tell us and we change the address. keep in mind that most of them are committers anyway, they will have an @apache.org address that will never change. I would like to see a little application, where a link in the mail points directly to the resource. The committer has to login and accept or reject the change. So conflict situations can also be much better handled and reverting changes should also be easier to be implemented. I dislike this, it stops me from doing auditing offline. This is still much easier than the current workflow with applying a patch, committing to CVS, and so on, but less error prone than the above approach. In the entire history of the wiki, we had only a few cases of severe vandalism on our wiki. That workflow is designed to prevent the vandalist acts but without slowing down the creative process. -- Stefano. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[proposal] Doco
First of all, sorry for the massive cross-post, but I think it is going to be a great opportunity for all the communities involved to show off their potentials with the apache infrastructure. The proposal is about the creation of a content management system for apache projects, codenamed Doco The features will be: 1) super-easy editing (should satisfy wiki lovers) 2) minimal, efficient and secure workflow (should satisfy board@ security concerns) 3) should allow the creation of static content (should satisfy infrastructure@ and mirror@ concerns) 4) should not be a distributed product (should avoid sensations of forking from existing projects) 5) should reuse more than reinvent 6) should come up with structured XML content well organized in a repository Here is my proposed architecture: static site --- frontend/staging --- repository --- backend Static site is httpd running on minotaur (the main ASF web) Frontend/staging will be Forrest. The idea, in order to satify intrastructure@ concerns is 1) forrest runs on top of the repository and generates a staged version of the web site (not on minotaur!) 2) a cron job on minotaur will a) download the entire site from the staging area (using rsync, wget or similar tools) b) commit it on the site module CVS c) move it on the public site folder The reason for such an inverted architecture is to avoid having an SSH access directly from the machine that runs forrestbot. This guarantees *COMPLETE* isolation of minotaur from the rest of the system. There is *NO* way for somebody to hack into any parts of doco and obtain access to minotaur. The reason for committing a copy on CVS is to allow infrastructure@ to have a fresh copy of the web site in case something happens and Doco is down. [they have expressed concerns about this] Repository will be Subversion (for now, in the future, the idea is to move into JSR-170-based repository, but this will take a while) Backend will be a lenya publication. It will incorporates Linotype as a Cocoon Form widget and wiki-syntax/wysiwig cross-editing capabilities. The workflow will be both web and/or email driven. Here is a tentative workflow: 1) everybody can edit a page and submit the changes. the pages on the main site will have a link that connects to the pages on the backend (which will be probably hosted on another machine, either nagoya or moof) 2) if a user can log in (doco committer), no workflow takes place and changes to directly into the repository. [the repository will generate diffs for changes and send email to the docs list for public oversight] 3) if a user cannot log in, workflow starts: a) the changes are stored into a local spool b) email is sent to the 'doco committer' list, this is going to be a private list c) the email will have reply-to set to the allow action and a continuation ID. and will have the CC: header set to reject with the continuation ID. So, by replying to the email d) by hitting reply, the workflow will approuve the changes e) by hitting reply to all, the workflow will reject them with no further action The idea is to use the Lenya workflow manager in coordination with James. Basically: 1) lenya sends an email to an account managed by james 2) this account is mapped to a mailing list mailet that will dispatch this message to the people subscribed 3) the reply-to fields of the email sent will be mapped to another account handled by james 4) this other account is mapped to another mailet that will parse the email address, extract the continuation ID, understand the action to take and trigger an HTTP request to lenya with the action and continuation ID that will re-initiate the workflow. [the message of the mail is discarded] TODO list: 1) create a lenya publication that uses linotype (the 'doco' publication) 3) configure doco with the workflow described above and write the necessary java classes that drive it 4) write the HTTP-triggering mailet (reuse the mail list mailet in james) 5) write the subversion implementation of the repository component [this will be sent back to the repository block in cocoon] 6) write the cron scripts for minotaur 7) install lenya, james, forrest and forrestbot on Moof. I propose the creation of a new CVS module called cocoon-doco to host the scripts, installation instructions and doco-specific code. I also propose the discussions to take place on lenya-dev, given that Lenya is the community focused on content management. Interested people are invited to join [EMAIL PROTECTED] [in case of community-oriented you reply to this email, please, keep all listed cross-posted, but in case of technical discussions, please, let's move it over to lenya-dev to avoid mailbombing people that don't really care] Awaiting for your comments. -- Stefano. - To unsubscribe, e-mail: [EMAIL
