Re: [Alois] Setting up the podling
Hi Christian Thanks for your support. One question: Is it sufficient if the employer of the contributors signs the Software Grant, or is a grant required by every of the two developer? Best Urs Am Dienstag, den 28.09.2010, 07:56 +0200 schrieb Christian Grobmeier: > Hello Alois team, > > welcome to Apache! > > I have just added ALOIS to the Incubator website, which will appear > there in a few hours. The status site will guide us through the > incubation process. > > However, before we request your accounts and infrastuff, can you > please check if you have already signed and sent a CCLA. > http://www.apache.org/licenses/icla.txt > > Since ALOIS has been published as GPL software, a signed Software > Grant would not hurt too: > http://www.apache.org/licenses/software-grant.txt > > Please come back to me once this is done with your preferred apache > user id. Some examples for user ids can be found here: > http://people.apache.org/committer-index.html > > The user id will also be your apache email address. Make sure you send > some alternatives in case you preferred one is already taken :-) > > Best regards, > Christian > signature.asc Description: Dies ist ein digital signierter Nachrichtenteil
[RESULT] ALOIS to enter the incubator
Hi The following votes were recieved (I hope I did get the binding and non-binding right): binding: +1 Benson Margulies +1 Craig L Russell +1 Niclas Hedhman non-binding: +1 Christian Grobmeier +1 Scott Deboy +1 Mohammad Nour El-Din The vote has passed, thank you very much. You can find the proposal on the proposal wiki page: http://wiki.apache.org/incubator/AloisProposal We have the following two mentors: - Scott Deboy - Christian Grobmeier Best Urs signature.asc Description: Dies ist ein digital signierter Nachrichtenteil
Re: Real-time communication (was [VOTE] ALOIS to enter the incubator)
Hi Glen Thanks for your input and a different view. I took the chat channell off the proposal, because I don't want that the project to be assessed on this single point. If a chat is useful for the followers, why not, as long as it is transparent and decisions are not made that way. Best Urs Am Freitag, den 17.09.2010, 09:58 -0400 schrieb Glen Daniels: > On 9/17/2010 9:41 AM, Urs Lerch wrote: > > To cut a long story short: ALOIS is _not_ about a chat channel, it's a > > tool for security incident and event management. Since the chat channell > > in the required resources list was only a wish, I gladly dropped it off > > the proposal. > > Hi Urs, > > While I certainly don't think a chat channel needs to be on the proposal (for > one thing, Apache projects tend to just use freenode's IRC network), I'd like > to strongly reiterate Bertrand's points. Off-list conversation is going to > happen any time you have multiple devs working at the same company, living in > the same town, or attending the same ApacheCon. Real-time chats are often > the source of very valuable insights, and having an online "hang-out" spot > for a project has in the past been hugely worthwhile to the projects I've > been involved with, both for devs and users. > > As long as no serious decisions are made without consulting the list, and > someone posts summaries of all conversations that significantly affect the > project, real-time channels are fine. The point is that someone looking back > at the project from five years down the road should be able to really see > what happened by looking at the archives -- not that real-time is a Bad Thing. > > Thanks, > --Glen > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > signature.asc Description: Dies ist ein digital signierter Nachrichtenteil
Re: Real-time communication (was [VOTE] ALOIS to enter the incubator)
Hi To cut a long story short: ALOIS is _not_ about a chat channel, it's a tool for security incident and event management. Since the chat channell in the required resources list was only a wish, I gladly dropped it off the proposal. Best Urs Am Freitag, den 17.09.2010, 07:32 -0400 schrieb Jim Jagielski: > I still don't see how that gets around the perception, and the > reality, that development is being done outside the list. > > So I don't see that proposal as helping out at all... > > On Sep 16, 2010, at 3:27 PM, Scott Deboy wrote: > > > I understand the concern raised by the use of real-time communication for > > Apache projects - that decisions may be made off-list, and that folks who > > aren't a party to the real-time communication do not have the opportunity to > > benefit from or impact the decisions that result from the real-time > > communication. > > > > The proposal does offer what seems to be a reasonable compromise: 'we would > > send the logs daily to the mailing list.' > > > > Daily chat logs posted to the dev list, coupled with good mentoring and > > guidance that decisions need to be made on the mailing list, would seem to > > minimize the risk. > > > > I'm interested in what others think of their proposal for supporting > > real-time communication, and curious what others are doing, if anything, to > > support the growing interest in real-time communication between project > > participants. > > > > Scott > > > > > > On Thu, Sep 16, 2010 at 10:49 AM, Craig L Russell > > wrote: > > > >> Hi Urs, > >> > >> My only concern is the request to have a chat channel. There's wide use of > >> chat channels in Apache (the periodic board and members' meetings make use > >> of them, and infrastructure uses channels to advantage). > >> > >> But for an incubating project, I'd strongly discourage use of chat as a > >> communication channel. > >> > >> +1 > >> > >> Craig > >> > >> > >> On Aug 26, 2010, at 9:09 AM, Urs Lerch wrote: > >> > >> Hi, > >>> > >>> I would like to call a vote for accepting "ALOIS" for incubation in > >>> the Apache Incubator. The full proposal is available below and on the > >>> proposal wiki page (http://wiki.apache.org/incubator/AloisProposal). We > >>> ask the Incubator PMC to sponsor it, with Scott Deboy volunteering as > >>> Champion and Mentor. Additional mentors are warmly welcome. > >>> > >>> Please cast your vote: > >>> > >>> [ ] +1, bring ALOIS into Incubator > >>> [ ] +0, I don't care either way, > >>> [ ] -1, do not bring ALOIS into Incubator, because... > >>> > >>> This vote will be open for 72 hours and only votes from the Incubator > >>> PMC are binding. > >>> > >>> Thanks, > >>> Urs > >>> > >>> > >>> > >>> > >>> > >>> = Preface = > >>> > >>> ALOIS is a log collection and correlation software with reporting and > >>> alarming functionalities. It has been implemented by the Swiss company > >>> IMSEC for a customer about five years ago. GPL-licenced, implemented in > >>> Ruby and completely based on other OSS-licensed components, it was > >>> designed for the open source community right from the start. Now that > >>> the software has shown its functioning over several years in production > >>> with the one customer and one IMSEC-internal installation, it seems to > >>> be the right time to open it to a wider community. > >>> > >>> > >>> = Abstract = > >>> > >>> ALOIS stands for „Advanced Logging and Intrusion Detection System“ and > >>> is meant to be a fully implemented open source SIEM (security > >>> information and event management) system. > >>> > >>> > >>> = Proposal = > >>> > >>> While almost all other SIEM software, be it closed or open source, > >>> concentrate on the technological part of security monitoring, ALOIS is > >>> aimed to monitor the security of the content. It intends to be > >>> pro-active in the detection of potential loss, theft, mistaken > >>> modification or unauthorized access. ALOIS works on log mess
Re: [VOTE] ALOIS to enter the incubator
Hi Craig Thanks for you input and your vote. In my view, the chat is not a must, but a proposition. I understand your concern and I certainly will keep it in mind. But since ALOIS should become a community project, I still think the followers of it should decide which communication channell they prefer. (By the way, I myself sure am no fan of chats.) I hope you can live with this. Best Urs Am Donnerstag, den 16.09.2010, 10:49 -0700 schrieb Craig L Russell: > Hi Urs, > > My only concern is the request to have a chat channel. There's wide > use of chat channels in Apache (the periodic board and members' > meetings make use of them, and infrastructure uses channels to > advantage). > > But for an incubating project, I'd strongly discourage use of chat as > a communication channel. > > +1 > > Craig > > On Aug 26, 2010, at 9:09 AM, Urs Lerch wrote: > > > Hi, > > > > I would like to call a vote for accepting "ALOIS" for incubation in > > the Apache Incubator. The full proposal is available below and on the > > proposal wiki page (http://wiki.apache.org/incubator/ > > AloisProposal). We > > ask the Incubator PMC to sponsor it, with Scott Deboy volunteering as > > Champion and Mentor. Additional mentors are warmly welcome. > > > > Please cast your vote: > > > > [ ] +1, bring ALOIS into Incubator > > [ ] +0, I don't care either way, > > [ ] -1, do not bring ALOIS into Incubator, because... > > > > This vote will be open for 72 hours and only votes from the Incubator > > PMC are binding. > > > > Thanks, > > Urs > > > > > > > > > > > > = Preface = > > > > ALOIS is a log collection and correlation software with reporting and > > alarming functionalities. It has been implemented by the Swiss company > > IMSEC for a customer about five years ago. GPL-licenced, implemented > > in > > Ruby and completely based on other OSS-licensed components, it was > > designed for the open source community right from the start. Now that > > the software has shown its functioning over several years in > > production > > with the one customer and one IMSEC-internal installation, it seems to > > be the right time to open it to a wider community. > > > > > > = Abstract = > > > > ALOIS stands for „Advanced Logging and Intrusion Detection System“ and > > is meant to be a fully implemented open source SIEM (security > > information and event management) system. > > > > > > = Proposal = > > > > While almost all other SIEM software, be it closed or open source, > > concentrate on the technological part of security monitoring, ALOIS is > > aimed to monitor the security of the content. It intends to be > > pro-active in the detection of potential loss, theft, mistaken > > modification or unauthorized access. ALOIS works on log messages and > > thus contains all the basic functionality of a conventional SIEM, as > > centralized collecting, normalizing, aggregation, analyzing and > > correlating of all log messages, as well as reporting all security > > related events. Therefore it can be used as any other SIEM. > > > > ALOIS consists of five modules interacting to ensure a scaleable > > functionality of a SIEM: > > > > * Insink is the message sink, which is the receiving entry point for > > all the different log messages into ALOIS. It is partly based on the > > syslog-ng software. Insink listens for messages (UDP), waits for > > messages (TCP), receives message collections (files, emails) and > > pre-filters them to prevent from message flow overload. > > > > * Pumpy is the incoming FIFO buffer, implemented as a relational > > database tables. which contain the incoming original messages (in raw > > format). In a complex system setup, there may be several insink > > instances, e.g. for a group of hosts, for specific types of > > messages, or > > for high-avaliablity. > > > > * Prisma contains logic to split up the text of log messages into > > separate fields, based on regular expressions. Actually, "prisma" is a > > set of "prismi", each one prisma for one type of log message (apache, > > cisco etc. Several prismi can be applied to the same message. This > > allows for stacked messages, i.e. forwarded log messages contained in > > compressed files contained in e-mail messages. The data retrieved form > > the log messages is stored in a database called Dobby. Due to prisma > > being writt
Re: [VOTE] ALOIS to enter the incubator
Hi Tim Thanks for your interest and thanks to Christian for his effort. So far the project and therefore the code had only be used in two organizations. Therefore, the project is like closed, although licenced under the GPL. There is really almost no usable documentation (at least in English, because the organizations are located in the German speaking part of Switzerland) and the developers would like to go over the code before it is widely published. If I understand the incubator process right, these two tasks should or could be done during the podling phase (which in this case might take a little big longer than with other projects). Having no real project side so far wasn't a real problem for me, because I thought entering the Incubator is more about a good (matching) idea than about working code. Furthermore, I can imagine, that with a bigger community of developers, a good part of the now existing code will be replaced anyway. Since we are not in a hurry, and if it is common sense that there should be more available before entering the incubator, we could do this extra effort. I wonder, what others think of this issue. Best Urs Am Donnerstag, den 16.09.2010, 07:38 -0400 schrieb Tim Williams: > On Thu, Sep 16, 2010 at 7:16 AM, Christian Grobmeier > wrote: > > All, > > > > this vote will fail in three hours because nobody responds to it. Are > > there any objections against this proposal? Or why is this vote > > ignored? > > Hi Christian, > I ignored it because it was odd to me that there's essentially no > further info about the project. I'm wondering why they don't/haven't > begun open development (e.g. via Google Code or somesuch) on their own > initiative since the code's already GPL. I dunno, they don't have > licensing problems and, so, if they "believe in open source" it's not > clear what was stopping them from open development already. > > Well, that's part of the story anyway... I had those thoughts and, > upon reflecting on those thoughts, I began to realize that I have no > clue what the objective criteria is for entering incubation. I am > unsure what would cause us to say no. My vote here is now binding so > I thought it better to watch and learn how more experienced folk > rationalize this. Anyway, that's bordering on too much information > but that's why *I* ignored it:) > > --tim > > > On Wed, Sep 15, 2010 at 4:06 PM, Urs Lerch wrote: > >> Hi everybody out there > >> > >> The vote for ALOIS ends in about 24 hours. Are there any more comments > >> or votes? We would appreciate it to get to know your opinion. > >> > >> Best > >> Urs > >> > >> > >> > >> Am Montag, den 13.09.2010, 11:33 -0400 schrieb Urs Lerch: > >>> Hi > >>> > >>> Since the first call a few weeks ago didn't suceed (more mentors were > >>> asked), I would like to call a second vote for accepting the security > >>> information and event management tool "ALOIS" for incubation in the > >>> Apache Incubator. Thanks Christian Grobmeier we now have two mentors at > >>> least. But any additional mentors are still warmly welcome. The full > >>> proposal is available below and on the proposal wiki page > >>> (http://wiki.apache.org/incubator/AloisProposal). > >>> > >>> Please cast your vote: > >>> > >>> [ ] +1, bring ALOIS into Incubator > >>> [ ] +0, I don't care either way, > >>> [ ] -1, do not bring ALOIS into Incubator, because... > >>> > >>> This vote will be open for 72 hours and, at least that's the way I > >>> understood, only votes from the Incubator PMC are binding. > >>> > >>> Thanks, > >>> Urs > >>> > >>> > >>> > >>> --- > >>> > >>> > >>> = Preface = > >>> > >>> ALOIS is a log collection and correlation software with reporting and > >>> alarming functionalities. It has been implemented by the Swiss company > >>> IMSEC for a customer about five years ago. GPL-licenced, implemented in > >>> Ruby and completely based on other OSS-licensed components, it was > >>> designed for the open source community right from the start. Now that > >>> the software has shown its functioning over several years in production > >>> with the one customer and one IMSEC-internal installation, it seems to > >>> be the right time to op
Re: [VOTE] ALOIS to enter the incubator
Hi everybody out there The vote for ALOIS ends in about 24 hours. Are there any more comments or votes? We would appreciate it to get to know your opinion. Best Urs Am Montag, den 13.09.2010, 11:33 -0400 schrieb Urs Lerch: > Hi > > Since the first call a few weeks ago didn't suceed (more mentors were > asked), I would like to call a second vote for accepting the security > information and event management tool "ALOIS" for incubation in the > Apache Incubator. Thanks Christian Grobmeier we now have two mentors at > least. But any additional mentors are still warmly welcome. The full > proposal is available below and on the proposal wiki page > (http://wiki.apache.org/incubator/AloisProposal). > > Please cast your vote: > > [ ] +1, bring ALOIS into Incubator > [ ] +0, I don't care either way, > [ ] -1, do not bring ALOIS into Incubator, because... > > This vote will be open for 72 hours and, at least that's the way I > understood, only votes from the Incubator PMC are binding. > > Thanks, > Urs > > > > --- > > > = Preface = > > ALOIS is a log collection and correlation software with reporting and > alarming functionalities. It has been implemented by the Swiss company > IMSEC for a customer about five years ago. GPL-licenced, implemented in > Ruby and completely based on other OSS-licensed components, it was > designed for the open source community right from the start. Now that > the software has shown its functioning over several years in production > with the one customer and one IMSEC-internal installation, it seems to > be the right time to open it to a wider community. > > > = Abstract = > > ALOIS stands for „Advanced Logging and Intrusion Detection System“ and > is meant to be a fully implemented open source SIEM (security > information and event management) system. > > > = Proposal = > > While almost all other SIEM software, be it closed or open source, > concentrate on the technological part of security monitoring, ALOIS is > aimed to monitor the security of the content. It intends to be > pro-active in the detection of potential loss, theft, mistaken > modification or unauthorized access. ALOIS works on log messages and > thus contains all the basic functionality of a conventional SIEM, as > centralized collecting, normalizing, aggregation, analyzing and > correlating of all log messages, as well as reporting all security > related events. Therefore it can be used as any other SIEM. > > ALOIS consists of five modules interacting to ensure a scaleable > functionality of a SIEM: > > * Insink is the message sink, which is the receiving entry point for > all the different log messages into ALOIS. It is partly based on the > syslog-ng software. Insink listens for messages (UDP), waits for > messages (TCP), receives message collections (files, emails) and > pre-filters them to prevent from message flow overload. > > * Pumpy is the incoming FIFO buffer, implemented as a relational > database tables. which contain the incoming original messages (in raw > format). In a complex system setup, there may be several insink > instances, e.g. for a group of hosts, for specific types of messages, or > for high-avaliablity. > > * Prisma contains logic to split up the text of log messages into > separate fields, based on regular expressions. Actually, "prisma" is a > set of "prismi", each one prisma for one type of log message (apache, > cisco etc. Several prismi can be applied to the same message. This > allows for stacked messages, i.e. forwarded log messages contained in > compressed files contained in e-mail messages. The data retrieved form > the log messages is stored in a database called Dobby. Due to prisma > being written in Ruby, prismi can be applied interactively (when having > system access). > > * Dobby is the central log database. It should be separated from the > Pumpy database for availability and performance reasons. The current > implementation is based on MySQL. > > * The Analyzer contains the two sub-systems Lizard and Reptor. Lizard > is the analysis engine and user interface of ALOIS, implemented in Ruby > on Rails using AJAX. It allows for interactive browsing through the > collected data, exclusion/inclusion/selection of data, data sorting, > data filtering, creation of views, ad-hoc textual and graphical > reporting. Reptor allows for automatic activation of views and > comparison of these views' results to a predefined result (pattern > matching). In case of mismatch, Reptor sends the result to predefined > e-mail add
Re: [VOTE] ALOIS to enter the incubator
Hi Tim Unfortunately, so far there is no link to the existing code. Until now the software has been used only in two organizations. Be asured that we are aware that we have to do a lot of basic work to release the code to the community. Nontheless, if you are interested I can send you the code as it is. Best Urs Am Dienstag, den 14.09.2010, 07:54 -0400 schrieb Tim Williams: > I might be missing it, but is there a link to the existing GPL project/code? > Thanks > --tim > > On Mon, Sep 13, 2010 at 11:33 AM, Urs Lerch wrote: > > Hi > > > > Since the first call a few weeks ago didn't suceed (more mentors were > > asked), I would like to call a second vote for accepting the security > > information and event management tool "ALOIS" for incubation in the > > Apache Incubator. Thanks Christian Grobmeier we now have two mentors at > > least. But any additional mentors are still warmly welcome. The full > > proposal is available below and on the proposal wiki page > > (http://wiki.apache.org/incubator/AloisProposal). > > > > Please cast your vote: > > > > [ ] +1, bring ALOIS into Incubator > > [ ] +0, I don't care either way, > > [ ] -1, do not bring ALOIS into Incubator, because... > > > > This vote will be open for 72 hours and, at least that's the way I > > understood, only votes from the Incubator PMC are binding. > > > > Thanks, > > Urs > > > > > > > > --- > > > > > > = Preface = > > > > ALOIS is a log collection and correlation software with reporting and > > alarming functionalities. It has been implemented by the Swiss company > > IMSEC for a customer about five years ago. GPL-licenced, implemented in > > Ruby and completely based on other OSS-licensed components, it was > > designed for the open source community right from the start. Now that > > the software has shown its functioning over several years in production > > with the one customer and one IMSEC-internal installation, it seems to > > be the right time to open it to a wider community. > > > > > > = Abstract = > > > > ALOIS stands for „Advanced Logging and Intrusion Detection System“ and > > is meant to be a fully implemented open source SIEM (security > > information and event management) system. > > > > > > = Proposal = > > > > While almost all other SIEM software, be it closed or open source, > > concentrate on the technological part of security monitoring, ALOIS is > > aimed to monitor the security of the content. It intends to be > > pro-active in the detection of potential loss, theft, mistaken > > modification or unauthorized access. ALOIS works on log messages and > > thus contains all the basic functionality of a conventional SIEM, as > > centralized collecting, normalizing, aggregation, analyzing and > > correlating of all log messages, as well as reporting all security > > related events. Therefore it can be used as any other SIEM. > > > > ALOIS consists of five modules interacting to ensure a scaleable > > functionality of a SIEM: > > > > * Insink is the message sink, which is the receiving entry point for > > all the different log messages into ALOIS. It is partly based on the > > syslog-ng software. Insink listens for messages (UDP), waits for > > messages (TCP), receives message collections (files, emails) and > > pre-filters them to prevent from message flow overload. > > > > * Pumpy is the incoming FIFO buffer, implemented as a relational > > database tables. which contain the incoming original messages (in raw > > format). In a complex system setup, there may be several insink > > instances, e.g. for a group of hosts, for specific types of messages, or > > for high-avaliablity. > > > > * Prisma contains logic to split up the text of log messages into > > separate fields, based on regular expressions. Actually, "prisma" is a > > set of "prismi", each one prisma for one type of log message (apache, > > cisco etc. Several prismi can be applied to the same message. This > > allows for stacked messages, i.e. forwarded log messages contained in > > compressed files contained in e-mail messages. The data retrieved form > > the log messages is stored in a database called Dobby. Due to prisma > > being written in Ruby, prismi can be applied interactively (when having > > system access). > > > > * Dobby is the central log database. It should be separated from the > > Pumpy database for availability a
[VOTE] ALOIS to enter the incubator
discussed that the filter creation engine would make a good tool for any kind of structured data, and thus could be separated from ALOIS and standardized as a stand-alone tool. = Background = It's not simple to know what happens in a bigger network. There's a multitude of applications, services and appliances working together. Many of them provide some kind of events or state information. The network administrator needs to get hands on all of them. But they come in many different flavors and multiple canals. Therefore, it's hard to get the big picture. Furthermore, we have learned that it's impossible to protect a system against all malicious attacks and to keep all the possible faulty handling away. A monitoring of the systems to guarantee a pro-active handling is therefore needed.. Therefore, more and more organizations collect and analyze all logfiles in a centralized system, called a SIEM (security information and event management). The technology provides two major functions for security events from networks, systems and applications: log management and compliance reporting (SIM – security information management) and real-time monitoring and incident management (SEM – security event management). = Rationale = Why another security information and event management system? It's true, there's already plenty of them. While the proprietary software is way too expensive for smaller to mid-sized companies, we find that the open source solutions are either too simple or not completely open. For example, behind each of the well known systems “OSSIM” and “Prelude”, there is a company that either closes central functionality for its own business or has dual licensing and therefore asks the full copyright for all contributed code. ALOIS is aimed to be totally free and open for all contributions. The openness provided for other programming languages is certainly proof of this. The plug-ability - yet to be further developed - is meant to guarantee that individual needs can be realized without stressing the whole system too much. In our opinion, the Linux kernel is a good example that this can work very well. Since we are in accordance with „the Apache way“, we would be very pleased if ALOIS could become part of the Apache community. In Addition, the Apache Logging Services would be a perfect home for the software. Furthermore, it's not the intention to compete with the already existing log viewer and analyzing tool „Chainsaw“. Since Chainsaw is a relatively easy tool, it meets a rather different need. Nevertheless, if the two projects use synergies, both can profit. = Initial Goals = When this project started ins 2005, there was no proven SIEM open source software and the commercial tools were way too expensive for the needed environment. Therefore, we decided together with a customer of ours to implement an open source SIEM tool from scratch. Now the software has run in a production environment for several years and has proven its functionality and reliabilty. = Current Status = == Meritocracy == As already mentioned, ALOIS is already in production use in two organizations. All the code has been written by two persons of the same company in a paid employment relationship. It is obvious that this is way different from the open source approach within Apache. But nevertheless, the two developers have always worked as a team and the decisions were made in consensus whenever possible. But it is no secret, that these developers have to learn to behave in an open community. Understanding this potential problem, they already got support by a freelance consulter, who has the corresponding experience and knowledge. == Community == Until today there is no real community, because the project hasn't been published officially, although it had been completely published on the web site for a couple of months (until a server relaunch). Convinced by the concept and design of the software, we are open and hope to reach many contributors and users. We think that it is realistic, because the SIEM issue has yet not been resolved in the OSS space. == Core Developers == ALOIS was developed by Simon Hürliman and Flavio Pellanda, both employed by the company IMSEC. Concerning Design and Architecture, Marcus Holthaus, owner of IMSEC, gave his input as security specialist. Since the beginning of this year, Urs Lerch, a doctorate on the subject of commercial open source software development, supports the team with his knowledge. Simon Hürlimann has left the company three years ago, but is still active in the OSS environment (although not for ALOIS). Current employee Daniel Lutz (a Debian Developer) has also contributed to the project. == Alignment == Besides that we strongly believe in the „Apache way“, we think that although that Apache hosts the Logging Services and different security projects, there is a gap when it comes to a superordinate security view. We therefore think it a goo
Re: [MENTORS WANTED] for ALOIS incubation
Dear Christian Thanks a lot for your offer! Also Scott Deboy, who acts as a champion and denominated mentor for ALOIS, will appreciate it. Since Scott is ready to do the main part of mentoring, we are still looking for at least one or two (experienced) mentors who could assist him, whenever he has questions. Best Urs Am Freitag, den 03.09.2010, 11:49 +0200 schrieb Christian Grobmeier: > Hey guys, > > I would like to help as a mentor for ALOIS. > I am involved into Logging and Commons. Ruby is on my current interest > list, but I am not an expert. > > Cheers, > Christian > > On Fri, Aug 27, 2010 at 4:44 PM, Urs Lerch wrote: > > Hi > > > > The SIEM (Security Incident and Event Manager) project called ALOIS > > would like very much to join the Apache community. We therefore have > > posted a proposal and started the voting a day ago. Since we are all new > > to the incubator process, we think it would be a good idea to share the > > role of a mentor by at least three volunteers to reduce the work of > > each. We know you are all very busy, the more we would apreciate your > > willingness. Thanks in advance! > > > > Please find the proposal in the wiki: > > http://wiki.apache.org/incubator/AloisProposal > > > > Best > > Urs > > > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > signature.asc Description: Dies ist ein digital signierter Nachrichtenteil
[MENTORS WANTED] for ALOIS incubation
Hi The SIEM (Security Incident and Event Manager) project called ALOIS would like very much to join the Apache community. We therefore have posted a proposal and started the voting a day ago. Since we are all new to the incubator process, we think it would be a good idea to share the role of a mentor by at least three volunteers to reduce the work of each. We know you are all very busy, the more we would apreciate your willingness. Thanks in advance! Please find the proposal in the wiki: http://wiki.apache.org/incubator/AloisProposal Best Urs signature.asc Description: Dies ist ein digital signierter Nachrichtenteil
Re: [VOTE] ALOIS to enter the incubator
Good point! I agree with you that three mentors would be perfect. Since I've seen other votes that have reported more volunteers as mentors, I had the hope that it happens here, too. I did already ask other persons for support, but didn't succeed yet. Do you have any advice how to be more attractive for mentors? Best Urs Am Freitag, den 27.08.2010, 10:52 +0200 schrieb Bernd Fondermann: > as soon as you come up with 2 more mentors, you have my +1. > > Bernd > > On Thu, Aug 26, 2010 at 18:09, Urs Lerch wrote: > > Hi, > > > > I would like to call a vote for accepting "ALOIS" for incubation in > > the Apache Incubator. The full proposal is available below and on the > > proposal wiki page (http://wiki.apache.org/incubator/AloisProposal). We > > ask the Incubator PMC to sponsor it, with Scott Deboy volunteering as > > Champion and Mentor. Additional mentors are warmly welcome. > > > > Please cast your vote: > > > > [ ] +1, bring ALOIS into Incubator > > [ ] +0, I don't care either way, > > [ ] -1, do not bring ALOIS into Incubator, because... > > > > This vote will be open for 72 hours and only votes from the Incubator > > PMC are binding. > > > > Thanks, > > Urs > > > > > > > > > > > > = Preface = > > > > ALOIS is a log collection and correlation software with reporting and > > alarming functionalities. It has been implemented by the Swiss company > > IMSEC for a customer about five years ago. GPL-licenced, implemented in > > Ruby and completely based on other OSS-licensed components, it was > > designed for the open source community right from the start. Now that > > the software has shown its functioning over several years in production > > with the one customer and one IMSEC-internal installation, it seems to > > be the right time to open it to a wider community. > > > > > > = Abstract = > > > > ALOIS stands for „Advanced Logging and Intrusion Detection System“ and > > is meant to be a fully implemented open source SIEM (security > > information and event management) system. > > > > > > = Proposal = > > > > While almost all other SIEM software, be it closed or open source, > > concentrate on the technological part of security monitoring, ALOIS is > > aimed to monitor the security of the content. It intends to be > > pro-active in the detection of potential loss, theft, mistaken > > modification or unauthorized access. ALOIS works on log messages and > > thus contains all the basic functionality of a conventional SIEM, as > > centralized collecting, normalizing, aggregation, analyzing and > > correlating of all log messages, as well as reporting all security > > related events. Therefore it can be used as any other SIEM. > > > > ALOIS consists of five modules interacting to ensure a scaleable > > functionality of a SIEM: > > > > * Insink is the message sink, which is the receiving entry point for > > all the different log messages into ALOIS. It is partly based on the > > syslog-ng software. Insink listens for messages (UDP), waits for > > messages (TCP), receives message collections (files, emails) and > > pre-filters them to prevent from message flow overload. > > > > * Pumpy is the incoming FIFO buffer, implemented as a relational > > database tables. which contain the incoming original messages (in raw > > format). In a complex system setup, there may be several insink > > instances, e.g. for a group of hosts, for specific types of messages, or > > for high-avaliablity. > > > > * Prisma contains logic to split up the text of log messages into > > separate fields, based on regular expressions. Actually, "prisma" is a > > set of "prismi", each one prisma for one type of log message (apache, > > cisco etc. Several prismi can be applied to the same message. This > > allows for stacked messages, i.e. forwarded log messages contained in > > compressed files contained in e-mail messages. The data retrieved form > > the log messages is stored in a database called Dobby. Due to prisma > > being written in Ruby, prismi can be applied interactively (when having > > system access). > > > > * Dobby is the central log database. It should be separated from the > > Pumpy database for availability and performance reasons. The current > > implementation is based on MySQL. > > > > * The Analyzer contains the two sub-systems Lizard and Reptor. Lizard > > is the analysis engine and user interface
Re: [VOTE] ALOIS to enter the incubator
Hi There is, at least in my opinion, a very clear statement regarding the licencing: = Source and Intellectual Property Submission Plan = ALOIS is currently under a GPL licence. Since there are only two contributors so far, both from the same company, there is no problem to re-licence the code and contribute it to Apache. The commitment of the company's owner has been granted. The names of the two contributors are listed elsewhere in the proposal. Do you think that ain't enough? Best Urs Am Donnerstag, den 26.08.2010, 12:17 -0400 schrieb Benson Margulies: > I don't see anything explicit in here about relicensing from GPL to > ASL. Perhaps that was hashed out before I joined the PMC? > > I'm +0 tending toward -1 without an explicit statement that the > copyright owners are known and on board with the license change. > > On Thu, Aug 26, 2010 at 12:09 PM, Urs Lerch wrote: > > Hi, > > > > I would like to call a vote for accepting "ALOIS" for incubation in > > the Apache Incubator. The full proposal is available below and on the > > proposal wiki page (http://wiki.apache.org/incubator/AloisProposal). We > > ask the Incubator PMC to sponsor it, with Scott Deboy volunteering as > > Champion and Mentor. Additional mentors are warmly welcome. > > > > Please cast your vote: > > > > [ ] +1, bring ALOIS into Incubator > > [ ] +0, I don't care either way, > > [ ] -1, do not bring ALOIS into Incubator, because... > > > > This vote will be open for 72 hours and only votes from the Incubator > > PMC are binding. > > > > Thanks, > > Urs > > > > > > > > > > > > = Preface = > > > > ALOIS is a log collection and correlation software with reporting and > > alarming functionalities. It has been implemented by the Swiss company > > IMSEC for a customer about five years ago. GPL-licenced, implemented in > > Ruby and completely based on other OSS-licensed components, it was > > designed for the open source community right from the start. Now that > > the software has shown its functioning over several years in production > > with the one customer and one IMSEC-internal installation, it seems to > > be the right time to open it to a wider community. > > > > > > = Abstract = > > > > ALOIS stands for „Advanced Logging and Intrusion Detection System“ and > > is meant to be a fully implemented open source SIEM (security > > information and event management) system. > > > > > > = Proposal = > > > > While almost all other SIEM software, be it closed or open source, > > concentrate on the technological part of security monitoring, ALOIS is > > aimed to monitor the security of the content. It intends to be > > pro-active in the detection of potential loss, theft, mistaken > > modification or unauthorized access. ALOIS works on log messages and > > thus contains all the basic functionality of a conventional SIEM, as > > centralized collecting, normalizing, aggregation, analyzing and > > correlating of all log messages, as well as reporting all security > > related events. Therefore it can be used as any other SIEM. > > > > ALOIS consists of five modules interacting to ensure a scaleable > > functionality of a SIEM: > > > > * Insink is the message sink, which is the receiving entry point for > > all the different log messages into ALOIS. It is partly based on the > > syslog-ng software. Insink listens for messages (UDP), waits for > > messages (TCP), receives message collections (files, emails) and > > pre-filters them to prevent from message flow overload. > > > > * Pumpy is the incoming FIFO buffer, implemented as a relational > > database tables. which contain the incoming original messages (in raw > > format). In a complex system setup, there may be several insink > > instances, e.g. for a group of hosts, for specific types of messages, or > > for high-avaliablity. > > > > * Prisma contains logic to split up the text of log messages into > > separate fields, based on regular expressions. Actually, "prisma" is a > > set of "prismi", each one prisma for one type of log message (apache, > > cisco etc. Several prismi can be applied to the same message. This > > allows for stacked messages, i.e. forwarded log messages contained in > > compressed files contained in e-mail messages. The data retrieved form > > the log messages is stored in a database called Dobby. Due to prisma > > being written in Ruby, prismi can be applied interactiv
[VOTE] ALOIS to enter the incubator
= It's not simple to know what happens in a bigger network. There's a multitude of applications, services and appliances working together. Many of them provide some kind of events or state information. The network administrator needs to get hands on all of them. But they come in many different flavors and multiple canals. Therefore, it's hard to get the big picture. Furthermore, we have learned that it's impossible to protect a system against all malicious attacks and to keep all the possible faulty handling away. A monitoring of the systems to guarantee a pro-active handling is therefore needed.. Therefore, more and more organizations collect and analyze all logfiles in a centralized system, called a SIEM (security information and event management). The technology provides two major functions for security events from networks, systems and applications: log management and compliance reporting (SIM – security information management) and real-time monitoring and incident management (SEM – security event management). = Rationale = Why another security information and event management system? It's true, there's already plenty of them. While the proprietary software is way too expensive for smaller to mid-sized companies, we find that the open source solutions are either too simple or not completely open. For example, behind each of the well known systems “OSSIM” and “Prelude”, there is a company that either closes central functionality for its own business or has dual licensing and therefore asks the full copyright for all contributed code. ALOIS is aimed to be totally free and open for all contributions. The openness provided for other programming languages is certainly proof of this. The plug-ability - yet to be further developed - is meant to guarantee that individual needs can be realized without stressing the whole system too much. In our opinion, the Linux kernel is a good example that this can work very well. Since we are in accordance with „the Apache way“, we would be very pleased if ALOIS could become part of the Apache community. In Addition, the Apache Logging Services would be a perfect home for the software. Furthermore, it's not the intention to compete with the already existing log viewer and analyzing tool „Chainsaw“. Since Chainsaw is a relatively easy tool, it meets a rather different need. Nevertheless, if the two projects use synergies, both can profit. = Initial Goals = When this project started ins 2005, there was no proven SIEM open source software and the commercial tools were way too expensive for the needed environment. Therefore, we decided together with a customer of ours to implement an open source SIEM tool from scratch. Now the software has run in a production environment for several years and has proven its functionality and reliabilty. = Current Status = == Meritocracy == As already mentioned, ALOIS is already in production use in two organizations. All the code has been written by two persons of the same company in a paid employment relationship. It is obvious that this is way different from the open source approach within Apache. But nevertheless, the two developers have always worked as a team and the decisions were made in consensus whenever possible. But it is no secret, that these developers have to learn to behave in an open community. Understanding this potential problem, they already got support by a freelance consulter, who has the corresponding experience and knowledge. == Community == Until today there is no real community, because the project hasn't been published officially, although it had been completely published on the web site for a couple of months (until a server relaunch). Convinced by the concept and design of the software, we are open and hope to reach many contributors and users. We think that it is realistic, because the SIEM issue has yet not been resolved in the OSS space. == Core Developers == ALOIS was developed by Simon Hürliman and Flavio Pellanda, both employed by the company IMSEC. Concerning Design and Architecture, Marcus Holthaus, owner of IMSEC, gave his input as security specialist. Since the beginning of this year, Urs Lerch, a doctorate on the subject of commercial open source software development, supports the team with his knowledge. Simon Hürlimann has left the company three years ago, but is still active in the OSS environment (although not for ALOIS). Current employee Daniel Lutz (a Debian Developer) has also contributed to the project. == Alignment == Besides that we strongly believe in the „Apache way“, we think that although that Apache hosts the Logging Services and different security projects, there is a gap when it comes to a superordinate security view. We therefore think it a good idea to add our SIEM project to the Apache repository. On the other side, Apache would become an even more complete software repository. = Known Risks = == Orphaned products == Since t
Re: [PROPOSAL] ALOIS Project
Hi A little more than a week ago I posted the proposal concerning the incubation of ALOIS. Unfortunatly there wasn't much of a discussion so far. Furthermore, we are still looking for mentors. Any feedback is welcome! Best Urs P.S.: http://wiki.apache.org/incubator/AloisProposal. Am Samstag, den 14.08.2010, 10:31 +0200 schrieb Urs Lerch: > Greetings All > > I would like to formally propose that the ALOIS Project be considered > for inclusion in the ASF Incubator as a new podling. ALOIS is a log > collection and correlation software with reporting and alarming > functionalities (a so-called SIEM). The full details of this proposal > are available below. I hope that the length of the text doesn't prevent > you from reading it. > > Furthermore, we are looking for Mentors, and any additional contributors > that we can get. And we gracefully ask the Incubator PMC for sponsoring > this project. > > We were happy to receive your feedback about our proposal. > > Best regards > Urs > > > > Here the full text of the proposal: > > > = Preface = > > ALOIS is a log collection and correlation software with reporting and > alarming functionalities. It has been implemented by the Swiss company > IMSEC for a customer about five years ago. GPL-licenced, implemented in > Ruby and completely based on other OSS-licensed components, it was > designed for the open source community right from the start. Now that > the software has shown its functioning over several years in production > with the one customer and one IMSEC-internal installation, it seems to > be the right time to open it to a wider community. > > > = Abstract = > > ALOIS stands for „Advanced Logging and Intrusion Detection System“ and > is meant to be a fully implemented open source SIEM (security > information and event management) system. > > > = Proposal = > > While almost all other SIEM software, be it closed or open source, > concentrate on the technological part of security monitoring, ALOIS is > aimed to monitor the security of the content. It intends to be > pro-active in the detection of potential loss, theft, mistaken > modification or unauthorized access. ALOIS works on log messages and > thus contains all the basic functionality of a conventional SIEM, as > centralized collecting, normalizing, aggregation, analyzing and > correlating of all log messages, as well as reporting all security > related events. Therefore it can be used as any other SIEM. > > ALOIS consists of five modules interacting to ensure a scaleable > functionality of a SIEM: > > * Insink is the message sink, which is the receiving entry point for > all the different log messages into ALOIS. It is partly based on the > syslog-ng software. Insink listens for messages (UDP), waits for > messages (TCP), receives message collections (files, emails) and > pre-filters them to prevent from message flow overload. > > * Pumpy is the incoming FIFO buffer, implemented as a relational > database tables. which contain the incoming original messages (in raw > format). In a complex system setup, there may be several insink > instances, e.g. for a group of hosts, for specific types of messages, or > for high-avaliablity. > > * Prisma contains logic to split up the text of log messages into > separate fields, based on regular expressions. Actually, "prisma" is a > set of "prismi", each one prisma for one type of log message (apache, > cisco etc. Several prismi can be applied to the same message. This > allows for stacked messages, i.e. forwarded log messages contained in > compressed files contained in e-mail messages. The data retrieved form > the log messages is stored in a database called Dobby. Due to prisma > being written in Ruby, prismi can be applied interactively (when having > system access). > > * Dobby is the central log database. It should be separated from the > Pumpy database for availability and performance reasons. The current > implementation is based on MySQL. > > * The Analyzer contains the two sub-systems Lizard and Reptor. Lizard > is the analysis engine and user interface of ALOIS, implemented in Ruby > on Rails using AJAX. It allows for interactive browsing through the > collected data, exclusion/inclusion/selection of data, data sorting, > data filtering, creation of views, ad-hoc textual and graphical > reporting. Reptor allows for automatic activation of views and > comparison of these views' results to a predefined result (pattern > matching). In case of mismatch, Reptor sends the result to predefined > e-mail addresses. > > Its modular design guarantees ALOIS to scale from little to large >
Re: [Proposal] ALOIS Project
Hi > IANAL either. At Apache we encourage the use of dependencies that are > licensed with an Apache-compatible license but we are not strict about > it. We have projects with dependencies on such things as Microsoft > Windows. We have had projects with hard dependencies on Java before > Java was open source. Since we have learned that MySQL is not the best solution for our purpose - i.e. tools with other backends get a better performance on standard hardware -, the replacement of MySQL is on our roadmap. > Since ALOIS is written in Ruby, it might be easy enough to add a > database-independence layer to remove the hard dependency on MySQL. Craig, I too think it is a very good idea to implement a database-independent layer. Even more, it is my designated goal to add APIs to all of the five moduls, so we get more flexibility. > Not a reason to turn away the project. Happy to hear that! Best regards Urs signature.asc Description: Dies ist ein digital signierter Nachrichtenteil
Re: [Proposal] ALOIS Project
Hi > For me, the challenge during Incubation would revolve mostly around > community. Perhaps it is exactly what Logging project needs to infuse > more interest and activity, and that we will see folks from that > background joining in... I don't think the current "weak" community is > reason enough to prevent incubation, only pointing it out as a > substantial challenge which may slow down the process. I am very thankful for this feedback. It is exactly what we think. The SIEM market needs a truly open source tool. And in our opinion, open source does mean, beside other aspects as the licence of course, independence and community. And that is what lacks all existing tools, at least as far as we know. The company behind ALOIS is ready to give up the control on the tool. We know that it is hard work to build a sustaining community, but we are willing to face this challenge. Best regards Urs signature.asc Description: Dies ist ein digital signierter Nachrichtenteil
Re: [Proposal] ALOIS Project
Hi Thanks for the feedback. > If it's been GPL-licensed, you'll need the permission of all contributors > to re-license it. Have you kept meticulous records of all contributions > over the lifetime of the project so you can expect to be able to contact > everyone? Since the software has been developed by only two persons, both in paid employment, and the their employer has a declared interest in entering the Apache Incubator, I see no problem concerning the re-licencing. > What about the required OSS-licensed components? You mention > MySQL - you'd (probably) want a specific exception for that, in the > manner of the APR's one. Although I am no lawyer, in my understanding the database is not part of the software and therefore the licence of MySQL (or any other database) is no subject here. Maybe someone with legal know how might clarify this point. Best regards Urs signature.asc Description: Dies ist ein digital signierter Nachrichtenteil
Re: [PROPOSAL] ALOIS Project
Greetings All In addition to my posting of yesterday, I send you the link to the proposal wiki page: http://wiki.apache.org/incubator/AloisProposal Best regards Urs P.S.: Since the prefix of the subject in my first posting has been in lower case, and I don't know if it is important if it is in upper case, I've corrected that. signature.asc Description: Dies ist ein digital signierter Nachrichtenteil
[Proposal] ALOIS Project
und = It's not simple to know what happens in a bigger network. There's a multitude of applications, services and appliances working together. Many of them provide some kind of events or state information. The network administrator needs to get hands on all of them. But they come in many different flavors and multiple canals. Therefore, it's hard to get the big picture. Furthermore, we have learned that it's impossible to protect a system against all malicious attacks and to keep all the possible faulty handling away. A monitoring of the systems to guarantee a pro-active handling is therefore needed.. Therefore, more and more organizations collect and analyze all logfiles in a centralized system, called a SIEM (security information and event management). The technology provides two major functions for security events from networks, systems and applications: log management and compliance reporting (SIM – security information management) and real-time monitoring and incident management (SEM – security event management). = Rationale = Why another security information and event management system? It's true, there's already plenty of them. While the proprietary software is way too expensive for smaller to mid-sized companies, we find that the open source solutions are either too simple or not completely open. For example, behind each of the well known systems “OSSIM” and “Prelude”, there is a company that either closes central functionality for its own business or has dual licensing and therefore asks the full copyright for all contributed code. ALOIS is aimed to be totally free and open for all contributions. The openness provided for other programming languages is certainly proof of this. The plug-ability - yet to be further developed - is meant to guarantee that individual needs can be realized without stressing the whole system too much. In our opinion, the Linux kernel is a good example that this can work very well. Since we are in accordance with „the Apache way“, we would be very pleased if ALOIS could become part of the Apache community. In Addition, the Apache Logging Services would be a perfect home for the software. Furthermore, it's not the intention to compete with the already existing log viewer and analyzing tool „Chainsaw“. Since Chainsaw is a relatively easy tool, it meets a rather different need. Nevertheless, if the two projects use synergies, both can profit. = Initial Goals = When this project started ins 2005, there was no proven SIEM open source software and the commercial tools were way too expensive for the needed environment. Therefore, we decided together with a customer of ours to implement an open source SIEM tool from scratch. Now the software has run in a production environment for several years and has proven its functionality and reliabilty. = Current Status = == Meritocracy == As already mentioned, ALOIS is already in production use in two organizations. All the code has been written by two persons of the same company in a paid employment relationship. It is obvious that this is way different from the open source approach within Apache. But nevertheless, the two developers have always worked as a team and the decisions were made in consensus whenever possible. But it is no secret, that these developers have to learn to behave in an open community. Understanding this potential problem, they already got support by a freelance consulter, who has the corresponding experience and knowledge. == Community == Until today there is no real community, because the project hasn't been published officially, although it had been completely published on the web site for a couple of months (until a server relaunch). Convinced by the concept and design of the software, we are open and hope to reach many contributors and users. We think that it is realistic, because the SIEM issue has yet not been resolved in the OSS space. == Core Developers == ALOIS was developed by Simon Hürliman and Flavio Pellanda, both employed by the company IMSEC. Concerning Design and Architecture, Marcus Holthaus, owner of IMSEC, gave his input as security specialist. Since the beginning of this year, Urs Lerch, a doctorate on the subject of commercial open source software development, supports the team with his knowledge. Simon Hürlimann has left the company three years ago, but is still active in the OSS environment (although not for ALOIS). Current employee Daniel Lutz (a Debian Developer) has also contributed to the project. == Alignment == Besides that we strongly believe in the „Apache way“, we think that although that Apache hosts the Logging Services and different security projects, there is a gap when it comes to a superordinate security view. We therefore think it a good idea to add our SIEM project to the Apache repository. On the other side, Apache would become an even more complete software repository. = Known Risks = == Orphaned products == Si
