Re: [Mailman-Developers] [GSoC 2012] Metrics
Hello, This is probably my last report. Any comments about the project are very appreciated. :) GSoC is almost over and i just pushed all the work i’ve done these three months. It was one of the greatest experiences and i’ve learned a ton of stuff. I feel proud of my work and i can’t wait to see it used in Mailman working environments. For those who don’t know what is all about, a live demo of the app exists here [1]. The code is hosted in Launchpad [2] and if you are interested to learn more, i would suggest you start reading the documentation, from the installation [3], to metrics collector [4] and metrics display [5]. To learn even more about the functionality of the app, please check the tests that lie in the ‘test app’ directory for each submodule. I would like to thank Wacky (a great mentor) and the rest of Mailman developers for their help. Mailman has a great community and of course i’ll stay here for anything regarding improvements of my app or anything else Mailman-related. [1]: http://sophron.latthi.com/metrics/ [2]: https://code.launchpad.net/~sophron/mm3-metrics/trunk [3]: http://bazaar.launchpad.net/~sophron/mm3-metrics/trunk/view/head:/installation.rst [4]: http://bazaar.launchpad.net/~sophron/mm3-metrics/trunk/view/head:/MM3_metrics_collector/__init__.py [5]: http://bazaar.launchpad.net/~sophron/mm3-metrics/trunk/view/head:/MM3_metrics_display/templatetags/__init__.py -- George Chatzisofroniou sophron.latthi.com ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Greetings, Following a new report. I've finished a coalecor daemon and i wrote some generators for KittyStore too. I'm now into establishing a real MM3 connection. The first version of coalescor is ready! Coalescor is a custom Django admin command that is responsible for maintaining the number of entries in the database at a reasonable level. If the number of entries in the database is not a concern, its use is completely optional. There are five parameters which determine the behavior of the coalesence: DAILY_DETAIL_RETAINED, WEEKLY_DETAIL_RETAINED, MONTHLY_DETAIL_RETAINED and ANNUAL_DETAIL_RETAINED. Each one of them represents the number of grains for which that level of detail is maintained. The daemon coalesces only the new data from its last operation date and it was designed in a manner not to be memory intensive. The last two days i also worked on generators. I had already created a script that generates metrics from a mailbox. There is a KittyStore API availabe, so it wasn’t hard to also generate metrics from SQL or MongoDB. -- George Chatzisofroniou sophron.latthi.com ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Hello, Here's my new report. I've finally published my code and created some samples for all of you. Youhou! :) Any feedback is very welcome. I’m happy to report that the first version of the software is now public. I have set up the app with some graph samples here [1], and hosted the code in Launchpad [2]. Above each graph, there is the code snippet that is being used to query the database. There are many syntax rules (most of them are already working) that allow a rich vocabulary for the expression of queries. You may see the BNF language description in the section ‘How to use them’ in the documentation [3]. The JS library that renders the graphs is jqplot. It’s not difficult (even for someone with minor experience to Django) to use another plotting library. The data structure returned by the extraction tag is very flexible and can be used in any manner. See the section ‘Configuring the output’ in the documentation [3]. My next step will be to implement the coalescor which (optionally) merges database entries in order to limit the growth of the number of entries stored in the database Thanks for reading and testing. I would highly appreciate any comments. [1]: http://sophron.latthi.com/metrics/ [2]: https://code.launchpad.net/~sophron/mm3-metrics/trunk [3]: http://bazaar.launchpad.net/~sophron/mm3-metrics/trunk/view/head:/MM3_metrics_display/templatetags/__init__.py -- George Chatzisofroniou sophron.latthi.com ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Hi everyone,
Here's my new report [1] just before the midterm evaluation.
---
Ι am before the midterm evaluation and i’m having fun coding and
learning.
The project has reached the first version and achieves most of the
targets set. Specifically:
Metrics store is ready and it is designed in a way to make it easy to
add new metrics. Every metric is either an event counter (eg posts
sent) or a measured level (eg number of total subscribers) and it is
stored in the database representing an interval of a granularity.
Special methods that retrieve counts for the specified interval or tally
the events with their effects (like a message sent or a new
subscription) are also completed.
Most of the graph tags are completed. In order to make it easy for the
designer to output the metrics he wishes, i have created a language and
its parser. For example, if the designer wishes to output the data of
monthly number of posts for the last two years, he can easily extract
these counts as follows: “ {% EXTRACT AS graph1 %} posts MONTHLY FOR 2
YEARS {% ENDEXTRACT %} “. The whole syntax of tag’s language is defined
by a BNF Language description (you may find an earlier version of this
language in a previous post [2]).
Usually, the designer will add a template specification for the graph
tag and the returned values will be placed in this context, but is is
also possible to customize the way this context is presented or create
his own template (For example, he can easily change the date format or
publish the metrics through a table), since the ’EXTRACT’ tag actually
returns a complex data structure that can be used in any manner.
However, some templates are already created for the designer, with the
most significant to be the one that outputs the graphs. I used jqplot
library [3] (minimal and fast rendered graphs) to render the graphs. So, in
the previous example, to build the graph from the extracted data, the
designer simply has to add the following to his template: “ {% include
“MM3/line_graph.js” with dataset=graph1 title=’List Activity’ %}”, after
loading the appropriate plotting libraries (there are ready templates
that can be included to do that).
For both metrics store and graph tags, i have already created a number
of tests. But i still need to add even more.
There are some things that have not need be done yet and i’ll work on
them in the following days. I need to complete the implementation of the
language (there are some rules that my parser does not handle yet). I
also want to create a template for bar graphs as well, design a
coalescor that will maintain the number of entries in the database at a
reasonable level and finish my generator that will generate the metrics
from scratch (a very first version is done). Last, i need to use
IArchiver to make a real connection with MM3 core (i currently use a
simulator for testing purposes).
---
[1]: http://sophron.latthi.com/gsoc-mailman/?p=70
[2]: http://sophron.latthi.com/gsoc-mailman/?p=62
[3]: http://www.jqplot.com/
--
George Chatzisofroniou
sophron.latthi.com
___
Mailman-Developers mailing list
Mailman-Developers@python.org
http://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives:
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe:
http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org
Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
* Patrick Ben Koetter p...@state-of-mind.de: let me throw in some thoughts just to annoy you ;) Like with most statistical data I mostly see the figures being used to give statements on quantity - top poster, number of threads etc. Do you think it would be possible to also make some statements on quality? Let me give an example: Mailing lists are often places where people go to ask for advice. Someone asking usually starts a thread and continually keeps replying. That easily makes a person top poster and might make the same person a thread starter, but number of posts and threads started gives no indication of that persons knowledge (concerning the mailing lists topic). OTOH someone who has been on the list for ages, who replies more often than starting threads and who ends threads often after she has replied might very well be a very knowledgeable person, because she gives the one answer that solves the problem. Do you think it would be possible to deduct such quality oriented statements? As a follow-up: I just stumbled across http://www.mentby.com/patrick-ben-koetter/, which is nice because it also gives an overview over all (here: some) mailing lists an identity posts to. The second pie chart seems to try to say something about quality. It splits posts in 'relevant' and 'passive', which are not exactly opposites, but well … Actually I'd say they still need to work on their rating: http://www.mentby.com/barry-warsaw/ ;) p@rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666 Amtsgericht MünchenPartnerschaftsregister PR 563 ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Hello Stephen, On Sat, May 19, 2012 at 09:12:03PM +0900, Stephen J. Turnbull wrote: I don't see why. I would think quality metrics would be usefully presented via the same application as quantity metrics. It would be interesting to correlate quality and quantity, for example. What i was trying to say is that the quality metrics need some designing on a posts rating system first. Of course they will be presented by my app. Do you mean to say this is out of scope of my project? As much as I'd like to see quality metrics provided, I'd have to agree with you that it's out of scope of your project (maybe you could do one or more quality metrics on a time-permitting basis at the end of the period). Yes, i think it is out of my GSoC project, but i would like to implement this after finish with quantity metrics this summer. ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
George Chatzisofroniou writes: Author This model represents an author of the mailing list. It mostly keeps track of the number of postings and number of threads started. It has the following fields: - authorid – IntegerField AFAIK every Django object has an internal ID. Why do authors need a separate, human-unfriendly authorid? - authormail – CharField Authors are people. They typically have nameswink/ and often multiple email addresses. There may also be other information (organization, etc) that is available from the headers. - totalmails – IntegerField - totalthreads – IntegerField - firstmsgdate – DateTimeField - lastmsgdate – DateTimeField MailingList This model counts the total number of postings and threads started. - totalmails – IntegerField - totalthreads – IntegerField longestthread? Month Year Views To display the metrics the Django template system will be used. To output the charts i will create some custom tags. The three following views will be used: - General page – On top, there will be general metrics about total authors, total mails and total threads and below three charts (AJAX based) AJAX based doesn't belong in the spec; it's an implementation detail. that represent number of posts per author, number of threads per author and mailing list’s yearly usage. Even below there will be a number of charts (equal to the number of years of list’s existence) that output monthly usage. Why multiple charts? If you can afford a 640x480 chart area, with 4 pixel wide bars you can have 160 months 13 years in one chart. I personally wouldn't hesitate to go to pixel width bars, which gives you 53 years. I don't think people will be looking at charts for precision, but rather to get an overview. At the end, there will be tabular data representing the authors - Author page – Each user will have his own page with his own metrics. Django Admin page – A ‘Generate’ button will be added to the Django admin page. Settings The Django app should handle the following configuration parameters: - Host – Message store data host - Port – Message store data port - Masking – A multi-state variable (None, abbreviated, full) for masking email addresses at the results (we don’t want the emails to be spammed) (1) If at all possible, this should be inherited from the list configuration (DRY). It's not useful if the addresses are available from the archives or by subscribing to the list. (Actually, a really sophisticated spammer might want to attack by spoofing frequent posters on the assumption they're more trusted and more read, but that seems second-order to me.) (2) It would be preferable if authors could supply nicknames, full names, or avatars for this purpose. Interface to the Mailman core - Metrics class - Generate class ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
George Chatzisofroniou writes: Hello Patrick, On Fri, May 18, 2012 at 12:09 AM, Patrick Ben Koetter p...@state-of-mind.de wrote: Like with most statistical data I mostly see the figures being used to give statements on quantity - top poster, number of threads etc. Do you think it would be possible to also make some statements on quality? +1 For example, I think this would be really useful for class discussion lists and the like (on the theory that the best way to learn a subject is to teach it to others). The metrics will primarily extract the activity of a mailing list and its users. It is possible to emphasize on the quality of the posts but i think this is a different app I don't see why. I would think quality metrics would be usefully presented via the same application as quantity metrics. It would be interesting to correlate quality and quantity, for example. Do you mean to say this is out of scope of my project? As much as I'd like to see quality metrics provided, I'd have to agree with you that it's out of scope of your project (maybe you could do one or more quality metrics on a time-permitting basis at the end of the period). ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
On May 19, 2012, at 7:12 AM, Stephen J. Turnbull wrote: George Chatzisofroniou writes: Hello Patrick, On Fri, May 18, 2012 at 12:09 AM, Patrick Ben Koetter p...@state-of-mind.de wrote: Like with most statistical data I mostly see the figures being used to give statements on quantity - top poster, number of threads etc. Do you think it would be possible to also make some statements on quality? +1 For example, I think this would be really useful for class discussion lists and the like (on the theory that the best way to learn a subject is to teach it to others). The metrics will primarily extract the activity of a mailing list and its users. It is possible to emphasize on the quality of the posts but i think this is a different app I don't see why. I would think quality metrics would be usefully presented via the same application as quantity metrics. It would be interesting to correlate quality and quantity, for example. Do you mean to say this is out of scope of my project? As much as I'd like to see quality metrics provided, I'd have to agree with you that it's out of scope of your project (maybe you could do one or more quality metrics on a time-permitting basis at the end of the period). I agree that he should think in terms of having additional quality-based metrics and presentations. This ties directly into some of the things that the hyperkitty guys are doing. We need to develop a single approach to the collection, storage, and display of this information. However, other than considering how the hooks would work, I think that this is beyond the scope of this phase of development. Implementation should be considered to be part of a different contract. Richard ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
On May 19, 2012, at 6:58 AM, Stephen J. Turnbull wrote: George Chatzisofroniou writes: Author This model represents an author of the mailing list. It mostly keeps track of the number of postings and number of threads started. It has the following fields: - authorid – IntegerField AFAIK every Django object has an internal ID. Why do authors need a separate, human-unfriendly authorid? Since George will not own the information about the author, this is his foreign key link into that data. - authormail – CharField Authors are people. They typically have nameswink/ and often multiple email addresses. There may also be other information (organization, etc) that is available from the headers. I think that we should remove ALL of the author information from the MM core and create a separate service to collect and manage it. The mail handling core can subscribe to this service for the little necessary information that it requires about the persons. In the real world, the relationship between the organization and the people subscribed to a mailing list often is not centered on the mailing list. They are customers, employees, participants, or such. From the POV of the mailing list, other than authentication of posting/subscription status, those details are not important. There is no reason why the mailing list handler should be the authority/repository for some, but not all, of the information about these persons. - totalmails – IntegerField - totalthreads – IntegerField - firstmsgdate – DateTimeField - lastmsgdate – DateTimeField MailingList This model counts the total number of postings and threads started. - totalmails – IntegerField - totalthreads – IntegerField longestthread? Month Year Views To display the metrics the Django template system will be used. To output the charts i will create some custom tags. The three following views will be used: - General page – On top, there will be general metrics about total authors, total mails and total threads and below three charts (AJAX based) AJAX based doesn't belong in the spec; it's an implementation detail. Agreed. George is still working on understanding the abstraction distinction. He still wants to expose things that should remain under the hood. He should use more black paint. that represent number of posts per author, number of threads per author and mailing list’s yearly usage. Even below there will be a number of charts (equal to the number of years of list’s existence) that output monthly usage. Why multiple charts? If you can afford a 640x480 chart area, with 4 pixel wide bars you can have 160 months 13 years in one chart. I personally wouldn't hesitate to go to pixel width bars, which gives you 53 years. I don't think people will be looking at charts for precision, but rather to get an overview. I would hope that he creates a chart widget (Django custom template tag) that will allow the site designer to choose the level of detail and duration covered by a particular instance. At the end, there will be tabular data representing the authors - Author page – Each user will have his own page with his own metrics. Django Admin page – A ‘Generate’ button will be added to the Django admin page. Settings The Django app should handle the following configuration parameters: - Host – Message store data host - Port – Message store data port - Masking – A multi-state variable (None, abbreviated, full) for masking email addresses at the results (we don’t want the emails to be spammed) (1) If at all possible, this should be inherited from the list configuration (DRY). It's not useful if the addresses are available from the archives or by subscribing to the list. (Actually, a really sophisticated spammer might want to attack by spoofing frequent posters on the assumption they're more trusted and more read, but that seems second-order to me.) (2) It would be preferable if authors could supply nicknames, full names, or avatars for this purpose. Agreed. This needs to be a part of the person data store. Interface to the Mailman core - Metrics class - Generate class ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
On May 19, 2012, at 08:18 AM, Richard Wackerbarth wrote: I think that we should remove ALL of the author information from the MM core and create a separate service to collect and manage it. The mail handling core can subscribe to this service for the little necessary information that it requires about the persons. This should be possible with today's Mailman 3, though it might not be obvious (and certainly isn't tested ;). To do this, you'd implement the IUserManager interface with whatever external-service-consulting implementation you'd like to come up with. Then you'd associate that component implementation with the utility via the zope.configuration file. Once you've done this, Mailman will always get its users and addresses from the your separate service. Come to think of it though, you probably also need to re-implement the various IRoster implementations as well. It might take some fiddling and experimentation, but the architecture intends to make this kind of thing possible. Cheers, -Barry ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Hello Patrick, On Fri, May 18, 2012 at 12:09 AM, Patrick Ben Koetter p...@state-of-mind.de wrote: Like with most statistical data I mostly see the figures being used to give statements on quantity - top poster, number of threads etc. Do you think it would be possible to also make some statements on quality? The metrics will primarily extract the activity of a mailing list and its users. It is possible to emphasize on the quality of the posts but i think this is a different app (but it interacts with the Metrics one). If the users were able to rank the posts through the archiver it wouldn't be hard to indicate one user's offering to his community. This looks like the first idea on this page [1]. P.S. Do you also plan to deliver a tool that analyzes a mailing list archive in order to gather your statistical data? Having the statistical data might be a good reason for people to upgrade their MMx installation to MM3. Yes, there will be a special Generate button in case of an existing archive or in case of a system crash. [1]: http://blog.linuxgrrl.com/2012/03/13/mailman-brainstorm/ -- George Chatzisofroniou sophron.latthi.com ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
The following document is the lowest level of my design concept. You may also read it in my blog [1]. Of course, comments are very welcome. -- Models In order to store statistical data, the app will use some Django models: Author This model represents an author of the mailing list. It mostly keeps track of the number of postings and number of threads started. It has the following fields: - authorid – IntegerField - authormail – CharField - totalmails – IntegerField - totalthreads – IntegerField - firstmsgdate – DateTimeField - lastmsgdate – DateTimeField MailingList This model counts the total number of postings and threads started. - totalmails – IntegerField - totalthreads – IntegerField Month This model associates the author and the mailing list with each month. - author – ForeignKey - month – CharField - postscount – IntegerField - threadscount – IntegerField - mailinglist – Boolean (if this is true it corresponds to the whole mailing list) Year This model is similar to month. It has a year field instead of a month field. Views To display the metrics the Django template system will be used. To output the charts i will create some custom tags. The three following views will be used: - General page – On top, there will be general metrics about total authors, total mails and total threads and below three charts (AJAX based) that represent number of posts per author, number of threads per author and mailing list’s yearly usage. Even below there will be a number of charts (equal to the number of years of list’s existence) that output monthly usage. At the end, there will be tabular data representing the authors of the mailing list along with their number of posts, number of threads started and the date of their last post. The user will be able to order the tabular data (alphabetically, ascending/descending on number of posts, number of threads, date of last message) by clicking on the table’s headings (Mail, Mails Sent, Threads Started, Last Message). - Author page – Each user will have his own page with his own metrics. On top, there will be the email of the author, number of posts, number of threads started and the dates of first and last message. Below there will be monthly usage charts for each year the user is subscribed to the mailing list. Django Admin page – A ‘Generate’ button will be added to the Django admin page. Settings The Django app should handle the following configuration parameters: - Host – Message store data host - Port – Message store data port - Masking – A multi-state variable (None, abbreviated, full) for masking email addresses at the results (we don’t want the emails to be spammed) Interface to the Mailman core - Metrics class – When a new post is sent, the Metrics class will receive it through the IArchiver interface. The Posts field of the Mailing List model (as well as the the related rows on the Month and Year models) will increase by one. If the author’s email is not in the database, it will query the mailman core database with the email, grab the author’s id and a new Author row will be created. Otherwise if the author is already in the database, the Posts field and the two foreign fields (Month and Year) will increase by one - Generate class – When the ‘Generate’ button on the Admin page is pressed: * The Django models will be initialized (the metrics will go back to zero). A progress bar will inform the administrator that the operation is being processed. * All the messages of the archive will be parsed by performing a direct Python call to the IArchiver. Another instance to the IArchiver will grab any mails sent while the parsing is going on. * The metrics will be generated from scratch. * The administrator will be informed with a success message when the process is over. [1]: http://sophron.latthi.com/gsoc-mailman/ -- George Chatzisofroniou sophron.latthi.com ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
On Sat, May 5, 2012 at 11:26 PM, Barry Warsaw ba...@list.org wrote: Do you know exactly what kind of information you need? With my current implementation plan i only need author and date of the message to export my metrics. The suggestion to use the IArchiver interface is convenient, but might not tell you much other than who posted what to which list on what date. It won't tell you things like the number of recipients, how long that message took to deliver, if there were any failures, etc. Some of the things you mentioned sound pretty interesting and it would great to be on the metrics. It would certainly be possible to hook in a zope.event notification with those metrics for each successfully posted message. From there, a plugin could register a subscriber that put the event data on a message bus. Or you could grep the logs. :) Hooking the zope.event notification sounds good. Anyway, I think it would be useful to improve the support for this in the core. I'd like that! I'll come up with another more-detailed report this week. Thanks Barry! -- George Chatzisofroniou sophron.latthi.com ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Terri Oda writes: So... If AJAX is the fastest way to get some initial prototypes going, that's a good place to start. As the author of the original suggestion about AJAXing the charts, I don't actually think it is the easiest way. I think it's easier to just generate a fixed-size chart and slam it in as an IMG, and generally good enough for the purpose. The point of the suggestion was the coolness (IMHO) and robustness (to window resizes and the like) of charts that expand to fit the width of the window by adding data for more items (users, Months, whatever), *without* asking the user to fill in a form to say how wide, just resize the browser window. It should be possible to do this with CSS or similar, but then you'd need to generate a chart as wide as you think anyone could want and serve that to everybody. If you and George disagree with me on the coolness factor, great! I'll be looking forward to some beautiful thigns I didn't imagine! ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Am 05.05.12 17:21, schrieb Stephen J. Turnbull: Terri Oda writes: So... If AJAX is the fastest way to get some initial prototypes going, that's a good place to start. As the author of the original suggestion about AJAXing the charts, I don't actually think it is the easiest way. I think it's easier to just generate a fixed-size chart and slam it in as an IMG, and generally good enough for the purpose. Creating JS-/AJAX-ified graphs doesn't have to be hard. There are some *really* awesome libraries out there (such as flot[1], Raphael[2] or d3[3]) that make stuff like this pretty easy. (Those three are all MIT licensed so I guess there should be no collision with the GPL?). But I would consider carefully where to use or not to use actual AJAX (meaning: asynchronous server requests). If there is a lot of data or if the data depends on on-the-fly user inputs (like drill-down charts) asynchronous requests are very useful. But charts can be fed from many sources and AJAX is not always the best choice. For example, the JavaScript application could read the data from a JSON string that is delivered within the page content. Or it could use HTML5 data- attributes. Or it could DOM-traverse an HTML table and build the chart from its contents (that one could also make a nice bare-bones fallback for non-JS browsers). As for Geoff's non-JS request: I agree that it's perfectly reasonable to ask for a non-JS version. But first let's let George be creative and have some fun. If he comes up with a visualization that gives us insights we would not get from a simple img or html table I surely wouldn't want to miss that for the sake of general JS-freeness and 100% accessibility. There's a reason modern data visualizations don't look like good ol' analog[4] stats anymore... :-) That said, there are a lot of ways to create beautiful fallback versions for non-JS (and even text-based) browsers, and we should take care of that at *some* point for sure. Cheers Florian [1] http://code.google.com/p/flot/ [2] http://raphaeljs.com, also gRaphael for analytics: http://g.raphaeljs.com [3] https://github.com/mbostock/d3/wiki/Gallery [4] http://www.analog.cx ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
On May 03, 2012, at 02:04 AM, George Chatzisofroniou wrote: Interface to the MM core The app needs to interact with the Mailman core. I think the best idea is to implement a message bus that will send a notification every time a message is sent (the same way an archiver works). Based on this notification -which will carry the information about the sent message- the app will be ready to update the counts. Although, in some cases (eg the app is installed after an existing archive or there was an unexpected crash), the message bus should deliver (triggered by a button) more than one notifications (for the whole archive) to the app in order to initialize/recover the metrics. In those cases, the models will be initialized and the generation of the metrics will start over. Do you know exactly what kind of information you need? The suggestion to use the IArchiver interface is convenient, but might not tell you much other than who posted what to which list on what date. It won't tell you things like the number of recipients, how long that message took to deliver, if there were any failures, etc. The core does know this, and in fact logs it, but all you get at the IArchiver interface is the posted message. It would certainly be possible to hook in a zope.event notification with those metrics for each successfully posted message. From there, a plugin could register a subscriber that put the event data on a message bus. Or you could grep the logs. :) Anyway, I think it would be useful to improve the support for this in the core. Cheers, -Barry ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Richard Wackerbarth writes: I will take the blame for any misunderstanding in this area. There's no blame to be assigned, really, unless to me for being a busybody. :-) If George is making these plans while consulting his mentor, that's what this is all about (but he didn't say that, so I stuck my nose in!) And maybe I misinterpreted the word coding which, as I continue to explore the GSoC documentation, seems to be a catchall for GSoC work. It's just been my experience in supervising economics and business students' research (confirmed by a lot of the academic and practical literature on managing software development) that people facing a scheduling issue tend to skimp on planning, hoping to get lucky with a first draft and as a backup, planning to revise during implementation. This doesn't usually save time in the end, and often ends with a lower quality product. How George chooses to accomplish the planning, design, and coding, I'm happy to leave up to him (and you). I just wanted to warn against hoping that producing a plan and design more quickly than he otherwise would will save time. Time taken liberally in the early stages will conversely probably save time later. ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Steve, I am in complete agreement with your points about adequate planning. However, I rarely see large projects that do enough research to plan things properly from the start. Usually, a few people, who think that they understand the whole problem, make early design decisions that often become obstacles in the future. It is only after the prototype has been developed that others are able to point out weaknesses in the initial design. As such, I advocate for a planned revise during implementation to the extent that you schedule a reimplementation for a new generation of the product rather than continually attempting to add on to the previous design. Richard On May 4, 2012, at 4:36 AM, Stephen J. Turnbull wrote: Richard Wackerbarth writes: I will take the blame for any misunderstanding in this area. There's no blame to be assigned, really, unless to me for being a busybody. :-) If George is making these plans while consulting his mentor, that's what this is all about (but he didn't say that, so I stuck my nose in!) And maybe I misinterpreted the word coding which, as I continue to explore the GSoC documentation, seems to be a catchall for GSoC work. It's just been my experience in supervising economics and business students' research (confirmed by a lot of the academic and practical literature on managing software development) that people facing a scheduling issue tend to skimp on planning, hoping to get lucky with a first draft and as a backup, planning to revise during implementation. This doesn't usually save time in the end, and often ends with a lower quality product. How George chooses to accomplish the planning, design, and coding, I'm happy to leave up to him (and you). I just wanted to warn against hoping that producing a plan and design more quickly than he otherwise would will save time. Time taken liberally in the early stages will conversely probably save time later. ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Hello Stephen, On Thu, May 3, 2012 at 8:04 AM, George Chatzisofroniou soph...@latthi.com wrote: I’m also thinking to rearrange the GSoC schedule a bit. I’ll start writing code on the bonding period, It's up to your coding mentors, but it's generally not a good idea to try to move up the coding stage by too much. Make sure you have a clear spec before you start coding anything, and at least a rough sketch of a design. Without those two pieces, there's no standard to evaluate progress, or whether your code is doing the right thing. As wacky mentioned i'm not planning to skip any step of the process. The idea is to press myself a bit more now, so i can have some more time to study later. I should have explained myself better. so i can have more time during my university’s exams (starting on half of June). I'd say just take the time as needed, after negotiating with your mentors. Settings The Django app should handle some configuration parameters, like: - Maximum number of the subscribers of a mailing list the user wants to be shown in the charts I don't know how feasible it would be to implement, but if you're willing to use AJAX, you could simply build up bar and line charts on the fly, adding slices of say 5 users at a time on the right side of the chart in progress. That might be cool. It could be set in a viewport and scrolled if it gets too big for the viewport. Yes, AJAX can do the job here. I will implement the AJAX bar as you described, although i think it's better to primarily have a non-Javascript output. - Month – This model will store total posts and threads for each month - Year – Similar to month model, this one will store total posts and threads for each year It's not clear to me why year views can't be generated as an aggregate of monthly data? This would allow years to start with arbitrary months without too much redundancy. Generating the year views from monthly data is some more calculations while displaying the metrics. That's why they should be stored somewhere. Interface to the MM core The app needs to interact with the Mailman core. I think the best idea is to implement a message bus that will send a notification every time a message is sent (the same way an archiver works). Why not just use the iArchiver interface? I think it should be possible to have this be a standalone app serving the list itself, or be hosted by a particular archive, which wouldn't need to be the list host (or even run by the same organization). Yes, i agree. The iArchiver should be the origin of the interface. Thank you for your feedback, -- George Chatzisofroniou sophron.latthi.com ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Richard Wackerbarth writes: Usually, a few people, who think that they understand the whole problem, make early design decisions that often become obstacles in the future. It is only after the prototype has been developed that others are able to point out weaknesses in the initial design. I bow to your superior experience in the field (there are no large projects needed in mine! ;-) As such, I advocate for a planned revise during implementation to the extent that you schedule a reimplementation for a new generation of the product rather than continually attempting to add on to the previous design. I see. So the idea is that that's the stage that George is at anyway? He built one to throw away, and now it's time to progress to the reimplementation? (I forgot about that aspect; indeed, the fact that he's already built a project similar to this one should speed up the planning and design stage.) ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
George Chatzisofroniou writes: It's not clear to me why year views can't be generated as an aggregate of monthly data? This would allow years to start with arbitrary months without too much redundancy. Generating the year views from monthly data is some more calculations while displaying the metrics. That's why they should be stored somewhere. OK, but these are pretty cheap calculations if I understand Django's design correctly. Which design do you think is simpler, in the context of your overall plan? ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
On Fri, May 4, 2012 at 4:06 PM, Stephen J. Turnbull step...@xemacs.org wrote: OK, but these are pretty cheap calculations if I understand Django's design correctly. Yes, these are cheap calculations. Although, the number of calculations is based on the years of list's existence (e.g. a 10 year list would need 10 calculations). Which design do you think is simpler, in the context of your overall plan? I think it's simpler to just add a year model. I would like to hear more opinions on this issue. -- George Chatzisofroniou sophron.latthi.com ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
On Fri, 4 May 2012, George Chatzisofroniou wrote: On Thu, May 3, 2012 at 8:04 AM, George Chatzisofroniou soph...@latthi.com wrote: I don't know how feasible it would be to implement, but if you're willing to use AJAX, you could simply build up bar and line charts on the fly, adding slices of say 5 users at a time on the right side of the chart in progress. That might be cool. It could be set in a viewport and scrolled if it gets too big for the viewport. Yes, AJAX can do the job here. I will implement the AJAX bar as you described, although i think it's better to primarily have a non-Javascript output. I would just like to restate my plea for the ability to manage Mailman without needing javascript. Note that I'm not saying don't use any, I'm merely asking for it to be possible to use without it. I use lynx as my primary browser, and while general web surfing can be a bit tricky these days, it's very quick and useful for doing admin-type things, and I for one would like to be able to continue to do so. Certainly I have no problems using Mailman 2'.1's interface with lynx, and have been doing so for many years. If my memory serves me correctly, this was generally felt to be a reasonable request when I first mentioned it. I'm sure it's in the archives somewhere. Geoff. ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Hello Geoff, On Sat, May 5, 2012 at 12:17 AM, Geoff Shang ge...@quitelikely.com wrote: I would just like to restate my plea for the ability to manage Mailman without needing javascript. Note that I'm not saying don't use any, I'm merely asking for it to be possible to use without it. I use lynx as my primary browser, and while general web surfing can be a bit tricky these days, it's very quick and useful for doing admin-type things, and I for one would like to be able to continue to do so. Certainly I have no problems using Mailman 2'.1's interface with lynx, and have been doing so for many years. If my memory serves me correctly, this was generally felt to be a reasonable request when I first mentioned it. I'm sure it's in the archives somewhere. Yes, i also agree with you. That's why i mentioned a primarily non-Javascript output. -- George Chatzisofroniou sophron.latthi.com ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
On 12-05-04 3:17 PM, Geoff Shang wrote: I would just like to restate my plea for the ability to manage Mailman without needing javascript. Note that I'm not saying don't use any, I'm merely asking for it to be possible to use without it. This is totally a reasonable request, but I'd like to point out that what George is doing is not really on the management path. If people without JavaScript can't see his graph prototypes ... oh well. Just put up a note saying JavaScript is required for metrics. Not a big deal as long as it doesn't break other functionality. So... If AJAX is the fastest way to get some initial prototypes going, that's a good place to start. If you want to do something else, go for that! But with the compressed GSoC schedule, I want to make sure that George spends most of his time working on on the 90% of the project that's faster and fun and interesting, rather than spending inordinate amounts of time in that 10% that involves the harder finicky bits like testing in IE4. It's always a good idea to discuss this stuff and to consider backwards compatibility, but as an experienced GSoC mentor I've seen a few GSoC projects fall off the rails due to starting with the 10% and never reaching the 90%. There just isn't enough time in the 12 weeks. So as one of George's mentors I'm going to encourage him to focus on making beautiful things. No fiddly bits 'till after midterms at the earliest, and it is a-ok to just leave some of that stuff for the rest of the dev team to figure out later. Terri ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Hi George, Thanks for the report! A couple of quick comments: On Thu, May 3, 2012 at 8:04 AM, George Chatzisofroniou soph...@latthi.com wrote: I’m also thinking to rearrange the GSoC schedule a bit. I’ll start writing code on the bonding period, It's up to your coding mentors, but it's generally not a good idea to try to move up the coding stage by too much. Make sure you have a clear spec before you start coding anything, and at least a rough sketch of a design. Without those two pieces, there's no standard to evaluate progress, or whether your code is doing the right thing. so i can have more time during my university’s exams (starting on half of June). I'd say just take the time as needed, after negotiating with your mentors. Settings The Django app should handle some configuration parameters, like: - Maximum number of the subscribers of a mailing list the user wants to be shown in the charts I don't know how feasible it would be to implement, but if you're willing to use AJAX, you could simply build up bar and line charts on the fly, adding slices of say 5 users at a time on the right side of the chart in progress. That might be cool. It could be set in a viewport and scrolled if it gets too big for the viewport. - A multi-state variable (None, abbreviated, full) for masking email addresses at the results (we don’t want the emails to be spammed) This is pointless if the archives show addresses. So archives and list should coordinate with the chart module on this. I suppose this should be a query to the list config (and maybe per user configs). - Month – This model will store total posts and threads for each month - Year – Similar to month model, this one will store total posts and threads for each year It's not clear to me why year views can't be generated as an aggregate of monthly data? This would allow years to start with arbitrary months without too much redundancy. Interface to the MM core The app needs to interact with the Mailman core. I think the best idea is to implement a message bus that will send a notification every time a message is sent (the same way an archiver works). Why not just use the iArchiver interface? I think it should be possible to have this be a standalone app serving the list itself, or be hosted by a particular archive, which wouldn't need to be the list host (or even run by the same organization). ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] [GSoC 2012] Metrics
Steve, I will take the blame for any misunderstanding in this area. The GSoC calendar is based on the assumption that the students will have completed their spring academic work prior to the serious coding phase. Since conditions in Greece have delayed George's academic year, his spring final exams will now occur during the summer. I suggested that, in order to meet his mid-term goals in a timely fashion, he might expend some extra hours now, hopefully be ready to start coding a little early, and then be able to take a planned period of reduced effort during his finals. I am not, in any manner, suggesting that he skip any steps in the process. Richard On Thu, May 3, 2012 at 8:04 AM, George Chatzisofroniou soph...@latthi.com wrote: I’m also thinking to rearrange the GSoC schedule a bit. I’ll start writing code on the bonding period, It's up to your coding mentors, but it's generally not a good idea to try to move up the coding stage by too much. Make sure you have a clear spec before you start coding anything, and at least a rough sketch of a design. Without those two pieces, there's no standard to evaluate progress, or whether your code is doing the right thing. so i can have more time during my university’s exams (starting on half of June). I'd say just take the time as needed, after negotiating with your mentors. ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
[Mailman-Developers] [GSoC 2012] Metrics
Hello, I'm sending a report of my progress for the last two weeks (containing an outline of my project). The following report is also posted in my blog [1]. -- The last two weeks i’m into learning the Django web framework in more detail. I’m also thinking to rearrange the GSoC schedule a bit. I’ll start writing code on the bonding period, so i can have more time during my university’s exams (starting on half of June). Finally, i came up with an outline of my project. Here it is: Settings The Django app should handle some configuration parameters, like: - Maximum number of the subscribers of a mailing list the user wants to be shown in the charts - A multi-state variable (None, abbreviated, full) for masking email addresses at the results (we don’t want the emails to be spammed) Apart from these, some parameters to establish a connection to the message store (such as host or port) will be added. Models In order to store statistical data, the app will use some Django models: - Author – This model will store specific information for each subscriber such as total posts, total threads started, date of last message, subscription date, average of posts sent per day - MailingList – General information about the mailing list will be stored here such as total posts, total threads and two foreign keys pointing the month and year models - Month – This model will store total posts and threads for each month - Year – Similar to month model, this one will store total posts and threads for each year Views To display the metrics some views will be used: - A generic view for the mailing list and the top subscribers - A more detailed page about the participation of a specific subscriber The graphs will be generated using the PyChart library. (Custom tags are the way that the app will embed the graphs) Interface to the MM core The app needs to interact with the Mailman core. I think the best idea is to implement a message bus that will send a notification every time a message is sent (the same way an archiver works). Based on this notification -which will carry the information about the sent message- the app will be ready to update the counts. Although, in some cases (eg the app is installed after an existing archive or there was an unexpected crash), the message bus should deliver (triggered by a button) more than one notifications (for the whole archive) to the app in order to initialize/recover the metrics. In those cases, the models will be initialized and the generation of the metrics will start over. -- Any comments are highly appreciated. [1]: http://sophron.latthi.com/gsoc-mailman/?p=12 -- George Chatzisofroniou sophron.latthi.com ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
