Re: Analyze This!
> Date: Thu, 8 Nov 2007 14:23:09 -0500 > From: Christopher Schmidt <[EMAIL PROTECTED]> > BeautifulSoup is just one .py file -- it can be grabbed fro the website > and dropped into the directory where you're working with it, and then > you can do 'import BeautifulSoup'. (That's why I've never bothered to > package it.) Though it does appear that someone has packaged it for > Debian: http://packages.debian.org/python-beautifulsoup Yeah. One .py file... in a tar archive... gzipped... alongside other tarballs... in an ar archive... in a/directory/somewhere/way/under/nothing which is really just a symlink to one of a bazillion files in /pool. Yeah, I used to do that Debian thing too. :) ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: Analyze This!
Gonna do it in ruby anyway; I'd rather use hpricot. However, I'm pretty sure there's already a mechanism to generate the CSV; either way, shouldn't be a big deal. Take it easy, David Berube Berube Consulting [EMAIL PROTECTED] (603)-485-9622 http://www.berubeconsulting.com/ Christopher Schmidt wrote: > On Thu, Nov 08, 2007 at 02:16:55PM -0500, Ben Scott wrote: >> On Nov 8, 2007 7:19 AM, David J Berube <[EMAIL PROTECTED]> wrote: >>> Sure. I'm assuming there's no Ruby, so I'll have to write it in Perl or >>> Python or something. >> Thanks to the beauty of YUM, Ruby 1.8.1 is now installed on liberty. :) >> >> Python 2.3.4 is also present. It appears, however, that >> BeautifulSoup is not. Nor can I find it in the rpmforge repository. >> Nor on CentOS 5. You might have to do whatever Python's equivalent of >> a local build/install is for that. > > BeautifulSoup is just one .py file -- it can be grabbed fro the website > and dropped into the directory where you're working with it, and then > you can do 'import BeautifulSoup'. (That's why I've never bothered to > package it.) Though it does appear that someone has packaged it for > Debian: http://packages.debian.org/python-beautifulsoup > > > Regards, ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: Analyze This!
On Nov 8, 2007 7:19 AM, David J Berube <[EMAIL PROTECTED]> wrote: > Sure. I'm assuming there's no Ruby, so I'll have to write it in Perl or > Python or something. Thanks to the beauty of YUM, Ruby 1.8.1 is now installed on liberty. :) Python 2.3.4 is also present. It appears, however, that BeautifulSoup is not. Nor can I find it in the rpmforge repository. Nor on CentOS 5. You might have to do whatever Python's equivalent of a local build/install is for that. > Not sure what output format is good; probably straight HTML, unless I > can find an easy way to post it to the TWiki. I'd go with HTML or even plain text. Like I said, the plan is for TWiki to go away, so that's a bad choice (MediaWiki uses a different syntax for tables). You can place your output somewhere under /sites/gnhlug/main/static/ (root for http://www.gnhlug.org/) and we can link to it from anything. -- Ben ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: Analyze This!
On Thu, Nov 08, 2007 at 02:16:55PM -0500, Ben Scott wrote: > On Nov 8, 2007 7:19 AM, David J Berube <[EMAIL PROTECTED]> wrote: > > Sure. I'm assuming there's no Ruby, so I'll have to write it in Perl or > > Python or something. > > Thanks to the beauty of YUM, Ruby 1.8.1 is now installed on liberty. :) > > Python 2.3.4 is also present. It appears, however, that > BeautifulSoup is not. Nor can I find it in the rpmforge repository. > Nor on CentOS 5. You might have to do whatever Python's equivalent of > a local build/install is for that. BeautifulSoup is just one .py file -- it can be grabbed fro the website and dropped into the directory where you're working with it, and then you can do 'import BeautifulSoup'. (That's why I've never bothered to package it.) Though it does appear that someone has packaged it for Debian: http://packages.debian.org/python-beautifulsoup Regards, -- Christopher Schmidt Web Developer ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: Analyze This!
On Nov 8, 2007 3:58 PM, <[EMAIL PROTECTED]> wrote: > Yeah, I used to do that Debian thing too. :) Please keep the distro wars off this list. They're off-topic. (We actually have a topic charter for *this* list, so we can say that.) http://mail.gnhlug.org/mailman/listinfo/gnhlug-org -- Ben ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: Analyze This!
Ben Scott wrote: > > Python 2.3.4 is also present. It appears, however, that > BeautifulSoup is not. Nor can I find it in the rpmforge repository. My FC6 distro let me install it with yum install python-BeautifulSoup, funky capitalization and all... -- Ted Roche Ted Roche & Associates, LLC http://www.tedroche.com ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: Analyze This!
On Nov 9, 2007 1:17 PM, Ted Roche <[EMAIL PROTECTED]> wrote: > My FC6 distro let me install it with yum install python-BeautifulSoup, > funky capitalization and all... Interesting. I see that, too, in "extras". The package does not appear to exist in CentOS (RHEL), nor the rpmforge repository for same. Fortunately, this doesn't matter: BeautifulSoup is apparently a trivial install, and David's using Ruby anyway. :-) -- Ben ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: Analyze This!
> Date: Fri, 9 Nov 2007 12:44:55 -0500 > From: "Ben Scott" <[EMAIL PROTECTED]> > Please keep the distro wars off this list. They're off-topic. (We > actually have a topic charter for *this* list, so we can say that.) You're right. I did not notice that this was on the -org list. I'm gonna have to start paying more attention to my headers. :/ ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: Analyze This!
Ben Scott wrote: > Python 2.3.4 is also present. It appears, however, that > BeautifulSoup is not. Nor can I find it in the rpmforge repository. > Nor on CentOS 5. You might have to do whatever Python's equivalent of > a local build/install is for that. Yet another reason to upgrade the OS! (No, not really.) For those following along at home, the GNHLUG PastEvents2007 page [1] has been updated with events through the end of October, and the CSV file refreshed to also contain that data. GNHLUG-Nashua is still in the lead, but PySIG is nipping at its heels! -- Ted Roche Ted Roche & Associates, LLC http://www.tedroche.com ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: Analyze This! [Installing BS on CentOS]
> Interesting. I see that, too, in "extras". The package does not > appear to exist in CentOS (RHEL), nor the rpmforge repository for > same. Fortunately, this doesn't matter: BeautifulSoup is apparently a > trivial install, and David's using Ruby anyway. :-) For the record, there are two easy install methods on an RPM-based distro that doesn't have it in a repository. First get the tarball 'version' of BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/download/BeautifulSoup.tar.gz Untar it, cd into the directory. You can now use the generic python setuptools method of install (similar to Perl's 'perl Makefile.PL ; make ; make install'): python setup.py install There's an even "cooler" way. (This will not work with some more complicated python module unless the author has specifically provided a manifest file for this purpose...but it works with BS) python setup.py bdist_rpm You will then have a "dist" directory with the tarball, an rpm, and an srpm:[EMAIL PROTECTED] BeautifulSoup-3.0.4]$ ls -l dist/ total 124 -rw-rw-r-- 1 shawn shawn 60213 Nov 9 13:40 BeautifulSoup-3.0.4-1.noarch.rpm -rw-rw-r-- 1 shawn shawn 28680 Nov 9 13:40 BeautifulSoup-3.0.4-1.src.rpm -rw-rw-r-- 1 shawn shawn 26131 Nov 9 13:40 BeautifulSoup-3.0.4.tar.gz Install RPM and enjoy BeautifulSoup -Shawn ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
[Fwd: Re: Analyze This!, was: Re: Attendance: A business-like approach]
David J. Berube did a little analysis and here's what he got (with his permission, forwarding to the group) +---+--+--++ | location | meetings | total_attendance | average_attendance | +---+--+--++ | MerriLUG |8 | 207 | 25.8750| | PySIG |7 | 100 | 14.2857| | DLSLUG|7 | 69 | 9.8571 | | CentraLUG |8 | 57 | 7.1250 | | GNHLUG|5 | 57 | 11.4000| | MonadLUG |8 | 56 | 7. | | SLUG |5 | 48 | 9.6000 | | RubySIG |5 | 32 | 6.4000 | +---+--+--++ +-+--+--++ | speaker | meetings | total_attendance | average_attendance | +-+--+--++ | Jarod Wilson|1 | 61 | 61.| | Dave Rowell |1 | 23 | 23.| | Christoph Doerb |2 | 43 | 21.5000| | Máirín Duffy|1 | 21 | 21.| | Everybody! |1 | 20 | 20.| | Todd Underwood |2 | 39 | 19.5000| | LloydKvam |2 | 37 | 18.5000| | MattBrodeur |2 | 30 | 15.| | Alex Hewitt... |1 | 14 | 14.| | KentSJohnson... |1 | 13 | 13.| | William Henders |1 | 13 | 13.| | Ric Werme |1 | 13 | 13.| | GuyPardoe |1 | 13 | 13.| | MadDog... |1 | 13 | 13.| | TedRoche|1 | 13 | 13.| | Bill Stearns|2 | 25 | 12.5000| | BillSconce |1 | 12 | 12.| | PySIG group |1 | 12 | 12.| | SethCohn|2 | 24 | 12.| | Roger Trussell |1 | 10 | 10.| | MadDog ... |1 | 10 | 10.| | Rob Anderson|2 | 20 | 10.| | RobertAnderson |2 | 18 | 9. | | Brian DeLacey |1 | 9| 9. | | Group Discussio |1 | 9| 9. | | BenScott|2 | 17 | 8.5000 | | Ed Haynes...|1 | 8| 8. | | Members |4 | 32 | 8. | | The Community |1 | 7| 7. | | Roger K. Trusse |1 | 7| 7. | | Public |1 | 7| 7. | | Group |1 | 7| 7. | | AndyBair|1 | 7| 7. | | Nick Plante, Sc |1 | 7| 7. | | ScottGarman |1 | 6| 6. | | Nick Plante and |1 | 6| 6. | | Happy Holidays |1 | 0| 0. | | Nobody |3 | 0| 0. | | Canceled|1 | 0| 0. | +-+--+--++ David Berube Berube Consulting [EMAIL PROTECTED] (603)-485-9622 http://www.berubeconsulting.com/ ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: [Fwd: Re: Analyze This!, was: Re: Attendance: A business-like approach]
*shrug*. That's based on the data on the TWiki; I doubt all of the meetings are represented, and doubtless some of the data is inaccurate. As far as 2008, that's in the future, which has historically been more difficult to report on. Take it easy, David Berube Berube Consulting [EMAIL PROTECTED] (603)-485-9622 http://www.berubeconsulting.com/ [EMAIL PROTECTED] wrote: >> Date: Mon, 05 Nov 2007 09:46:54 -0500 >> From: Ted Roche <[EMAIL PROTECTED]> > >> +-+--+--++ >> | speaker | meetings | total_attendance | average_attendance | >> +-+--+--++ > > > >> | Nobody |3 | 0| 0. | > > I don't think these statistics are accurate. I know I had more than 0 > people at the presentation which I gave in 2007. Are these stats only > for 2008? > ___ > gnhlug-org mailing list > gnhlug-org@mail.gnhlug.org > http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/ > > ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: [Fwd: Re: Analyze This!, was: Re: Attendance: A business-like approach]
On Monday 05 November 2007 09:46, Ted Roche wrote: > David J. Berube did a little analysis and here's what he got (with > his permission, forwarding to the group) Nice job, David Berube! An additional dimension to map is the attendance against some measures of the topic. This might be done by assigning a 1 to 5 (or 0 to 4) score to each topic in each measure, 1 (0) being no relevance. Then sort the topic table by score to see how strong the correlation is in the attendance column. (A mathematical, true correlation is an optional exercise.) It is expected that some topics will score high (or low) in several measures. Some measures to consider: 1) useful for professional computer systems work 2) useful for home computer enthusiast 3) involved some audience participation or hardware demo 4) hardware: installing, building, troubleshooting 5) software: language, programming, diagnostics, web 6) system security, privacy 7) user oriented application: graphics, word processing, file processing 8) Linux at the command line level These are just some seeds to get thinking started; better measures may come to mind as the topics are examined more closely. Note also, that some presentations must have been missed in the reports. (Actually, it is mostly thanks to Ted that there are so many attendance reports.) I know, for example that Andy Bair gave more than one presentation, and the one I attended had 12 people or so, as I recall. Also, Mad Dog = MadDog and Rob Anderson = RobertAnderson. Nevertheless, three cheers for the Python crew. Jim Kuzdrall ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: [Fwd: Re: Analyze This!, was: Re: Attendance: A business-like approach]
> Date: Mon, 05 Nov 2007 09:46:54 -0500 > From: Ted Roche <[EMAIL PROTECTED]> > +-+--+--++ > | speaker | meetings | total_attendance | average_attendance | > +-+--+--++ > | Nobody |3 | 0| 0. | I don't think these statistics are accurate. I know I had more than 0 people at the presentation which I gave in 2007. Are these stats only for 2008? ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: [Fwd: Re: Analyze This!, was: Re: Attendance: A business-like approach]
> Date: Mon, 05 Nov 2007 17:33:18 -0500 > From: David J Berube <[EMAIL PROTECTED]> > > I don't think these statistics are accurate. I know I had more than 0 > > people at the presentation which I gave in 2007. Are these stats only > > for 2008? > As far as 2008, that's in the future, which has historically been more > difficult to report on. LOL. Indeed, you're right! s/2007/2006/ s/2008/2007/. I must have forgotten to set my clocks back a year when we switched to standard time yesterday morning. :) ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: [Fwd: Re: Analyze This!, was: Re: Attendance: A business-like approach]
Jim Kuzdrall wrote: > Note also, that some presentations must have been missed in the > reports. (Actually, it is mostly thanks to Ted that there are so many > attendance reports.) I know, for example that Andy Bair gave more than > one presentation, and the one I attended had 12 people or so, as I > recall. Also, Mad Dog = MadDog and Rob Anderson = RobertAnderson. Jim: The TWiki is read-write. If there are mistakes or clarifications you can make, please hit the "Edit" link and try your best to fix it. If you're uncomfortable with that, please let me or Ben Scott know, and we'll be glad to try to make the change ourselves. However, you can save us some time but giving it a shot yourself. The editing lets you preview the changes you've made and abandon them if you're not happy with them; it also automatically makes version-controlled backups so we can always restore something if a page is accidentally wiped. You can't break anything. When we do some data cleanup, we can run the scripts again, upload better CSV data files and get better analyses from them. > Nevertheless, three cheers for the Python crew. hear, hear!!! I was impressed with the process, and the results. -- Ted Roche Ted Roche & Associates, LLC http://www.tedroche.com ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: [Fwd: Re: Analyze This!, was: Re: Attendance: A business-like approach]
SQL. I edited the CSV with VIM a bit, LOAD DATA INFILE'd it into MySQL, and then dropped a few SQL statements, and copy/pasted them into VIM again. David Berube Berube Consulting [EMAIL PROTECTED] (603)-485-9622 http://www.berubeconsulting.com/ Jim Kuzdrall wrote: > On Wednesday 07 November 2007 11:32, Ted Roche wrote: > >> The TWiki is read-write. If there are mistakes or clarifications you >> can make, please hit the "Edit" link and try your best to fix it. > > I would have given the Edit feature a try, but my "data" was too > inexact to justify an edit. It was just my recollection of the > evening, not an actual head count. > > It is not clear to me what insight can be gained from just the raw > data table, but I am curious as to what an analysis by topic might > yield. I will try to give that a try as time permits. > > If David Berube did his analysis and chart in some standard language > and would be willing to share the program, it could be run again as new > data gets added. > > Would it be possible, in the new Wiki, to add a column in the > "Resent Events" table for the attendance? Even better, could that > table get at least some of its entries from a MySQL data base? Then > analysis for this or for different purposes come be easily rerun from > time to time. > > (This is not intended to be a new thread.) > > Jim Kuzdrall > ___ > gnhlug-org mailing list > gnhlug-org@mail.gnhlug.org > http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/ > > ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: [Fwd: Re: Analyze This!, was: Re: Attendance: A business-like approach]
On 11/7/07, Jim Kuzdrall <[EMAIL PROTECTED]> wrote: > Would it be possible, in the new Wiki, to add a column in the > "Resent Events" table for the attendance? ... There's one in the present table. Titled "How many". At least one of us is confused. :) > Even better, could that table get at least some of its entries from a > MySQL data base? If someone wants to write a web front-end to said database, and also write a program to generate hypertext and/or wiki markup from same, then sure. :) Right now, the tables are just wiki tables maintained by hand. Somewhat inelegant, but has the benefit of something we have now. :) -- Ben ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: [Fwd: Re: Analyze This!, was: Re: Attendance: A business-like approach]
how about a program that pulls from the CSV file and does the analysis on the fly? Take it easy, David Berube Berube Consulting [EMAIL PROTECTED] (603)-485-9622 http://www.berubeconsulting.com/ Ben Scott wrote: > On 11/7/07, Jim Kuzdrall <[EMAIL PROTECTED]> wrote: >> Would it be possible, in the new Wiki, to add a column in the >> "Resent Events" table for the attendance? > >... There's one in the present table. Titled "How many". At least > one of us is confused. :) > >> Even better, could that table get at least some of its entries from a >> MySQL data base? > > If someone wants to write a web front-end to said database, and also > write a program to generate hypertext and/or wiki markup from same, > then sure. :) Right now, the tables are just wiki tables maintained > by hand. Somewhat inelegant, but has the benefit of something we have > now. :) > > -- Ben > ___ > gnhlug-org mailing list > gnhlug-org@mail.gnhlug.org > http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/ > > ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: [Fwd: Re: Analyze This!, was: Re: Attendance: A business-like approach]
On 11/7/07, David J Berube <[EMAIL PROTECTED]> wrote: > how about a program that pulls from the CSV file and does the analysis > on the fly? In other words, automate the process of scraping the HTML to CSV, and then automate the analysis of the CSV, and automatically post the results of that somewhere? Sounds like it might be doable with some crontab entries, given what we have now. Want a shell account on liberty (the server)? -- Ben ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: [Fwd: Re: Analyze This!, was: Re: Attendance: A business-like approach]
On 11/7/07, Ted Roche <[EMAIL PROTECTED]> wrote: > When we do some data cleanup, we can run the scripts again, upload > better CSV data files and get better analyses from them. If anyone is interested, we also have the ability to install software, run server-side databases and scripts, etc. So if you've got the time, talent, and inclination, we've got the technology to support it. That said, the general plan is to migrate to MediaWiki as the wiki software Real Soon Now, so I wouldn't invest too much in anything dependent on our existing software. -- Ben ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: [Fwd: Re: Analyze This!, was: Re: Attendance: A business-like approach]
On Wednesday 07 November 2007 11:32, Ted Roche wrote: > The TWiki is read-write. If there are mistakes or clarifications you > can make, please hit the "Edit" link and try your best to fix it. I would have given the Edit feature a try, but my "data" was too inexact to justify an edit. It was just my recollection of the evening, not an actual head count. It is not clear to me what insight can be gained from just the raw data table, but I am curious as to what an analysis by topic might yield. I will try to give that a try as time permits. If David Berube did his analysis and chart in some standard language and would be willing to share the program, it could be run again as new data gets added. Would it be possible, in the new Wiki, to add a column in the "Resent Events" table for the attendance? Even better, could that table get at least some of its entries from a MySQL data base? Then analysis for this or for different purposes come be easily rerun from time to time. (This is not intended to be a new thread.) Jim Kuzdrall ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
Re: [Fwd: Re: Analyze This!, was: Re: Attendance: A business-like approach]
Hi Ben, Sure. I'm assuming there's no Ruby, so I'll have to write it in Perl or Python or something. Not sure what output format is good; probably straight HTML, unless I can find an easy way to post it to the TWiki. Take it easy, David Berube Berube Consulting [EMAIL PROTECTED] (603)-485-9622 http://www.berubeconsulting.com/ Ben Scott wrote: > On 11/7/07, David J Berube <[EMAIL PROTECTED]> wrote: >> how about a program that pulls from the CSV file and does the analysis >> on the fly? > > In other words, automate the process of scraping the HTML to CSV, > and then automate the analysis of the CSV, and automatically post the > results of that somewhere? Sounds like it might be doable with some > crontab entries, given what we have now. > > Want a shell account on liberty (the server)? > > -- Ben > ___ > gnhlug-org mailing list > gnhlug-org@mail.gnhlug.org > http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/ > > ___ gnhlug-org mailing list gnhlug-org@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-org/
