Code4Lib 2009 -- February 25, 2009 Draft notes: Breakout Session: Evaluating Open Source
Input welcome from attendees and anyone else. ---------------- This breakout session started from an informal discussion at a C4L wine and cheese. The group brainstormed questions about questions to ask for evaluating open source. These questions are not absolutes and some of them (perhaps many of them) will elude clearly objective answers. Also, again and again, the group pointed out that the presence of a question did not translate to a requirement or a judgment — these are assessment questions, many of which will not be relevant to every project and will only translate to meaningful criteria on a selective basis. Additionally, while many of these same questions could be applied to any software, the consensus appeared to be that it was helpful to ask these questions specifically in the context of OSS. 1. “Openness” of open source a. Describe the license(s) used b. Is the code freely and publicly available? Is it easy to find? 2. Growth and growth management a. How widely is the code used? i. How many organizations are known to use it operationally ii. How many times has it been downloaded iii. Is usage information tracked and reported? b. How long has it been in use? c. How many developers are actively involved in the project? d. What is the commit activity? i. How are commits reported? ii. Can commits be tracked in real-time? How? e. Describe the enhancement process. i. Are enhancement decisions publicly available? Who decides? f. Describe bug-tracking: what tools, how bugs are evaluated and prioritized i. Is the bug-tracking system publicly available? g. Describe QA/testing processes. h. How is the software updated? i. Is there a migration path to the next version? j. Describe the development planning model. Is there upgrade planning? A commitment to a migration path? k. What tools are provided for migrations and upgrades? l. Has the project forked, and if so, describe 3. Community engagement… a. Are there user groups? How large are they? How often do they meet (f2f, virtually, etc.)? b. Discussion groups, chat channels, etc.—presence, traffic, availability c. Activity of support forum, length of support d. Other characteristics of the software community: size, diversity 4. Governance a. Describe the governance model (nonprofit, foundation, etc.) b. Is the governance transparent? Describe. 5. Code and standards a. Describe the architecture—languages, structure, etc. b. Is the project using version control? c. How available is the version control system? d. Is there a commercial support option? e. Interoperability—describe. f. Error logging and reporting—describe g. Scalability? h. Security? Encryption? i. Does it provide security auditing tools? i. How are permissions set and what are the default permissions? j. What platforms does it run on, and how easy is it to implement on each platform? k. For dependencies, does it rely on current versions of those programs? l. Does the code hew to de facto or de jure standards? Which ones? m. Are key developers active in related standards work? n. Does the code include proprietary-source codex, flash players, etc.—and how is that handled? 6. Documentation a. Is it complete? b. Current? c. Open? d. Written to standards (e.g. Docbook or DITA)? 7. Innovation and quality a. Is it cool at what it does? Is it useful? What’s its karma? Does it work well? Does it solve a problem? that needs to be solved? b. Is it easy to use? c. Is it focused on end users (including librarians, if they are the software’s end users)? d. Ease of installation? Consistent results? e. Accessibility? f. Internationalization? g. Business intelligence functions? h. Incompatibilities? i. Failures and deficiencies? j. Awards, reviews, citations? k. Certifications? -- -- | Karen G. Schneider | Community Librarian | Equinox Software Inc. "The Evergreen Experts" | Toll-free: 1.877.Open.ILS (1.877.673.6457) x712 | k...@esilibrary.com | Web: http://www.esilibrary.com | Be a part of the Evergreen International Conference, May 20-22, 2009! | http://www.solinet.net/evergreen