Hi Tim, thanks for your interest in investigating collaborating with us. --- In openhealth@yahoogroups.com, Tim Churches <[EMAIL PROTECTED]> wrote: > NetEpi Analysis was designed to deal with the types of data and analyses > which you mention - for example, apart from supporting complex > cross-tabs (with good support for proportions), basic contingency table > analysis and standard statistical charts (again with confidence limits), > it also does direct and indirectly age-standardised population- based > rates (with confidence intervals), and we hope to shortly add support > for log-linear (Poisson and negative-binomial) models for counts/rates, > as well as possibly Bayesian smoothing of rates. And it works quickly > enough for interactive, real-time analysis even with many millions of > records, and the underlying architecture deals well with the sort of > multi-level data you describe eg it supports multi-valued records. And > it has full support for missing data handling. Oh, and it also does > Google-style indexed searching or free text fields. However, a major > downside is that it only accepts batch loads of data at the moment, > without incremental updates (although many data sources can be loaded > into one dataset, but they all have to be loaded in one go). That's > something we also plan to fix as soon as possible, but unless you have > many tens millions of records , periodic batch loading works OK (and is > simpler to set up and maintain than incremental updates, which can be > surprisingly tricky and fragile). But it does support dataset > versioning, so that the latest version of source data can be loaded into > a new dataset in the background while users continue to use an existing > dataset, and when the new load has completely it is transparently used > for all new analysis sessions. Of course, there are still plenty of > rough edges and gaps, but it might be a good start, or a partial > solution.
Thanks for this overview. There are so many layers to this whole data analysis aspect of medical record repositories. Thinking from left to right, there's the whole set of challenges around converting "stacked" database models (where there's one row per clinical observation) to the typical "flat" models, where the columns of a table are the observations, and each row is some higher level of measurement (ie, encounter, patient, etc). We're working quite hard on tools to go from our stacked OpenMRS data repository to flat data exports for analysis purposes. To generate the rows, we're developing a cohort builder which allows users to interactively look at patient/encounter counts for given clinical measurements, and in an OVID style, provide an interface to and/or them together to arrive at your final cohort. If you'd like to see this in a demonstration, let me know and I can pass some instructions. To generate the columns, we're developing a tool to select the observations of interest, and apply aggregations and/or rules to them (IE, last 4 hemoglobins where they occured in the past year). Once you have the data exported, there's the whole issue of what interfaces do you support to various analysis software packages. I think the OpenMRS group is fully in support of letting a thousand flowers bloom. As we @ Regenstrief are directly serving a large set of clinical sites in Western Kenya, we have to have a solution for our own backyard, and so the one we're investigating at this point is BIRT. That being said, we'd love to have a variety of choices, and perhaps NetEpi Analysis could be another one. We'd gladly advertise and document ways in which these softwares could work together. >Fairly easy to write hard-coded interfaces to OpenMRS, I > suspect, either in PHP or Python or both, although a generic interface > would be harder, but possible - maybe a few weeks work. It can certainly > load data directly from a database back-end, although some queries to > flatten the data appropriately might be needed. Further processing and > data transformation can be done in Python as the data are loaded. Tim, perhaps I don't understand the scope of your software, but does NetEpi analysis generate OLAP cubes or views on the fly as well as provide the interface for report generation? Oh, > you can use it via the Web front-end, no programming, or via its own > Python API, which is simple enough for interactive command-line analysis > as well as for calling from other things. I'd like to sick Justin Miranda, the developer currently leading up our reporting efforts, on top of what your team has done. Would you mind me passing some contact information along to him? > Yes. Actually, we don't support messaging in NetEpi, yet, but no reason > why it can't. No, that's wrong, one of our application, for real- time > ED-based public health surveillance, does use HL7 messaging and the data > is fed into NetEpi Analysis. We do have some very insistent calls for > interoperabilty, but they are for interoperabilty with paper-based data > collection forms: people want to be able to scan in and optical mark > read or OCR hand-written paper forms. I think OpenMRS uses SharePoint or > perhaps Adobe Acrobat forms for this? Do they support scanning? It's interesting that you bring this up. Prior to my work on OpenMRS, I led up an effort to build a decision support system, that actually uses paper as it's transaction device. We built a system with Arden Syntax as it's rule base, and integrated that framework with Cardiff Teleforms to generate what I call "Adaptive Turnaround Documents". The system that implements this technology is called CHICA (Child Health Improvement through Computer Automation). It just so happens that b/c I lead up technical development of both these projects, I'm in a good position to merge them together. So in fact, we're actively working on merging the work we've done on CHICA to live on top of OpenMRS. Stay tuned, likely within the next 12 or so months, we'll have full Adaptive Turnaround Document technology on top of OpenMRS. Once again, I'm happy to share more information with you if you'd like (I've written multiple papers on OCR technology for AMIA symposiums in the past). My guess is that there will be a lot of interest in such a feature set within the developing world as well. That's > what our users want. Naturally we are not very enthusiastic about > providing such facilities, not the least because there are few open > source optical character recognition packages which can be readily used > (yes, IBM released one a while ago, but it takes a lot to use it), and > also, form scanning just doesn't work as well in practice as people > think it should, I've found in the past. Perhaps it works better these days. Hah. Well, I'd challenge anyone to find a more fully featured suite than Teleforms. It's relatively expensive, but we've worked really hard at compartmentalizing what we "expect" of Teleforms, such that if an open source competitor came along that provided similar functionalities, that we could easily make that switch. Once again, let a thousand flowers bloom. No reason why OpenMRS couldn't have a Teleforms module and the "super FOSS OCR" module when that becomes available. :) Best, -Paul
