Hi Sebastian,
Many thanks for your input. Your contribution is particularly important as Code24 is, as you say, one of the early pioneers of managing openEHR data and related artefacts. You have raised an important and very real issue, but I agree with Sebastian Garde that the .V0 question itself, is actually not what is going to potentially cause you a problem. Firstly I would defend what has been done in CKM to date as being completely in line with the specs and with the revisioning rules (which actually have worked extremely well for published archetypes). It has always been clear that any archetypes not marked as 'published' must be considered unstable and cannot be guaranteed to follow the numbering rules. I understand why you might take a view that 'anything that appears in CKM is de-facto published' but that is not, and never will be the case. CKM is first and foremost a clinical review environment which will always contain a mix of draft, published, re-draft and re-publshed archetypes. Having said all that, none of us are happy that it has take so long to get to a stage where we could get sufficient resource to properly fund editorial resource to take many archetypes through to publication though thankfully, through the Industry Sprint that point has now arrived. I am also familiar with and have sympathy for the issue you have raised which I will try to elucidate here, as I agree with Sebastian that it is a separate issue from V0. Your current situation is that you have used .v1 archetypes in your production system, even though they are marked as 'draft' in the metadata and by definition will have unstable structures leading to a situation where ongoing draft development and eventual publication of the draft v1 archetype may well break existing query paths in your patient data and in related code and querying statements. Although you can differentiate the draft from published archetypes in the archetypes themselves, there is no way that you can tell whether the instantiation of problem_diagnosis.v1 in patient data refers to the published version or to the previous draft version (and this may well include a breaking change). The problem essentially is not that the archetypes themselves are mis-labelled but that the archetypeId carried in data is not sufficient to disambiguate a draft archetype from a published archetype. This is a situation that was discussed on the lists a couple of years ago around the import of the NHHTA lab archetype where it was recognised that calling it lab_test.v1, been though it had many structural changes was going to cause problems for a number of vendors who had used the original draft lab_test.v1 in their applications. We asked the vendor community for their response and the message we got was that "by using draft archetypes we recognised that this might happen" and that we should keep the archetype at v1. I should add that this is not purely an issue for V0/V1 archetypes. When a published v1 archetype goes back into draft we will face the same issue, and for v2 and v3 etc. If vendors are going to use draft archetype in systems and in tooling (and they will) we need to be able to clearly identify unstable archetypes inside patient data and query statements, and not just in the archetypes. Vendors who do this need to understand that such an archetype is unstable and will break the rules but at least it is unambiguously identified in the patient data and any queries. This obviously comes back to the main discussion about the use of -unstable in the Semver proposals and I will try to explain our thinking in response to others who have made alternative suggestions. It turns out to be very,very tricky, and I don't think it is an accident that we have come back to Semver as the basis for a solution. So the issue you raised is real and significant, and will be faced by any vendor who is using a v1 draft archetype in an operational system. Adopting the new naming/numbering system will solve the problem going forward but the question remains about how to ameliorate the problem now There are, I think, three approaches: 1. Implementors accept that the use of draft archetypes was that 'own risk' and that they will have to manage the transition from draft to published, which will entail making changes to program code or query statements which refer to draft archetypes. i.e. rename lab_test.v1 to lab_test.v1-unstable or lab_test.v0. This is clearly a major pain (as Sebastian has said) but vendors we have spoken to so far have indicate that they are prepared to do this. Note that it will not have to be done as a batch, only when the system need to use each published v1 archetype, rather than the draft version. 2. Implementers rely on the new metadata that will be introduced when we publish archetypes from now on. This will include the original namespace and an optional Unique ID which can be used internally in the place of the current archetypeId. So the draft archetypeId reference in the data would be left unchanged as: openEHR-EHR-OBSERVATION.lab_test.v1 and any data captured using the published archetype would be labelled in the data (and any associated queries) as org.openehr::openEHR-EHR-OBSERVATION.lab_test.v1.0.0 or 123453446rhtytyyu.1.0.0 if you prefer When this published v1 archetype needs to go back into review it gets labelled as org.openehr::openEHR-EHR-OBSERVATION.lab_test.v1.0.1-unstable+build.e34dgtj67856 or using the uid - 123rhturytu55634567.v.1.0.1-unstable+build.e34dgtj67856 It is up to vendors to decide just how much of that mouthful to record in data and queries but if you are going to use unstable archetypes in data that is the only unequivocal method of knowing exactly which specific version of the archetype you are dealing with. I doubt that carrying the build is really necessary but knowing that this is v1.0.1-unstable is important, and resolves the problem that we all face right now. 3. The final option is that in CKM publication we bend the rules and go straight from V1 drafts to V2. We have discussed this as an editorial team and are not keen on the idea because it is a bit confusing for our clinical reviewer audience. If I was implementor I would go for option 2. At some point very soon I am going to have to get to grips with the need to identify namespaced and unstable archetypes in data. I don't have to upgrade to the new published archetypes right away, or as a bulk process but when I do so it makes sense at that point to be able to handle the kind of extended archetype naming that is proposed. Sorry that this was a bit of a tome. This ends up being a pretty complicated subject and part of the reason it took so long to figure out! We really need feedback. Everyone feel free to pitch in and argue Ian On 1 October 2014 14:35, Sebastian Iancu <sebastian at code24.nl> wrote: > Dear all, > > I'm surprise to see such a low analysis of the impact of changing v1 to v0 > of the existing CKM archetypes. > Even though they are not 'published', or are in logical 'draft' mode, they > were conformant to openEHR standards for at least past 5 years or so. Some > of them are used already in production environments for more than 2-3 years > (at least in our case). > Changing them now on CKM will break logical binding with already existing > production data. This cost has to be eventually supported by industry > implementers, and I can assure you this is not trivial, and it is giving > the impression that openEHR standard is not reliable/stable enough. > > Basically the proposal is to rename existing CKM archetypes (published or > not) in contradiction with what the current openEHR specifications is > stating right now, which is not right. > > Other than this, I personally think that is nice to have v0 for 'drafts', > and v1 for stable-published; it might be a better indication about a state > of an archetype. > So my suggestion is to keep existing CKM archetypes the way they are and > only apply the new 'rule' for archetypes created from now one. Published > archetypes from the sprint can become v2 (or 3, 4 etc) depends on the need > (see specifications). > > Sebastian > Code24 > > > > > > > On 10/1/2014 2:57 PM, Thomas Beale wrote: > > On 01/10/2014 13:31, Sebastian Garde wrote: > > > > > > It seems to me a no-brainer that we should support v0, because it's the > clearest possible sign I can think of, that everyone already recognises, > that indicates an artefact to be in early chaotic/uncontrolled development. > So v0 should be the start version for any archetypes created new on > someone's own machine, or any similar location. > > Supporting it as part of the specs for uncontrolled/chaotic development is > one thing and I agree. > But what we are looking at here is that all CKM archetypes would have a v0 > extension until they are first published (as v.1.0.0). > Existing pre-publication CKM archetypes would be converted to have a v0 > extension (either as a batch or one-by-one when each one is updated the > next time) > > > that sounds right to me - is it a problem? > > > > > > > - I think that the openEHR.org CKM archetypes identified for the > industry sprint should stay at v1.x, and others that are of unknown quality > should go to v0, which finally establishes what is clinically trustworthy > and what is not. > > That's not really an option unless you want to make this migration even > more difficult. > What you are suggesting requires two different sets of rules and someone > needing to deciding which stays at v1 and which is converted to v0. > I don't think it makes sense - either we use v0 to indicate > pre-publication archetypes or we don't. > > > I would have thought that individual archetypes can have their version > modified? I don't think the world minds if there is a period of a few > months during the industry sprint when the rules are technical being > broken. Once it's done, there will be 70+ archetypes with v1.x, and the > rest with v0.x, and that will correctly represent the situation of the > archetypes in CKM. > > Then for every mirroring, copy, and reuse of what's in openEHR.org CKM, > there is no need to educate anyone on anything - it's obvious what the > situation is. So we are only talking about a limited period of time where > the rules are being broken (as they are right now in fact ;-) > > > > - I don't see the problem with v0 references in templates, since it's > the same problem between any major version. An archetype xxxx.v0.x can't be > assumed to be compatible with xxxx.v1.y - as for other major versions, we > treat them technically as different archetypes. So either the template > reference is v* (any major version you like) or it isn't, in which case it > points to a known major version - v0, v1, v2 or other. > - we should assume that future tools will make it easy to change > these template references - it won't be difficult to do. > > Sure, looking forward to tooling support on it, but realistically at the > moment it is a pain that is not needed for the first publication if you go > with v1 as a the initial major version. > > > well I think its a bigger danger, given the level of reuse of CKM > archetypes, that the version ids wouldn't correct reflect the state of the > archetypes. We could in theory revert everything that is not published to > v0 right now, and i guess that is breaking the rules for less time. But > there are still some dozens of fully reviewed and published archetypes that > have to retain their v1.x version anyway. So I think the only question is > to do with the industry sprint archetypes. How about doing this: > > - mark archetypes that have never been 'published' and are not in the > industry sprint as v0.5.0 > - mark the industry sprint archetypes as v1.0.0-unstable > - leave currently published archetypes on v1.x, i.e. where they are > today. > > If I receive a copy of archetypes marked like that in say the GitHub > mirror, or through a different CKM, I don't need any further education, > it's obvious what's going on. > > - thomas > > > > _______________________________________________ > openEHR-clinical mailing listopenEHR-clinical at > lists.openehr.orghttp://lists.openehr.org/mailman/listinfo/openehr-clinical_lists.openehr.org > > > > _______________________________________________ > openEHR-technical mailing list > openEHR-technical at lists.openehr.org > > http://lists.openehr.org/mailman/listinfo/openehr-technical_lists.openehr.org > -- Dr Ian McNicoll office / fax +44(0)141 560 4657 mobile +44 (0)775 209 7859 skype ianmcnicoll ian at freshehr.com Clinical modelling consultant freshEHR Director openEHR Foundation Honorary Senior Research Associate, CHIME, UCL BCS Primary Health Care www.phcsg.org -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.openehr.org/pipermail/openehr-technical_lists.openehr.org/attachments/20141003/e7c7e94d/attachment-0001.html>
