James Lawson wrote: > David Nickerson wrote: >> OK - now I'm really confused by all this talk of stars and simulation >> tools. Perhaps the key is to separate the model curation status from the >> simulation tools - i.e., to have a distinct "Curation Level" field at >> the top of the model page separate to the current Download options >> section. I think it is very import to ensure there is a clear >> distinction between model curation status and how good or bad particular >> simulation tools are in regard to a specific model. >> >> I'm not sure that we want to get rid of the multiple stars for >> simulation tools, as defined by >> http://www.cellml.org/repository-info/info. If someone comes along and >> wants to look into using a particular "uncurated" model, it would be a >> good starting point to know which tools can or cannot at least load the >> model even if it doesn't give the right answers. >> > > Sure, but you only need a binary system to describe that - it either > loads or it doesn't, one star, either there or not. More stars means > confusion.
but there is a big difference between it loading and it running...and then between it running and it giving the right answers. There are issues to do with units consistency, numerical integration, code generation, etc...that mean the model may not run at all or may give wildly incorrect results. This is why we came up with the confidence levels for simulation tools as described at http://www.cellml.org/repository-info/info >> I'm a bit worried that you seem to be saying that by default all models >> will get one star and then that star has to be manually removed? seems >> the wrong way round to me. > > Hmm, I'd be worried if I was saying that too :) If that's how it came > across I didn't mean that. Let me put it this way "the three *possible* > stars for each simulator will go, and be replaced by one *possible* > star." Is that clearer? its more whether that one star is on or off by default? > Okay, the idea is that: > there are two sets of stars. One for curation, one for the simulator. > For curation, there are three possible stars, as outlined here. For the > simulator, there is one possible star, which is given if the model loads > in that simulator. ok - its getting clearer :-) I still think there is value in keeping the current three levels of confidence for simulation tools. >>> Currently, I think the biggest issue is with getting models to produce >>> the output that the original model produced. I think sometimes the best >>> we'll be able to do is ensure that the model is mathematically >>> equivalent to what is described by the authors. >> Sure, and then the model sits at level 1 curation until someone comes >> along to fix it up. There is nothing wrong with that. > > apart from people coming along and thinking that because the models only > get 1/3 they aren't curated well. If we're having problems with our > system how are we going to expect other people to understand! :) but they aren't curated well, thats why they are at level 0 or level 1 curation? there isn't anything you can do about that unless someone curates the model to a higher level. I think the issue might be in the particular graphical representation of curation level that the repository is currently using. Typically on the internet, anywhere you see stars the more stars the better. While technically true for the curation status of a model, only have a curation level of 1 or 2 out of three is not necessarily a bad thing. Perhaps we need to come up with some other graphical rendering? maybe a tabular format with curation levels for the columns and tick marks in the appropriate column... >> hopefully Tommy's design for moving the repository forward will help >> resolve the whole versions of variants and variants of versions issue. >> > > I think Matt needs to work on the naming convention for this to be > cleared up. This is an issue that keeps coming up. I would expect the new design to have quite a different naming convention, but until we see something... _______________________________________________ cellml-discussion mailing list cellml-discussion@cellml.org http://www.cellml.org/mailman/listinfo/cellml-discussion
