Inappropriate but informative. ----- Original Message ----- From: "Marcallee Jackson" <[EMAIL PROTECTED]> To: "WEDI SNIP Testing Subworkgroup List" <[EMAIL PROTECTED]> Sent: Monday, November 25, 2002 8:34 AM Subject: RE: VALIDATION or Certification
> I don't think this is an appropriate use of the list serve. Though I > understand Kepa's message was in response To Rama's, this kind of email > is too close to a sales pitch. It gives other vendors no choice but to > respond in kind. Now, if we want the list to turn into a forum for > vendors to discuss the quality of their products and benefits of its > features in comparison to others, we can make that choice, but unless > we're ready to open the door to everyone with a testing product to plug, > I suggest this discussion change direction. > > Whether you call it testing or certification, it seems the real topic > here is to what extent does analysis need to occur? That's a valid > discussion but product comparisons do not belong here. > > -----Original Message----- > From: Kepa Zubeldia [mailto:[EMAIL PROTECTED]] > Sent: Sunday, November 24, 2002 1:31 AM > To: WEDI SNIP Testing Subworkgroup List > Subject: Re: VALIDATION or Certification > > Rama, > > I agree with you that the work of comparing results between the three > largest > testing vendors was very much needed. Patrice and Mike are to be > commended > for all the work they put into this, and I hope there will be more > extensive > studies in the future. > > You bring up some great points. And since you invited me to address > them, > here it goes... > > First, in spite of the title of the session being "Validation or > Certification", the specific instructions from Patrice to all three > vendors > were that we were ONLY to discuss testing. We were not supposed to > discuss > certification, only testing. I specifically asked Patrice about this > because > it seemed odd, given the title of the session and she specifically > forbade me > to discuss certification or HIPAAmetrics. My presentation was to be > ONLY > about testing outbound transactions using Velocedi. > > Since I was the first presenter, I stuck to the script. The ONLY thing > we > were supposed to do in our 10 minutes each was to test/analyze/validate > the > one file that Patrice had provided us with, and no other. No other > discussion or presentation was to take place, per Patrice's > instructions. > Much to my dismay, the other two presenters chose to present their > entire > product lines rather than sticking to the rules. Patrice, given the > situation, passed me a note inviting me to take a second turn of 5 > minutes to > make a "sales pitch", but I declined. > > Had I know that I could present not only about outbound testing but also > about > certification and inbound testing and inbound certification, I would > have > used my 10 minutes very differently. > > So, given that the only topic was testing/analysis/validation, you can > probably understand why when you brought the topic of the difference > between > testing and certification from the audience, Patrice diverted your > question > and never addressed the certification part of it. She was not > interested in > discussing certification in this meeting, only testing. I hope this > email > addresses your question. > > Comparing the offering of these three vendors by only looking at the > results > of testing outbound EDI files is equivalent to comparing automobiles by > only > looking at their tires. Yes, they do all have four tires on the road. > Yes, > all tires are made out of rubber, steel, plastic, cotton and other > fibers. > Yet, some tires will separate the thread under certain conditions and > other > tires will work better in the snow or may have studs for ice. But, > would you > buy a car based on the tires? There are many other aspects that are > much > more important than the tires. The Minnesota case study only looked at > the > single aspect of outbound transactions test results, with the explicit > exclusion of other important aspects, such as certification or inbound > testing. > > So as far as testing is concerned, Patrice showed that the three main > contenders can test the files with some differences. However, had we > tested > a different file, a more moderately sized file for example, we would had > seen > a number of significant differences in the quality and completeness of > the > results. This is left for a future exercise. For example, I tested a > medium size file with 3,500 professional claims and it took 133 seconds > (2m13s) using the Foresight software but only 3 seconds using Claredi's > Velocedi, using exactly the same hardware. This sort of differences are > not > visible when the test files are as small as the file Mike and Patrice > gave > us. I will stop here, as I don't want to start benchmarking > discussions. > > As for your request for clarification about the disclaimer in the test > report... The test report is exactly that, a TEST report. It is NOT > certification. For almost two years now I have been making a big > distinction > between testing and certification. Some people understand the > difference, > others don't understand the difference. I think it is clearly expressed > in > the white paper, but obviously it is not as clear as I think it is, or > this > topic would not come up so often. I will try to explain it in this > email, > but I suspect it is going to need a more detailed description, so I am > going > to start working on a Claredi white paper to make sure people understand > what > WE mean by certification. I am aware that most of our competitors do > NOT > understand the difference and therefore try to equate the two concepts > as if > they were one and the same. In their eyes, if you pass a test without > errors, that equates to being certified HIPAA compliant. I believe that > is > not the case. Please keep reading. > > When Patrice approached Claredi about the case study, she was very > explicit > about wanting to show only outbound test results. She did NOT want to > show > inbound testing (with Claredi's Wibbler) and she did not want to show > certification in either direction. So we gave her a temporary test > account > of the same kind we offer in our "trial run" accounts. Those accounts > are > established with only a limited set of capabilities and do NOT have > access to > Claredi's certification, they have access only to testing tools. In a > "normal" (that includes certification) account, the report summary that > contains the disclaimer you are citing also contains a few more sections > > between the "Action" and the "Support" areas of the summary. These > other > sections are labeled "Certification", "HIPAAmetrics", and "File Specific > > Info". Again, these sections are only available to Claredi customers, > not to > test accounts. Also, test accounts are restricted in other ways. I can > give > you the restriction details off-line. > > Since I don't think it would be appropriate to turn this email into a > sales > pitch, I will only describe the outbound "Certification" process. We > also > have a completely different process for inbound certification, and I > could > describe it another day if there is enough interest. > > Once a file has passed the Claredi testing/verification/validation (all > synonyms) process without HIPAA errors, it becomes eligible for > certification > by Claredi. The "Certification" section of the summary page then shows > a > button labeled "Submit this file for certification". If the file has > HIPAA > errors the button does not appear. Unless the user clicks this button, > the > file is not taken through Claredi's certification process. It is only > tested, > not certified. This is why the summary report MUST NOT be construed as > a > certification report, even if it shows that a file does not have "HIPAA > errors". And thus the disclaimer. > > As you can tell, the summary report shows three types of "errors": the > "HIPAA > Errors", with the standard 7 types defined in the SNIP white paper, the > "Business Errors" section, and the "Warnings" section. Only the "HIPAA > Errors" section must be clean for the file to be a candidate for > certification as HIPAA compliant. The file could have "Business Errors" > and > "Warnings" and still be certified as HIPAA compliant, since the > certification > of compliance measures against the requirements of the HIPAA > implementation > guides. > > In the next few weeks we will be relaxing the cleanliness requirement > even > further. For a file to qualify as a candidate for certification, it > will > need to have at least one "business unit" (e.g. claim) without "HIPAA > Errors" > rather than the entire file being error free. The HIPAAmetrics > certification > will give a detail of the contents of the file, including the percentage > of > compliant and non-compliant claims. This is currently in "beta" with a > restricted number of accounts and will appear in all the Claredi > accounts in > a few weeks, as well as in the certification details through the > directory. > > But, back to the certification process... Once the file passes the > testing > clean enough to be "certifiable", and the user clicks on the "Submit for > > certification" button, Claredi takes the file through a process that > looks at > the data content. Based on the data content of the file we can identify > > patterns, such as (I will use the claim as an example) the presence of > Billing Provider and/or Pay-To provider, or the presence of both > Subscriber > and Patient or only Subscriber, or the presence of secondary coverage, > or the > presence of a prior adjudication (COB claims), or the presence of a > referring > physician, or the maximum number of service lines submitted by this > entity, > or the presence of a "cluster of data" that indicates this is an > anesthesiology claim, or a DME claim, or an ambulance claim, etc. > > We identify these data clusters through a process we call HIPAAmetrics. > This > is a unique Claredi pattern recognition process that has never before > been > applied to EDI data, and we have applied for a patent to cover it. The > result of the process gives us the "metrics" of this particular file. I > am > sure you have heard me say many times that the fact that a file does not > > contain errors is not enough. You have to know what was actually > contained > in the file. > > Using HIPAAmetrics we can determine exactly what is in the file, from > the > business perspective. The testing phase looks at the EDI perspective > and > determines that the file (or the individual claim within the file) does > not > contain errors. The HIPAAmetrics looks at the healthcare contents and > "characterizes" it. At the completion of the HIPAAmetrics phase Claredi > can > say: > We have seen a file that contained the following data > - Office visits, 234 without errors, 3 with errors > - Primary claims, 180 without errors, 20 with errors > - Primary claims with a second payer, 0 without errors, 20 with errors > - Secondary claims, 0 without errors, 97 with errors > - Consultations, 5 without errors, 40 with errors > - etc. > Claredi is making a statement of what we have seen. Because there are > some > claims that are "compliant", the provider could claim to be compliant. > However, the statement from Claredi clearly shows that although this > provider > has the ability to generate HIPAA compliant office visits for "single > payer" > situations, the provider does not have the capability to create correct > claims when two payers are involved. > > The HIPAAmetrics describe the profile that we have seen reflected in the > EDI > data itself. The HIPAAmetrics certification gives the detailed data > profile > necessary to make a determination of compliance. Without this sort of > detail, all you would know is that the provider has generated something > that > did not have errors, but you don't know what that "something" was. > Certainly > none of the other testing entities (neither our competitors nor the > translator validators) provide these details, and I suspect that they do > not > yet understand the difference between passing a "clean test" and being > certified as HIPAA compliant. Both EDIFECS and Foresight show a > pass/fail > approach in their reports. Of course, by the end of this email their > eyes > will be wide open. > > So, there is a significant difference between a testing or validation > service > and Claredi's certification. Big difference. The problem is that if > you > don't understand the difference and assume that any entity that can > produce > an error free file is automatically "certified" you will certainly find > a > real problem when you look at the quality of the data that entity sends. > And > thus the bad rap that the "pseudo-certification" provided by the > pass/fail > approach of our competitors is giving to the word "certification". > > And this is also the reason why a vendor cannot really be certified > unless all > of its clients are individually certified. It is just too easy for the > vendor to produce certain clean transactions, and for the clients to not > be > able to produce the transactions THEY need in a compliant form. Ditto > for > clearinghouses. > > >From the usability side, a clean test that does not have the > HIPAAmetrics type > of data analysis, still needs to be reviewed by an expert to determine > whether the test is sufficient in volume and quality, and to determine > what > is it that was actually tested. Did the test include sufficient variety > of > business scenarios to make it relevant? Were those scenarios complete? > Is > the volume of the sample for each scenario big enough? These are some > of the > sort of things the EDI folks at the payer's end have been looking in > provider's tests. Using an EDI test tool may tell you that the EDI data > does > not contain errors, but that is only a small part of the answer. Most > translators can do that too, so, if that is all the EDI test tool does, > it is > not much better than the translator rules. > > The Claredi HIPAAmetrics answer these questions. The payer can then > define > what are the required parameters. Things as simple as saying "we need > to see > 95% clean claims in a test containing at least 200 claims" can be > answered by > HIPAAmetrics and the Claredi certification, and cannot be answered by > any of > the other test tools in the market today, without human intervention. > > I hope this email sheds some more light on these concepts. I have been > talking about this for almost two years. It is (washed out) in the SNIP > > white paper. And yet, the concept is still a mystery to most, even a lot > of > experts. The reality is that it will take a while to grasp the concept. > > Those of us that have done extensive testing of healthcare EDI before > HIPAA > (NSF, UB92, etc.) understand the concept easily. For some reason the > traditional non-healthcare EDI community (X12) has more difficulty > understanding the concept. > > Now let me add another important difference between the "test results" > and the > HIPAAmetrics. The test results, by their nature, will contain PHI. If > there > are any errors, the PHI will most likely be part of the error message or > > description. Claredi's test results also contain PHI. So we have a > bold > statement on the test results report saying that they must be handled > according to the appropriate privacy and security procedures. And this > in > spite or requiring a Business Associate Agreement with all our > customers. > The HIPAAmetrics reports, on the other hand, do NOT contain PHI and thus > are > shareable with the world. In fact, in a few weeks we will be releasing > the > HIPAAmetrics results as part of the Claredi directory describing the > details > of each certification obtained through Claredi. > > There is a concept of "community" testing that is being touted lately. > The > problem that I see is that if the "community" is comprised by more than > one > payer (e.g. several payers sharing the results of all the providers in > the > community) then all the payers in the "community" are seeing the PHI > from all > the testers. Unless the provider creates a separate test account for > each > one of the payers in the "community" and then sends the appropriate test > > files to each one of those accounts, so each of the payers will only see > the > test files that corresponds to that payer, there is a problem... All > the > payers in the "community" will be seeing (through the "community" > testing > center) the PHI that belongs to the other payers. As far as I can tell > this > is not kosher for healthcare privacy reasons. Even if the provider has > a > "business associate agreement" with the community test center, and a BAA > with > each one of the payers, and each payer has a BAA with each one of the > payers > in the "community", I still think that the "minimum necessary" rules > would > preclude this sort of bulleting board sharing of PHI within the testing > "community". > > Yes, you guessed right. If instead of sharing the test results, like > these > "communities" do today, they were to share the HIPAAmetrics > certification, > the PHI would never be shared. > > Some people have seen the vision. We have offered it to other testing > vendors > and translator vendors. Some have expressed interest, others are not > interested. Clearly we need to do a lot more education on this topic. > The > American Hospital Association understood the concept and is now > endorsing > Claredi as the exclusive HIPAA transactions testing and certification > service > to their members. > > So, going back to Patrice's case study, you cannot compare testing and > certification if the only thing you look at is testing. Testing your > own > transactions is important. Some people will focus only on testing. > That is > just fine, as there are many different approaches to putting a trading > partner into production. We believe out test tools are better than our > competitors', but I am sure they believe theirs are better. But testing > is > only part of the picture. To establish a business relationship for > HIPAA > transactions, you must go beyond compliance testing. Knowing that the > data > you receive is clean is important, but you must also know what is > contained > (from the business perspective) in that clean data stream. Only Claredi > > gives you an automated way to determine that. We call it HIPAAmetrics > Certification. For the last 20 years we have done without it, and we > have > been happy with just test tools. Some people are just happy with the > current > tools, other people want to explore a more automated way to put a > trading > partner into production. This country was built on freedom of choice. > No > need to put someone else down just because you don't like what they have > to > offer. > > If you want to see it in action, give me a call and I can show it to > you. It > is really simple... > > Kepa > > > > On Saturday 23 November 2002 12:54 pm, [EMAIL PROTECTED] wrote: > > For the benefit of those who could not make it to the WEDI SNIP HIPAA > IMPLEMENTATION SUMMIT held in Phoenix from Nov 18 - 21, 2002. > > > > There was a session on Nov 20th between 9.45 AM - 11.45 AM on the > topic > "Validation or Certification", moderated by Patrice Thaler of Allina > Health > System and Mike Ubl of Blue Cross of Minnesota and the main participants > > being three vendors of testing tools represented by their big chiefs... > Namely Sunny Singh from EDIFECS, ED Hafner from Foresight Corp, Kepa > Zubeldia > from Claredi, presenting a case study of three vendors. > > > > First of my sincere thanks to Patrice, Mike and whoever helped in this > case > study, besides the vendors. > > > > Being the author of the vendor-neutral statement in the Testing and > Certification White Paper (Hey John, why is my name missing in the > acknowledgements section? ;-)), and being the first (one of the?) to > suggest > the formation of Vendor (of testing tools AND TRANSLATORS) Consortium in > one > of the teleconference calls in May/June of 2002, I am really glad that > this > case study was done. It is the first step in that direction of forming > the > vendor consortium to help the CEs be able to pick and choose any one of > them > vendors, for they ALL have to offer the same basic VALIDATION (not > certification) capability by coming up with the same ERROR messages and > similar warning messages for the same test files. This case study > proved > just that point and am truly delighted and am happy to share it with one > and > all. > > > > One more interesting thing and I would like Kepa Z to clarify for all > of > us... > > > > On page 3(of the handout we were given there), on a "Claredi - Passed > Claim", there is a heavy disclaimer in BOLD, which reads "This is not a > certification or verification of HIPAA compliance Summary of test > report for > confidential use by Claredi customer only". I know and agree that it is > NOT > CERTIFICATION. I can also understand why the disclaimer is there. But > Kepa > should explain to us why he touts Certification with an evangelical > pitch and > then this disclaimer... saying it is NOT CERTIFICATION! > > > > Another important point, as suggested by Larry Watkins, as a member of > the > X12N committee (correct me Larry if I am not saying this right), that > the > discrepancies in the standards are not that many and there is a very > good > process to address them. I can't agree more. > > > > The bottom line is, it is Testing, Verification and Validation and NOT > > CERTIFICATION. End of discussion. I have been saying this all along > and one > more time won't hurt ;-). That said, I really do believe all the > vendors, > have a lot to offer in the validation part of transaction testing. I > have no > bias or preference towards one vendor. The CEs (or their testing > people) > should make the choice. Don't let anyone fool you that it is soooooooo > big > and only I (my org) can lead you(r org) to salvation ;-). > > > > Accreditation, creating entities to certify and the process of > certification > is a long one and can be done, but vendor consortium will eliminate the > need > for certification all together, IMHO. > > > > This message is not intended to be against any individual or a flame > and > rake up any unnecessary things but to share the joy of possible > formation of > vendor consortium and also request the translator vendors to join in the > > vendor consortium. This will really help in reducing the chaos and > eliminate > the obfuscation and confusion that is there and being created... We > don't > have time and anybody who can chip in to make things simple to whatever > extent should be commended... like Patrice and Mike and all those who > participated in the case study. In Patrice's words, paraphrased, "It is > easy > and you can do it, and I have done it". > > > > Thanks for reading up to this point. > > > > Best Regards, --Rama. > > > > --- > > The WEDI SNIP listserv to which you are subscribed is not moderated. > The > discussions on this listserv therefore represent the views of the > individual > participants, and do not necessarily represent the views of the WEDI > Board of > Directors nor WEDI SNIP. If you wish to receive an official opinion, > post > your question to the WEDI SNIP Issues Database at > http://snip.wedi.org/tracking/. These listservs should not be used for > > commercial marketing purposes or discussion of specific vendor products > and > services. They also are not intended to be used as a forum for personal > > disagreements or unprofessional communication at any time. > > > > You are currently subscribed to wedi-testing as: > [EMAIL PROTECTED] > > To unsubscribe from this list, go to the Subscribe/Unsubscribe form at > > http://subscribe.wedi.org or send a blank email to > [EMAIL PROTECTED] > > If you need to unsubscribe but your current email address is not the > same as > the address subscribed to the list, please use the Subscribe/Unsubscribe > form > at http://subscribe.wedi.org > > > > > > -- > This email contains confidential information intended only for the named > > addressee(s). Any use, distribution, copying or disclosure by any other > person is strictly prohibited. > > > --- > The WEDI SNIP listserv to which you are subscribed is not moderated. The > discussions on this listserv therefore represent the views of the > individual participants, and do not necessarily represent the views of > the WEDI Board of Directors nor WEDI SNIP. If you wish to receive an > official opinion, post your question to the WEDI SNIP Issues Database at > http://snip.wedi.org/tracking/. These listservs should not be used for > commercial marketing purposes or discussion of specific vendor products > and services. They also are not intended to be used as a forum for > personal disagreements or unprofessional communication at any time. > > You are currently subscribed to wedi-testing as: [EMAIL PROTECTED] > To unsubscribe from this list, go to the Subscribe/Unsubscribe form at > http://subscribe.wedi.org or send a blank email to > [EMAIL PROTECTED] > If you need to unsubscribe but your current email address is not the > same as the address subscribed to the list, please use the > Subscribe/Unsubscribe form at http://subscribe.wedi.org > > > --- > The WEDI SNIP listserv to which you are subscribed is not moderated. The discussions on this listserv therefore represent the views of the individual participants, and do not necessarily represent the views of the WEDI Board of Directors nor WEDI SNIP. If you wish to receive an official opinion, post your question to the WEDI SNIP Issues Database at http://snip.wedi.org/tracking/. These listservs should not be used for commercial marketing purposes or discussion of specific vendor products and services. They also are not intended to be used as a forum for personal disagreements or unprofessional communication at any time. > > You are currently subscribed to wedi-testing as: [EMAIL PROTECTED] > To unsubscribe from this list, go to the Subscribe/Unsubscribe form at http://subscribe.wedi.org or send a blank email to [EMAIL PROTECTED] > If you need to unsubscribe but your current email address is not the same as the address subscribed to the list, please use the Subscribe/Unsubscribe form at http://subscribe.wedi.org > --- The WEDI SNIP listserv to which you are subscribed is not moderated. The discussions on this listserv therefore represent the views of the individual participants, and do not necessarily represent the views of the WEDI Board of Directors nor WEDI SNIP. If you wish to receive an official opinion, post your question to the WEDI SNIP Issues Database at http://snip.wedi.org/tracking/. These listservs should not be used for commercial marketing purposes or discussion of specific vendor products and services. They also are not intended to be used as a forum for personal disagreements or unprofessional communication at any time. You are currently subscribed to wedi-testing as: [email protected] To unsubscribe from this list, go to the Subscribe/Unsubscribe form at http://subscribe.wedi.org or send a blank email to [EMAIL PROTECTED] If you need to unsubscribe but your current email address is not the same as the address subscribed to the list, please use the Subscribe/Unsubscribe form at http://subscribe.wedi.org
