Introduction
A key issue in external quality assurance (EQA) or proficiency testing is to ensure high quality schemes and to avoid that participation in such schemes does more harm than good. Ideally, any EQA program should provide the participants information of whether their measurement procedure has a bias from a true value (1). An EQA organisation distributing control material should strive to obtain and use native commutable materials where the target values are set by using a reference method or a certified reference material (1). However, in many cases this is not possible, control material that is not commutable is used, and thus peer group target values must be established. Such materials have severe limitations since the material may also not be commutable between reagent lots within the same method (2, 3). Circulation of unsuitable EQA materials could in such cases generate harm by misclassifying participant performance.
It is often more difficult to obtain a commutable material when the EQA scheme has a high number of participants since commutable material is based on native patient samples and have a limited stability. In such cases, smaller (national/regional) schemes with fewer participants can be preferred. For point-of-care (POC) testing, it is even more difficult to obtain commutable control materials since the matrix generally is whole blood. Often different control materials have to be circulated to the different POC instruments, and no control materials are available for some POC instruments, e.g. as has been shown for POC international normalized ratio (INR) testing (4). An alternative EQA approach has been developed in situations where commutable control materials are not available (5), in which a limited number of selected general practitioner (GP) offices perform a split sample comparison with a central laboratory method using native whole blood patient samples. In addition, non-commutable EQA materials are circulated to all participants. In this way, method performance is addressed by the split samples system and participant performance is addressed by the non-commutable material, and the EQA provider does not need to circulate native materials to all the participants (5).
EQA for POC testing is in many ways similar to EQA for larger hospital laboratories. There is, however, one important difference that is not always acknowledged; the participants. Whereas the participants in the EQA schemes for hospital laboratories usually are specialists in laboratory medicine or medical laboratory scientists, the participants in EQA for POC testing are often the end users of the tests, i.e. health care personnel with little or no knowledge of laboratory medicine. The implication of this is that the EQA organiser has a) to convince the participants that participation in EQA schemes are important, b) be able to circulate materials with time intervals that are acceptable both for the participants and from the organisers point of view, c) produce feedback reports that are understandable by the participants, and d) offer help and guidance to the participants when needed. In addition, it is important to e) address the pre-examination, the examination and the post-examination processes, and f) offer schemes for measurement procedures using interval or ordinal scale. The aim of the present paper is to highlight these essential aspects of EQA for POC testing.
Convince the participant that EQA is important
The common opinion in laboratory medicine is that EQA is useful. This opinion is more or less part of our education and we are so used to it that we do not question the value of it. However, since we with EQA of POC testing are addressing people without much knowledge of laboratory medicine we have to explain why they should participate in this system and what they can gain from it. In fact, there is little evidence that participation in EQA is useful to improve the quality of the results and no evidence concerning the benefit for the patients. The reason for this is partly that it is difficult to isolate the “EQA factor” from other factors that can contribute to the improvement of the quality of the examination processes. In a recent paper by Bukve et al., looking at the development of the analytical quality for POC analyses during a period of 9 years, it could be shown that the number of times the participants participated in an EQA program for POC glucose, haemoglobin and C-reactive protein (CRP) testing, was an independent factor associated with improved analytical quality of these analytes (6). Other independent factors associated with improved analytical quality were type of instrument, performing internal quality control weekly, performing 10 or more patient tests weekly, and having laboratory-qualified personnel performing the tests. Another important factor, which was not investigated in the study by Bukve et al. was that the POC testing laboratories participated in a quality improvement follow-up system in which they always had somebody to contact if they had problems with the tests. To the authors knowledge, this study is the first evidence that EQA for POC testing is useful (6).
Frequency of EQA schemes
The optimal frequency of EQA schemes is often debated and the evidence for an optimal frequency is difficult to find. Looking through the catalogues of different EQA providers, it is easy to see that the frequency of EQA surveys varies, and the scientific reason for this is not given. In these authors opinion, a high quality scheme with commutable material and reference target values and thoroughly elaborated and understandable feedback reports are much more important than schemes with a high number of surveys. The theoretical reason for this opinion is that EQA should not be a substitute for internal quality control, but should concentrate on finding systematic deviations of measurement procedures preferably from a true target value. In cases where deviations are found, the users of the tests should be followed up by direct contact. This is of course even more important with POC testing since the users are clinicians, nurses and health care personnel with little or no education in laboratory medicine. One important factor that promotes a high frequency of schemes is money. It is common that EQA providers are paid by each survey they circulate, meaning that more money is earned if surveys are circulated more frequently. An alternative approach could be that the EQA providers are paid for the expertise they provide so that professional reasons could determine the optimal frequency of surveys and the content of the feedback reports. A working group in the European Organisation for External Quality Assurance Providers in Laboratory Medicine (EQALM) is currently investigating the optimal frequency of EQA schemes, and the goal is to provide guidance on evidence based models for EQA design (7).
Feedback reports
Feedback reports from EQA schemes are often comprehensive and complicated, and can be difficult to understand. However, such comprehensive reports can be very useful for skilled personnel in large laboratories, e.g. it can help them look at time trends, concentration effects, calibration, etc. For the users of POC tests on the other hand, only two basic aspects are important: 1) is my result correct and 2) if not, what can I do to improve it? The feedback reports must therefore be simple and educational. It is important to explain for the participants if a deviant result is caused by the instrument they are using, by the reagent lot they are using or by their own performance. Therefore, the participants should report which instrument and reagent lot they have used in addition to the control results. It is then easier to identify the reason for a deviant EQA result and to describe this in the feedback report. Reagent lot information is often not given in EQA, probably because it is assumed that lot variation is detected by internal quality control. However, this can be more difficult to achieve for POC testing. In a recent paper dealing with POC tests for urine albumin:creatinine ratio (ACR) and INR it was shown that there indeed could be large lot variations for these analytes, detected by the EQA control material (8). Concerning INR, a non-commutable material was used and it could be shown that the material was even not commutable within the method studied, i.e. the material was not commutable between reagent lots. The lot variation found did not reflect results obtained from native patient samples. Thus, if this had not been discovered, the feedback report could easily have resulted in “more harm than good”. In contrast, for ACR where a commutable material was used, the lot variation was also found using patient samples and this variation could therefore have clinical implications. Again, information about the reagent lots was of the utmost importance for the understanding of the EQA results and the feedback reports.
It could be valuable if the EQA provider asks for different characteristics of the participants in order to understand which factors are associated with good quality. Such factors could for example be the frequency of running internal quality control, the number of tests performed per week, the profession of the test operator, and the number of employees (6).
Help and guidance to the participants
In general, EQA participants should have the opportunity to seek help and guidance when needed. For participants in primary health care it is even more important to have someone they can address when they have problems to interpret the feedback reports, and most importantly to decide what actions should follow a deviant result. The EQA provider should have the responsibility to establish such follow-up system. In a hospital, however, the central laboratory could have responsibility for the POC analyses at the wards and thus also to educate and guide the POC users in EQA issues. Such supervision system might be easier to organize within a hospital environment than in primary health care. The POC manufacturers are also responsible to have a system to educate, guide and follow-up the POC users.
In 1992, the Norwegian Quality Improvement of Primary Care Laboratories (Noklus) (9) was established and one pre-requisite for starting this organisation was that there should be a system where the users could get sound advice. Therefore, in addition to establish an EQA organization, a system with more than 40 laboratory advisers sited all over Norway was established. Each laboratory advisor has the responsibility for about 50-100 units like e.g. GP offices, nursing homes, emergency healthcare centres, occupational healthcare centres, and oil platforms. 99.8% of all GP offices and 96% of all nursing homes in Norway participates in Noklus voluntary. The main tasks for the laboratory advisors are to visit the participants, organize courses, offer advice concerning which instrument to buy, and follow up results from EQA schemes. In principle, these advisors contact all participants in Norway with poor performance. Since the choice of instrument is very important for the quality of patient results, the Scandinavian Evaluation of laboratory equipment for primary health care (SKUP) (10, 11) was established.
EQA of the total examination process
Whereas most attention from EQA providers has been focused on the analytical examination, more and more attention is drawn to the pre- and post-examination processes. Many EQA providers are now circulating pre- and post-examination schemes alone or embarked in the analytical schemes. Especially the post-examination schemes are important for POC testing since there is a direct communication with the end-users of the tests. It is then possible to examine how the test results influence their clinical decisions and indirectly also ascertain what performance they believe they have (12). By combining the feedback of analytical quality with feedback on how the clinicians use the test, it is possible to generate a strong educational material concerning the value of laboratory tests and how important it is that tests conform to given performance specifications. Noklus has run a series of case history based EQA surveys over years (13) showing that clinicians have widely different knowledge of the analytical quality of the tests they are using.
Measurement procedures using the nominal or ordinal scale
A nominal scale deals with classification of a quantity irrespective of magnitude, e.g. type of virus, bacteria, or mutations whereas an ordinal scale deals with all types of grading, e.g. urine strips for glucose or human chorionic gonadotrophin (hCG) (14, 15). Generally, measurements performed on an ordinal scale are measurements that can also be performed on a ratio or interval scale. The quantities are often measured on an ordinal scale because a more rapid test result can be obtained and because people without much laboratory experience can perform such tests. Thus, these tests are commonly used as POC tests in for example GP offices, nursing homes and rural areas (16, 17).
EQA of the ordinal scale can be used for different types of POC tests, such as for example HIV, malaria and tuberculosis (18). EQA of such tests raise specific challenges because the users have more difficulties to understand the value of such EQA (19). In an EQA for POC on the ordinal scale, samples are typically circulated with concentrations that are expected, with a very high probability to give “positive” or “negative” results. In addition, samples with an intermediate concentration are circulated. The participants will get an evaluation only with respect to the “positive” or “negative” samples since they are supposed to classify these samples correct (18). Samples with intermediate concentrations will give results that are expected to be both “positive” and “negative.” This information is useful to assess and to monitor the performance of the POC tests, but not to assess the user performance. Therefore, it is important to also circulate samples with intermediate concentrations, but this must be thoroughly explained to the users so that they can see a benefit from it.
Conclusion
EQA for POC testing is in principle similar to EQA for larger hospital laboratories, but there are some important differences. The participants are often the end-users of the tests (e.g. clinicians and nurses), they have usually little or no knowledge of laboratory medicine and the number of participants is often high. This gives the EQA providers some extra challenges; they must convince the participants that participation in EQA schemes are important, be able to circulate materials with reasonable time intervals, produce feedback reports that are understandable by the participants, and offer help and guidance to the participants when needed. It is also important that EQA for POC testing address the total examination process, and that schemes for measurement procedures using interval or ordinal scale are offered.
