Expert Systems In Medicine Research Paper

1. Introduction

Medical expert systems are a class of artificial intelligence systems that make predictions or treatment recommendations as is done by a medical expert. The idea that a computer system could act as a medical expert is an exciting concept. Images of universal access to this expert via the Internet are part of this fantasy, as is the hope that such a system would provide state-of-the-art (perhaps someday even flawless) patient care. Much progress has been made, but important challenges remain.

It is important to differentiate MESs from other systems, such as decision support systems. A MES actually makes a prediction or treatment recommendation, as would a human expert (i.e., a clinician). This is the key difference from a decision support system, which is designed to help the clinician make a better decision than he or she would in the absence of the MES. The MES can actually stand alone with its own prediction or recommendation. At the conclusion of the MES encounter, the clinician may accept or reject the output of the MES, but in theory, the MES recommendation with no further refinement could be compared to that of the human expert. Thus, there would really be three potential predictions or recommendations: the human expert’s alone, the MES alone, and the human expert’s (possibly revised) recommendation after viewing the prediction made by the system. With a decision support system, only the first and last recommendations could be compared.

Physical representations of MESs vary greatly. While in general the representational form is of little concern, it occasionally may affect MES accuracy or ease of use. For example, a table may oversimplify relative to a graph, text, or computerized algorithm, resulting in a loss of accuracy because the table may collapse levels of continuous variables into ranges. The patient with an age of 40 may have a prognosis different from the patient of age 50, yet they would be treated identically if the table grouped patients as ‘Age 40–50.’ Note that computerization of the MES is not necessarily required, unless ease of use or accuracy are affected. For example, tedious manual calculations may become inaccurate if not computerized, though having to have a computer at the patient’s bedside may complicate ease of use. The bottom line, however, is quality of the recommendations, which is most often measured as predictive accuracy. For this reason, the reader is referred to Decision Support Systems for technology issues, since they are not particularly unique to MESs.

The focus on accuracy of the MES recommendations and predictions is not meant to dismiss issues of innovation diffusion or political championship. Clearly, a perfectly good MES, which does not make the clinician more efficient and/or more effective, will never be widely adopted. The system must be at least as effective as the human expert. An MES that is more efficient but makes more mistakes than its human counterpart will not, and indeed should not, be utilized in a litigious medical environment where quality of care is scrutinized. However, a system which is of comparable effectiveness but higher efficiency would be particularly welcome in a capitated provider system where the clinician is not directly compensated for each patient encounter or procedure. Alternatively, a system that is more effective than the human expert might still be enthusiastically accepted even if efficiency is impaired. The reason for this is that some patients will seek the most effective care for their condition and look for the clinician or provider who employs the most effective technology if it produces superior patient outcomes. Of course, MESs which increase both efficiency and effectiveness will have an easy time with diffusion, but examples of such MESs are rare.

2. Classes Of Medical Expert Systems

There are two major classes of medical expert systems in development or use today. The most popular are predictive MESs. This type of MES strictly makes predictions of some medical endpoint or outcome that is not immediately known. Diagnostic systems would be included under this category because these systems predict what disease the patient currently has, which is not immediately known until a (perhaps expensive, time-consuming, or painful) gold standard evaluation is performed. Generally, such MESs rely on readily available predictor variables, and the outcome being predicted usually requires an invasive procedure or follow-up time before it can be known with certainty. Thus, the clinical values usually associated with being able to predict the endpoint are that of (a) lead time that a future event will or will not occur or (b) possibly sparing the patient from a painful, delayed, or expensive procedure. The second most popular MES class is prescriptive. Prescriptive MESs make treatment recommendations and possibly partially implement them. A distinction between predictive and prescriptive MESs is that the outcome is not affected by the predictive MES but may be affected by the prescriptive MES since a treatment is attached to the latter recommendation.

2.1 Predictive Medical Expert Systems

Numerous examples of predictive MESs exist. These are essentially prediction tools, and appear in a variety of forms. Some of the more common forms are charts, tables, graphs, or ‘nomograms’ which are scales with numbers. Figure 1 is a nomogram that serves as a predictive MES for prostate cancer (Kattan et al. 1998). The reason for the graphical rather than tabular depiction is to avoid, as much as possible with printed media, categorizing the continuous variables, which would have to be done if a table were used. Alternatively, predictive MESs may be computerized as traditional software or enabled to run on dedicated handheld calculators.

The most desirable feature of a predictive MES is predictive accuracy. For a new predictive MES to become popular, it must predict more accurately than the traditional method of prediction in that clinical setting, which may be an older prediction tool or simply clinical judgment. With the latter, a substantial body of research has been devoted to studying human prediction and its limitations (Hogarth 1988). It has been shown quite clearly that human beings, even clinical experts, have biases when it comes to prediction. For example, humans do not remember all cases in the past with equal ability; certain cases stand out, and these are given more weight when trying to predict outcomes of future cases. Also, humans tend to predict the outcome they want to occur, rather than the outcome with the highest actual probability. For these reasons and others, predictive MESs often outperform human experts, even when the experts have more information available than that utilized by the predictive MES. The likely reason for this seems to be that humans do not properly weight the additional information when mentally revising the predictive MES output.

Predictive MESs are typically created based on statistical analyses which form the underlying prediction model. The statistical analysis may have been performed using traditional statistical procedures (such as multiple regression analysis) or with computationally intensive procedures (such as neural net- works). Typically, the actual outcome is modeled by attempting to predict it using other variables contained in a dataset. Alternatively, one could model the judgments of a human expert. With this approach, what the expert decided is the outcome being modeled, rather than what actually occurred with the patient. However, the resulting model would not have much chance of being more accurate than the expert(s) modeled, so this type of MES would only succeed in the setting where (a) a premier expert were modeled and (b) the predictive MES was needed for application outside that original setting. An example of this could be a treatment recommendation, where what the expert decided serves as the gold standard, and where success or failure of the expert’s recommendation might be difficult to judge. In settings where the prediction task is deterministic (i.e., 100 percent predictive accuracy is possible with existing data), statistical predictions models may not be the preferred basis for the MES. Here, traditional MES chaining algorithms (Ignizio 1991) could be implemented in lieu of a statistical prediction formula. However, interesting deterministic medical settings are extremely rare.

The studies that have compared human experts with predictive MESs have been quite convincing. Ignoring the issue of potential publication bias, human experts only rarely outperform a predictive MES, and quite often predict less accurately than a predictive MES. This has been illustrated in a variety of medical settings, usually involving clinical experts. Dawes et al. (1989) present one of the more comprehensive reviews of this literature. In most cases, it appears that the predictive MES is not routinely embedded in electronic medical record system software. Rather, a separate data entry step is required for the human expert to obtain the predictive MES prediction. Despite this slight decrease in efficiency of the expert, numerous applications exist where this tradeoff is apparently worthwhile. An example is the commercially successful APACHE software for predicting the probability of death for patients admitted to intensive care (Knaus et al. 1995).

2.2 Prescriptive Medical Expert Systems

Prescriptive MESs are more comprehensive than predictive MESs since the former make treatment recommendations that incorporate predictions in their recommendation. With a prescriptive MES, a judgment is made regarding appropriate care, not just anticipated outcome. Many prescriptive MESs at least partially implement their recommendations, such as a drug dosing application. In this setting, the prescriptive MES prescribes a treatment as well as monitors its effectiveness, possibly making adjustments and revising its predictions based on feedback within the same patient. The common example from this class is drug dosing, where numerous repeated predictions at the individual patient level may be made (i.e., the level of drug to deliver). The system manages those predictions over time based on its performance within the individual patient (i.e., how well the last dose level controlled the medical condition and what future dose changes need to be made).

As with predictive MESs, the literature is quite consistent concerning prescriptive MESs. Properly developed and validated prescriptive MESs seem to perform at a level that is at least as accurate as a clinical expert, and typically exceed that of a clinical novice (Raschke et al. 1996). Also, as with predictive MESs, clinicians in the presence of a prescriptive MES tend to perform better than they did prior to the prescriptive MES. However, prescriptive MESs are available only in limited domains, where the medical process has been well studied and understood. Furthermore, it seems likely that this literature has the strong potential to suffer from publication bias, in that poorly performing prescriptive MESs are underreported. In addition, results of a prescriptive MES are somewhat more difficult to validate and analyze. The reason for this is that usually the patient can only be treated by the MES or by the clinician, but not by both. With a predictive MES, the same patient can usually have his outcome predicted by both the MES and the clinician who is blind to the MES prediction. With a prescriptive MES, patients must be randomized to receive treatment from either the MES or the clinician. Since the same patient can no longer serve as his own control, the sample size requirements and study design become more complicated in studies of prescriptive MES performance. Furthermore, patients may have difficulty with consenting to be randomized to treatment by either a computer or a doctor. An alternative study design is to randomly provide the clinician with the MES recommendation and observe whether outcomes are improved when the clinician has this recommendation, but interpretation of this design is also complicated because it is not a true test of the prescriptive MES performing alone.

3. Future Development Of Medical Expert Systems

Predictive MES development is likely to continue to flourish. These systems have been well established as valuable, and much research is directed to the development and comparison of alternative systems. Adoption of these systems takes time, but the Internet has facilitated this with promotion of particular systems as well as offering end user and physician interaction with these systems. For example, women can now estimate their risk of being diagnosed with breast cancer using a National Cancer Institute web page. While it seems that prescriptive MESs also should play a larger role in healthcare delivery, this process will be slower. Since they are more comprehensive, they are harder to develop and validate. Nonetheless, these systems have good potential to improve efficiency and effectiveness of patient care.

An important challenge for prescriptive MESs is to incorporate patient preferences into medical decision-making. Quality of life can be as important as quantity of life in particular medical situations. It is clear that the goal of a medical test or treatment cannot be survival with complete disregard to quality of life. The medical decision is not complex when, for example, two treatments have comparable survival but differing quality of life: the treatment with higher quality of life would be preferred. However, it is not uncommon to find medical options where one treatment has better survival but worse quality of life than the other treatment. Obviously, the magnitudes of the survival and quality of life differences become an issue. A minor decrease in quality of life might be worth a major difference in survival, but the simultaneous consideration of these measures is nonetheless required.

The current paradigm for simultaneous consideration of quantity and quality of life, when comparing treatment or testing options, is that of clinical decision analysis. This technique incorporates outcome probabilities and patient preferences to compute quality adjusted survival times. This is typically done by assuming that the patient progresses through a series of health states, each with its own utility (measured on a scale where 0 equals death and 1 equals perfect health). Quality adjusted survival is computed by weighting the survival time in a health state by its utility, and summing those products across the life expectancy of the patient. The appeal of this approach is that the preferred treatment can be judged even when treatments differ with respect to quantity and quality of life. Via the quality adjusted survival computation approach, magnitudes of survival probabilities and quality of life preferences are considered.

Though the decision analytic approach is an apparently rational one, many challenges to its implementation remain. The preferred method of measuring utility varies greatly. Unfortunately, there is little data on longitudinal utility assessment in the literature, so questions of how well patients adapt to their health are largely unanswered. This body of literature is conflicting, with some studies indicating that patients rate future health states lower than when patients rate the health state upon entering it. Also of concern is the patient’s attitude towards risk, or similarly, how he or she discounts future health. For these reasons, it is not clear the individual patient should even want to maximize his quality adjusted life years without first incorporating some of these factors. Until these theoretical issues are resolved, or until the decision analytic approach is demonstrated to improve outcomes in a randomized clinical trial, adoption of this method as a basis for prescriptive MESs will be hampered.

4. Future Evaluation Of Medical Expert Systems

In light of the difficulty associated with judging what an optimal treatment should provide when both quality and quantity of life are important, alternative evaluation methods of medical expert systems will become necessary. Survival and quality of life by themselves are often limited endpoints, much the same way intermediate endpoints do not necessarily translate into final endpoints. Thus, judging whether an MES does better than status quo can be challenging. Work is just beginning in the definition of more global endpoints, such as regret-free survival. Here, a patient is followed until death or regret of treatment choice (either indicates treatment failure), and when applied in the context of a randomized trial, would appear to offer a measure which also incorporates quantity and health-related quality of life but with possibly fewer technical implementation and interpretation problems.

5. Conclusions

Predictive (predict outcomes) and prescriptive (make treatment recommendations) MESs have potentially valuable roles in the delivery of medical care. Validation of these systems is critically necessary and difficult, but numerous examples of successful medical expert systems are available.

Bibliography:

1. Dawes R M, Faust D, Meehl P E 1989 Clinical versus actuarial judgment. Science 243: 1668–74
2. Hogarth R M 1988 Judgement and Choice: The Psychology of Decision Making, 2nd edn. Wiley, New York
3. Ignizio J P 1991 An Introduction to Expert Systems. McGrawHill, New York
4. Kattan M W, Eastham J A, Stapleton A M F, Wheeler T M, Scardino P T 1998 A preoperative nomogram for disease recurrence following radical prostatectomy for prostate cancer. Journal of the National Cancer Institute 90: 766–71
5. Knaus W A, Harrell F E, Lynn J, Goldman L, Phillips R S, Connors A F Jr, Dawson N V, Fulkerson W J Jr, Califf R M, Desbiens N et al. 1995 The SUPPORT prognostic model. Objective estimates of survival for seriously ill hospitalized adults. Annals of Internal Medicine 122: 191–203
6. Raschke R A, Gollihare B, Peirce J C 1996 The effectiveness of implementing the weight-based heparin nomogram as a practice guideline. Archives of Internal Medicine 156: 1645–9