home
boxmenumenumenumenumenumenumenumenumenuyellowbox

 

 

 

The Cidi-Auto:
A Computerised
Diagnostic Interview For Psychiatry

A discussion paper by Gavin Andrews and Lorna Peters

World Health Organization Collaborating Centre for Mental Health and Substance Abuse, at St Vincent's Hospital, Sydney, Australia, 2010.

Contents
pixel

pixel

Structured Diagnostic Interviews

Accurate diagnosis is the first step in treatment. Specification of diagnostic criteria in the form of the Diagnostic and Statistical Manual for Mental Disorders (DSM-IV) (1) and the International Classification of Diseases (ICD-10) (2) has provided a common language. The specification of criteria for diagnoses embodied in such classifications does not automatically ensure that clinicians will become perfectly reliable when making diagnoses. For example, in the field trials of DSM-III, the inter-rater reliability for the major disorders ranged from kappas of 0.28 to 0.92 (3), indicating that the agreement between two clinicians trained in the application of the DSM-III criteria was at worst poor and at best excellent. The problem is that although the criteria for particular diagnoses are specified in the classification schemes, there are no guidelines for how the information required for diagnoses is to be elicited, nor are there rules for how the obtained information is to be amalgamated to reach a decision about the presence or absence of a criteria for a disorder. Individual clinicians apply their own implicit rules about eliciting and amalgamating information, rules which may differ from those applied by other clinicians, and which may differ from occasion to occasion even with the same clinician, thus rendering clincial diagnoses unreliable.

It was the unreliability of clinician diagnoses which resulted in the advent of structured diagnostic interviews for psychiatric diagnosis (4). Structured interviews usually inquire about symptoms in a standard manner and the data obtained from the interview are then combined to form diagnoses according to a standard algorithm. Thus, two interviewer sources of unreliability are eliminated, even though the validity of the 'standard' decisions about manner of enquiry and combining of that information to inform the diagnostic criteria of the classifications have necessarily been made by the developers of the instrument. It is for this latter reason that structured diagnostic interviews have to have their reliability and validity established before being widely used.

Two types of structured diagnostic interviews have been developed. The first type give structure as to the questions to be asked, but require the interviewer to make clinical judgements based on agreed criteria as to whether the answers to questions fulfill diagnostic criteria and hence whether there is a need to ask additional questions about that diagnosis. The Schedules for Clinical Assessment in Neuropsychiatry (SCAN) is an example of this genre (5). Unless the interviewer is very highly trained such interviews still allow for variability in the clinical decision making and hence inter-rater reliability will necessarily be less than perfect.

In the second type of interview, questions are fully specified, no clinical judgement is required, and interviewers are required to follow fully specified routes in asking questions according to the yes/no responses of subjects (e.g., the Composite International Diagnostic Interview, CIDI, 6). The scoring is based solely on the subject's responses and does not allow interviewer judgement to intervene. This type of interview greatly reduces inter-rater and test-retest unreliability. It is the second type of interview, and in particular the CIDI, which is the focus of this paper, because only fully structured interviews can be properly computerised. While both types of diagnostic interview are amenable to computerised scoring (e.g., the CATEGO-5 program for scoring of data obtained from the SCAN, and the WHO data entry and scoring programs for the CIDI), only the fully structured interviews in which questions are completely spelled out are amenable to computerised administration.

The Composite International Diagnostic Interview

The CIDI is the product of a joint project undertaken by the World Health Organization (WHO) and the former United States Alcohol, Drug Abuse and Mental Health Administration. It is a comprehensive, fully structured diagnostic interview for the assessment of mental disorders which provides, by means of computerised algorithms, lifetime and current diagnoses according to the accepted definitions of ICD-10 and DSM-IIIR. The paper-and-pencil CIDI can be administered by trained lay interviewers and is, like its predecessor (the Diagnostic Interview Schedule (DIS)) (7), the interview of choice for large epidemiological studies (8, 9). Revisions to the CIDI are carried out by an international advisory committee, both to keep abreast of updates in the diagnostic classification schemes and to improve the reliability and validity of the instrument. The members of that advisory committee with specific content responsibilities as at 1-3-96 are Gavin Andrews (Chair), Linda Cottler, Mohan Isaac, Ron Kessler, Lorna Peters, Charles Pulls, Lee Robins, Maritza Rubio-Stipec, Rob Smeets, Claudio Torres de Miranda, and Uli Wittchen, with Bedirhan Ustun as the responsible scientist from WHO, Geneva. The present version, CIDI 2.1, addresses both DSM-IV and ICD-10 criteria and is available in lifetime and twelve-month versions.

During a CIDI interview, respondents are asked questions about symptoms of psychiatric disorders. Positive responses to symptom questions are followed by questions from the Probe Flow Chart which determine whether the symptom is a possible psychiatric symptom (i.e., it is clinically significant and is not due to medication, drugs or alcohol or to a physical illness or injury). Negative responses to symptom questions will often lead to later questions being skipped. If enough symptoms have been endorsed, and these symptoms occur in a pattern which suggests a diagnosis might be present, respondents are asked about the onset and the recency of the particular cluster of symptoms which they have endorsed. Administration of the interview requires training in following skip instructions, in invoking the Probe Flow Chart when necessary, in assembling lists of the symptoms which have been endorsed for the onset and recency questions, and in the use of the data entry and scoring program. Training in administering this structured interview is conducted at nine WHO endorsed centres around the world.

The paper-and-pencil version of the CIDI has been shown to be reliable in a large international field trial (10, 11) and in a number of smaller studies (12). Similarly, the validity of the CIDI has been demonstrated in a number of small scale studies (12). Despite the acceptable reliability and validity of the paper-and-pencil version the CIDI, the extensive training required, its lengthy administration time (an average of 75 minutes in a general population sample), and the clerical time required for data entry and scoring are barriers to its use in routine clinical practice.

The Computerised Interview: the CIDI-Auto

The structure and logic of the CIDI facilitates computerisation because the computer can implement the skip patterns and the probe flow chart decisions and assemble symptom texts. Most persons, even those who are not computer literate, can complete the interview themselves, without the need for an interviewer. In addition, having the data from the interview fed directly by the computer to scoring algorithms removes the clerical errors which can be made during data entry. Thus, the computerised CIDI-Auto is a low cost means of confirming psychiatric diagnoses, whether in epidemiological surveys or in clinical practice.

The computerised version of the CIDI is known as CIDI-Auto (13). CIDI-Auto can be self-administered by the respondent, or administered by an technician interviewer who reads the questions as they appear on the screen. CIDI-Auto is a faithful representation of all modules of the CIDI interview, with the wording of questions that appear on the screen being identical to that of the paper-and-pencil interview, and the probe flow chart and skip decisions being implemented by the program. The coded responses to all questions are written to a file in a form that allows them to be scored using the same computerised scoring algorithms as are used for the paper-and-pencil interview. Other output files contain the information about diagnoses met (in either text or data form) and about all of the responses made to the questions throughout the interview. The files containing information about responses made during the interview are in two forms: one that is suitable for printing out as a hard copy and one that is suitable for reading into a database or other statistical package. The principal advantages of the CIDI-Auto over the paper-and-pencil CIDI are increased reliability of administration and significant reduction in staff time required to administer the interview and to enter and score the data. CIDI-Auto is currently being used for research, training and clinical support. The CIDI-Auto is available in Dutch, English, French, German, Portuguese, and Spanish. Other language versions are in preparation.

The program is supplied to users in a compiled form with a detailed Administrator's Guide and Reference Manual that covers issues from installation and running of the program to procedures for backing up data. Each copy of the CIDI-Auto is licensed to an individual at an institution. The licensee may make copies of the program for colleagues at that institution, provided that the licensee can vouch for the appropriate use of the program and for the confidentiality of the data. To date, 140 licences have been sold to users in Australia, New Zealand, the United States, the United Kingdom, Norway, Iceland and South Africa. Source code for the CIDI-Auto has been released to eight approved translators of the CIDI-Auto in other countries. The source code is fully documented. Once installed, the CIDI-Auto programs occupy approximately 1.5 Mb of disk space. In addition, the CIDI-Auto requires a further 250Kb of disk space and 500Kb of free RAM in order to run.

Testing And Maintenance

The third version of CIDI-Auto, version 2.1, was released in January 1997 and includes the Cognitive Impairment Module and the new Post Traumatic Stress Disorder Module. The source code for the current version consists of approximately 250 files containing about three million characters. Obviously error can occur in such a large document. The programs used to turn source code into executable programs can identify almost all errors in syntax but errors in logic can only be detected by thorough field testing. The testing of the programs involved staff who were familiar with the CIDI spending long hours at the computer screen to determine whether the CIDI-Auto program accurately implemented the paper and pencil interview and whether the outputs of the program accurately reflected the inputs. This involved running through modules many times giving different combinations of responses to test that the probe flow chart was implemented correctly, that appropriate skips were made, and that the correct symptom lists were compiled.

Every program requires maintenance, to correct errors in the code, and to improve awkward sequences or to add features. The CIDI-Auto has not been an exception. Since version 1.1 was released the detected fault rate has been less than 0.001 per function point per year, indicating that the software complies with the ISO-9000 standards (14). The program currently runs under the DOS operating system in or out of a Windows environment.

Training And Support

The administrator's guide and reference has been written so that a user who has a limited amount of knowledge of the DOS environment (e.g., is able to create directories and copy files) can install and run the program. Telephone, Fax or email support is provided by the Sydeny Centre. The staff providing support have knowledge of computers in general, of the CIDI interview, of the CIDI-Auto program, and can offer help on some broader research issues.

Patient Acceptance

Acceptability of computerised interviews has been examined for the computerised version of the DIS. Levitan et al. (15) reported that patients who completed the self-administered computerised DIS rated it as easy to operate and efficient. Similarly, Greist et al. (16) reported that patients who completed both the self-administered computerised version of the DIS and paper-and-pencil DIS found the computerised administration of the interview acceptable. In particular, Greist et al. (16) found that patients felt more comfortable answering the questions on the computer than answering them to a human interviewer and our experience with the CIDI-Auto is consistent with this. The computerised version of the CIDI also appears to be quite acceptable to patients whether the interview is self-administered or administered by an interviewer. Many patients report that the interview provides them with an opportunity to reveal symptoms about which they have never before been asked. At the Anxiety Disorders Unit at St Vincent's Hospital, Sydney, the CIDI-Auto is completed by every patient who has been accepted for treatment. We now have over 500 patients who have completed the CIDI-Auto in its self-administered form. No one has refused or complained.

Patients are provided with 12 screens of tutorial instructions prior to embarking upon the interview that provide information about the types of questions that the patient will be asked and how the patient is to answer those questions. There are interactive examples which allow the patient to practise making responses to the different sorts of questions they will be asked. These interactive examples also allow patients who are unable to comply with the instructions on the screen to be identified. Once the tutorial has been completed patients can be left to complete the interview at their own pace. There is the opportunity for the patient to take short breaks at the end of each module, and when the interview is completed, an instruction will appear on the screen for the patient to call the interview administrator.

Reliability And Validity Of The CIDI-Auto

Like any new assessment instrument, the CIDI-Auto must be tested for its reliability (the consistency of the diagnoses made across time, and for the interviewer-administered version, across interviewers) and its validity (the accuracy of the diagnoses made). The test-retest reliability of the CIDI-Auto version 1.1 was investigated in two sites (Sydney, Australia and San Juan, Puerto Rico) as part of the WHO reliability and validity study (17) and the results were excellent. In a recent comparison of the CIDI-Auto with the standard administration of the paper-and-pencil CIDI, Peters (in preparation) found that the agreement between CIDI-Auto (self-administered) and CIDI delivered by a human was good to excellent with Kappas ranging from 0.65 for Social Phobia to 0.83 for Panic Disorder. The subjects found the computerised format acceptable, with the majority saying that their comfort level was equal for the computerised and human delivery formats. Fewer subjects found the computerised format embarrassing. In another comparison at this Centre the draft of version 2.0 was adminstered on two occasions a week apart to 150 volunteers. The test retest reliability Kappas were excellent to acceptable for the vast majority of items. The items which were not reliable were identified and rewritten for the final version.

The validity of any psychiatric diagnostic interview is more difficult to assess because, as noted earlier, the standard against which the structured interview is compared, a clinician's diagnosis, is itself not perfectly reliable. In order to enhance the standard against which structured diagnostic interviews are compared, Spitzer (20) has suggested use of a LEAD standard diagnosis. LEAD is an acronym representing the components of the clinical diagnosis: information is collected over a LONGITUDINAL period by EXPERTS who come to a consensus diagnoses on the basis of ALL DATA available to them. A comparison between the self-administered version of the CIDI-Auto and LEAD standard diagnoses has recently been conducted (21). The results of that study show that the CIDI-Auto detects 88% of the LEAD standard diagnoses, but that agreement between the CIDI-Auto and the LEAD standard diagnoses (overall kappa = 0.40) is lower that expected. This is because the CIDI-Auto identifies more diagnoses than do clinicians, but nevertheless the Kappa values are similar to that found for comparisons between the paper-and-pencil CIDI and clinician diagnoses (12). The CIDI-Auto has acceptable validity.

Current Work

The CIDI was designed for lay interviewers in epidemiological surveys. The CIDI-Auto, because no clinician time is involved, is a promising, and indeed is the only contender for a second opinion instrument for clinical practice. Even though hundreds of copies are in use around the world there are things that need to be finalised before one can accept that this instrument has won an accepted place as a laboratory test for psychiatry.

Together with Dr Pull in Luxembourg we have begun to test the validity of the CIDI-Auto module for the schizophrenic disorders by comparing it with the results of semi-structured clinician based diagnostic interviews like the Structured Clinical Interview for DSM and the Schedules for Clinical and Neuropsychiatric Assessment.

We are in the process of exploring the possibility that computer logic could shorten personality disorder interviews such as the International Personality Disorder Examination and make such modules part of the CIDI-Auto.

We have shortened the administration time in the anxiety and affective disorder sections by skipping to the next section whenever the criteria for a diagnosis cannot be met. We are exploring the possibility of doing the same with the other sections.

Structured diagnostic interviews always allow people to identify the presence of more comorbid disorders than clinicians do. We need to develop a report structure that will mark the principal diagnosis identified by the patient.

Dr Kessler developed a brief ten minute version of CIDI 1.1. With him, we are currently updating this to match version 2.1. and will make a CIDI-Auto version available once the necessary reliability and validity studies are complete. We anticipate that it will be widely used in screening surveys and in primary health care.

We have developed a survey instrument whereby the CIDI-Auto can be routinely administered, in conjunction with measures of symptoms, disability, health service utilisation and perceived health needs. This was designed for use in health surveys or to be incorporated into clinical information systems to serve the needs of routine consumer outcome measurement.

We have yet to demonstrate something that has seldom been done, even for common laboratory investigations, that the use of the CIDI-Auto in routine clinical practice significantly improves patient outcomes. This should be done.

Conclusion

The CIDI-Auto is a computerised diagnostic interview that has become the standard way of conducting epidemiological surveys, simply because computerisation reduces errors in administration and reduces the costs of editing and data entry. Epidemiological surveys that do not use CAPI instruments will soon be things of the past. In addition to providing a cheap and reliable research tool, CIDI-Auto provides the opportunity for improving diagnostic accuracy in routine clinical practice. This computerised interview could be used in psychiatry as laboratory tests are used in other branches of medicine. Improved diagnostic accuracy should lead to better treatment and thus, to more effective patient outcomes. Click here Contact Us for enquiries about the CIDI-Auto:

A recent abstract on the psychometric properties of the CIDI

References

  1. American Psychiatric Association. (1994). Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition). Washington, DC: APA.
  2. World Health Organization. (1993). The ICD-10 Classification of Mental and Behavioural Disorders: Diagnostic Criteria for Research. Geneva: WHO.
  3. American Psychiatric Association. (1980). Diagnostic and Statistical Manual of Mental Disorders (Third Edition). Washington, DC: APA.
  4. Robins, L.N. (1989). Diagnostic grammar and assessment: Translating criteria into questions. Psychological Medicine, 19, 57-68.
  5. World Health Organization. (1992). Schedules for Clinical Assessment in Neuropsychiatry. Geneva: WHO.
  6. World Health Organization. (1993). Composite International Diagnostic Interview - Version 1.1. Geneva: WHO.
  7. Robins, L., Helzer, J.E., Croughan, J. & Ratcliff, S. National Institute of Mental Health Diagnostic Interview Schedule, 38, 381-390.
  8. Robins, L.N., Regier, D.A. (1991). Psychiatric disorders in America. New York: MacMillan.
  9. Kessler, R.C., McGonagle, K.A., Zhao, S., Nelson, C.B., Hughes, M., Eshleman, S., Wittchen, H.U. & Kendler, K.S. (1994). Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States: Results from the National Comorbidity Survey. Archives of General Psychiatry, 51, 8-19.
  10. Cottler, L.B., Robins, L.N., Grant, B.F., Blaine, J., Towle, L.H., Wittchen, H.U., Sartorius, N., and the participants in the WHO/ADAMHA Field Trial. (1991). The CIDI-Core substance abuse and dependence questions: Cross-cultural and nosological issues. British Journal of Psychiatry, 159, 653-658.
  11. Wittchen, H.U., Robins, L.N., Cottler, L.B., Sartorius, N., Burke, J.D., Regier, D., & participants in the multicentre WHO/ADAMHA field trials. (1991). Cross-cultural feasibility, reliability and sources of variance of the Composite International Diagnostic Interview (CIDI). British Journal of Psychiatry, 159, 645-653.
  12. Wittchen, H.U. (1994). Reliability and validity studies of the WHO Composite International Diagnostic Interview (CIDI): A critical review. Journal of Psychiatric Research, 28, 57-84.
  13. World Health Organization. (1993). CIDI-Auto Version 1.1: Administrator's Guide and Reference. Sydney: Training and Reference Centre for WHO CIDI.
  14. Jones, C. (1994). Assessment and control of software risks. Englewood Cliffs, NJ: Prentice Hall.
  15. Blouin, A.G., Perez, E.L. & Blouin, J.H. (1988). Computerized administration of the Diagnostic Interview Schedule. Psychiatry Research, 23, 335-344.
  16. Greist, J.H., Klein, M.H., Erdman, H.P., Bires, J.K., Bass, S.M., Machtinger, P.E. & Kresge, D.G. (1987). Comparison of computer- and interviewer-administered versions of the Diagnostic Interview Schedule. Hospital and Community Psychiatry, 38, 1304-1311.
  17. World Health Organization. (1993). WHO/NIH Joint Project on Diagnosis and Classification of Mental Disorders, and Alcohol- and Drug-related Problems: Reliability and Validity Study. Geneva: WHO.
  18. Levitan, R.D., Blouin, A.G., Navarro, J.R. & Hill, J. (1991). Validity of the computerized DIS for diagnosing psychiatric inpatients. Canadian Journal of Psychiatry, 36, 728-731.
  19. Erdman, H.P., Klein, M.H., Greist, J.H., Skare, S.S., Husted, J.J., Robins, L.N., Helzer, J.E., Goldring, E., Hamburger, M. & Miller, J.P. (1992). A comparison of two computer-administered versions of the NIMH Diagnostic Interview Schedule. Journal of Psychiatric Research, 26, 85-95.
  20. Spitzer, R.L. (1983). Psychiatric diagnosis: Are clinicians still necessary? Comprehensive Psychiatry, 24, 399-411.
  21. Peters, L. & Andrews, G. (1995). A procedural validity study of the computerised version of the Composite International Diagnostic Interview. Psychological Medicine, 25, 1269-1280.

Edited by Gavin Andrews MD, UNSW, Jan 03
© 2007 CRUfAD

 

HealthInsite is Australia's online gateway for easy access to quality health information

ad
legals