EFFECTIVE DECISION SUPPORT
IN THE NEONATAL INTENSIVE CARE UNIT

RESEARCH PROPOSAL
 

INTRODUCTION
RELATIONSHIP TO EXISTING WORK
OBJECTIVES
OVERALL METHODOLOGY
DETAILED METHODOLOGY
ACCESS TO VOLUNTEER STAFF
ETHICAL CONSIDERATIONS
REFERENCES

INTRODUCTION

The interpretation of clinical patient data is the cornerstone of medical practice. In the modern Intensive Care Unit the data now available includes signals sampled every few seconds (such as heart rate, blood pressure, O₂ and CO₂ levels in the blood, respiration rate, etc.), other respiratory channels and laboratory results. The volume of these data has increased to the point that those responsible for clinical care (the nurses and doctors) are unable to make full use of it. It has been shown that simply displaying complex time series data does not automatically lead to improvements in patient care (McIntosh et al. 1994, Cunningham et al. 1998, McIntosh and Becher 1999). The recent COGNATE project (Logie et al. 1997; Alberdi et al. 1999a, 1999b, 1999c) concluded that some assistance in the form of additional data processing is necessary to support the decisions made by the clinical staff.

Decision support systems exist to assist decision-makers in achieving task-based performance that is as close to optimal as possible. If such systems are to be effective, their designers must be aware when the decision-maker's performance is most often or most critically sub-optimal; there is little point in investing effort in situations where the decision-maker has been demonstrated to be consistently competent. In addition, in a complex environment in which there are decision-makers with different rôles (e.g. consultants, junior doctors and nurses), the notion of optimal performance needs to be defined for each rôle - decision-makers with different types and levels of responsibility will have different types and levels of performance expected of them.

In this project we propose to identify these situations where sub-optimal performance tends to occur, to develop a number of data processing algorithms aimed at alerting the clinical staff to those situations, and to evaluate which approaches are most effective in bringing about improvements in performance. Our domain of study will be the Neonatal Intensive Care Unit (NICU) at the Simpson Maternity Hospital (Edinburgh). However we expect the outcomes of the project to be applicable in other domains where staff have to make rapid decisions informed by complex time-series data.

RELATIONSHIP TO EXISTING WORK

A number of studies have dealt with mechanisms for reporting critical incidents in the ICU (Beckmann et al. 1996, Buckley et al. 1997, Wright et al. 1991); analysis of these reports identifies human error as by far the most significant factor.

All three applicants have been active collaborators in the recently completed COGNATE project (ESRC funded: L 127 25 1019) which examined the reasoning processes of clinical staff on a NICU who had access to cotside computers displaying complex time-series data. Although it was not the goal of COGNATE to look at all aspects of clinical reasoning in the NICU, it indicated certain areas where performance is sub-optimal. COGNATE showed that those who spend most time on the unit in direct contact with the baby (the nurses and the junior doctors) do not make effective use of the computer system nor are they well equipped to interpret the data displayed when they do so - even after extended exposure to the system. However in COGNATE we concentrated on the identification (or diagnosis) of patient state. This project will focus on the decisions made as a result of this interpretation as manifested by the resulting actions. It should be noted that very little resource was available within COGNATE for the development of computer-based decision support algorithms. However we were able to implement a software platform - the Time Series Workbench (TSW) - which is designed for experimentation with ICU time series data. The TSW will be used extensively in this project.

Patient data recording has been in place for the past ten years in the Simpson NICU. As a result a large database of time-series information from more than 2000 intensive care babies is available. Using this we have established reference centiles of physiological data for infants in the NICU (based on both gestation and postnatal age) and have information on how illness and conditions of prematurity affect the physiology. This knowledge of the normal variability of physiology in high risk infants is unique and is fundamental to a statistical evaluation of physiological data in order to differentiate physiology, pathology and artefact. We have developed a number of techniques to identify respiratory events using transcutaneous measurements of O₂ and CO₂. Using a moving window, various detectors (based on limits, deviations and correlations) have been developed in Edinburgh and Boston to detect the presence of artefact and pneumothorax (Cao et al. 1998, 1999a and 1999b, McIntosh et al. 2000, Tsien et al. 2000). A somewhat different approach using interval merging and rules derived from clinical experts has been developed in Aberdeen; it has been applied to the detection of the probe-changes needed when employing the combined transcutaneous O₂ and CO₂ electrodes in the infant (Salatian and Hunter 1999, Hunter and McIntosh 1999). In addition we are in close contact with the group at the University of Vienna who developed the VIE-VENT system for advice on weaning newborn infants from ventilators; they too have developed other techniques for signal validation (Horn, Miksch et al. 1997).

It is highly significant that the Badger monitoring system, developed by Edinburgh Telemedicine Solutions, has been introduced in the Simpson NICU since the end of the COGNATE project. This new system will provide us with access to additional detailed time-series data from the ventilator and to respiratory function data. The TSW is already capable of handling data captured by the Badger system.

The applicants have worked together successfully since 1996. Given modern electronic communications, JANET and good rail and road links, the physical separation of the groups has not provided any significant obstacle. We believe that our collaboration which brings together groups with considerable expertise in Artificial Intelligence and Cognitive Psychology as applied to Medicine, and a clinical group with immediate access to large volumes of on-line neonatal ICU data, is unusual if not unique.

OBJECTIVES

Our goals in this project are:

• To investigate on a systematic basis, a wider range of situations in the NICU than hitherto studied where the actions taken are sub-optimal;

• To use the results of this investigation to implement and evaluate computerised aids which, in these situations, will support the decision-making process leading to more appropriate actions;
• To identify verbal descriptors of patient state used by staff at different levels and types of expertise;
• To identify those elements of the above activities which have a more general relevance for Cognitive Engineering in medicine and related areas.

1. Collect detailed information on a number of real patients, (heart rate, blood pressure O₂ and CO₂, laboratory results, appearance, etc) and on the actions that the attending staff (nurses, junior and senior doctors) take (ordering further tests, alterations to ventilator settings, administration of drugs, etc.). Some of this information will be collected automatically by the Badger system, the remainder by a suitably qualified and trained research nurse.
2. Based on this information, identify examples of sub-optimal performance and select a subset of the data for further study.
3. Present this subset to a larger number of clinical participants in `off-ward' experiments, inviting them to `manage' the patient and to record the actions that they would have taken.
4. Investigate in greater detail the situations which appear to give rise to sub-optimal performance and to develop and implement computer-based algorithms to support clinical decisions in these situations.
5. Evaluate the effectiveness of these procedures by carrying out step 3 again with the same (or equivalent) group of clinical participants but with decision support tools in place; check for any improved performance.

If these judgements are to be made `off-ward' then the person making the judgement must have access to information which is as close as possible to that available on the ward.

We could limit ourselves to `off-ward' judgements on `on-ward' decisions, but this in turn raises the following issues:

• We believe that cases of sub-optimality will be, if not rare, then at least infrequent; infrequent events can, of course, have extremely serious consequences. By selecting an appropriate subset of the data in order to focus on the more frequent and/or serious incidents and by conducting `off-ward' experiments in which a larger number of subjects are exposed to each sample in the subset, we should be able to improve the statistical power of our data.
• We are reluctant to test computerised decision support on real patients before evaluating them `off-ward' first. If the algorithms are successful, we would clearly want to proceed to a full `on-ward' clinical trial, but this would be outwith the scope of the current proposal.

The different phases of the project are described in more detail in the following sections.

DETAILED METHODOLOGY

Phase A: Prepare for data collection (Months 1-6): During this phase we will take all necessary steps to prepare for the on-ward data collection.

• Establish and codify the actions which the decision-makers - nurses (junior and senior) and junior doctors - are expected to take by reviewing the COGNATE data and interviewing participants, at all levels of medical and nursing responsibility. We are not concerned at this stage with the detailed circumstances under which these actions should be taken, but rather with building up a lexicon of the actions and whether they would be taken on a routine basis or as a result of detecting adverse changes in patient state. It is clear that different grades will have different actions expected of them. Actions will include requests for tests (e.g. X-rays) which are only available on demand.
• Augment the TSW to allow hand entry of the above data.
• Identify the different types of patient which are to be the focus of attention of the research nurse. We need to achieve a balanced mixture of stable babies to and babies which are likely to pose complex problems.
• Establish clear guidelines for the data collection activities of the research nurse and train the person appointed.

It is important to note that despite the extensive data available in the Simpson archive, it is not suitable for this project in that it is not as comprehensive (in terms of the different sources of data recorded), nor is it annotated with the events which actually took place on the ward. The data used in COGNATE were not collected specially for that project but were taken from the archive.

Phase C: Develop schemes for codifying visual and tactile data (Months 7-12): This phase will run in parallel with Phase B and will focus on the visual and tactile information derived from physical examination of the baby. As part of the routine data collection, the research nurse will ask staff to provide brief verbal descriptions for these physical aspects of the baby’s condition. This process will continue over a period of at least three months to ensure that multiple exemplars are collected for a range of physiological conditions and from several junior and senior nurses and physicians. Subsequently, the different verbal descriptors will be shown on cards in different random orders to different grades of staff who will be asked to sort the cards into groups according to their judged similarity/degree of association. The groupings identified will be analysed to detect semantic clusters of terms that will indicate how the descriptor terms are mentally organised and used by nurses and physicians who have different roles on the ward. We will then use the results of this phase to code the verbal descriptors for individual babies.

Phase D: Carry out preliminary analysis and selection (Months 7-12): The senior clinician on the project will review the data on a regular basis using the TSW. He will annotate any actions (and absences of action) that he considers to be sub-optimal and attach a degree of seriousness to each event; these annotations will provide our `gold standard'. Towards the end of the period he will review all these annotations and select the data periods that are to be used in Phase E.

Phase E. Conduct and analyse off-ward experiments (Months 13-18): The data collected above will contain as accurate a picture as possible of the time-varying state of the patient. The TSW will be used to replay the data in a controlled manner to other junior and senior nurses and junior doctors who will be instructed to use the interactive facilities of the system to enter any actions that they would have taken; direct entry will avoid the overhead of manual transcription incurred in COGNATE. The information presented will consist of that normally available - it will not include the results of tests that have to be specially ordered nor the actions taken by the on-ward staff. The subjects will only be able to enter actions at the time of the last data available to them - they will not be allowed to `look into the future'. They will be able to ask to see any actions taken on the ward (and the results of that action if it is to order a test), but such a request will be noted and will prohibit the requester from taking the same action at that time.

We will then analyse the actions (and absences of action) in the light of the gold standard (taking degree of seriousness into account) in order to identify those situations where performance is sub-optimal and where decision support should be concentrated. These experiments will also provide the reference results for the experiments carried out in Phase G. They will differ from those carried out within COGNATE in that descriptors of appearance and the results of physical examination will be available. However it may be possible to capitalise on these differences to study the extent to which the provision of these descriptors improves the identification of critical events compared with the performance observed in COGNATE.

Phase F: Design and implement decision support software (Months 13-27): In developing the decision support software which could ultimately be incorporated into the on-line Badger system, we must limit ourselves to using only those data which will be collected automatically by that system (i.e. excluding the data collected specially as part of this project by the research nurse). This software will be designed to assist users in improving their performance in situations identified in Phase E; it will need to recognise these situations and then to draw them to the attention of the clinical staff. In such a complex environment, we must take care that an automatic decision support system respects the fact that the clinical staff will have access to information denied to the system (in particular the results of observation and examination).

In computational terms, this problem can be viewed as one of generating higher level abstractions of the raw data. Tasks include the identification and removal of artefacts (e.g. probe changes, handling, suction), and noise, identifying meaningful trends in single channels, defining and recognising across multiple channels the patterns which characterise events of clinical significance (e.g. blocked tube, pneumothorax, etc.). As a result of the COGNATE project and other work, we have already made considerable progress on all of these fronts, but they need to be brought together and linked to the appropriate displays. The knowledge of what was actually happening to the patient at all times (mainly in terms of the actions taken) will be invaluable in developing and testing these algorithms.

Possible ways of alerting the clinical staff include:

• the enhancement of different aspects of the display (e.g. by drawing best fit straight lines to draw attention to trends which might otherwise be missed);
• the use of mimic diagrams or metaphor graphics (e.g. Green, Logie, Gilhooly et al. 1996; Horn et al. 1998);

• displaying appropriate pre-stored clinical guidelines.

Phase G: Evaluate the effectiveness of the decision support aids (Months 28-33): With the decision-support software installed on the TSW, we will re-run the same off-ward experiments as were carried out in Phase E to see if the overall performance of nursing and medical staff at various grades is improved by the presence of this software. As in Phase E, performance will be measured by comparison with the gold standard established in Phase D. Wherever possible we will invite the same subjects, but rotations among junior doctors will make this difficult, and we will attempt to test subjects with similar amounts of experience. When the subjects are the same, we consider that sufficient time will have elapsed between the experiments that recall is unlikely to be a problem.

Phase H: Write up and disseminate results (Months 31-42):We plan to make the results of this research available to the Artificial Intelligence, Medical and Cognitive Psychology communities through publication in peer-reviewed journals such as: Artificial Intelligence in Medicine, Pediatric Research, International Journal of Clinical Monitoring and Computing, Cognitive Science, International Journal of Human-Machine Interaction, etc., and at suitable conferences such as: Conference on Artificial Intelligence in Medicine (biennial meetings in Europe), Symposium on Computer Applications in Medical Care (annual meetings in the USA), Conference of the American Pediatric Research Society, Neonatal Society (UK), Royal College of Paediatrics and Child Health, Experimental Psychology Society Conference (UK), European Society for Cognitive Psychology. We request that the formal duration of the project be extended for six months beyond the main activities; this will allow the project to fund the applicants to present the results at the appropriate national and international conferences in the spring/summer of 2004.

We would expect that successful decision support software would be incorporated into the real time Badger system for routine use and further evaluation. Given that this system and the TSW use the same underlying technology, transfer will be not be difficult and there is therefore a significant likelihood that at least one commercial organisation will exploit our results.

ACCESS TO VOLUNTEER STAFF

Through the Senior Clinician we shall have access to staff in the Simpson Maternity Hospital ICU. The staff will be asked to participate on a voluntary basis outwith their normal working shift. Junior medical staff and nursing staff will be offered a small honorarium for their participation. A number of the senior staff have already indicated their willingness to take part in the project on a purely voluntary basis. We anticipate no difficulties in accessing a sufficient number of volunteers, but individuals will be free to decline our invitation to take part. We will in addition have access, through applicant McIntosh, to a range of other neo-natal intensive care wards elsewhere in the UK in which the card sort of the verbal descriptors (Phase C) will also be carried out to maximise the size of the participant sample at each staff grade and to seek an element of generalisability in the project.

ETHICAL CONSIDERATIONS

There will be no direct contact with patients but real time patient data will be used. The data will be depersonalised and the Reproductive Sub-committee of the Ethics Advisory Committee of the Lothian Health board accepted that this did not require individual patient consent. The data will be handled appropriately as recommended by the Data Protection Act. The clinical medical and nursing staff will be fully informed of the reasons for the research. Ward observation periods will be with their co-operation. Outside the ward, at the off ward interviews and experiments, they will give their time voluntarily. This procedure worked well and with no problems for the COGNATE study. For data collection in Phases B and C parental consent will be sought when each baby is first admitted to the ward and again after the information has been recorded. The information will be matched with computerised data recorded for that baby, and the baby identity will then be removed for subsequent use of the data in later Phases. In no case will any recording of information from staff on the ward be allowed to interfere with delivery of medical or nursing care to the baby concerned.
 

REFERENCES

Alberdi E, J-C Becher, KJ Gilhooly, JRW Hunter, RH Logie, A Lyon, N McIntosh and J Reiss, 'Decision Support in the Neonatal Intensive Care Unit: Expertise Differences in the Interpretation of Monitored Physiological Data', in Engineering Psychology and Cognitive Ergonomics: Integration of Theory and Application, D Harris (Ed.), Ashgate, pp 397-404, 1999a.

Alberdi E, Gilhooly KJ, Hunter JRW, Logie RH, Lyon A, McIntosh N and Reiss J, 'The Role of Computerised Monitoring in neonatal Intensive care: A Cognitive Engineering Approach', submitted to International Journal of Clinical Monitoring and Computing, 1999b.

Alberdi E, Becher J-C, Gilhooly KJ, Hunter JRW, Logie RH, Lyon A, McIntosh N and Reiss J, 'Expertise and the Interpretation of Computerised Physiological Data: Implications for the Design of Medical Decision Support in Neonatal Intensive Care', submitted to International Journal of Human Computer Studies, 1999c.

Beckmann U, West LF, Groombridge GJ, Baldwin I, Hart GK, Clayton DG, Webb RK and Runciman WB, 'The The Australian Incident Monitoring Study in Intensive Care: AIMS-ICU. The Development and Evaluation of an Incident Reporting System in Intensive Care', Anaesth Intensive Care, Vol 24(3), pp 314-9 1996.

Buckley TA, Short TG, Rowbottom YM and Oh TE, 'Critical Incident Reporting in the Intensive Care Unit', Anaesthesia, Vol 52(5), pp 403-9, 1997.

Cao C and McIntosh N, 'Empirical Study of Artifact Identification in Clinical Monitoring Data', Proceedings of the 1998 Annual American Medical Informatics Association Symposium, pp 983-983, 1998.

Cao C, McIntosh N and Kohane IS, 'Artifact Detection in the Cardiovascular Time Series Monitoring Data from Preterm Infants', Proceedings of the 1999 Annual American Medical Informatics Association Symposium, pp 207-211, 1999a.

Cao C, McIntosh N and Kohane IS, 'Artifact Detection in the PO₂ and PCO₂ Time Series Monitoring Data from Preterm Infants', International Journal of Clinical Monitoring and Computing, Vol 15, pp369-378, 1999b.

Cunningham S, Deere S, Symon A, Elton RA and McIntosh N, 'A Randomized, Controlled Trial of Computerized Physiologic Trend Monitoring in an Intensive Care Unit', Crit.Care Med, Vol 26, pp 2053-2060, 1998.

Green CA, Logie RH, Gilhooly KJ, Ross DG, Ronald A, 'Aberdeen Polygons: Computer Displays of Physiological Profiles for Intensive Care', Ergonomics, Vol 39, pp 412-428, 1996.

Horn W, Popow C and Unterasinger L, 'Metaphor Graphics to Visualize ICU Data over Time', Intelligent Data Analysis in Medicine and Pharmacology (IDAMAP-98), Workshop Notes of the ECAI-98 Workshop, Brighton, UK, 1998.

Horn W., Miksch S., Egghart G., Popow C. and Paky F., 'Effective Data Validation of High Frequency Data: Time-Point-, Time-Interval-, and Trend-Based Methods', Computers in Biology and Medicine, Vol. 27, No. 5, pp 389-409, 1997.

Hunter JRW and McIntosh N, 'Knowledge-Based Event Detection in Complex Time Series Data', AIMDM'99: Joint European Conference on Artificial Intelligence in Medicine and Medical Decision Making, Horn W et al. (Eds.), Springer Verlag, pp 271-280, 1999.

Logie R, Hunter JRW, McIntosh N, Gilhooly K, Alberdi E and Reiss J, 'Medical Cognition and Computer Support in the Intensive Care Unit: A Cognitive Engineering Approach', in Engineering Psychology and Cognitive Ergonomics: Integration of Theory and Application, D Harris (Ed.), Ashgate, pp 167-174, 1997.

McIntosh N, Becher JC, Stenson BJ, Laing IA, Lyon AJ, Badger P, 'The clinical diagnosis of pneumothorax is late: use of trend data and decision support might allow preclinical detection' Pediatric Research (in press) 2000.

McIntosh N, Cunningham S and Elton R, 'Computerised Trend Monitoring', Pediatric Research, Vol 36, 28A, 1994.

Salatian A and Hunter JRW, 'Deriving Trends in Historical and Real-Time Continuously Sampled Medical Data', Journal of Intelligent Information Systems, Vol 13, pp 47-71, 1999.

Tsien CL, Kohane IS and McIntosh N, 'Multisignal Integration by Decision Tree Induction to detect false alarms in the Intensive Care Unit', Artificial Intelligence in Medicine, in press, 2000.

Wright D, Mackenzie SJ, Buchan I, Cairns CS and Price LE, 'Critical Incidents in the Intensive Therapy Unit', Lancet, Vol 338(8768), pp 676-8, 1991.