Voice Recognition System Report Accuracy [QSI Ref: XR-508]

This audit is to assess the number of errors in voice recognition system generated reports due to the failure of software to correctly identify words and failure of user to spot error in text.

Voice recognition software has been around for many years but until the widespread introduction of PACS few radiology departments employed it. This has however changed with increasing numbers of departments using it for at least part of their report output. Performance of individual systems has improved in recent years and in part this is contributing to an increase in usage. All systems however, make errors in speech recognition and users need to be aware of problem areas and review reports carefully before finalising.

The standard: 

Voice recognition system generated reports should be accurate and clear.

Target: 

• An overall error rate of less than 5% in VRS reports

• There should be no major errors where the report makes no sense, i.e. target 0%

Indicators: 

Percentage of reports containing an error

Data items to be collected: 

Series of consecutive reports for each voice recognition system user. Ideally this should include all types of reports from plain film to more complex reports such as CT or MRI.

Suggested number: 

50 per user

• Review types of errors made

• Classify into minor - e.g. 'am' instead of 'an', moderate where incorrect words / syntax are used making interpretation more difficult and major where the report makes no sense.

• Feedback individual reports to users

• Circulate information on the common types of errors and issue advice on how to reduce / eliminate errors such as use of macros, minimizing interruptions and background noise, deferred review of reports

Report review - depends on size of department approx 1 hour per user for 50 reports.

1. The effect of voice recognition software on comparative error rates in radiology reports. S McGurk, K Brauer, T V McFarlane, K A Duncan Br J Radiol. 2008 Oct; 81(970): 767-70.

K A Duncan, S McGurk, K Brauer. Updated by K A Duncan & J Parikh