The content on this page is being archived for historic and reference purposes only. The content, links, and pdfs are no longer maintained and might be outdated.
Conducting Population Behavioral Health Surveillance by Using Automated Diagnostic and Pharmacy Data Systems*
Julie A. Pavlin,1 P.
Murdock,2 E. Elbert,1 C. Milliken,1 S. Hakre,1 J. Mansfield,1 C. Hoge1
Corresponding author: Julie A. Pavlin, Chief, Department of Field Studies, Walter Reed Army Institute of Research, 503 Robert Grant Ave., Silver Spring, MD 20910. Telephone: 301-319-9346; Fax: 301-319-9104; E-mail: email@example.com.
Introduction: The Walter Reed Army Institute of Research used the Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE) to conduct population-based behavioral health surveillance among military-health--system beneficiaries. The study analyzed the effectiveness of using prescribing patterns of psychotropic medications to monitor changes in a community's behavioral health status.
Objectives: The objectives of this study were to 1) determine the feasibility of tracking psychiatric illnesses by monitoring prescriptions for psychiatric medications; 2) assess how often psychiatric medications are prescribed for patients with no record of psychiatric illness; 3) determine at what types of clinics these medications are prescribed most often and what other diagnoses are attributed to these patients; and 4) analyze data for potential changes in the population's mental health after high-stress events.
Methods: Correlation analysis and calculations of sensitivity and specificity were used to determine how well prescription medications correlate with outpatient diagnoses and how well they serve as proxies for outpatient diagnoses. A descriptive analysis was conducted of the types of clinics (e.g., primary care, behavioral health, or other specialty clinics) treating patients and the associated percentage of concurrence between prescriptions and diagnostic codes.
Results: In military treatment facilities, a diagnosis of depression or anxiety correlated significantly (r = 0.82) with antidepressant or anxiolytic prescriptions. Sensitivity of prescriptions when compared with outpatient visits was 0.76, and specificity was 0.94. Among those patients who visited a primary care clinic either the day before or the same day as an antidepressant or anxiolytic prescription was filled, 60.1% did not receive a diagnosis of any mental health disorder. Behavioral health clinics had the highest correlation between diagnoses and prescriptions; specialty clinics had the lowest.
Conclusions: Behavioral health trends in a population can be monitored by automated analysis of prescribing patterns alone. This method might be a rapid indicator of needed mental health interventions after acute stress-inducing events and be more sensitive than tracking diagnoses alone.
New approaches to public health surveillance that use automated, and often unconventional, data sources have focused on the threat of emerging infections and biologic terrorism. However, other uses for these technologies exist beyond traditional surveillance of infectious disease. Mental health surveillance using de-identified data has the potential to estimate the prevalence of certain mental illnesses, especially among persons who are sensitive to events that cause stress in their communities (e.g., natural or man-made disasters, regional unemployment, or deployments at military bases). These systems can support planning for more resource-intensive traditional mental health surveillance activities (1) and signal a need for community-based mental health interventions that emphasize normalization of responses to stress.
Automated public health surveillance systems are an inexpensive and timely augmentation to traditional health surveillance methodologies and can enhance provider alertness to public health threats (2,3). By using routinely collected electronic data (4--6) from different traditional and nontraditional sources (e.g., administrative, clinical, pharmacy, and retail databases, and school and work absenteeism data) (4,7--11), such systems can detect increases in the number of cases above that normally expected, with varying degrees of specificity.
Certain mental illnesses might not require as urgent a response as infectious-disease outbreaks and therefore might not seem to justify use of automated data sources in surveillance. However, few, if any, active mental health surveillance tools exist, primarily because finding measures of mental health changes in a community is difficult. Thus, data on routine outpatient visits or pharmacy prescriptions might prove useful for determining a community's mental health status and assessing the effectiveness of interventions. Furthermore, communitywide increases in mental illness might require rapider intervention when illness could result in suicidal or homicidal behavior.
In 1997, the U.S. Department of Defense (DoD) instituted the Standard Ambulatory Data Record (SADR) to record demographic and diagnostic data on all military outpatient visits, including International Classification of Diseases, Ninth Revision (ICD-9) codes for each visit. In 1999, the Walter Reed Army Institute of Research (WRAIR) created the Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE) to detect and track infectious-disease outbreaks among military-health--system beneficiaries (12,13). Using the SADR database, ESSENCE automatically collects ICD-9 codes that potentially indicate infectious diseases and groups them into clinical diagnostic categories based on clusters of similar ICD-9 codes. Codes are grouped to reduce the variability and increase the sensitivity of administrative diagnostic data (14) and to improve baseline data-monitoring capability.
To study utilization of mental health services among military beneficiaries in the Washington, D.C., area after the September 11, 2001, attack on the Pentagon, WRAIR adapted the ESSENCE model to include psychiatric ICD-9 codes. Although no overall increase in utilization of mental health care was identified, the study did detect a significant change in the distribution of diagnoses, including relative increases in the median number of visits for adjustment reactions, anxiety, and acute-stress reactions during the first 5 months after the attack (15).
For the current study, groupings of mental health outpatient diagnostic data were correlated with pharmacy data and used to monitor changes in the mental health status of military communities. Diagnostic data based on ICD-9 codes might not be the best indicator of mental illness in a community, for multiple reasons. Anecdotal evidence published previously indicated that physicians might code only one diagnosis even when patients are seen for multiple conditions or make coding decisions based on codes most available or frequently used (10). In addition, diagnostic coding for mental health can be affected by stigma and employment culture. Stigma associated with a mental illness diagnosis is well-documented in the literature (1,16--20). In the military, a mental illness diagnosis can affect a service member's security clearance, flight status, and authorization to carry a weapon (17,19). A patient whose recorded diagnosis does not indicate a mental disorder might still receive a prescription for a psychiatric condition; therefore, prescriptions might be a better reflection of true mental health than the recorded diagnosis.
Pharmacy data provide insight into a clinician's treatment focus and might more accurately represent a patient's true condition. In addition, prescriptions are often renewed or refilled for chronic conditions regardless of the patient's primary complaint at the time of the visit (10,21). However, these data can be complicated by multiple indications for the same medication and are also sensitive to treatment setting (6). Because military patients, like other populations that have been studied systematically, receive a substantial percentage of their psychiatric care from primary care providers rather than mental health providers, measures of mental health treatment in primary care settings are needed.
This study's objectives were to 1) determine the feasibility of tracking psychiatric illnesses through the monitoring of psychiatric medication prescriptions by correlating diagnoses and the drugs prescribed; 2) assess how often psychiatric medications are prescribed to patients with no diagnostic record of psychiatric illness, particularly to estimate underreporting of psychiatric illnesses and determine whether pharmacy data might be a better indicator of mental health treatment; 3) determine at what types of clinics (i.e., primary care, specialty care, or behavioral health) psychiatric drugs are most often prescribed without corresponding mental health diagnoses, and identify what other diagnoses are attributed to these patients; and 4) evaluate whether any increases in anxiety or depression among family members of deployed military personnel could be detected.
Data were obtained for all outpatient visits at fixed military-treatment facilities (MTFs) and for prescriptions for all military beneficiaries during July 2001--August 2002. Approximately 8.8 million active-duty personnel, family members, and retirees are eligible for care of MTFs. Of this population, approximately 4.5 million are enrolled in the military's health-care system, Tricare Prime, which usually indicates they intend to receive care at MTFs (although some do access care outside of military hospitals and clinics). Those not enrolled in Tricare Prime can also receive treatment at MTFs on a different payment schedule.
The outpatient SADR database consists of <4 ICD-9 codes for every MTF visit. These codes are entered approximately at the time of the patient encounter, usually by the provider but also by professional coders at certain locations. The data include visits to all fixed MTFs worldwide but do not include deployed forces involved in military operations. All mental disorder ICD-9 codes in the 291--318 range, as well as related ICD-9 codes used by behavioral health clinics (e.g., mental health or substance abuse counseling; problems related to partner relationships, family circumstances, life circumstances, maltreatment, or abuse) were grouped according to established methods (22,23). In addition, a subset was created of all ICD-9 codes related to depression and anxiety, as those conditions would be more likely to increase during times of stress (Table 1).
The code for tension headache (307.81) was excluded from the analysis because this diagnosis is commonly used for headache unrelated to a mental disorder. The code for tobacco use disorder (305.1), which is included in the ICD-9 mental disorder category but not typically treated as a mental disorder, was also excluded. Deleting 305.1 also excluded use of certain antidepressants (e.g., bupropion) used as smoking cessation aids that could confound the analysis.
Outpatient pharmacy prescriptions at all MTFs and Tricare network pharmacies are collected in the Pharmacy Data Transaction Service database at the time they are filled (24). Prescriptions of medications used primarily to treat depression and anxiety (Table 2) were correlated with outpatient diagnoses. Certain medications were excluded to limit potential confounding factors. For example, trazodone, a potential antidepressant, is highly sedating and almost always used as a sleep aid. Hydroxizine has an anxiolytic indication but is almost always used for its antihistamine properties as an allergy medication. Amitriptyline is sedating and has cardiovascular side effects and is therefore rarely used as an antidepressant, although it is often used at low doses for pain conditions (e.g., headaches).
The strengths of correlations between antianxiety and antidepressant prescription medications and outpatient visits for mental health, anxiety, and depression were measured by using Pearson's correlation coefficient (25). Data were grouped by week to decrease the effect on the correlation of the usual weekly pattern of visits and prescriptions.
The two databases were then matched by using a code provided by Tricare that is uniquely assigned to each patient but does not allow patient identification. A match was determined for those patients who 1) had a new prescription written for one of the medications listed (Table 2) and 2) also had a recorded outpatient visit the day (or the day before) the prescription was written.
Prescriptions and outpatient visits were expected to have a correlation based on holiday and seasonal effects (e.g., fewer persons saw a health-care provider or were prescribed medications on holidays, compared with more persons during the winter influenza and seasonal affective disorder seasons); for this reason, the sensitivity, specificity, and positive predictive value of the prescription data were calculated by using outpatient visits both for depression and anxiety only and for all mental health concerns as the standard.
For those patients who were prescribed antidepressants or anxiolytics and who also had an outpatient diagnostic code from the same visit, the numbers of patients receiving depression or anxiety diagnoses, any mental health diagnoses, and all other diagnoses were calculated. The clinical setting was taken into account by grouping clinics into three categories: 1) mental health, 2) primary care (i.e., family practice, urgent-care clinics, emergency departments, internal medicine, and pediatrics), and 3) all other clinics (e.g., orthopedic, cardiology, or physical therapy). Prescription refills were not used in this analysis because the purpose of this surveillance was to detect acute psychiatric illness. However, certain prescriptions that appeared to be new and that were included might have represented dosage changes, brand changes, or renewals after all refills had been used (i.e., were not first-time prescriptions for the drug category).
Finally, outpatient visits and drug prescriptions among military beneficiaries were monitored to determine whether any increases in depression or anxiety had occurred; this was particularly relevant during 2003, when U.S. military deployments likely increased stresses on active-duty military and their families.
Although data for deployed forces were not available, data were examined from three installations from which substantial numbers of troops had been deployed to Iraq for Operation Iraqi Freedom (OIF). Trends in outpatient visits for anxiety and depression and filled prescriptions for antianxiety and antidepressant medications during July 2001--September 2003 were analyzed, by military beneficiary category, at all MTFs at the three installations. On any given day during surveillance, only the initial visit for anxiety and depression or the first prescription filled at the installation was included. To best reflect those who live at or near the installations, the analysis included only anxiolytics and antidepressants filled at pharmacies within a 50-mile radius of any of the three installations. The percentages of mental health visits and prescriptions for spouses (out of total outpatient visits or prescriptions) at that MTF were determined to decrease the effect of a changing population size. The Wilcoxon's rank-sum test (25) was used to test the alternative hypothesis that anxiety and depression visits and prescriptions differed significantly after January 9, 2003 (i.e., the date of deployment for OIF).
During July 2001--August 2002, a total of 2,343,684 anxiolytic and antidepressant prescriptions were written for 894,922 unique patients. A total of 1,588,081 outpatient visits for 408,083 unique patients were given an ICD-9 code for any mental health disorder, and 675,564 (42.5%) of these visits, representing 224,459 unique patients, were for depression or anxiety, as defined previously (Table 1). Records containing the code for tension headache (2,712) or tobacco use disorder (44,828) were excluded.
The correlation coefficient was 0.82 when only new prescriptions for anxiolytics and antidepressants were compared with diagnoses of depression or anxiety. The coefficient was 0.85 when prescriptions were compared with all mental health diagnoses. Including prescription refills in the analysis increased the correlation coefficient to 0.85 for diagnoses of depression or anxiety and to 0.88 for all mental health diagnoses.
Of all antidepressant or anxiolytic prescriptions, 934,220 (40.0%) matched with a recorded outpatient visit. This number includes 650,100 patients who received one or more prescriptions and had a matching outpatient visit, with 87% of visits occurring the same day as and 13% the day before the prescription was written. Of those prescriptions that matched an outpatient visit, 37.4% were for refills and the remainder for new prescriptions. For prescriptions that did not match an outpatient visit, the percentage attributable to refills increased to 54%.
In the sensitivity and specificity analysis, prescription data were relatively sensitive (0.76) and highly specific (0.94) (Table 3). However, the positive predictive value was low (0.26). This result was expected because visits for anxiety or depression are relatively rare (8.0%) compared with all outpatient visits. The result also reflects the substantial number of prescriptions given without a corresponding diagnosis of depression or anxiety. If prescriptions without a corresponding diagnosis truly represent a mental illness, then prescriptions might be a better indicator of mental illness than the gold standard of outpatient visits. Because the outpatient codes chosen are broad and medications are more specific, these findings probably are conservative, and the correlation between prescriptions and mental health diagnoses might be stronger. When the gold standard is expanded to include all mental health visits, the sensitivity decreases (0.52) and the positive predictive value increases (0.31), indicating that antidepressants and anxiolytics are not as sensitive an indicator of any mental health condition.
Among patients with a matched prescription and visit, 62.4% who were prescribed anxiolytics or antidepressants in primary care clinics did not receive a diagnosis of any mental health disorder (Table 4). Behavioral health clinics had the highest correlation between prescription and diagnosis, and other specialty clinics had the lowest, with 11.5% and 91.2%, respectively, having no mental health diagnosis. This discrepancy might differ in civilian health-care settings that link diagnostic codes more closely to reimbursement and prescription justification.
The majority of the diagnostic codes for patients receiving medications were codes for common medical illnesses (e.g., hypertension and diabetes) or generic codes for counseling. In addition, certain diagnoses (e.g., insomnia, myalgia, and myositis) were not psychiatric but could justify the prescriptions given. Nonpsychiatric diagnoses for which an antidepressant or antianxiety drug could appropriately be used (e.g., back and joint pain and strains, urticaria and rash, headache and migraines, counseling, or insomnia) constituted approximately 19% of the codes.
ICD-9-code and prescription data were then used retrospectively for surveillance of military-health--system beneficiaries. Although use of psychotropic medications increased gradually during July 2001--September 2003, since the start of OIF deployments on January 9, 2003, or the start of OIF hostilities on March 19, 2003, no acute increases in outpatient visits for anxiety or depression or for prescriptions across the total military beneficiary population were determined. However, if the data are grouped by beneficiary category and if military installations with higher rates of deployment are isolated, certain trends become apparent. The percentage of total outpatient visits attributed to depression or anxiety and the percentage of total prescriptions for antidepressants or anxiolytics for spouses at the three installations that had high rates of OIF deployment were calculated (Figure). The rates of both outpatient visits and prescriptions during January 9, 2003--September 25, 2003, differed significantly (p <0.0001 for both visits and prescriptions; Wilcoxon's rank-sum test) compared with the previous period of July 7, 2001--January 8, 2003.
The analysis demonstrated a strong correlation between mental health outpatient diagnoses and prescription of antidepressants and anxiolytics. It also indicated that additional patients are prescribed these medications without a corresponding diagnosis for depression, anxiety, or any mental disorder. This result is similar to findings published previously that 55% of Medicaid beneficiaries who received psychotropic medication did not receive a mental health diagnosis (21). These results indicate that tracking outpatient medications might be a more sensitive means for detecting changes in the mental health of a population.
However, multiple potential confounders exist. First, 24% of patients who had an ICD-9 diagnosis for anxiety or depression were not prescribed psychotropic medications; therefore, if only prescriptions are surveyed, those patients will be overlooked. Second, in certain instances, the matching of outpatient visits and prescriptions might not have been for the same encounter. For example, a patient could have had a telephone consultation or other provider interaction that resulted in a prescription; meanwhile, on the same day or the previous day, the same patient could have made an office visit for an entirely unrelated medical complaint. In such a case, the matching process would have linked the records and made the prescribed medication appear linked to an unrelated diagnosis. Such an error can make discrepancies between diagnosis and prescribing behavior appear greater than they are. However, such a situation also indicates that prescriptions might be a more sensitive surveillance tool, given that a prescription record exists despite no corresponding mental health diagnosis.
A third potentially confounding situation involves prescription of medications for disorders other than mental illness. Certain medical conditions (e.g., insomnia, pain conditions, urticaria, or migraine) can merit the prescription of antidepressants or anxiolytics. To adjust for this confounder, certain medications used more commonly for such conditions were removed and the analysis rerun. In that analysis, the percentage of patients receiving antidepressants or anxiolytics who had not been given a mental health-related diagnosis decreased by <1% in primary care clinics.
A fourth potentially confounding situation involves patients who take psychotropic medications for chronic conditions but who are treated for a different chief complaint during an office visit. A provider might code the visit accurately for the presenting complaint while also renewing the prescription for the chronic condition, which would make the number of coding errors appear greater and would decrease the specificity of using prescriptions for surveillance of acute events. Now that a longer historical record of patient visits and prescriptions is available, future studies will attempt to exclude from the analysis anyone who has ever received a medication in the anxiety or depression category.
Surveillance among military beneficiaries at three Army posts indicates that distress levels related to deployments might have increased in the population. Increases in mental health visits by military spouses were apparent. The increase in the rate of visits for anxiety or depression was greater than the increase in rate of psychotropic drug prescriptions. This finding highlights a potential limitation of relying on pharmacy data for mental health surveillance in a population. Deployment-related stress is common, and various counseling services that do not involve pharmacologic intervention are available to service members and families. Prescription-based indicators of distress might be less helpful in this context than they would be if a traumatic or terrorist event occurred in a population.
Automated analysis of prescribing patterns of psychotropic medications can be used to monitor behavioral health trends in a population. This surveillance method has potential to be a rapid and sensitive indicator of needed mental health interventions after acute stress-inducing events, especially in combination with surveillance of outpatient diagnoses. The importance of this surveillance is in its ability to react quickly to an increased need for mental health services. As with any other surveillance system that relies on data not originally gathered for surveillance purposes, any apparent increases in either prescribing behavior or outpatient visits should be verified by discussions with the provider or a review of records. If an increase in mental health needs is confirmed, early interventions can include community outreach and increased advertisement of available resources.
* The views expressed are those of the contributors and do not reflect the position of the U.S. Army or the U.S. Department of Defense.
Return to top.
Return to top.
Return to top.
Return to top.
Return to top.
Disclaimer All MMWR HTML versions of articles are electronic conversions from ASCII text into HTML. This conversion may have resulted in character translation or format errors in the HTML version. Users should not rely on this HTML document, but are referred to the electronic PDF version and/or the original MMWR paper copy for the official text, figures, and tables. An original paper copy of this issue can be obtained from the Superintendent of Documents, U.S. Government Printing Office (GPO), Washington, DC 20402-9371; telephone: (202) 512-1800. Contact GPO for current prices.**Questions or messages regarding errors in formatting should be addressed to firstname.lastname@example.org.
Page converted: 9/14/2004
This page last reviewed 9/14/2004