Process Monitoring Reporting Requirements
Developing Data Collection Tools
Collecting Client-level Data
Reporting Risk Population Served
Documenting the Number of Sessions Received
Data Reporting and Management Systems Overview
Choosing a Data Reporting and Management System

This chapter:

• Reviews process monitoring reporting requirements;
• Distinguishes process monitoring and process evaluation;
• Presents strategies for data collection including developing data collection tools, collecting client-level data, documenting the risk population served by the intervention, and tracking the number of intervention sessions received by clients; and
• Describes three systems for process monitoring data collection and reporting.

Process Monitoring Reporting Requirements

Process Monitoring is the routine documentation of data describing the characteristics of the population served, the services provided, and the resources used to deliver those services. Health departments are required to report annually to CDC aggregate process monitoring data on their CDC-funded interventions. The core set of data to be reported for all interventions includes:

• Type of agency;
• Number of clients served, categorized by race/ethnicity and gender³ (except for HC/PI);
• Number of full time equivalent (FTE) staff used to provide the intervention; and
• Expenditures for the intervention.

Some data are only reported for certain interventions. These intervention-specific data are listed below. See the Evaluation Guidance volume 1, chapter 4, for a complete description of core and intervention-specific data reporting requirements for process monitoring.

Distinguishing Process Monitoring and Process Evaluation

For this Guidance, process monitoring is distinct from process evaluation. Additional data are collected for process evaluation to answer more detailed questions about implementation of the intervention. Questions may include:

• Was the intervention implemented in a manner consistent with its design?
  
• Did the intervention reach the population most at risk?
  
• What barriers did clients experience in accessing the intervention?

Process evaluation is strongly encouraged, but not required, by the Guidance. See the Guidance, volume 2, chapter 4 for more information about process evaluation.

Data Collection Strategies

A health department should choose a process monitoring data collection method that is best suited to their jurisdiction. This section describes strategies for process monitoring data collection in four areas:

• Developing data collection tools,
  
• Collecting client-level data,
  
• Documenting the risk population served by the intervention, and
  
• Tracking the number of intervention sessions received by clients.

Developing Data Collection Tools

Contractors use various data collection tools to collect and report process monitoring data including simple tally sheets for documenting aggregate data about clients served through outreach or questionnaires for collecting detailed demographic and risk behavior data for each individual client receiving GLI, ILI, or prevention case management.

Data collection tools may be the same for all contractors in a jurisdiction or they may vary from one contractor to another, even when the same type of intervention is being implemented. Health departments may choose either way, but it is important to consider the advantages and limitations of each approach.

See the Appendix, p. 87-90, for examples of a data collection tools from Wisconsin, Virginia, Maryland, and New Jersey. Suggestions for developing data collection tools and sample questions can be found in the Guidance, volume 2, chapter 6.

Contractors Use the Same Data Collection Tools

Health departments and their funded contractors can collaborate to standardize process monitoring data collection tools. Tools can be designed to collect data needed to meet Guidance reporting requirements, as well as to gather other information of interest in the jurisdiction (e.g., client zip code, sexually transmitted disease history). Data collection tools that were in place prior to release of the Guidance may be modified to meet reporting requirements or new data collection tools may need to be developed.

Using the same data collection tools facilitates collecting uniform data throughout the jurisdiction and enables comparisons across interventions and populations. Development of these tools provides an opportunity for collaboration between the health department and contractors to define local data needs and helps cultivate "buy-in" to evaluation.

Some contractors may have invested resources in developing their own data collection prior to the Guidance. These contractors may not want to adopt new data collection tools and requiring them to do so can erode their support for conducting evaluation. Health departments should carefully consider the balance between retaining elements of existing data collection tools and establishing new standardized tools for use throughout the jurisdiction.

"Part of the problem with developing a common data collection system was that we had two real distinct groups of contractors. We had people who needed tons of help and any system was going to be a challenge for them because they didn't have anything. And we had people that already had something in place and were really resentful that we were replacing their system. We sort of made a decision in the beginning that if we're going to have a system, it's going be a system everybody uses. Otherwise we can't pool the data and make planning decisions." Health Department Staff Member

Contractors Use Different Data Collection Tools

Health departments may permit variation in how process monitoring data are collected, with each contractor developing and using its own data collection tools. Contractors may continue to use data collection tools that preceded the Guidance, or they may modify their tools or develop new ones to better meet Guidance reporting requirements. Regardless of the data collection tools used, health departments need to ensure that contractors collect the data required for reporting.

Allowing contractors to use different data collection tools helps avoid the risk of upsetting those who are using data collection tools developed prior to the Guidance. However, variation in data collection tools will likely yield variation in data quality, limiting comparisons across interventions and populations within the jurisdiction. In addition, contractors that do not currently have data collection tools, or lack tools that collect data required by the Guidance, may not have the capacity to develop these tools on their own. Health departments can provide technical assistance to help contractors develop their own tools or they may develop optional data collection tools for those contractors that need them.

Collecting Client-level Data

Client-level data collection involves gathering data about each individual client and maintaining that information in a database. Client data can then be retrieved, sorted, grouped, and analyzed across different variables of interest. In contrast, aggregate data collection combines information about all clients served by an intervention and does not retain client-specific data in a database. Client-level data can be pooled to yield aggregate data; however, information collected in aggregate form cannot be converted to client-level data.

Health departments can decide if they want to collect client-level data and for which interventions. Client-level data collection is usually limited to GLI, ILI, and prevention case management because these interventions usually provide sufficient interaction with clients to collect this information. Client-level data is rarely, if ever, collected during outreach or HC/PI interventions.

Client-level data collection typically involves assigning a unique identifier or code to each client. Linking a client's code and the client's data permits tracking of the individual client over time, as well as the aggregation, analysis, and reporting of data from multiple clients. The client code may be included on questionnaires and other forms for collecting data on client demographics, risk behaviors, intervention services received, and other variables of interest. Client codes can be generated by the client or the contractor.

Client-Generated Codes: The client creates a code by responding to a series of prompts, such as client initials, birth date, and mother's first name. With client-generated codes, the code is designed so that clients know all the information needed to complete the code themselves, though contractors may assist if necessary.

Contractor-Generated Codes: The contractor assigns a code to a client based on a series of prompts, such as provider initials and a number for each consecutive client seen by the provider. With contractor-generated codes, the contractor must create the code for the client because the client may not have all the information needed (e.g., provider initials). A master list is often maintained linking client codes and client names to ensure that clients are assigned the same code during subsequent contacts.

Client Code Examples

Methods used by different jurisdictions to create client codes are described below. Following these, a method suggested by the Health Resources Service Administration (HRSA) is described. The examples provided are for a white male, non-Hispanic client named John Doe, born on March 16, 1963. John is the fifteenth client served by a provider named Mary Smith.

HRSA creates a client code, called a Unique Record Number (URN), using the following method: 1st and 3rd letter of first name (if blank, use the middle initial), 1st and 3rd letter of last name (if blank, use the middle initial), birth month, last two digits of birth year, and gender code (1=male, 2=female). For example, JHDE03631. After this number is created, it is encrypted, or scrambled, using a complex algorithm. The resulting nine-digit code does not resemble the original information in any way. It is virtually impossible to retrace the information in the URN or retrace any personal information about a client. Decoding a URN is not feasible; too much of the original information is removed during the encryption process to be able to work backwards to the original 11 digit information. ⁴

Client Confidentiality

Concerns about confidentiality can hinder efforts to collect client-level data and should be considered. Client codes typically avoid using complete names, portions of social security numbers, or any other information that may reveal the client's identity. Even in the absence of information that could reveal client identity, clients may perceive the potential for breeches of confidentiality and therefore be hesitant to report risk behaviors or to utilize prevention services that collect client-level data. These concerns may be particularly salient for clients engaged in illegal or stigmatized behaviors. Contractors may also be concerned about confidentiality issues and resist collecting client-level data.

Confidentiality concerns can be addressed in different ways. One health department conducted focus groups with clients and learned that they would feel more comfortable if contractors did not see client-level data. In this jurisdiction, clients generate their own code and complete questionnaires. These questionnaires are placed in a sealed envelope, which the contractor collects without seeing the information and sends to the health department for data entry and analysis. In a different jurisdiction, clients did not want the health department to have access to client-level data. In this case, the contractor collects and aggregates client-level data. Only aggregate reports are submitted to the health department. Both approaches show a positive response to the particular concerns in each jurisdiction.

Benefits of Client-Level Data

Collecting client-level data facilitates reporting several process monitoring data elements required by the Guidance, including client race, ethnicity, gender, and age; risk population served; number of clients served; and number of intervention sessions received for ILI, GLI, and PCM. (See p. 38 for more information about documenting the number of intervention session received.) In the absence of client-level data, health departments may not be able to report this information accurately. These data also may be useful for local evaluation and planning purposes beyond the Guidance requirements.

The advantages of client-level data are contrasted with aggregate data in the following example. A three session GLI targeting heterosexuals serves six clients. Jurisdiction A collects aggregate data, and Jurisdiction B collects client-level data. Both jurisdictions collect data on risk, race, gender, and the number of intervention sessions received.

Aggregate Data Collection: Jurisdiction A collected and reported the following aggregate data upon completing the three sessions GLI.

In this example, aggregate data do not permit reporting of client race by gender because these demographic data were collected and reported independently. There is no way to identify the race of the one MSM client served by the intervention nor is it possible to know how many intervention sessions were received by each client (i.e., only the number of clients attending each session is known). Without this information it is difficult for Jurisdiction A to report all required Guidance data.

Client-Level Data Collection: Jurisdiction B collected and reported the following client-level data upon completing the three session GLI.

In contrast to aggregate data, client-level data reveals greater demographic detail about clients served. These data show that the one MSM client was Black and that none of the White clients received more than two intervention sessions. These data also reveal that three clients received two intervention sessions and three clients received three intervention sessions. This more detailed information makes it easier for Jurisdiction B to report Guidance data.

Client-level data can reduce the need for contractors to collect data each time they see a repeat client. For example, if contractors collect demographic and other data during the first contact with a client, they can then use the client's code to access the information for reporting subsequent visits.

Challenges of Client-Level Data

Client codes must protect client confidentiality. Ideally, these codes are a unique, unduplicated identifier for each client. Using last names or portions of social security numbers (e.g., last four digits) in the code can decrease the possibility of code duplication. However, these elements are usually avoided to mitigate client concerns about confidentiality. Efforts to maintain confidentiality can make it difficult to avoid duplication, reducing the quality of client-level data.

Codes may also be unstable over time for the same client. For example, a client may have multiple names or nicknames that undermine the consistency of a code that uses initials or letters from their name. It may not be possible, therefore, to create client codes that eliminate the possibility of duplication and that are completely stable over time. Duplicate and unstable codes can compromise data quality. Health departments should be mindful of these problems and try to minimize their occurrence.

Reporting Risk Population Served

Contractors report data to the health department about which risk populations were served by their interventions (e.g., MSM, IDU). For some intervention types (e.g., PCM), conducting a risk assessment can provide data to be used to report on the risk population served. With other interventions (e.g., outreach), a risk assessment cannot always be conducted and other methods must be used to report on the population served. Three strategies can be used to document the risk of clients served by an intervention:

• Client self-report,
• Contractor perception, and
• Intervention intention.

Although some approaches produce better quality data than others, health departments may choose which method they will use. One strategy may be used by all contractors throughout a jurisdiction or strategies may vary across interventions and contractors within a jurisdiction.

Client Self-Report

With this strategy, contractors gather self-reported data about risk from the clients they serve. Risk-assessment data should always be collected with PCM interventions. Risk assessment may also be conducted for clients in GLI and ILI if there is sufficient contact with the client to perform an assessment. Data may be collected as part of a comprehensive risk assessment appropriate for PCM, may involve a shortened risk assessment tool, or may be a simple questionnaire in which clients indicate the risk population to which they belong. Questionnaires may be completed by clients themselves or the contractor can elicit the information and complete the questionnaire for them. Client codes may be included on the questionnaire to facilitate collection of client-level data. (See p. 33 for more information about collecting client-level data.)

Clients may not always be truthful about reporting their risk behaviors, and the sensitive nature of risk behavior questions may alienate some clients. Despite these concerns, self-reported data are likely to yield the most reliable information about the risk populations served. Contractors may find that clients are more truthful about reporting their risk behaviors as trust and rapport develops after several contacts. Under these circumstances, self-reported risk data may give the appearance that a client's risk is initially increasing during the course of the intervention. For this reason, contractors may consider waiting until the second or third contact with the client to collect risk data; however, if a client does not return for services, then an opportunity to collect data during the first contact will have been missed. Health departments are encouraged to carefully consider the circumstances under which they would allow a contractor to delay collecting data on self-reported risk.

Contractor Perception

When it is not possible to collect self-reported data, the risk population served may be based on a contractor's perception of a client's risk. This approach is often used during outreach interventions because it is difficult to conduct a risk assessment during the brief contacts with clients typical in this intervention. Accurate reporting depends on contractor knowledge of the prevalence of risk behaviors in an outreach location (e.g., gay bar, shooting gallery) or knowledge of the particular risk population with whom they are working. Although reporting risk based on contractor perception minimizes the possibility of alienating a client by asking explicit questions about risk, it is likely to be less accurate than client self-reported data.

Intervention Intention

When risk assessment is not possible, and the contractor is unsure of client risk, the risk population may be reported based on the intention of the intervention. For example, if an intervention is designed to serve MSM, then everyone reached by the intervention is assumed to be MSM and is reported as such. This approach may misclassify the risk of some clients and yield unreliable data. Although reporting risk in this manner is in compliance with the minimum expectations of the Guidance, this method should only be used when there is no alternative.

Documenting the Number of Sessions Received

Health departments are required to report the number of clients receiving only 1, only 2, or 3 or more intervention sessions for GLI, ILI and PCM. The number of sessions received by a client is one measure of the intensity of the intervention. Documenting the number of sessions is fairly easy for ILI and PCM. The one-on-one nature of the interaction between contractor and client simplifies record keeping to collect this data.

GLI can present some unique challenges to documenting the number of sessions because attendance may be fluid during the course of the intervention. For example, in a four session group-level intervention, some clients may drop out after the first session, others may attend just the first and third, and others will participate in all four. With several participants in a group and multiple sessions over time, it can be difficult to track the number of sessions received by clients.

Health departments can use the method they prefer to collect data documenting the number of intervention sessions received. The following four strategies are discussed below:

• Client-level data,
• Sign-in sheets,
• Contractor recall, and
• Client recall.

One strategy may be used by all contractors in a jurisdiction or health departments may allow variation in how these data are collected. Using one strategy throughout the jurisdiction will likely yield uniform data, facilitating comparison across contractors and interventions. Variation in methods allows flexibility to accommodate the needs of different interventions, but may yield data of varying quality. Health departments are encouraged to consider the advantages and disadvantages of each strategy and to collect these data in a manner most appropriate for their jurisdiction.

Client-Level Data

Collecting data on each individual client is one way to document the number of sessions received. (See p. 33 for more information about collecting client-level data.) These data can be tabulated to determine the distribution of sessions received by all clients. In a variation of this method, some contractors use client codes without collecting the risk behavior data that usually accompanies client-level data collection. Clients maintain their confidentiality by using their code to sign-in for each intervention session attended and these data are used to track the number of sessions received. See the Appendix, p. 91, for an example of a sign-in sheet from Maryland.

This data can be used to determine the number of intervention sessions received, e.g., 2 clients attended 1 session, 1 client attended 2 sessions, and 3 clients attended 3 or more sessions

"One of the biggest problems was collecting information about the number of sessions. It was really difficult to track that without having a unique identifier for each client. We were asking questions like how many clients completed a one session ILI, how many completed two sessions, and that relied on the contractor having the ability to track that information. What we found was they just couldn't do it without a client code." Health Department Staff Member

Sign-In Sheets

Clients can use their name to sign an attendance sheet for each intervention session received. Attendance is tracked over time to determine the number of sessions received by each client and aggregated to determine the distribution for all clients. Although this approach is simple to use, clients may not feel comfortable revealing their names or may use different names each time they sign in (e.g., nicknames), compromising the quality of the data collected. Any identifying information used on the sign-in sheet must be kept confidential.

Contractor Recall

This approach relies on the contractors' ability to recognize clients and remember who did and did not attend sessions. Alternatively, contractors may rely on the staff at the agency or institution hosting the intervention to remember how many sessions clients attended. Similar to the other methods, this information is used to determine the number of sessions received by each client and aggregated to determine the distribution for all clients. Although this approach is unobtrusive, poor recall can compromise data quality.

Client Recall

Contractors can ask clients to self-report how many intervention sessions they have attended. For example, at the end of a three session GLI, clients can be asked to report how many of the three sessions they attended. Although the simplicity of this approach may be appealing, data cannot be collected from clients who do not attend the last session. Data quality may also be compromised by poor client recall and the possibility that some clients will be biased to over-report the number of sessions they attended.

Data Reporting and Management Systems Overview

Data reporting and management systems are used by health departments to collect, aggregate, and analyze process monitoring data. These systems establish procedures for monitoring data quality, transmitting data from contractors to the health department, creating a database, and producing CDC- and jurisdiction-specific reports. Three data reporting and management systems are described below:

• Health Department-Based Systems,
• Contractor-Based Systems, and
• Web-Based Systems.

The key features of each system are summarized on p. 42. No one system is best for all jurisdictions. Health departments are encouraged to consider the strengths and limitations of each system as they develop a data reporting and management system for their jurisdiction.

Health Department-Based System

Health departments use data management software such as Microsoft Access or Excel to enter and manage process monitoring data reported to them by their contractors. Contractors typically collect process monitoring data on paper forms and submit these records to the health department for data entry.

Jurisdictions using this approach have established different schedules for data submission. Data may be sent to the health department after each intervention event or it may be accumulated and submitted in batches monthly, quarterly, or bi-annually. In some cases, the contractor collects and aggregates data over time and submits a summary report to the health department. This approach, however, requires that the contractor has a method for tabulating data.

Data submitted to the health department are reviewed by health department staff, checked for missing or inconsistent data, and either scanned or entered manually into a database. Missing or inconsistent data may be identified by visually reviewing the data for apparent errors (e.g., no risk population is reported) or by using a computer-based data edit program. This process allows the health department to identify and correct reporting errors and to assist contractors in improving the quality of future data submissions.

The health department uses these data to produce reports for CDC in compliance with Guidance requirements and may also generate reports for use by health department staff, contractors, and other stakeholders. Data management software may be preprogrammed to conduct data analysis and produce reports that meet CDC and jurisdiction-specific information needs.

Contractor-Based System

Contractors use data management software housed at their agency to enter and manage their own process monitoring data. This approach requires that all contractors have access to a computer and data management software and that they are all collecting and reporting the variables specified in the Guidance, regardless of the software they are using. Some health departments have made standard software available to all contractors. Others have allowed them to use the software of their choice (e.g., Microsoft Excel, Access). Contractors typically collect process monitoring data on paper forms and later enter the information into a database. In some cases, data are entered directly into the computer while the intervention is being conducted, eliminating the need for a paper form.

Data entry is performed by the staff member delivering the intervention or by other staff within the agency. Data entry screens can be designed to minimize data entry errors by incorporating skip patterns and menus with set options for data entry. After data entry, data can be submitted to the health department by e-mail or on diskette, usually on a monthly, quarterly, or bi-annual schedule. These data are reviewed, checked for errors, and then combined to create a master database at the health department. By checking for missing or inconsistent data, the health department can correct mistakes and assist contractors in improving the quality of future data submissions. However, since the original data collection forms are usually not submitted to the health department, it may be difficult to detect data errors that occurred during the data collection or entry process.

The master database is used to produce reports for CDC in compliance with Guidance requirements. Additional reports may also be generated for use by health department staff, contractors, and other stakeholders. Data management software may be preprogrammed to conduct data analysis and produce these reports.

Web-Based System

Contractors may access a web-based system at their agency to enter process monitoring data and to transmit the data to the health department. This approach requires that all contractors have access to a computer with an Internet connection linked to a web page for data entry. Similar to the contractor-based system described above, contractors collect process monitoring data on paper forms and later enter the information into the Internet system. Alternatively, they may enter the data directly into the computer while the intervention is being conducted, eliminating the need for a paper form.

Data entry screens can be designed to minimize data entry errors by incorporating skip patterns and menus with set options for data entry. Data entry and submission to the health department generally occurs immediately after the intervention, or on a monthly or quarterly schedule. Data are automatically aggregated by the system to create a master database at the health department. This database is used to produce reports for CDC in compliance with Guidance requirements. Additional reports may also be generated for use by health department staff, contractors, and other stakeholders. The system may be preprogrammed to conduct data analysis and produce these reports.

Combining Systems

These three systems may be used alone or in combination depending on the needs of the jurisdiction. For example, a health department may use a contractor-based system with most of their contractors and use a health department-based system for those contractors that do not have the capacity to enter and manage their own data. This combination may be used as an interim strategy while data entry and management capacity of contractors is further developed. Likewise, these strategies may be sequenced over time as part of a developmental approach for the jurisdiction. For example, a health department-based system might be used for all contractors as a way to closely monitor and improve data quality as a first step in a long-range plan to establish a web-based system.

"The grantees have a choice about doing data entry. Some want to do it and some don't want to do it because it is too complicated for them. They just want to fill out the forms, and we'll get somebody to do the data entry for them here at the health department. Those who have the ability and enough staff to actually enter data will just do the data entry on site." Health Department Staff Member

Choosing a Data Reporting and Management System

Health departments should consider the strengths and limitations of each approach when choosing a data reporting and management system. There are important differences across systems in terms of the technology required, responsibility for data entry, mechanisms to monitor data quality, implications of the frequency of data submissions, and the ability to access data for analysis. These five issues are described below.

Technology

Technology needs should be considered when choosing a data reporting and management system. Health department-based systems do not require the contractor have computer access or literacy; however, health department staff will need skills to establish and manage the database. Contractor-based and Web-based systems both require the contractor to have access to a computer; the latter is also dependent on an Internet connection linked to a web page for data entry. Contractors must also have staff that are literate in the data management software or Internet system used. Maintaining staff computer capacity can be challenging given the frequent turnover experienced by many contractors. Web-based systems have an added advantage over contractor-based systems in that the former avoids the challenge of identifying and deploying software compatible with different computers and operating systems.

Data Entry

Consideration should be given to contractor and health department capacity to conduct data entry. In health department-based systems, health department staff conduct data entry; with contractor-based and web-based systems, data entry is performed by the contractor. While the burden on the contractor is reduced when the health department assumes responsibility for data entry, this task can require significant time and resources from the health department and may limit opportunities to develop contractor data entry capacity. However, some health departments perform data entry as an interim strategy while simultaneously developing contractor capacity to perform data entry in the future.

Data Quality

Systems vary in how they monitor data quality. Contractor-based and web-based systems can be designed to facilitate correct data entry and reporting. Data entry screens can be constructed with menus, internal checks, contractor-specific access codes, and other features that prevent entry of spurious data. Data entry screens may even be tailored to the specific intervention the contractor is funded to deliver, ensuring that the intervention type and risk population are correctly reported. These strategies require standardization of the data entry screen. This is easily accomplished with web-based systems or when the health department provides the same data management software to all contractors.

"With the web-based system, there'd be a log-in screen were contractors would put in information, like their agency name, and up would pop a list of the interventions that matched whatever they were funded to do. This would take them directly to the page where they need to enter the data, so there wouldn't be any of that thought process anymore about how or where to report the information." Health Department Staff Member

Health department-based systems manage data quality by reviewing paper data collection forms submitted by their contractors. The same features for data entry screens described above can be used with health department data entry to improve data quality. Health department-based systems have the added advantage of allowing health department staff to review and identify errors on the original paper forms completed by contractors. Although the other two systems typically do not involve reviewing these forms, health departments may ask their contractor to submit the paper forms so they can be compared with the electronic data submitted.

Reporting Schedule

The frequency of data submission from contractors to the health department is not dependent on a particular system, and the implications of reporting frequency vary depending on which system is used. For contractor-based and web-based systems, the longer the delay between collecting intervention data and reporting it to the health department, the greater the need for contractors to store the data until it is reported. Contractors may vary, however, in their capacity to keep their data collections forms organized and secure. With quarterly data submissions, contractors may be inclined to save their process monitoring data for three months and then enter it all at once. If records were not well maintained, the quality of the data may be compromised. With health department-based systems, frequent data submissions help avoid a backlog of data waiting to be entered at the health department. Regardless of the systems used, frequent reporting enables health departments to monitor data quality, intervene quickly when there are problems, and conduct interim analysis to help monitor progress in meeting objectives.

"We have them do the reports monthly because it is hard to respond to problems if you are finding out three months after the fact that there is a problem and also because of what we know about the reliability of recall. Unless people are filling out their forms during the intervention, which some do, some have it on clipboard, it is hard to remember and report the data accurately." Health Department Staff Member

Data Analysis

Although all three systems can be designed to automatically conduct data analysis and produce reports, the ability to access the data and produce these reports varies. With health department-based systems, the database resides at the health department and contractors depend on the health department to conduct data analysis and produce reports for their use. In contrast, contractor-based systems permit contractor access to the database and allow them to generate their own reports, as needed. Web-based systems also permit contractor access to the database; access can be restricted so that contractors view data only from their own interventions.