Free-text Coding in NSSP–ESSENCE: Part 4
This is the fourth article in the series about how to write ESSENCE free-text queries. We thank Senior Data Analyst Zachary Stein for developing this series.
Part 1. Wildcards
Part 2. Underscores_and Brackets [ ]
Part 3. Inclusionary Terms
Part 5. A “Starter” Fall-related Injury Query and Examples of Complex Queries
Part 6. Wrapping Things Up
Part 7. Additional Tips: ISBLANK and ISNULL
Part 8. Additional Tips: !TERM! Syntax
Introduction
The search criteria for ESSENCE free-text queries are built around Boolean logical operators and regular expressions. Free-text queries are not case-sensitive and may contain “^” for wildcards; “,” for multiple entries; “ISBLANK” to look for blanks; “ISNULL” to look for nulls; [COMMA] to look for commas; and operators “and,” “or,”, “andnot,” and parentheses “()” to define order and grouping. This series will cover all these topics in-depth.
Free-text queries are what makes syndromic surveillance practice, particularly practice using NSSP–ESSENCE, adaptable to different data sources and types. By using free-text queries, analysts and epidemiologists can exercise a high level of customization. They can quickly code free-text queries and rapidly respond to outbreaks, disasters, and events that unfold. Such capabilities empower users to customize queries to fit their level of data, ensuring accurate results.
Free-text coding in ESSENCE, which is accessible to all users, follows distinct patterns. Learning to read these patterns allows users to take queries from many places and repurpose them to suit their unique needs. Syndromic surveillance depends heavily on sharing methods, and practitioners must understand the language.
Part 4. Exclusion Terms and Parentheses
So far in this series, we have covered basic query notation, use of Carets, Underscores, and Brackets, and “OR” and “AND” statements. The final two parts of a query’s structure can involve “ANDNOT” statements and parentheses. We have demonstrated the power of inclusion terms, and the “ANDNOT” statement is the next step to leveraging the system’s query resources. We will also show how to add parentheses to specify how these “OR”, “AND”, and “ANDNOT” statements are applied and interact with each other.
“ANDNOT” statements
“ANDNOT” statements, also referred to as negations, exclusions, or exclusionary statements, are a common way to specify what you do not want your query to return. These statements are often leveraged in a syndrome definition to exclude common false positives. An “ANDNOT” statement without parentheses connects the word before it and the word after and will remove any returns where both criteria are met. To demonstrate this, we can work through the following very simple query.
THIS,OR,THAT,ANDNOT,THERE
In this query, ESSENCE will return fields containing the term “THIS” or the term “THAT” and then look through the “THAT” returns for the presence of the term “THERE” and remove these records where both criteria are met. “ANDNOT” statements do not specify the order in which the terms appear. Also note that the example’s “THERE” exclusion is only applied to “THAT” and does not apply to “THIS”. “THIS” is unaffected by the exclusion statement. Similar to “AND” and “OR” statements, ESSENCE queries are not case sensitive, neither are the operators, and all operators should be surrounded by commas.
Let’s assume the following Chief Complaints (used in Parts 1–3) and a desire to create a query for fall-related injuries.
- Fall
- Fell getting out of car
- Left arm injury; Fall
- Falling out with friends; Suicidal
- Feels crestfallen
- Patient brought in after falling on face
- Fall; Left wrist injury
- Feels congested; Allergies
You may reasonably assume the boldfaced CCs 1, 2, 3, 6, and 7 are the intended cases and 4, 5, and 8 are false positives.
Here’s a table that shows examples of the “ANDNOT” query:
| “ANDNOT” Query Examples | |
|---|---|
| Code | Description |
^Fall^ |
This is a simple query of a term surrounded by carets taken from Part 1. It returns CCs 1, 3, 4, 5, 6, and 7. |
^Fall^,ANDNOT,^Crestfallen^ |
This query negates the false positive in CC 5. Returns CCs 1, 3, 4, 6, and 7. |
^Fall^,OR,^Fell^,ANDNOT,^Crestfallen^ |
This covers both forms of “Fall” in our CCs, but the negation of “Crestfallen” follows the term ^Fell^. Since “Fell” and “Crestfallen” don’t appear in the same CC, no CCs are negated. Returns CCs 1, 2, 3, 4, 5, 6, and 7. |
^Fall^,ANDNOT,^Falling out^,ANDNOT,^Crestfallen^ |
“ANDNOT” statements can be written in series. Multiple exclusionary statements in a row apply to the inclusionary term they immediately follow. Both negation terms apply to the ^Fall^ term. Returns CCs 1, 2, 3, 6, and 7. |
Parentheses
Complex syndrome definitions often use parentheses to group terms and apply statements in different ways. “ANDNOT” statements are frequently used alongside parenthetical groupings. Regarding notation in ESSENCE, all parentheses should be surrounded by commas, except when the parenthesis starts or ends the statement. To demonstrate this notation and how parentheses can redistribute statements, we can work through the following simple query.
(,THIS,OR,THAT,),ANDNOT,(,THERE,OR,THEN,)
In this query, since both sections are surrounded in parentheses, ESSENCE will apply both the “ANDNOT” negations “THERE” and “THEN” to both the initial inclusion terms “THIS” and “THAT”. That means it will return fields containing either term “THIS” or the term “THAT” so long as there is not also the presence of either term “THERE” or “THEN”. Notice the second “(“ is surrounded by commas while the first one isn’t since it is the first character in the query. Likewise, notice the first “)” is surrounded by commas while the last one isn’t since it’s the last character in the query. Misplaced commas are one of the most frequent reasons a query doesn’t work as intended.
Let’s assume the following Chief Complaints (used in Parts 1–3) and a desire to create a query for fall-related injuries.
- Fall
- Fell getting out of car
- Left arm injury; Fall
- Falling out with friends; Suicidal
- Feels crestfallen
- Patient brought in after falling on face
- Fall; Left wrist injury
- Feels congested; Allergies
You may reasonably assume the boldfaced CCs 1, 2, 3, 6, and 7 are the intended cases and 4, 5, and 8 are false positives.
Here’s a table that shows examples of queries with parentheses:
| Query Examples with Parentheses | |
|---|---|
| Code | Description |
^Fall^ |
This is a simple query of a term surrounded by carets taken from Part 1. It returns CCs 1, 3, 4, 5, 6, and 7. |
(,^Fall^,) |
Sometimes parentheses won’t change anything. This query functions exactly the same as the query above it. Returns CCs 1, 3, 4, 5, 6, and 7. |
(,^Fall^,OR,^Fell^,),ANDNOT,^Crestfallen^ |
This covers both forms of “Fall” in our CCs, and the parentheses ensure the negation applies across both initial terms. Returns CCs 1, 2, 3, 4, 6, and 7. |
^Fall^,ANDNOT,(,^Falling out^,OR,^Crest fallen^,) |
“ANDNOT” statements can be written and applied to a series of “OR” statements, excluding the whole statement in the parentheses. Returns CCs 1, 2, 3, 6, and 7. |
(,^Fall^,OR,^Fell^,),ANDNOT,(,^Cre stfallen^,or,^Falling Out^,ANDNOT,(,^Out a window^,),) |
Syndromes can get complicated with multiple-nested parentheses. Notice this query attempts to negate the “Falling out with friends” CC while also trying to include visits related to “Falling out a window.” In this instance, the “ANDNOT” term is double negated by a second “ANDNOT” term. Returns CCs 1, 2, 3, 6, and 7. |
Putting two sets of parentheses with different criteria linked by an “AND” statement means at least one item from each set of criteria must be met. In the example below, both coded statements will return the same visits, but the second statement is far more concise and easier to read.
^Fall^,AND,^Injur^,OR,^Fall^,AND,^Face^,OR,^Fell^,AND,^Injur^,OR,^Fell^,AND,^Face^
(,^Fall^,or,^Fell^,),AND,(,^Injur^,or,^Face^,)
Here are a couple examples to show how “ANDNOT” and parentheses can be used in practice:
Example 1:
^measl^,ANDNOT,^titer^,ANDNOT,^measles mumps rubella^,ANDNOT,^mmr^,ANDNOT,^vacc^,ANDNOT,^shot^,ANDNOT,^immun^,ANDNOT,^rubel^,ANDNOT,^proph^,ANDNOT,^room b05^,ANDNOT,^german^,ANDNOT,^homesless^,ANDNOT,^rule out measles^,OR,^meez^,ANDNOT,^titer^,ANDNOT,^measles mumps rubella^,ANDNOT,^mmr^,ANDNOT,^vacc^,ANDNOT,^shot^,ANDNOT,^immun^,ANDNOT,^rubel^,ANDNOT,^proph^,ANDNOT,^room b05^,ANDNOT,^german^,ANDNOT,^homesless^,ANDNOT,^rule out measles^,
Example 2:
(,^[;/ ]T40.1X1A^,OR,^[;/ ]T401X1A^,OR,^[;/ ]T40.1X4A^,OR,^[;/ ]T401X4A^,OR,^[;/ ]965.01[;/]^,OR,^[;/ ]96501[;/]^,OR,^[;/ ]E850.0^,OR,^[;/ ]E8500^,or,^295174006^,or,^295175007^,or,^295176008^,),or,(,(,(,^narcan^,or,^naloxo^,or,^poison^,or,^verdo[se][se]^,or,^over dose^,or,^overose^,or,^nodding^,or,^ nod ^,or,^snort^,or,^in[gj]est^,or,^intoxic^,or,^unresponsiv^,or,^loss of consciousness^,or,^syncop^,or,^shortness of breath^,or,^short of breath^,or,^altered mental status^,),and,(,^her[io][oi]n^,or,^ hod ^,or,^speedball^,or,^speed ball^,or,^dope^,),),andnot,(,^no loss of consciousness^,or,^denie[sd] loss of consciousness^,or,^negative loss of consciousness^,or,^denies any loss of consciousness^,or,^denies her[io][oi]n^,or,^deny her[io][oi]n^,or,^denied her[io][oi]n^,or,^denying her[io][oi]n^,or,^denies drug^,or,^deny drug^,or,^denied drug^,or,^denying drug^,or,^denies any drug^,or,^with dra^,or,^withdra^,or,^detoxification^,or,^detos^,or,^detoz^,or,^dtox^,),)
This example is the entire CDC Heroin Overdose v4 CCDD Category. Notice how the green statement is linked to the yellow by an “AND” statement. Then, both statements are contained within parentheses and negated by the red statement with an “ANDNOT” operator.
Try it out!
Compare the results of the following queries in the CCDD field of ESSENCE (results should be exactly the same):
^Fall^,AND,^W19.XXXA^,OR,^Fall^,AND,^W19XXXA^
^Fall^,AND,(,^W19.XXXA^,OR,^W19XXXA^,)
Try writing a set of negation terms (expected false positives) for the following query. Check your answer with the CDC Pertussis v1 CCDD Category in NSSP ESSENCE.
(,Whooping^,OR,^Pertuss^,),ANDNOT,(_______________)
We thank Senior Data Analyst Zachary Stein for volunteering to write a series of articles about free-text coding. Stein, formerly with the Kansas Department of Environment and Health, does epidemiologic work to support NSSP efforts. Stein is an active participant in the NSSP CoP. He initially wrote about free-text coding as an entry on the NSSP CoP Syndrome Definition Committee forum. The forum generated considerable interest, inspiring this series. Stein acknowledges input provided by others who contributed to the forum post.
