(introduction...)

Report to the
World Health Organization Collaborating Centers on the Classification of Disease

September 2000

Department of Health & Human Services
Centers for Disease Control and Prevention
National Center for Disease Prevention and Control

Prepared by:

J. Lee Annest, PhD, Director
Chester L. Pogostin, DVM, MPA, Deputy Director

Office of Statistics and Programming
National Center for Injury Prevention and Control (NCIPC)
Centers for Disease Control and Prevention (CDC)
Atlanta, GA, USA

Pilot Study Collaborators:

Judy Conn, MS, Patricia Holmgreen, MS, and Marcie-jo Kresnow, MS, NCIPC, CDC; Torie Ozonoff, PhD, Jennifer Taylor, MPH, Laurie Jannelli, Elizebeth Castagneto, and Stacy Klingler, Massachusetts Department of Public Health, Boston, MA; Art McDonald, MA, Joel Friedman, Stephanie Drury, Linda Swenson, Verlene Swenson, Storm Randall, Carol Strunk, Kirby Riffey, Karen Wozniak, Robert Hayden, and Bea Belton, U.S. Consumer Product Safety Commission, Bethesda, MD.

We would also like to acknowledge our colleagues who participated in the development of CDC’s short version of the ICECI:

Dan Pollock, MD, NCIPC, CDC, Atlanta, GA;

Lois Fingerhut, MA, and Donna Pickett, RRA, MPH,
National Center for Health Statistics, CDC, Hyattsville, MD

Executive Summary

This report summarizes the findings of a pilot study to assess the usefulness of the Centers for Disease Control and Prevention’s (CDC) short version of the International Classification of External Causes of Injury (ICECI) in capturing external-cause-of-injury data from hospital emergency department (ED) records. CDC’s short ICECI was designed to capture major mechanisms of injury (e.g., fall, motor vehicle traffic crash, struck by/against, cut/pierce, fire/burn, poisoning, firearm, and others), intent of injury, locale of the injury incident, activity at the time of the injury, work-relatedness, safety equipment use, consumer products involved, and a narrative describing the circumstances of the injury incident. For this pilot study, coders used the short ICECI data collection form to code external-cause-of-injury data from a standard set of case scenarios and from hospital ED records. Coded data based on the case scenarios were then analyzed for percent agreement with a gold standard code set (i.e., test of validity), percent agreement among coders (i.e., test of inter-coder reliability), and percent agreement for repeated cases for individual coders (i.e., test of intra-coder reliability). Coded data based on hospital ED records were used to assess inter-coder reliability, timeliness, and completeness of coded data. The study used two independent ED surveillance systems: the Massachusetts (MA) Department of Public Health’s Emergency Department Surveillance and Coordinated Injury Prevention Program (ED-SCIP) and the U.S. Consumer Product Safety Commission’s National Electronic Injury Surveillance System (NEISS). The findings from this study suggest that CDC’s short ICECI can be a useful and efficient injury surveillance tool for coding valid and reliable external-cause-of-injury data. The pilot study also helped identify some limitations of the current version; therefore, we plan to revise and develop a version 2.0 of CDC’s short ICECI for further testing and refinement.

We identified some important aspects of this classification system that have implications for the further development and implementation of the full ICECI and various short versions of the ICECI for use in injury morbidity data systems. These aspects, which are discussed later in the report, are listed below:

1. Protocols and instruction manuals for orientation and training of coders and for ongoing quality assurance activities are essential for ensuring complete and valid coding of external-cause-of-injury data.

2. A detailed, user-friendly coding manual is necessary for specifying the case definition, coding rules and definitions, and guiding principles for coding.

3. The data collection instrument should make collecting and processing short ICECI data from medical records more efficient; electronic versions of the data collection form are needed.

4. Coding of mechanism of injury should be limited to two mechanisms: the precipitating cause (i.e., the cause that initiated the chain of events leading to the injury) and the immediate cause of injury (i.e. the direct cause of the most severe injury being treated).

5. A narrative description of the injury incident should be included as an essential ICECI data element. Narratives can be very helpful in conducting quality assurance reviews of assigned codes and for capturing further details about injury-related circumstances.

6. Training and orientation of hospital medical staff is critical to improve the quality of information on intent of injury, mechanism of injury, and other injury-related circumstances contained in the medical record.

7. Proposed short ICECI data elements and code sets should be cross-walked to the full ICECI, ICD-10, and ICD-10-CM code sets as a measure of compatibility and cross-reference. The full ICECI should serve as a standard coding system for all proposed short ICECIs.

Development of an injury surveillance tool like CDC’s short ICECI can provide a useful instrument for routinely capturing and coding population-based data on external causes of injury for the large number of injury-related cases treated in hospital emergency departments. However, if this tool is to be useful and appropriate, it must be consistent with existing international classification standards for morbidity data systems. With careful planning and development, pilot testing, and implementation efforts, this type of surveillance tool can open new avenues for obtaining detailed data important for both clinical practice and injury prevention efforts. Clearly, we need to step beyond the limits of data captured using ICD-10 external cause codes and derive compatible coding systems and software that will allow for routine and timely capture of more detailed data about the injury incident. Timely access to population-based, injury-related data is critical in designing and evaluating injury prevention programs. Because of advances in information technology, now is the time for a concerted effort to develop injury surveillance standards and tools.

Introduction

Background on ICECI and Short Versions of the ICECI

An international effort is underway to develop a new multi-axial classification system for external causes of injury designed for use in mortality and morbidity data systems. This system will provide a standard for coding more detailed information about injury circumstances than is possible using the current ICD-10 external-cause-of-injury coding system. This new system, called the International Classification of External Causes of Injury (ICECI),¹ is designed to capture details about the place of occurrence, activity at the time of injury, alcohol and drug involvement, objects or substances involved, intent of injury, and mechanism of injury. Specific modules are being developed to capture more detailed information about injuries related to violence, transportation, sports, and work (occupational). Several short versions of the ICECI (subsequently referred to as short ICECIs) are being developed for use as injury surveillance tools in different settings. For example, a set of short ICECIs has been proposed by Dr. Yvette Holder, Dr. Etienne Krug, and colleagues for use in countries or societies with limited resources for injury surveillance.² Also, CDC has developed a short ICECI for use in capturing external-cause-of-injury data from hospital emergency department (ED) records in the United States.³ This report will focus on a pilot study that was conducted in the United States to test the feasibility of CDC’s short ICECI for abstracting data on external causes of injury using hospital ED records. The remainder of this report will provide a description of CDC’s short ICECI as well as the methods, results, discussion, and recommendations stemming from the pilot study.

Purpose of the Pilot Test of CDC’s Short ICECI

The purpose of the short ICECI pilot study was to test the reliability of capturing external-cause-of-injury data using the short ICECI data elements and code sets. The pilot study had two parts: (1) a case scenario test to measure validity, timeliness, and inter-coder and intra-coder reliability; and (2) a field test to measure inter-coder reliability and completeness of external-cause-of-injury data obtained from hospital ED records.

CDC’s Short ICECI and Its Components

CDC’s short ICECI has five major components: (1) type of incident (work-related or not), (2) locale of injury incident, (3) type of activity when the injury occurred, (4) intent of injury, and (5) mechanism of injury.³ Each of the major components or data elements consists of a code set with specified categories pertinent to the injury incident. For intent of injury and mechanism of injury, there are also subdata elements with code sets designed to capture more information about specific types of injury-related incidents. For instance, if a patient was treated for an injury resulting from a motor vehicle crash, the short ICECI captures information on traffic-relatedness, type of motor vehicle involved, occupant status (e.g., driver or passenger, boarding or alighting), and counterpart to the crash (e.g., another vehicle, pedestrian, or fixed object). In addition, the short ICECI data collection form provides space for recording safety equipment use, consumer products involved, and a narrative description of the injury incident.

The short ICECI data elements and code sets were derived from the full ICECI data elements and codes sets and the ICD-10 external-cause-of-injury code set. The short ICECI code sets have been cross-walked to code sets for both of these classification systems. The short ICECI consists of a hard copy check-box type data collection form, a manual with coding definitions and rules, and a training module. The current or modified versions of CDC’s short ICECI are under consideration or are being used for other injury surveillance activities in the United States (e.g., voluntary reporting from hospital EDs in Michigan; data linkage study at the Grady Hospital ED in Atlanta). The short ICECI data elements and code sets for intent of injury and mechanism of injury components also have been modified to capture external-cause-of-injury data in the National Electronic Injury Surveillance System (NEISS), operated by the U.S. Consumer Product Safety Commission (CPSC). CPSC, in collaboration with CDC is expanding NEISS to capture nationally-representative data on all types and causes of injuries treated in U.S. hospital EDs.

Methods and Procedures

Massachusetts ED-SCIP and NEISS Substudies

The short ICECI Pilot Test consisted of two independent substudies involving hospital EDs in two different injury surveillance systems: the Massachusetts (MA) Emergency Department Surveillance and Coordination Project (ED-SCIP) and CPSC’s National Electronic Injury Surveillance System (NEISS). The Massachusetts Department of Public Health is developing ED-SCIP as a statewide surveillance system that is based on voluntary reporting from a representative sample of hospitals. The reporting system is being implemented in a stratified random sample of 20 of the state’s 79 hospital EDs. The NEISS is comprised of 100 hospital EDs that are a stratified probability sample of hospitals in the United States that have at least six beds and provide 24-hour emergency care.

The ED-SCIP substudy involved 4 public health professionals from the MA Department of Public Health who coded medical records in 16 of the 20 ED-SCIP hospital emergency departments. The NEISS substudy involved 7 on-site data abstractors who routinely code data from ED records for product-related injuries at 7 NEISS hospital EDs. Sample hospitals in both substudies consisted of hospitals located in rural, urban, suburban, and inner-city areas.

Training of Coders and Quality Assurance Methods

The training of coders was conducted independently for the two substudies. For the ED-SCIP substudy, coders were mailed a coding manual and the coding rules and definitions for their review approximately 10 days in advance of the training session. The ED-SCIP coders attended an 8-hour training session with a detailed presentation and discussion of the study protocol, coding rules and definitions, and specific training exercises. For the NEISS substudy, coders were not given training material prior to attending a 2-hour orientation to the study protocol and coding rules and definitions.

For the ED-SCIP substudy, coders were asked to conduct a prepilot test at two study hospitals using the short ICECI data collection form and to provide feedback to the CDC investigators. A conference call was conducted to clarify how to interpret and apply coding rules and definitions. For the NEISS substudy, coders were asked to proceed without prepilot testing. Both ED-SCIP and NEISS substudy coders then followed similar study protocols for the case scenario test and the field test. Completed data collection forms were reviewed for completeness and mailed to CDC for data processing and analysis. At CDC, data were key-entered, checked electronically for valid codes, and visually reviewed for accuracy of data entry. In some cases, coders did not follow the skip patterns correctly in filling out the data collection form. All forms were key-entered as they were completed by the coder; therefore, these errors were included in the analysis of percent agreement.

Protocol for the Case Scenario Test

One hundred case scenarios and “gold standard” code sets were prepared for this test. (Note: These same case scenarios were also used by Malinda Steenkamp and James Harrison, Australia’s National Injury Surveillance Unit, to conduct a pilot study of inter-coder reliability using the full ICECI.) Case scenarios represented a wide variety of injury-related circumstances including unintentional injuries, intentionally self-inflicted injuries, assaults, and legal interventions. Seven of the case scenarios did not involve injuries at all. Gold standard code sets were established by having three coinvestigators code each case independently and then arrive at consensus on the appropriate codes based on the coding rules and definitions.

Coders participating in the short ICECI pilot study were asked to code 50 case scenarios prior to the field test and 50 case scenarios at the end of the field test. About two weeks after the coders completed the second set of 50 case scenarios, they were asked to code an additional 20 cases. Coders were not told that these additional cases represented recodes of cases selected from the second set of 50 case scenarios. These data were then used to measure validity and reliability by estimating percent agreement for (1) codes assigned by each coder with the gold standard (to measure accuracy in applying coding rules and definitions), (2) codes assigned by multiple coders (to measure inter-coder reliability), and (3) codes assigned by the same coder (to measure intra-coder reliability).

Protocol for the Field Test

For the field test, coders were asked to review medical records of approximately 100 injury-related ED cases and to code all pertinent data elements using the short ICECI data collection form. Adverse effects of therapeutic use of drugs and adverse effects of medical and surgical care were excluded. For the ED-SCIP substudy, ED cases were randomly selected from cases with a principal diagnosis of an injury or poisoning (ICD-9-CM diagnosis codes of 800–999), which were in the preexisting ED-SCIP injury surveillance database of each sample hospital. For one of the larger inner-city hospitals, assaults were oversampled to increase the number of intentional injuries in our study. In addition, 5–10 medical records of injury-related cases at each hospital were randomly chosen to be coded independently by at least two coders for use in measuring inter-coder reliability. Because the ED-SCIP hospital ED data obtained in this field study are not representative of all ED injury-related visits in the state, they cannot be used to project statewide injury incidence by external cause of injury.

For the NEISS substudy, ED cases were selected to represent a broad spectrum of external causes of injury. This was done to examine the use of the short ICECI for capturing information on a wide variety of injury-related circumstances for injured persons treated in hospital EDs.

Statistical Methods Used to Assess Gold Standard, Inter-coder, and Intra-coder Comparisons

The kappa statistic, expressed as a percent, was used as a measure of agreement to test for validity and reliability. This statistic provides a new dimension to the percent observed agreement by assuming that, except in most extreme cases, some degree of agreement is to be expected by chance alone. The estimated kappa was calculated as [(p_o – p_e)/(1–p_e) x 100], is the expected agreement based on chance.⁴ where p_o is the observed agreement and p_e Their difference, (p_o – p_e), represents the obtained excess agreement beyond chance, while the maximum possible excess agreement beyond chance is represented by the quantity (1–p_e). The ratio of these two, or the kappa statistic, can be interpreted as the percent agreement among coders beyond that which is expected by chance. The kappa statistic was used to compare (1) each individual coder’s ratings with the gold standard (validity), (2) all substudy coders’ ratings simultaneously (inter-coder reliability) and (3) each coder’s rating with his or her own ratings for repeated cases (intra-coder reliability). Landis and Koch⁵ suggest that values greater than 75% may be taken to represent excellent agreement beyond chance, values below 40% to represent poor agreement beyond chance, and values between 40% and 75% to represent fair to good agreement beyond chance. Standard errors of kappas were calculated using a method, as described by Fleiss et. al.,⁶ that accounts for different sets of coders for different cases. Standard errors were used to compute 95% confidence intervals (CIs) for all kappa statistics.

For the analysis of mechanism of injury, we created an analytic variable with 15 categories, excluding adverse effects of drugs and adverse effects of surgical and medical care, with all 14 major mechanisms of injury and an “other specified” category. This variable was constructed to reflect the immediate or most direct cause of the most severe injury being treated. The most severe injury was determined by the principal diagnosis of the physician at the time of the ED visit. Only first-time visits for an injury were included in the study.

Pilot Study Participant Survey

Each coder was asked to complete a survey to provide coinvestigators with feedback on his or her experience in using the short ICECI.

Results

Results of the Case Scenario Test

Ninety-three of the 100 case scenarios were injury-related cases; the remaining seven did not meet the definition of an injury case. The latter case scenarios were purposely included to see if coders would be able to distinguish injury-related cases from non-injury cases. The ability to classify a case as an injury versus non injury varied substantially among coders. Among the four Massachusetts coders, two recognized 5 of 7 non-injury cases and two recognized only 1 of the 7 non-injury cases. Also, 1 or 2 of 93 injury-related case scenarios were classified as non-injury cases by two of the four Massachusetts coders. Among the seven NEISS coders, three recognized all 7 non-injury cases, and the others recognized between 1 and 5 of the 7 non-injury cases. From 1 to 3 of 93 injury-related case scenarios were classified as non-injury cases by 3 of the 7 NEISS coders.

For injury-related cases, we found no significant differences in percent observed agreement or kappa statistics for case scenarios coded before or after the field test. Therefore, all 93 injury-related case scenarios were combined for further analysis.

For the 93 injury-related case scenarios, the percent observed agreement among MA ED-SCIP coders and between MA ED-SCIP coders and the gold standard for five major data elements (i.e., work-relatedness, locale of injury incident, activity at time of injury, intent of injury, and mechanism of injury) were observed to be quite high (Table 1). The observed inter-coder agreement ranged from 85.7% to 97.8%. The average percent observed agreement for coder and gold standard code sets ranged from 80.6% to 94.1%. However, a better measure of reliability can be obtained by using the kappa statistic, a measure of percent agreement beyond what can be expected by chance alone.

The average kappa (in percent) between coder and gold standard code sets and among coders for the five major short ICECI data elements were all in the good to excellent range for both the Massachusetts ED-SCIP and NEISS substudies (Figure 1, Tables 2 and 3). Kappas for MA coders were consistently higher than for NEISS coders for all five major short ICECI data elements (Table 4). Overall, excellent agreement was observed among the MA coders and between the MA coders and the gold standard.

Intra-coder reliability for the Massachusetts ED-SCIP substudy, as measured by average kappas, was also found to be in the excellent range or borderline excellent range for all five major data elements (Table 5).

Results of the Field Test

Inter-coder reliability, as measured by kappa, for randomly selected pairs of MA coders for a set of 127 injury-related ED cases shows percent agreement in the good to excellent range for most of the categories reported within four of the five major data elements (Table 6). Intent of injury was not included in this Table because all but four of the 127 cases were unintentional injuries; therefore the expected agreement was very high. For mechanism of injury, inter-coder reliability achieved 100% agreement for motor vehicle occupant, pedestrian-vehicle crash, poisoning, and foreign body. The lowest kappa was 83.5 for struck by/against.

The median coding time for abstracting data from ED records and completing the data collection form was 3 minutes for the Massachusetts ED-SCIP substudy and 4 minutes for the NEISS substudy. Average coding time was significantly higher for NEISS coders (mean= 4.9 minutes, 95% CI= [4.7%, 5.1%], range=1–15 minutes) than for Massachusetts coders (mean=3.4 minutes, 95% CI= [3.3%, 3.5%], range=1–18 minutes). NEISS coders were much more likely to record multiple mechanisms of injury (i.e., 2 or more mechanisms for the same case) than MA coders. For NEISS coders, 22.9% of 705 injury-related ED cases were coded with multiple mechanisms of injury. For MA coders, 2.6% of 1,710 injury-related ED cases were coded with multiple mechanisms of injury. Most of the cases with multiple mechanisms involved combinations of “struck by/against” and “fall” or of “struck by/against” and “cut/pierce.” This difference in recording multiple mechanisms of injury did not affect our calculations of percent observed agreement or kappa statistics for mechanism of injury (presented in Tables 1–6), because we limited our analysis to only the immediate, or most direct, cause of injury.

A description of short ICECI data for 1,841 injury-related ED patients obtained from review of medical records at 16 Massachusetts ED-SCIP hospitals is presented in Table 7. (NOTE: This number is higher than the 1,710 reviewed by MA coders; it includes an additional 131 cases reviewed by CDC coinvestigators during a site visit to two of the 16 hospitals.) A majority of data abstractions took between 2 and 5 minutes to complete. The sex and age of the patient were readily available in the medical record. Almost 15% of the cases abstracted were determined to be work-related. However, about 20% of the time, no information was available to determine work-relatedness. Locale of injury incident and activity at the time of injury could not be classified in about 40% of cases. Intent of injury and mechanism of injury could be classified in over 98% of cases. Of 200 assault cases, the relationship of perpetrator to victim and the reason for assault could be classified for about 55% of injury incidents. For 218 motor vehicle occupant-related cases, traffic-relatedness, type of motor vehicle in which the patient was riding, and the occupant status of the patient could be determined for over 90% of the injury incidents. Counterpart for the motor vehicle crash was classified in 60% of cases. The number of pedestrian, motorcyclist, and pedal cyclist cases was relatively small in this sample. However, counterpart for the injury incident could be classified in over 90% of these types of transportation-related cases. Traffic-relatedness could not be determined for 55% of pedal cyclist injuries. For cases where the mechanism of injury was struck by/against, the source of force and type of force could be determined in over 95% of injury incidents. The type of firearm used in gunshot injuries was recorded for only about half of the cases. Injuries where the mechanism of injury was stab/cut/pierce, fire/burn, or poisoning could be further characterized by type in over 89% of cases. Other specified mechanisms of injury were predominantly bites and stings.

Use of safety equipment was indicated for 238 of the 1,841 Massachusetts ED-SCIP cases. Most of these cases were associated with seat belt use and air bag deployment. Of the 218 motor vehicle occupant-related cases, seat belt use was indicated for 127 cases (58.3%), and air bag deployment was recorded for 19 cases (8.7%).

A listing of consumer products and a narrative description of the injury incident were provided, where applicable, on completed short ICECI data collection forms. NEISS and MA coders, in general, provided good details about the injury incident that further characterized the intent of injury, mechanisms of injury, and other circumstances of the injury incident. NEISS coders routinely recorded the principal diagnosis of the injury in the narrative field, as this is common practice in their injury surveillance activities.

Discussion

This pilot study suggests that CDC’s short ICECI can be a useful surveillance tool for capturing external-cause-of-injury data on hospital emergency department visits through the use of medical records. The current version needs to be revised and refined to improve its utility and efficiency. However, this pilot study did demonstrate that with the use of this coding system in the United States, valid and reliable data can be captured in a timely manner. Further development of CDC’s short ICECI as an ED surveillance tool will need to consider compatibility with the full ICECI, ICD-10, and ICD-10-CM coding systems. CDC’s short ICECI has code sets for each of the fundamental components of these systems, including work-relatedness, locale of the injury incident, activity at the time of injury, intent of injury, and mechanism of injury. It also has components to capture data on safety equipment use, consumer products involved, and provides a narrative description of the injury circumstances. Information in the narrative could be used for more detailed coding that uses the full ICECI and the ICD coding systems.

Based on data from a 1997 pilot study on expanding NEISS to capture all injuries, we estimate that for every injury death in the United States, approximately 190 injured persons are treated and released from hospital emergency departments.⁷ Interest in having a surveillance tool for timely capture of external-cause-of-injury data from hospital ED records to establish state and local injury morbidity data systems is increasing. CDC’s short ICECI could be a useful tool for obtaining population-based data on injury-related ED visits in state and local jurisdictions.

This pilot study of CDC’s short ICECI brought to light a number of aspects that are relevant to future development and implementation of the full ICECI and various short versions of the ICECI for use in injury morbidity data systems. Below is a discussion of each of these aspects based on our findings:

1. Protocols and instruction manuals for orientation and training of coders and for ongoing quality assurance activities are essential to ensuring valid and complete coding of external-cause-of-injury data.

Kappa statistics for the Massachusetts ED-SCIP substudy were consistently higher than those for the NEISS substudy. This difference in kappas could be attributed to a number of factors: coders in the Massachusetts ED-SCIP substudy had the training material in advance, took part in a substantially enhanced training session, and were subject to an ongoing quality assurance effort. A concerted effort was made to provide extensive up-front training and orientation and ongoing feedback to coders in the Massachusetts ED-SCIP substudy. This effort, including an 8-hour training session, a prepilot test at two hospitals, ongoing com-munication by conference call, and a site visit by CDC coinvesti-gators, resulted in excellent agreement for code sets among coders and between the gold standard and each coder for the case scenarios and hospital ED cases.

A detailed training module with practice case scenarios was used to instruct coders to properly apply coding rules and definitions. After the pilot study, coders commented that the training module would have been more effective if it had included practice exercises using actual ED records followed by immediate feedback from coinvestigators. Prepilot visits by coders to two Massachusetts ED-SCIP hospitals helped to reveal ambiguities in the coding guidelines. Follow-up discussions between CDC coinvestigators and the MA coders helped to clarify coding rules and definitions before proceeding to other study hospitals.

2. A detailed, user-friendly coding manual is needed to provide the case definition, coding rules and definitions, and guiding principles for coding.

A reference manual with clear definitions and detailed examples of inclusions and exclusions for each data element and associated response categories is essential for coders. This set of standard coding guidelines should be readily available to each coder to ensure consistency and uniformity in how codes are assigned. In addition, coders must refer to these documents often; otherwise, coders could have a tendency to base their coding decisions on their own intuitions, opinions, or preconceived notions.

In the pilot study, coders were given a detailed manual with coding rules and definitions, inclusions, and exclusions for use as a reference during the study. Whenever coders questioned how to code a case, they were instructed to refer to the manual. Then, if the manual was unclear, the CDC coinvestigators would clarify and interpret the coding rules and definitions. To maintain consistency in coding, coders were then notified about these coding decisions. Changes to the coding rules and definitions were then noted in the manual.

3. The data collection instrument should facilitate efficiency in the collection and processing of ICECI data from medical records; electronic versions are needed.

The check-box type format of the short ICECI data collection form worked well in recording and coding external-cause-of-injury data using medical records. The average time to code all short ICECI data elements from a medical record was about 3 to 4 minutes. However, coders sometimes failed to code fields or to follow skip patterns. These problems could be avoided by developing a computer data entry screen with drop-down windows revealing code choices. Selections could then easily be made by the click of a computer mouse. Automatic skip patterns and range checks could be programmed into this system to minimize recording errors. The data could be handled entirely electronically to perform additional checks for completeness and consistency. These electronic data could then be passed to a data base for further quality assurance analysis and feedback to the coders. Final edited data could be prepared for timely analysis and reporting.

CDC is currently working with CPSC to expand NEISS to capture all types and causes of injuries treated in hospital EDs. The new system is called the NEISS All Injury Program. CPSC is developing such an automated system for capturing data on an expected 600,000 injury-related ED cases annually from a nationally representative sample of 100 U.S. hospital EDs. NEISS is capturing data on intent of injury and mechanism of injury in addition to the routine injury-related data elements historically collected in NEISS (i.e., age, sex, race/ethnicity, diagnosis, primary body part affected, locale of injury incident, work-relatedness, consumer products, disposition at ED discharge, and a narrative description of the diagnosis and circumstances of the injury incident). Intent of injury and mechanism of injury were derived as modified data elements from CDC’s short ICECI.

4. Coding the mechanism of injury should be limited to two mechanisms: the precipitating cause (i.e., the cause that initiated the chain of events leading to the injury) and the immediate cause of injury (i.e., the direct cause of the most severe injury being treated).

Coding practices of NEISS and MA coders varied substantially in the number of ED cases that were assigned more than one mechanism of injury. NEISS coders were much more likely to use more than one mechanism. The data collection form was set up to capture as many mechanisms as applicable. Sometimes coders assigned several mechanism codes to account for multiple injuries noted in the hospital ED record, making it difficult to interpret the data. This problem could be eliminated by specifying the following rules:

Code the intent and mechanism of injury for the most severe injury being treated. This is usually the injury indicated as the principal diagnosis by the attending physician or other health care provider.

Code only two mechanisms of injury: the precipitating cause (i.e., the mechanism that initiated the chain of events that led to the injury) and the immediate cause (i.e., the mechanism directly responsible for the most severe injury being treated).

We plan to modify the short ICECI data collection form to accommodate these rules. The NEISS All Injury Program is using these rules to code intent and mechanism of injury.

5. A narrative description of the injury incident should be included as an essential ICECI data element. Narratives can be very helpful in quality assurance reviews of the assigned codes and for capturing further details about injury-related circumstances.

A narrative description of the injury incident should be an essential component of the full version and all short versions of the ICECI. We found that these brief descriptions of the injury circumstances were invaluable in reviewing how coding rules and definitions were applied by coders and allowed us to provide timely feedback to the coders. The narratives can also be useful to researchers who need more specific details about the causes and circumstances of the injury incident.

6. Training and orientation of hospital medical staff is imperative to improve information on intent of injury, mechanism of injury and other injury-related circumstances provided in the medical record.

To maintain objectivity in coding for the pilot study, coders were instructed to record information only if it was provided in the medical record. Therefore, coders were often not able to capture data on locale of the injury incident, activity at the time of injury, and other circumstances because the information was not provided in the medical record. Also, infor-mation provided in EMS reports, nurses’ notes, and doctors’ notes were sometimes in disagreement. Research needs to be conducted on methods to improve these data sources. EMS technicians, nurses, and doctors need to be educated about their important role in obtaining high-quality external-cause-of-injury data.

7. Proposed short ICECI data elements and code sets should be cross-walked to the full ICECI, ICD-10, and ICD-10-CM code sets as a measure of compatibility and cross-reference. The full ICECI should serve as a standard coding system for all proposed short ICECIs.

All proposed short ICECI coding systems must map to the full ICECI data elements and code sets. In this way, the full ICECI can serve as the standard coding system for all short ICECIs. A process is underway for further development of the ICECI for use in the ICD coding framework. Therefore, proposed short ICECIs also need to consider compatibility with ICD-10. CDC’s short ICECI has been cross-walked to the current version of the full ICECI and to the ICD-10. We also plan to cross-walk the short ICECI to ICD-10-CM once this classification system has been finalized in the United States.

Next Steps for CDC’s Short ICECI

This pilot study suggests that CDC’s short ICECI, in its current form, can be useful as a surveillance tool for obtaining external-cause-of-injury data from hospital ED records and can serve as a basic instrument for coding external-cause-of-injury data in national, state, and local ED surveillance systems. As mentioned previously, CDC’s short ICECI is being adapted for use in capturing intent of injury and mechanisms of injury in the newly expanded NEISS All Injury Program.

Modifications to CDC’s short ICECI will improve its usefulness and efficiency in capturing and coding high-quality data. We plan to develop version 2.0 of CDC’s short ICECI based on findings from this pilot study. Revisions will improve the data collection form; instruction manual with case definition, coding rules and definitions, and guiding principles for coding; and the training module with coding guidelines and practice exercises. After further pilot testing and refinement of version 2.0, CDC’s short ICECI will be made available in electronic and hard copy forms. Injury researchers and public health professionals should find CDC’s short ICECI to be a useful tool for injury surveillance activities at the local, state, and national levels in the United States, and internationally.

References

1. WHO-Working Group for Injury Surveillance Methodology Development and Its Technical Group. International Classification of External Causes of Injuries (ICECI). Amsterdam: Consumer Safety Institute, June 1999.

2. Holder Y, Krug E, et al. World Health Organization Injury Surveillance Guidelines for Less-Resourced Environments. Draft proposal released for comment at the 5th World Conference of Injury Prevention and Control; 2000 Mar 5–8; New Delhi (India). Atlanta, GA: National Center for Injury Prevention and Control, CDC, February 16, 2000.

3. Annest JL, Pogostin CL, Conn J, Fingerhut LA, Pickett D. Proposed short version of the International Classification of External Causes of Injuries (ICECI). Chapter 6. In: Proceedings of the International Collaborative Effort on Injury Statistics, Vol III, 2nd Symposium; 1999 June; Washington (DC). Available at www.cdc.gov/nchs/about/ otheract/ice/pro-iii.htm. Accessed April 2000.

4. Fleiss JL. Statistical Methods for Rates and Proportions. 2nd ed. New York, NY: John Wiley & Sons, 1981: 212–225.

5. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–174.

6. Fleiss JL, Nee JC, Landis JR. Large sample variance of kappa in the case of different sets of raters. Psychological Bulletin 1979;86:974–977.

7. Quinlan KP, Thompson MP, Annest JL, Peddicord J, Ryan G, Kessler EP, McDonald AK. Expanding the National Electronic Injury Surveillance System to monitor all nonfatal injuries treated in U.S. hospital emergency departments. Annals of Emergency Medicine 1999;34:637–645.

Table 1. Percent Observed Agreement among Massachusetts ED-SCIP Coders and between Coders and the Gold Standard for Five Major Data Elements, Overall and by Coder: 93 Injury-Related Case Scenarios, CDC’s Short ICECI Pilot Study

	Inter-coder*	Gold/Coder A	Gold/Coder B	Gold/Coder C	Gold/Coder D	Gold/Overall
Data Element	% Observed Agreement	% Observed Agreement	% Observed Agreement	% Observed Agreement	% Observed Agreement	Average % Observed Agreement
Work-Relatedness	97.8	94.6	94.6	95.7	91.4	94.1
Locale of Injury Incident	91.9	81.9	80.8	81.9	79.9	81.1
Activity at Time of Injury	85.7	80.7	86.2	77.5	79.7	81.0
Intent of Injury	96.9	89.3	91.4	89.2	90.3	90.1
Mechanism of Injury	86.5	76.6	87.4	79.7	78.8	80.6

* All four Massachusetts coders coded these 93 injury-related case scenarios. The inter-coder percent observed agreement was calculated as the percentage of cases where at least 3 of the 4 coders assigned the same code with respect to each of the five major data elements listed.

Table 2. Kappa Statistics^* among Massachusetts ED-SCIP Coders and between Coders and the Gold Standard for Five Major Data Elements, Overall and by Coder: 93 Injury-Related Case Scenarios, CDC’s Short ICECI Pilot Study

	Inter-coder	Gold/Coder A	Gold/Coder B	Gold/Coder C	Gold/Coder D	Gold/Overall
Data Element	Kappa (95% CI)	Kappa (95% CI)	Kappa (95% CI)	Kappa (95% CI)	Kappa (95% CI)	Average Kappa
Work-Relatedness	81.1	84.0	83.4	86.5	76.4	82.4
	(74.2-88.1)	(65.9-102)	(64.5-102)	(67.6-105)	(58.8-93.9)	(73.2-91.5)
Locale of Injury Incident	82.6	78.0	76.5	78.1	75.4	77.1
	(79.2-85.9)	(69.7-86.3)	(68.2-84.9)	(69.9-86.4)	(67.2-83.7)	(72.9-81.2)
Activity at Time of Injury	73.6	77.6	83.8	73.9	76.3	77.9
	(70.4-76.9)	(69.7-85.5)	(75.8-91.8)	(65.9-81.8)	(68.3-84.3)	(74.0-81.9)
Intent of Injury	89.7	81.2	84.3	80.3	82.5	82.1
	(84.2-95.2)	(68.8-93.7)	(71.3-97.3)	(67.1-93.4)	(69.7-95.4)	(75.6-88.5)
Mechanism of Injury	79.7	72.8	85.1	76.3	75.3	77.4
	(76.6-82.7)	(65.2-80.3)	(77.6-92.6)	(68.5-84.0)	(68.0-82.7)	(73.6-81.1)

* The kappa statistic measures the percent agreement beyond what can be expected by chance alone. Values of kappa >75% are considered to be excellent, and values between 40% and 75% represent fair to good agreement.⁵

^† The inter-coder kappa statistic measures agreement for all four coders simultaneously.

Table 3. Kappa Statistics^* among NEISS Coders and between Coders and the Gold Standard for Five Major Data Elements, Overall and by Coder: 93 Injury-Related Case Scenarios, CDC’s Short ICECI Pilot Study

	Inter-coder	Gold/Coder 1	Gold/Coder 2	Gold/Coder 3	Gold/Coder 4	Gold/Coder 5	Gold/Coder 6	Gold/Coder 7	Gold/Overall
Data Element	Kappa (95% CI)	Kappa (95% CI)	Kappa (95% CI)	Kappa (95% CI)	Kappa (95% CI)	Kappa (95% CI)	Kappa (95% CI)	Kappa (95% CI)	Average Kappa
Work- Relatedness	76.8	92.8	75.9	71.8	75.0	81.5	90.0	89.9	81.9
	(72.9-80.6)	(72.5-113)	(58.1-93.6)	(55.0-88.6)	(56.0-94.0)	(63.3-99.6)	(69.6-110)	(70.1-110)	(74.8-88.9)
Locale of Injury Incident	71.4	69.8	70.4	74.2	66.4	64.2	68.6	58.0	67.4
	(68.0-74.7)	(61.2-78.5)	(62.2-78.6)	(66.1-82.4)	(58.1-74.6)	(56.1-72.3)	(60.3-77.0)	(50.0-66.0)	(64.3-70.5)
Activity at Time of Injury	59.3	75.0	67.5	68.5	42.0	72.5	59.7	65.0	64.5
	(54.4-64.2)	(67.0-83.0)	(59.4-75.5)	(60.2-76.7)	(33.8-50.2)	(64.4-80.6)	(51.5-67.9)	(56.9-73.0)	(61.4-67.5)
Intent of Injury	74.2	85.0	83.1	75.6	72.1	76.5	77.2	74.4	77.7
	(71.5-76.9)	(72.3-97.6)	(70.7-95.6)	(63.2-88.0)	(59.7-84.6)	(64.2-88.8)	(64.5-89.9)	(61.9-86.9)	(73.0-82.4)
Mechanism of Injury	59.7	67.4	65.9	65.2	54.9	53.2	58.1	55.7	60.1
	(56.4-62.9)	(59.7-75.1)	(57.5-74.2)	(57.7-72.7)	(47.1-62.6)	(45.3-61.0)	(50.4-65.8)	(47.9-63.5)	(57.2-63.1)

* The kappa statistic measures the percent agreement beyond what can be expected by chance alone. Values of kappa >75% are considered to be excellent, and values between 40% and 75% represent fair to good agreement.⁵

^† The inter-coder kappa statistic measures agreement for all 7 coders simultaneously.

Table 4. Comparison of Kappa Statistics^* between Massachusetts ED-SCIP Coders and NEISS Coders for Five Major Data Elements: 93 Injury-Related Case Scenarios, CDC’s Short ICECI Pilot Study

	Inter-coder for MA	Inter-coder for NEISS	% Difference in Kappas	Gold/MA Coder	Gold/NEISS Coder	% Difference in Kappas
Data Element	Kappa	Kappa	(MA-NEISS) NEISS	Average Kappa	Average Kappa	(MA-NEISS) NEISS
Work-Relatedness	81.1	76.8	+5.6	82.4	81.9	+0.6
Locale of Injury Incident	82.6	71.4	+15.7	77.1	67.4	+14.4
Activity at Time of Injury	73.6	59.3	+24.1	77.9	64.5	+20.8
Intent of Injury	89.7	74.2	+20.9	82.1	77.7	+5.7
Mechanism of Injury	79.7	59.7	+33.5	77.4	59,1	+28.8

* The kappa statistic measures the percent agreement beyond what can be expected by chance alone. Values of kappa >75% are considered to be excellent, and values between 40% and 75% represent fair to good agreement.⁵

Table 5. Kappa Statistics^* for Massachusetts ED-SCIP Coders for Five Major Data Elements, Overall and by Coder: 18 Injury-Related Case Scenarios Coded Twice by the Same Coder at Different Times, CDC’s Short ICECI Pilot Study

	Coder A	Coder B	Coder C	Coder D	Intra-Coder Reliability
Data Element	Kappa (95% CI)	Kappa (95% CI)	Kappa (95% CI)	Kappa (95% CI)	Average Kappa
Work-Relatedness	87.7	100	67.2	79.3	83.2
	(48.6-127)	(60.6-139)	(29.8-105)	(43.3-115)	(64.3-102)
Locale of Injury Incident	100	100	84.5	92.4	94.2
	(77.7-122)	(77.7-122)	(60.7-108)	(70.4-114)	(83.1-105)
Activity at Time of Injury	93.1	86.0	72.2	100	88.0
	(70.9-115)	(63.1-109)	(49.4-94.9)	(75.2-125)	(77.0-98.9)
Intent of Injury	82.7	81.6	90.8	90.8	86.4
	(53.8-112)	(50.7-113)	(58.9-123)	(58.9-123)	(71.1-102)
Mechanism of Injury	70.8	77.7	78.6	70.3	74.4
	(48.7-92.9)	(55.6-99.8)	(57.1-100)	(48.4-92.2)	(63.5-85.3)

* The kappa statistic measures the percent agreement beyond what can be expected by chance alone. Values of kappa >75% are considered to be excellent, and values between 40% and 75% represent fair to good agreement.⁵

Table 6. Kappa Statistics^* among Randomly Selected Pairs of Massachusetts ED-SCIP Coders for Categories for Four Major Data Elements: 127 Injury-Related Hospital ED visits, CDC’s Short ICECI Pilot Study

		Inter-coder Reliability
Data Element and Categories†		Kappa (95% CI)
Work-Relatedness
	Overall	67.1 (54.3–80.0)
	Work-related	91.4 (74.0–109)
	Not work-related	67.6 (50.1–85.1)
Locale of Injury Incident
	Overall	73.5 (65.2–81.8)
	Home/mobile home	83.5 (65.9–101)
	Residential institution	100.0 (82.5–118)
	Farm/ranch	-----‡
	Street/highway	90.9 (73.4–108)
	Trade and service area	66.0 (48.4–83.5)
	Industrial/construction area	52.5 (34.9–70.0)
	School/educational area	66.3 (48.7–83.8)
	Other public building	100.0 (82.5–118)
	Sports and athletic area	65.9 (48.3–83.4)
Activity at Time of Injury
	Overall	67.5 (59.2–75.8)
	Sports	92.9 (75.5–110)
	Leisure	63.6 (46.2–81.0)
	Traveling	85.7 (68.3–103)
	Paid work	91.3 (73.9–109)
	Unpaid work	48.4 (31.0–65.8)
	Educational activity	-----
	Vital activity	-----
Mechanism of Injury
	Overall	91.4 (84.1–98.8)
	Motor vehicle (occupant)	100.0 (82.5–117)
	Pedestrian-vehicle crash	100.0 (82.5–117)
	Motorcycle	-----
	Pedal cycle	85.3 (67.9–103)
	Struck by/against or crushed	83.5 (66.0–101)
	Fall	94.1 (76.7–112)
	Gunshot, firearm-related	-----
	Stab/cut/pierce	89.5 (72.1–107)
	Fire/burn	88.5 (71.0–106)
	Smoke inhalation	-----
	Poisoning	100.0 (82.5–117)
	Near-drowning/drowning/submersion	-----
	Foreign body	100.0 (82.5–117)
	Overexertion	90.8 (73.3–108)

* The kappa statistic measures the percent agreement beyond what can be expected by chance alone. Values of kappa >75% are considered to be excellent, and values between 40% and 75% represent fair to good agreement.⁵

^† Intent of injury is not presented here because all but four cases were unintentional injuries; refer to Table 1 for percent observed agreement on this data element.

^‡ ----- denotes that no cases were reported for this category.

Table 7. Description of Short ICECI Data for Injury-Related ED Patients Obtained from 16 Massachusetts ED-SCIP Hospitals: CDC’s Short ICECI Pilot Study

				Number of	Percent
Data Element and Categories1				Patients	Distribution
Coding Time (in minutes)
	Total			1,841	100.0
		1 min		49	2.7
		2 mins		17	28.1
		3 mins		610	33.1
		4 mins		323	17.5
		5 mins		177	9.6
		6 mins		38	2.1
		7 mins		24	1.3
		8 mins		26	1.4
		9 mins		12	0.7
		10 mins		15	0.8
		11–18 mins		14	0.8
		Not recorded/unknown		36	2.0
Sex of Patient
	Total			1,841	100.0
		Male		1,088	59.1
		Female		748	40.6
		Not stated/unknown		5	0.3
Age of Patient (in years)
	Total			1,841	100.0
		00–04 yrs		189	10.3
		05–09 yrs		96	5.2
		10–14 yrs		132	7.2
		15–19 yrs		227	12.3
		20–24 yrs		228	12.4
		25–44 yrs		615	33.4
		45–64 yrs		228	12.4
		65+ yrs		115	6.2
		Not stated/unknown		11	0.6
Work-Relatedness
	Total			1,841	100.0
		Work-related		272	14.8
		Not work-related		1,210	65.7
		Not recorded/unspecified		359	19.5
Locale of Injury Incident
	Total			1,841	100.0
		Home/mobile home		401	21.8
		Residential institution		39	2.1
		Farm/ranch		2	0.1
		Street/highway		277	15.0
		Trade and service area		115	6.2
		Industrial/construction area		51	2.8
		School/educational area		47	2.6
		Other public building		20	1.1
		Sports and athletic area		83	4.5
		Other specified		82	4.5
		Not recorded/unknown		724	39.3
Activity at Time of Injury
	Total			1,841	100.0
		Sports		150	8.1
		Leisure		271	14.7
		Traveling		233	12.7
		Paid work		261	14.2
		Unpaid work		66	3.6
		Educational activity		4	0.2
		Vital activity		29	1.6
		Other specified		70	3.8
		Not recorded/unspecified		757	41.1
Intent of Injury
	Total			1,841	100.0
		Unintentional		1,580	85.8
		Intentionally self-inflicted		25	1.4
		Assault, confirmed or suspected		200	10.9
		Legal intervention		4	0.2
		Operations of war and civil insurrection		-----*	-----
		Not recorded/undetermined		2	1.7
Relationship of Perpetrator
to Victim for Assaults
	Total			200	100.0
		Spouse or partner		25	12.5
		Parent		3	1.5
		Other relative		11	5.5
		Unrelated care giver		-----*	-----
		Acquaintance or friend		17	8.5
		Official/legal authorities		1	0.5
		Multiple perpetrators		18	9.0
		Stranger		25	12.5
		Other specified person(s)		9	4.5
		Not recorded/unknown		91	45.5
Reason for Assault
	Total			200	100.0
		Altercation		91	45.5
		During illegal acquisition of		5	2.5
			money or property
		Drug-related		-----	-----
		Sexual assault		7	3.5
		Gang-related		-----	-----
		Other specified		8	4.0
		Not recorded/unknown		89	44.5
Mechanism of Injury
	Total			1,841	100.0
		Motor vehicle (occupant)		218	11.8
		Pedestrian-vehicle crash		12	0.7
		Motorcycle		21	1.1
		Pedal cycle		20	1.1
		Struck by/against or crushed		389	21.1
		Fall		416	22.6
		Gunshot, firearm-related		17	0.9
		Stab/cut/pierce		294	16.0
		Fire/burn		33	1.8
		Smoke inhalation		1	0.1
		Poisoning		35	1.9
		Near-drowning/drowning/submersion		-----	-----
		Foreign body		52	2.8
		Overexertion		191	10.4
		Other specified mechanism		132	7.2
		Not recorded/undetermined		10	0.5
Traffic-Relatedness for
Motor Vehicle Occupants
	Total			218	100.0
		Traffic-related		197	90.4
		Nontraffic-related		5	2.3
		Not recorded/unknown		16	7.3
Type of Motor Vehicle that Patient
Was Riding
	Total			218	100.0
		Automobile		183	83.9
		Pickup truck or van		8	3.7
		Heavy transport vehicle		1	0.5
		Bus		-----*	-----
		3-wheel motor vehicle		1	0.5
		Other specified		5	2.3
		Not recorded/unknown		20	9.2
Motor Vehicle Occupant Status
	Total			218	100.0
		Driver		141	64.7
		Passenger		69	31.7
		Person boarding or alighting		1	0.5
		Person on outside of motor vehicle		1	0.5
		Not recorded		6	2.8
Counterpart for Motor Vehicle Crash
Involving Motor Vehicle Occupant
	Total			218	100.0
		Automobile		73	33.5
		Pickup truck or van		3	1.4
		Heavy transport vehicle		1	0.5
		Bus		-----	-----
		3-wheel motor vehicle		-----	-----
		Motorcycle		-----	-----
		Railway train/vehicle		-----	-----
		Pedal cycle		-----	-----
		Pedestrian		-----	-----
		Animal or animal-drawn vehicle		-----	-----
		Fixed or stationary object		33	15.1
		No counterpart		16	7.3
		Other specified		5	2.3
		Not recorded/unknown		87	39.9
Traffic-Relatedness for
Pedestrian Injuries
	Total			12	100.0
		Traffic-related		9	75.0
		Nontraffic-related		2	16.7
		Not recorded/unknown		1	8.3
Counterpart for
Pedestrian-Vehicle Crash
	Total			12	100.0
		Automobile		8	66.7
		Pickup truck or van		-----*	-----
		Heavy transport vehicle		-----	-----
		Bus		-----	-----
		3-wheel motor vehicle		1	8.3
		Motorcycle		2	16.7
		Railway train/vehicle		-----	-----
		Pedal cycle		-----	-----
		Pedestrian		-----	-----
		Animal or animal-drawn vehicle		-----	-----
		Fixed or stationary object		-----	-----
		No counterpart		-----	-----
		Other specified		1	8.3
		Not recorded/unknown		-----	-----
Traffic-Relatedness for
Motorcyclist Injuries
	Total			21	100.0
		Traffic-related		14	66.7
		Nontraffic-related		2	9.5
		Not recorded/unknown		5	23.8
Motorcyclist Status
	Total			21	100.0
		Driver		13	61.9
		Passenger		8	38.1
		Person boarding or alighting		-----	-----
		Person on outside of motor vehicle		-----	-----
		Not recorded/unknown		-----	-----
Counterpart for Motorcycle Crash
	Total			21	100.0
		Automobile		3	14.3
		Pickup truck or van		-----*	-----
		Heavy transport vehicle		-----	-----
		Bus		-----	-----
		3-wheel motor vehicle		-----	-----
		Motorcycle		-----	-----
		Railway train/vehicle		-----	-----
		Pedal cycle		-----	-----
		Pedestrian		1	4.8
		Animal or animal-drawn vehicle		3	14.3
		Fixed or stationary object		3	14.3
		No counterpart		9	42.9
		Other specified		-----	-----
		Not recorded/unknown		2	9.5
Traffic-Relatedness for
Pedal Cyclist Injuries
	Total			20	100.0
		Traffic-related		8	40.0
		Nontraffic-related		1	5.0
		Not recorded/unknown		11	55.0
Counterpart for Pedal Cycle Crash
	Total			20	100.0
		Automobile		3	15.0
		Pickup truck or van		-----	-----
		Heavy transport vehicle		-----	-----
		Bus		-----	-----
		3-wheel motor vehicle		-----	-----
		Motorcycle		-----	-----
		Railway train/vehicle		-----	-----
		Pedal cycle		-----	-----
		Pedestrian		-----	-----
		Animal or animal-drawn vehicle		1	5.0
		Fixed or stationary object		2	10.0
		No counterpart		13	65.0
		Other specified		-----	-----
		Not recorded/unknown		1	5.0
Source of Force Applied for
Struck By/Against Injuries
	Total			389	100.0
		Human		111	28.5
		Animal		3	0.8
		Inanimate object or force		256	65.8
		Not recorded/unknown		19	4.9
Type of Force Applied for
Struck By/Against Injuries
	Total			389	100.0
		Struck by		219	56.3
		Crushed by		63	16.2
		Struck against		103	26.5
		Not recorded/unknown		4	1.0
Type of Firearm Used in
Gunshot Injuries
	Total			17	100.0
		Handgun		7	41.2
		Rifle		-----*	-----
		Shotgun		1	5.9
		Larger firearm		-----	-----
		Not recorded/unknown		9	52.9
Type of Stabbing Weapon, Instrument,
or Object in Cut/Pierce Injuries
	Total			294	100.0
		Knife		68	23.1
		Sharp instrument/tool other than knife		45	15.3
		Sharp glass		40	13.6
		Other specified		109	37.1
		Not recorded/unknown		32	10.9
Type of Burn in Fire/Burn Injuries
	Total			33	100.0
		Fire/flame		3	9.1
		Hot object		14	42.4
		Hot liquid		11	33.3
		Steam		1	3.0
		Chemical		1	3.0
		Other specified		2	6.1
		Not recorded/unknown		1	3.0
Type of Poisoning
	Total			35	100.0
		Drug (excludes alcohol)		25	71.4
		Alcohol		2	5.7
		Chemical		3	8.6
		Other specified		4	11.4
		Not recorded/unknown		1	2.9
Other Specified Mechanisms
	Total			132	100.0
		Railway/streetcar		-----*	-----
		Water transport		1	0.8
		Air transport		-----	-----
		Thrown or fallen from animal or		1	0.8
			animal-drawn vehicle
		Other transport		-----	-----
		Inhalation/ingestion of food		2	1.5
			(blocking airway)
		Inhalation/ingestion of other objects		3	2.3
			(blocking airway)
		Hanging or strangulation		3	2.3
		Suffocation		-----	-----
		Entrapment in closed space		-----	-----
		Venomous bite or sting		11	8.3
		Human bite		12	9.1
		Dog bite		24	18.2
		Bite by animal other than dog		31	23.5
		Sting		3	2.3
		Fireworks explosion		-----	-----
		Explosive blast		-----	-----
		BB or pellet gunshot		2	1.5
		Other firearm		-----	-----
		Lightning		1	0.8
		Electrical current		2	1.5
		Radiation		1	0.8
		Welding		1	0.8
		Machinery		6	4.5
		Exposure to excessive natural heat		2	1.5
		Exposure to excessive natural cold		1	0.8
		Sunlight		1	0.8
		Natural disaster		-----	-----
		Other specified		24	18.2

Figure 1. Average Kappa Statistic,^* with 95% Confidence Interval, between Coders and the Gold Standard, by Selected Data Elements and Substudy Groups: CDC’s Short ICECI Pilot Study

* The kappa statistic measures the percent agreement beyond what can be expected by chance alone.

Values of kappa >75% are considered to be excellent, and values between 40% and 75% represent fair to good agreement.⁵