Two strategies should be used: Analyze outcome indicators by year to see if rates of mortality and complications come down by system improvement. Also compare trauma system outcome between different districts/branches. We use confidence interval analysis to compare difference of proportions and means.
Trauma mortality by year/period
Trauma System X | Patients treated | Fatalities |
Year 1 | 330 | 64 |
Year 2 | 440 | 57 |
Load the proportions in a statistical calculator, e.g. the CIA software, and find that the trauma mortality rate was reduced from 19.4% in year 1 to 13.0% in year 2. The reduction is statistically significant at the 95% level, the confidence interval (CI) for the difference being 1.1% to 11.7%.
Notice: If the 95% CI for difference does not contain zero, the difference is considered statistically significant at that given level of confidence (equivalent to p < 0.05).
You can compare two separate systems in the same way:
YEAR 1 | Patients treated | Fatalities |
Trauma System X | 330 | 64 |
Trauma System Y | 110 | 17 |
This year System Y had mortality rate at 15.5% as compared to 19.4% in System X. It seems that System Y has the better quality of performance. However, the samples are small in numbers, therefore the difference is not statistically significant; 95% CI ranges from – 4.1% to 11.9%.
Effect of prehospital life support
We will examine the prehospital treatment effect by year using RTS differences (PEKER):
Trauma System X | RTS2 – RTS1 Mean SD |
Year 1 | 1.1 0.65 |
Year 2 | 1.55 0.8 |
The difference is significant, 95%CI for difference 0.34 – 0.56. System X seems to have improved the prehospital performance during the period.
Analyzing trauma system quality: are the samples really comparable?
Is it fair to compare Year 1 and Year 2? Or System X with System Y? It is fair enough on the single condition that the distribution of the main risk factors is the same in both samples.
For this you need a computer program for medical statistics to compare the mean values and the spread (standard deviation, SD) of the observations. Let us say that System X in Year 1 had 330 patients with a mean ISS of 12.5 (SD 10.3) and the system in Year 2 had 440 patients with mean ISS of 9.8 (SD 11,1). Let us also assume that the local scene of injury changed so that total prehospital transport time (Time 2 PEKER) was 5.5 hours ( SD 3.1) in Year 1 and 3,0 hours (SD 4,1) in Year 2. This is the result of confidence interval comparisons:
System X | 95%CI for ISS difference | 95%CI for time difference |
Year 1 versus Year 2 | 1.2 – 4.2 ISS points | 2.0 – 3.0 hours |
So, both anatomical severity and prehospital transit times are clearly significantly different in Year 1 and in Year 2. That makes the two patient populations not statistically comparable regarding outcome indicators. So, our statistical comparisons on the previous page are not valid; we have to stratify the two populations before doing outcome comparisons.
Stratification
Take out slices/subsets of the two populations that have (approximately) the same distribution of the main risk factors. E.g.:
| Subset 1 | Subset 2 | Subset 3 |
ISS | Moderate: < 9 | Serious: 9 – 15 | Major Trauma: > 15 |
Time 2 | < 2 hours | 2 – 4 hours | > 4 hours |
RTS | > 10 | 8 – 10 | < 8 |
Now we are ready to compare outcome indicators (mortality, sepsis etc.) by year for each subset separately. Notice that stratification criteria should be defined after careful studies of the distribution and the impact of the actual explanatory variables (scatter plots and ROC analysis PEKER).
Warning
Quality comparisons between trauma systems are invalid if the social context is significantly different. E.g. it is not fair to compare fatality rates of high-tech urban Western systems with grassroots Chain-of-Survival Model systems in the rural South – even after stratification. Also differences in distribution of statistics make inter-system comparisons invalid.