Using statistical and machine learning methods to evaluate the prognostic accuracy of SIRS and qSOFA

Akash Gupta, Tieming Liu, Scott Shepherd, William Paiva

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Objectives: The objective of this study was to compare the performance of two popularly used early sepsis diagnostic criteria, systemic inflammatory response syndrome (SIRS) and quick Sepsis-related Organ Failure Assessment (qSOFA), using statistical and machine learning approaches. Methods: This retrospective study examined patient visits in Emergency Department (ED) with sepsis related diagnosis. The outcome was 28-day in-hospital mortality. Using odds ratio (OR) and modeling methods (decision tree [DT], multivariate logistic regression [LR], and naïve Bayes [NB]), the relationships between diagnostic criteria and mortality were examined. Results: Of 132,704 eligible patient visits, 14% died within 28 days of ED admission. The association of qSOFA ≥2 with mortality (OR = 3.06; 95% confidence interval [CI], 2.96–3.17) greater than the association of SIRS ≥2 with mortality (OR = 1.22; 95% CI, 1.18–1.26). The area under the ROC curve for qSOFA (AUROC = 0.70) was significantly greater than for SIRS (AUROC = 0.63). For qSOFA, the sensitivity and specificity were DT = 0.39, LR = 0.64, NB = 0.62 and DT = 0.89, LR = 0.63, NB = 0.66, respectively. For SIRS, the sensitivity and specificity were DT = 0.46, LR = 0.62, NB = 0.62 and DT = 0.70, LR = 0.59, NB = 0.58, respectively. Conclusions: The evidences suggest that qSOFA is a better diagnostic criteria than SIRS. The low sensitivity of qSOFA can be improved by carefully selecting the threshold to translate the predicted probabilities into labels. These findings can guide healthcare providers in selecting risk-stratification measures for patients presenting to an ED with sepsis.

Original languageEnglish
Pages (from-to)139-147
Number of pages9
JournalHealthcare Informatics Research
Volume24
Issue number2
DOIs
StatePublished - Apr 2018

Fingerprint

Systemic Inflammatory Response Syndrome
Decision trees
Learning systems
Logistics
Sepsis
Decision Trees
Logistic Models
Hospital Emergency Service
Odds Ratio
Mortality
Labels
Confidence Intervals
Machine Learning
Sensitivity and Specificity
Hospital Mortality
ROC Curve
Health Personnel
Area Under Curve
Retrospective Studies

Keywords

  • Artificial intelligence
  • Medical informatics
  • Sepsis
  • Severity of illness index
  • Systemic inflammatory response syndrome

Cite this

@article{2680c35dbba648c98c043828ec268706,
title = "Using statistical and machine learning methods to evaluate the prognostic accuracy of SIRS and qSOFA",
abstract = "Objectives: The objective of this study was to compare the performance of two popularly used early sepsis diagnostic criteria, systemic inflammatory response syndrome (SIRS) and quick Sepsis-related Organ Failure Assessment (qSOFA), using statistical and machine learning approaches. Methods: This retrospective study examined patient visits in Emergency Department (ED) with sepsis related diagnosis. The outcome was 28-day in-hospital mortality. Using odds ratio (OR) and modeling methods (decision tree [DT], multivariate logistic regression [LR], and na{\"i}ve Bayes [NB]), the relationships between diagnostic criteria and mortality were examined. Results: Of 132,704 eligible patient visits, 14{\%} died within 28 days of ED admission. The association of qSOFA ≥2 with mortality (OR = 3.06; 95{\%} confidence interval [CI], 2.96–3.17) greater than the association of SIRS ≥2 with mortality (OR = 1.22; 95{\%} CI, 1.18–1.26). The area under the ROC curve for qSOFA (AUROC = 0.70) was significantly greater than for SIRS (AUROC = 0.63). For qSOFA, the sensitivity and specificity were DT = 0.39, LR = 0.64, NB = 0.62 and DT = 0.89, LR = 0.63, NB = 0.66, respectively. For SIRS, the sensitivity and specificity were DT = 0.46, LR = 0.62, NB = 0.62 and DT = 0.70, LR = 0.59, NB = 0.58, respectively. Conclusions: The evidences suggest that qSOFA is a better diagnostic criteria than SIRS. The low sensitivity of qSOFA can be improved by carefully selecting the threshold to translate the predicted probabilities into labels. These findings can guide healthcare providers in selecting risk-stratification measures for patients presenting to an ED with sepsis.",
keywords = "Artificial intelligence, Medical informatics, Sepsis, Severity of illness index, Systemic inflammatory response syndrome",
author = "Akash Gupta and Tieming Liu and Scott Shepherd and William Paiva",
year = "2018",
month = "4",
doi = "10.4258/hir.2018.24.2.139",
language = "English",
volume = "24",
pages = "139--147",
journal = "Healthcare Informatics Research",
issn = "2093-3681",
publisher = "Korean Society of Medical Informatics",
number = "2",

}

Using statistical and machine learning methods to evaluate the prognostic accuracy of SIRS and qSOFA. / Gupta, Akash; Liu, Tieming; Shepherd, Scott; Paiva, William.

In: Healthcare Informatics Research, Vol. 24, No. 2, 04.2018, p. 139-147.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Using statistical and machine learning methods to evaluate the prognostic accuracy of SIRS and qSOFA

AU - Gupta, Akash

AU - Liu, Tieming

AU - Shepherd, Scott

AU - Paiva, William

PY - 2018/4

Y1 - 2018/4

N2 - Objectives: The objective of this study was to compare the performance of two popularly used early sepsis diagnostic criteria, systemic inflammatory response syndrome (SIRS) and quick Sepsis-related Organ Failure Assessment (qSOFA), using statistical and machine learning approaches. Methods: This retrospective study examined patient visits in Emergency Department (ED) with sepsis related diagnosis. The outcome was 28-day in-hospital mortality. Using odds ratio (OR) and modeling methods (decision tree [DT], multivariate logistic regression [LR], and naïve Bayes [NB]), the relationships between diagnostic criteria and mortality were examined. Results: Of 132,704 eligible patient visits, 14% died within 28 days of ED admission. The association of qSOFA ≥2 with mortality (OR = 3.06; 95% confidence interval [CI], 2.96–3.17) greater than the association of SIRS ≥2 with mortality (OR = 1.22; 95% CI, 1.18–1.26). The area under the ROC curve for qSOFA (AUROC = 0.70) was significantly greater than for SIRS (AUROC = 0.63). For qSOFA, the sensitivity and specificity were DT = 0.39, LR = 0.64, NB = 0.62 and DT = 0.89, LR = 0.63, NB = 0.66, respectively. For SIRS, the sensitivity and specificity were DT = 0.46, LR = 0.62, NB = 0.62 and DT = 0.70, LR = 0.59, NB = 0.58, respectively. Conclusions: The evidences suggest that qSOFA is a better diagnostic criteria than SIRS. The low sensitivity of qSOFA can be improved by carefully selecting the threshold to translate the predicted probabilities into labels. These findings can guide healthcare providers in selecting risk-stratification measures for patients presenting to an ED with sepsis.

AB - Objectives: The objective of this study was to compare the performance of two popularly used early sepsis diagnostic criteria, systemic inflammatory response syndrome (SIRS) and quick Sepsis-related Organ Failure Assessment (qSOFA), using statistical and machine learning approaches. Methods: This retrospective study examined patient visits in Emergency Department (ED) with sepsis related diagnosis. The outcome was 28-day in-hospital mortality. Using odds ratio (OR) and modeling methods (decision tree [DT], multivariate logistic regression [LR], and naïve Bayes [NB]), the relationships between diagnostic criteria and mortality were examined. Results: Of 132,704 eligible patient visits, 14% died within 28 days of ED admission. The association of qSOFA ≥2 with mortality (OR = 3.06; 95% confidence interval [CI], 2.96–3.17) greater than the association of SIRS ≥2 with mortality (OR = 1.22; 95% CI, 1.18–1.26). The area under the ROC curve for qSOFA (AUROC = 0.70) was significantly greater than for SIRS (AUROC = 0.63). For qSOFA, the sensitivity and specificity were DT = 0.39, LR = 0.64, NB = 0.62 and DT = 0.89, LR = 0.63, NB = 0.66, respectively. For SIRS, the sensitivity and specificity were DT = 0.46, LR = 0.62, NB = 0.62 and DT = 0.70, LR = 0.59, NB = 0.58, respectively. Conclusions: The evidences suggest that qSOFA is a better diagnostic criteria than SIRS. The low sensitivity of qSOFA can be improved by carefully selecting the threshold to translate the predicted probabilities into labels. These findings can guide healthcare providers in selecting risk-stratification measures for patients presenting to an ED with sepsis.

KW - Artificial intelligence

KW - Medical informatics

KW - Sepsis

KW - Severity of illness index

KW - Systemic inflammatory response syndrome

UR - http://www.scopus.com/inward/record.url?scp=85047460209&partnerID=8YFLogxK

U2 - 10.4258/hir.2018.24.2.139

DO - 10.4258/hir.2018.24.2.139

M3 - Article

AN - SCOPUS:85047460209

VL - 24

SP - 139

EP - 147

JO - Healthcare Informatics Research

JF - Healthcare Informatics Research

SN - 2093-3681

IS - 2

ER -