Modification of the Renal Angina Index for identifying the need for renal replacement therapy in critically ill pediatric patients

Severe Acute Kidney Injury (AKI) is a common, serious problem affecting critically ill children that lacks effective treatment options. Currently, there are no treatment options for AKI other than supportive care. Continuous renal replacement therapy (CRRT) is employed to reduce Fluid Overload (FO) burden and treat metabolic disturbances in AKI. Identifying patients upon admission who may require CRRT has potential clinical care implications. The aim of this study was to determine if the RAI had diagnostic capabilities to identify patients who would require CRRT.

The analytic cohort consisted of patients who required CRRT and illness severity score matched controls who did not require CRRT at a single center. Patients who required CRRT had higher mortality rates, length of stay, and use of ventilatory and inotropic support. Sensitivity, specificity and area under the receiver operating characteristic curve (AUC) assessed and compared the discriminatory accuracy of three scores: 1) the renal angina index (RAI), 2) serum-creatinine (sCr) based AKI on day 0 and 3) modified RAI created with an additional RAI injury tranche that corresponded to severe stage 3 AKI sCr elevation.

Compared to Day₀AKI (AUC 0.78, 0.70-0.87; sensitivity 0.63, 0.45-0.79; specificity 0.93, 0.870.97) and RAI (AUC 0.76, 0.69-0.82; sensitivity 0.94, 0.81-0.99; specificity 0.57, 0.47-0.66), the modified RAI had the highest AUC (0.79; 0.72-0.85) with a high sensitivity (0.91; 0.77-0.98) and moderate specificity (0.65; 0.56-0.75) for prediction of CRRT requirements. As a more accurate tool for discriminating patients in need of CRRT, a modified RAI has numerous potential implications. Identifying patients who ultimately require CRRT at an earlier timepoint may influence timing of CRRT initiation in an attempt to avoid further FO, or may influence nephrotoxin administration. The diagnostic capabilities of the modified RAI may be refined by the addition of urinary biomarkers. These findings should be validated in a larger cohort.

Acute kidney Injury (AKI) is a common and serious problem among critically ill children admitted to the pediatric intensive care unit (PICU) and is a significant contributor to morbidity and mortality [1]. AKI occurs in as many as 25% of PICU patients, and severe AKI, defined by the Kidney Disease Improving Global Outcomes criteria (KDIGO) stage 2 or 3 affects approximately 12% of patients [1,2]. Greater AKI severity is associated with greater morbidity and risk of death than less severe AKI [1]. Currently there is no treatment for AKI and mainstays of therapy include supportive care: avoiding nephrotoxic medications and fluid overload (FO). Although serum creatinine (sCr) is an imperfect biomarker of AKI, clinicians continue to rely on its use which has led to the inability to institute therapeutic and supportive interventions in a timely fashion [3]. Earlier identification of patients who will develop AKI may enable providers to reduce AKI severity and improve or reduce the incidence of these poor outcomes.

The ability to identify future AKI has been improved by the development and utilization of the Renal Angina Index (RAI). The original RAI, which was designed to be calculated at 12 hours of PICU admission incorporates patient risk factors and early signs of renal injury [4]. The RAI is predictive of day-3 severe AKI (KDIGO stage 2 or 3) following PICU admission [5]. Variations of the RAI have also been developed, with an “acute” RAI having predictive capabilities for future AKI in septic Emergency Department subjects [6]. The injury score in both the original and acute RAI is based on estimated decline in creatinine clearance and corresponds to KDIGO AKI stages 0, 1, and ³2. Stage 3 AKI, which is characterized by severe elevation of sCr or need for renal replacement therapy (RRT) is not distinguished from stage 2 AKI in the original RAI prediction model.

Although the use of RAI has improved prediction of AKI, no tool has been developed to predict the most severe form of AKI – stage 3 AKI that requires continuous renal replacement therapy (CRRT), which is needed in approximately 1% - 2% of PICU population [1]. The most common indication for CRRT in the PICU population is FO, either alone or in conjunction with other AKI induced abnormalities such as acidosis and electrolyte disturbances [7]. While FO is independently associated with worse outcomes, there are also synergistic effects between FO and AKI that show an even larger effect on morbidity and mortality [8]. Compared to context-free creatinine increases, the RAI has improved AKI prediction accuracy by incorporating changes in both creatinine and FO and analyzing these changes in a stratified manner based on baseline patient risk factors [5]. The RAI could potentially be used to predict CRRT need as both the composition of RAI and indications for CRRT are heterogeneous and overlapping.

The purpose of this study is to compare three scoring modalities in their ability to discriminate between pediatric ICU patients in need of CRRT and those who do not need CRRT. We aimed to: 1) identify the performance of the RAI at 12 hours after PICU admission at various thresholds to identify an optimal threshold for discriminating between subjects who require CRRT during PICU admission and those who do not, 2) create a modified RAI score that delineated between creatinine associated injury scores corresponding to stage 2 and stage 3 AKI and 3) determine whether the modified RAI had improved diagnostic accuracy over that of the RAI and serum-creatinine based AKI on day 0 (Day₀AKI).

Study design

The study is a retrospective case control study that evaluated the ability of the renal angina index (RAI) to predict need for Continuous Renal Replacement Therapy (CRRT) at any point during PICU admission. At the time of this study, CRRT was initiated based on PICU and nephrology attendings’ joint decision and the RAI was not routinely assessed. Investigators first identified patients who underwent CRRT using an internal CRRT database initially developed for quality improvement methodology. Matching was then performed based on Pediatric Risk of Mortality III (PRISM-III) score which is an estimate of illness severity [9]. Pediatric Risk of Mortality III (PRISM-III) scores achieved prognostic population enrichment as previous RAI studies have shown that higher PRISM-III scores are associated with higher RAI scores. The Colorado Institutional Review Board approved the study with a waiver of informed consent.

Patient population

Inclusion criteria for cases of CRRT were: 1) age between 90 days and £25 years, 2) admitted to Children’s Hospital Colorado PICU between January 2009 and January 2017 and 3) record of CRRT after 12 hours and within 7 days from admission to the PICU. Controls were identified over the same time frame and matched with at least a 1:1 ratio to cases depending upon the available clinical data. Patients with chronic kidney disease (estimated glomerular filtration rate (eGFR) £15 ml/min/1.73m²), end stage renal disease on maintenance dialysis, need for extracorporeal membrane oxygenation, or who had history of kidney transplant within the preceding 90 days prior to admission were excluded. Patients who were identified as needing CRRT on admission (or who received CRRT prior to 12 hours after admission), including those with toxic ingestions, mitochondrial disorders, or hyperammonemia were also excluded.

Data extraction and predictor information: Data was manually extracted via chart review. Predictor variables included the RAI and Day₀AKI based on KDIGO serum creatinine staging [10]. The RAI was calculated at 12 hours after ICU admission. If there was no preceding baseline creatinine, one was estimated based on a previously validated method using patient height and an estimated baseline GFR of 120 ml/minute/1.73 m² [11].

A modified RAI was created post-hoc that delineated further extreme levels of kidney injury (Table 1). The original RAI’s upper injury tranche had a score of 8 which corresponded to a sCr of ³2x baseline and/or FO of ³15%. This injury tranche was unable to differentiate between moderate sCr increases of 2x baseline and more severe levels of injury, such as 3-4x baseline sCr elevations. Therefore, this injury tranche was divided into 2 potential new scores: 8 or 16. The modified score of 8 corresponds to a sCr ³2x baseline and < 3x baseline (and/or FO% ³15 and < 20%). The modified score of 16 corresponds to a sCr ³3x baseline and/or FO ³20%.

Outcomes

The primary outcome was the receipt of CRRT during PICU admission. Secondary outcomes included PICU length of stay, need for mechanical ventilation, need for vasoactive support, and hospital mortality rate.

Statistical analysis

Baseline demographics and clinical characteristics are summarized using count and proportion, mean and standard deviation, or median and interquartile. Fisher’s Exact, Cochrane-Mantell-Haenzel and Chi-square tests assessed for differences in categorical outcomes between groups. Wilcoxson Rank Sum Tests or unpaired t-tests compare continuous outcomes between groups. The receiver operating characteristic (ROC) curve was used to quantify the ability of various cutoffs on multiple modalities to identify subjects who required CRRT compared to those who did not. ROC analysis is a valid and accepted tool for evaluating the diagnostic accuracy on both a continuous and discrete scale [12]. Modalities included: 1) the Renal Angina Index (RAI), 2) serum-creatinine based AKI on day 0 (Day₀AKI) and 3) modified RAI created with an additional injury tranche that corresponded to severe stage 3 AKI serum creatinine (sCr) elevation Sensitivity, specificity and area under the receiver operating characteristic curve (AUC) assessed and compared the discriminatory accuracy of various scoring modalities. The largest Youden’s J identified the optimal cutoff for identifying future CRRT. Areas under the receiver operating characteristic curve (AUCs) for various composite cutoffs were compared via SAS procedure PROC LOGISTIC as has been previously describe [13]. Statistical significance for comparisons was set at 0.05. All analyses were conducted in SAS 9.3 (Cary, NC).

The study included 146 patients, with 35 patients requiring CRRT and 111 patients not requiring CRRT. Pediatric Risk of Mortality III (PRISM-III) scores ranged from 2 to 46 with the median score of 13.5 (IQR: 9 to 18). As expected by study design, there was no difference in PRISM-III scores between groups (p = 0.27). Table 2 summarizes the demographic and clinical characteristics of each group. Patients who required CRRT had higher serum creatinine values (sCr) on admission (p < 0.001), on day 3 (p < 0.001), and peak values (p < 0.001), as well as higher RAI scores (p < 0.001) when compared to patients who did not require CRRT. The most prevalent diagnosis group was shock, followed by central nervous system (CNS) disease and respiratory etiologies.

Patients who required CRRT had uniformly higher rates of morbidity and mortality (Table 3) with a longer length of stay (p < 0.001), higher rates of mechanical ventilation (p = 0.039), vasoactive medications (p < 0.001), and higher mortality rates (p = 0.024).

Association between RAI scores, Day₀AKI stage, and CRRT need: A comparison of diagnostic accuracies between the best cutoffs of established measures.

A larger proportion of patients who received CRRT had an elevated RAI score compared to those who did not (p < 0.001). Similar findings were seen with Day₀AKI where a higher proportion of patients who received CRRT had higher AKI stage (p < 0.001) (supplemental table 1).

The diagnostic accuracies at various cutoffs on either Day₀AKI Stage or RAI scores are shown in table 4. The optimal cutoff for creatinine-based Day₀AKI was found to be stage 3; the optimal cutoff for RAI was identified as ³8. AUCs were comparable at 0.78 (95% CI: 0.70, 0.87) and 0.76 (95% CI: 0.69, 0.82) respectively, although each scoring measure differed in sensitivity and specificity. The greatest specificity was 0.93 (95% CI: 0.87, 0.97) for Day₀AKI stage 3 while the greatest sensitivity was 0.94 (95% CI: 0.81, 0.99) for RAI score ³8. Day₀AKI stage 3 had the highest PPV of 0.76 (95% CI: 0.56, 0.90) while RAI ³8 had the highest NPV of 0.97 (95% CI: 0.89, 1.00). ROC curves for various cutoffs are shown in figure 1.

Modified RAI selectively increased the score in patients who required CRRT.

A modified RAI injury tranche to discriminate higher levels of injury that may be more likely to require CRRT was developed. This new injury tranche separated the previous highest RAI injury score into two separate injury tranche categories, which corresponded to moderate and severe renal injury. The modified RAI injury tranche is summarized in table 1, expanding the overall RAI score from 1-40 to 1-80. Creation of the modified RAI score resulted in an increased score in 24 (69%) of the 35 patients who required CRRT while 11 (31%) subjects did not change RAI score (Figure 2).

Of 111 patients who did not require CRRT, 22 (20%) had increased RAI score, while 89 (80%) did not change RAI score.

The optimal cutoff for the modified RAI was found to be ≥ 10 (Table 4); the modified RAI differentially shifted subjects who go on to need CRRT to higher modified RAI scores. The sensitivity and specificity at the modified RAI threshold of ≥ 10 are 0.91 (95% CI: 0.77, 0.98) and 0.65 (95% CI: 0.56, 0.75). Compared to the original RAI score, the modified RAI cutoff of ≥ 10 has a minimal decline in sensitivity with an improved specificity, resulting in a Youden’s index of 0.57 and an AUC of 0.79 (95% CI: 0.72, 0.85) (Figure 3). There was no significant difference in AUC between the modified RAI and either the original RAI (p = 0.12) or the Day₀AKI stage 3 cutoff (p = 0.94).

In this single center retrospective study, we found that a modified RAI score which includes an additional injury tranche can discriminate between patients who will or will not need future CRRT in the PICU. The modified RAI score had the greatest diagnostic accuracy as measured by AUC. Both Day₀AKI stage and original RAI score were able to discriminate these patients, although both had a slightly lower AUC when compared to the modified RAI. By using the modified RAI, the sensitivity for identifying patients with future CRRT need remains quite robust at > 0.9, and has improved specificity when compared to the original RAI. Similar to the ability of the original RAI to rule out subsequent AKI, the use of the modified RAI is able to effectively rule out the likelihood of requiring CRRT in this study. The ability to rule-out the need for CRRT could positively impact coordination of care, particularly around vascular access in the PICU.

This study also showed the dramatic increase in morbidity and mortality in patients who required CRRT which has been previously reported [7,14,15]. As expected by PRISM-III scores, the overall mortality and morbidity rates in this study were high overall, but CRRT patients had significant morbidity burden and mortality rates as compared to patients who did not require CRRT.

One might conclude that the observed decrease in sensitivity and increase in specificity observed at the optimal cutoff for the modified RAI simply result from an increase in the score of the cutoff. However, the differences in sensitivity and specificity, though expected by increasing the cutoff, do not originate from an increase in the score cutoff alone. Inclusion of more granular information for each given patient changes the ability to discriminate who did or did not need CRRT. We observe this by calculating the sensitivity and specificity for a RAI cutoff of 10 (0.66 and 0.72) and a modified RAI cutoff of 10 (0.91 and 0.65) and see a difference. Recall that the RAI cutoff of ≥ 8 had sensitivity and specificity of 0.94 and 0.57 respectively. Thus by using the modified RAI scale we maintain nearly equal AUC but increase the specificity by 0.10 with a modest drop in sensitivity of 0.03.

The poor outcomes of patients who require CRRT highlight the importance of being able to identify patients with early and severe signs of AKI who are likely to require CRRT, and the addition of a higher injury tranche to the modified RAI can identify these patients. The heterogeneous indications for CRRT are included in the modified RAI and thus are able to identify patients with extreme levels of kidney injury: severe FO and/or severe sCr elevation. Numerous studies have identified that severe FO is independently associated with higher mortality rates [16] and have found that patients with ³20% FO at CRRT initiation had a mortality rate of over 65% [17]. In pediatric ECMO patients, emerging evidence suggests that initiating CRRT at an earlier timepoint prior to the development of severe FO may improve mortality, and these findings may be generalizable to a general PICU population [18]. This highlights the potential improvement in patient outcomes by initiating CRRT at an earlier timepoint prior to the development of severe FO. Identifying these patients at the 12 hour timepoint with a modified RAI may therefore be able to improve their clinical care. Patients who are at risk of needing CRRT may also warrant closer monitoring of nephrotoxins or adjustment of medications that are renally metabolized.

Identifying these patients with early severe injury may also help guide targeted urinary biomarker measurement and may allow for even further improvement in the predictive capabilities for identifying either severe AKI or CRRT need. Previous incorporation of urinary biomarkers into the original RAI risk stratification dramatically improved the AUC for future AKI prediction by improving the specificity characteristics [19]. Since the sensitivity is similar between the original and modified RAI, the measurement of urinary biomarkers in this target group of severe early injury signs may improve the specificity and allow for further refinement of CRRT prediction. If the combination of the modified RAI+ urinary biomarkers has further improvement in diagnostic capabilities, this could be potentially be used to stratify subjects into an early versus late CRRT clinical trial.

There are numerous strengths of this study. This was the first analysis at utilizing RAI in a clinical context to identify subjects likely to require CRRT. Due to the inability of the original RAI to identify subjects with more extreme injury scores, a modified RAI was developed which fully incorporated the 3 stages of AKI based on KDIGO creatinine criteria. While this was a single center study, our findings provide an impetus to reanalyze a larger RAI database to determine whether these findings are generalizable to a much broader patient population [5].

This study has several limitations. Because of the study design, the population has a high prevalence rate of CRRT thus PPV may be overly optimistic and the NPV a conservative estimate of how the test threshold would perform in the general PICU population. The population analyzed in this study is small and from a single center and thus results may not be generalizable. A small number of our overall CRRT patients met our inclusion and exclusion criteria. However, the population was heterogeneous and encompassed a greater proportion of critically ill patients due to the higher PRISM-III scores. The measures of sensitivity and specificity remain consistent across smaller prevalence rates. The decision to start CRRT was not standardized and at the discretion of the intensivist in conjunction with the consulting nephrologist.

The creation of a modified RAI which delineated more severe forms of initial renal injury was able to improve diagnostic capabilities for prediction of future CRRT need. These initial findings should be assessed in a larger population to determine if they remain significant. The use of a modified RAI to predict CRRT needs has numerous potential and important clinical implications.

We thank Jenny Lusk, RN for maintenance of the quality improvement database of CRRT patients. Statistical support was supported by Children’s Hospital Colorado Department of Pediatrics and Research Institute via the Pediatric Kidney Injury & Disease Stewardship Group.

1. Kaddourah A, Basu RK, Bagshaw SM, Goldstein SL. Epidemiology of Acute Kidney Injury in Critically Ill Children and Young Adults. N Engl J Med. 2017; 376: 11-20. PubMed:  https://pubmed.ncbi.nlm.nih.gov/27959707/
2. Slater MB, Anand V, Uleryk EM, Parshuram CS. A systematic review of RIFLE criteria in children, and its application and association with measures of mortality and morbidity. Kidney Int. 2012; 81: 791-798. PubMed: https://pubmed.ncbi.nlm.nih.gov/22258324/
3. Goldstein SL. Acute kidney injury biomarkers: renal angina and the need for a renal troponin I. BMC Med. 2011; 9: 135.
4. Basu RK, Zappitelli M, Brunner L, Wang Y, Wong HR, et al. Derivation and validation of the renal angina index to improve the prediction of acute kidney injury in critically ill children. Kidney Int. 2014; 85: 659-667. PubMed: https://pubmed.ncbi.nlm.nih.gov/24048379/
5. Basu RK, Kaddourah A, Goldstein SL. Assessment of a renal angina index for prediction of severe acute kidney injury in critically ill children: a multicentre, multinational, prospective observational study. Lancet Child Adolesc Health. 2018; 2: 112-120. PubMed: https://pubmed.ncbi.nlm.nih.gov/30035208/
6. Hanson HR, Carlisle MA, Bensman RS, Byczkowski T, Depinet H, et al. Early prediction of pediatric acute kidney injury from the emergency department: A pilot study. Am J Emerg Med. 2020. PubMed: https://pubmed.ncbi.nlm.nih.gov/32024590/
7. Sutherland SM, Alexander SR. Continuous renal replacement therapy in children. Pediatr Nephrol. 2012; 27: 2007-2016. PubMed: https://pubmed.ncbi.nlm.nih.gov/22366896/
8. Gist KM, Selewski DT, Brinton J, Menon S, Goldstein SL, Basu RK. Assessment of the Independent and Synergistic Effects of Fluid Overload and Acute Kidney Injury on Outcomes of Critically Ill Children. Pediatr Crit Care Med. 2020; 21: 170-177. PubMed: https://pubmed.ncbi.nlm.nih.gov/31568240/
9. Pollack MM, Holubkov R, Funai T, Dean JM, Berger JT, et al. The Pediatric Risk of Mortality Score: Update 2015. Pediatr Crit Care Med. 2016; 17: 2-9. PubMed: https://pubmed.ncbi.nlm.nih.gov/26492059/
10. KDIGO Guidelines on AKI. Kidney International Supplements. 2012; 2.
11. Zappitelli M, Parikh CR, Akcan-Arikan A, Washburn KK, Moffett BS, et al. Ascertainment and epidemiology of acute kidney injury varies with definition interpretation. Clin J Am Soc Nephrol. 2008; 3: 948-954. PubMed: https://pubmed.ncbi.nlm.nih.gov/18417742/
12. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982; 143: 29-36. PubMed: https://pubmed.ncbi.nlm.nih.gov/7063747/
13. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988; 44: 837-845. PubMed: https://pubmed.ncbi.nlm.nih.gov/3203132/
14. Hayes LW, Oster RA, Tofil NM, Tolwani AJ. Outcomes of critically ill children requiring continuous renal replacement therapy. J Crit Care. 2009; 24: 394-400. PubMed: https://pubmed.ncbi.nlm.nih.gov/19327959/.
15. Riley AA, Watson M, Smith C, Guffey D, Minard CG, et al. Pediatric continuous renal replacement therapy: have practice changes changed outcomes? A large single-center ten-year retrospective evaluation. BMC Nephrol. 2018; 19: 268. PubMed: https://pubmed.ncbi.nlm.nih.gov/30340544/
16. Alobaidi R, Morgan C, Basu RK, Stenson E, Featherstone R, Majumdar SR, et al. Association Between Fluid Balance and Outcomes in Critically Ill Children: A Systematic Review and Meta-analysis. JAMA Pediatr. 2018; 172: 257-268. PubMed: https://pubmed.ncbi.nlm.nih.gov/29356810/
17. Sutherland SM, Zappitelli M, Alexander SR, Chua AN, Brophy PD, et al. Fluid overload and mortality in children receiving continuous renal replacement therapy: the prospective pediatric continuous renal replacement therapy registry. Am J Kidney Dis. 2010; 55: 316-325. PubMed: https://pubmed.ncbi.nlm.nih.gov/20042260/
18. Gorga SM, Sahay RD, Askenazi DJ, Bridges BC, Cooper DS, et al. Fluid overload and fluid removal in pediatric patients on extracorporeal membrane oxygenation requiring continuous renal replacement therapy: a multicenter retrospective cohort study. Pediatr Nephrol. 2020. 35: 871-882. PubMed: https://pubmed.ncbi.nlm.nih.gov/31953749/
19. Menon S, Goldstein SL, Mottes T, Fei L, Kaddourah A, et al. Urinary biomarker incorporation into the renal angina index early in intensive care unit admission optimizes acute kidney injury prediction in critically ill children: a prospective cohort study. Nephrol Dial Transplant. 2016; 31: 586-594. PubMed: https://pubmed.ncbi.nlm.nih.gov/26908772/