Visualizing Uncertainty in AI-Assisted Canine Acute-Pain Assessment: Dog-Grouped Risk Modeling and Information-Equivalent Interface Prototypes
DOI:
https://doi.org/10.51903/ijgd.v3i1.4272Keywords:
canine acute pain, uncertainty visualization, clinical decision support, probability calibration, information designAbstract
Canine acute-pain assessment combines behavioral observation, clinical context, and professional judgment, yet a model probability can be mistaken for a clinical pain score when the two lack a clear visual hierarchy. This study linked retrospective prediction-model findings to three information-equivalent interface prototypes. The structured database contained 594 observations from 184 dogs; 114 complete acute-pain observations from 65 dogs formed the primary cohort, with the short-form Glasgow Composite Measure Pain Scale (CMPS-SF) ≥ 6/24 as the reference outcome. A hypothetical pre-score feature set excluded direct score components, although the database did not establish recording order. Four calibrated models were evaluated with five repetitions of four-fold dog-grouped nested cross-validation. Logistic regression achieved an area under the receiver operating characteristic curve (AUROC) of .627 (95% CI [.474, .755]), an area under the precision–recall curve (AUPRC) of .476 [.317, .660], and Brier score .210 [.174, .249]. Sensitivity at 0.50 was .189, missing 30 of 37 threshold-positive observations. Full-development bootstrap fits supplied 90% model-stability ranges, distinct from patient-outcome intervals. Exploratory routing reduced Brier score but also reduced AUROC and changed retained event prevalence. An auxiliary vocabulary analysis identified 169 descriptors in 52 grouped rows. A separate transit-display analysis of 12,331 records from 299 participant identifiers did not establish an average payoff or duration advantage over a no-uncertainty display. The numeric dashboard, visual scale, and progressive evidence card retained identical information while varying hierarchy and disclosure. Their structural audit establishes content equivalence, not clinical effectiveness. The design contribution connects model limitations to visible range definitions, separate clinical-reference labels, missing-context cues, and a persistent structured-assessment action.
References
Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020). Explainability for artificial intelligence in healthcare: A multidisciplinary perspective. BMC Medical Informatics and Decision Making, 20, Article 310. https://doi.org/10.1186/s12911-020-01332-6
Ancker, J. S., Senathirajah, Y., Kukafka, R., & Starren, J. B. (2006). Design features of graphs in health risk communication: A systematic review. Journal of the American Medical Informatics Association, 13(6), 608–618. https://doi.org/10.1197/jamia.M2115
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324
Bremhorst, A. (2025). Dog pain database: A multidimensional dataset for investigating canine pain (Version 2) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.15304081
Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3. https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021). To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), Article 188, 1–21. https://doi.org/10.1145/3449287
Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. Chapman & Hall/CRC.
Fernandes, M., Walls, L., Munson, S., Hullman, J., & Kay, M. (2018). Uncertainty displays using quantile dotplots or CDFs improve transit decision-making. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Article 144, pp. 1–12). Association for Computing Machinery. https://doi.org/10.1145/3173574.3173718
Frans, N., Hummelen, B., Albers, C. J., & Paap, M. C. S. (2023). Visualizing uncertainty to promote clinicians’ understanding of measurement error. Assessment, 30(8), 2449–2460. https://doi.org/10.1177/10731911221147042
Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5), 1189–1232. https://doi.org/10.1214/aos/1013203451
Geurts, P., Ernst, D., & Wehenkel, L. (2006). Extremely randomized trees. Machine Learning, 63(1), 3–42. https://doi.org/10.1007/s10994-006-6226-1
Han, P. K. J., Klein, W. M. P., & Arora, N. K. (2011). Varieties of uncertainty in health care: A conceptual taxonomy. Medical Decision Making, 31(6), 828–838. https://doi.org/10.1177/0272989X11393976
Hansen, B. D. (2003). Assessment of pain in dogs: Veterinary clinical studies. ILAR Journal, 44(3), 197–205. https://doi.org/10.1093/ilar.44.3.197
Holton, L. L., Scott, E. M., Nolan, A. M., Reid, J., Welsh, E., & Flaherty, D. (1998). Comparison of three methods used for assessment of pain in dogs. Journal of the American Veterinary Medical Association, 212(1), 61–66. https://doi.org/10.2460/javma.1998.212.01.61
Holton, L., Pawson, P., Nolan, A., Reid, J., & Scott, E. M. (2001). Development of a behaviour-based scale to measure acute pain in dogs. Veterinary Record, 148(17), 525–531. https://doi.org/10.1136/vr.148.17.525
Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392
Lipkus, I. M. (2007). Numeric, verbal, and visual formats of conveying health risks: Suggested best practices and future recommendations. Medical Decision Making, 27(5), 696–713. https://doi.org/10.1177/0272989X07307271
Mathews, K., Kronen, P. W., Lascelles, D., Nolan, A., Robertson, S., Steagall, P. V. M., Wright, B., & Yamashita, K. (2014). Guidelines for recognition, assessment and treatment of pain. Journal of Small Animal Practice, 55(6), E10–E68. https://doi.org/10.1111/jsap.12200
Monteiro, B. P., Lascelles, B. D. X., Murrell, J., Robertson, S., Steagall, P. V. M., & Wright, B. (2023). 2022 WSAVA guidelines for the recognition, assessment and treatment of pain. Journal of Small Animal Practice, 64(4), 177–254. https://doi.org/10.1111/jsap.13566
Morton, C. M., Reid, J., Scott, E. M., Holton, L. L., & Nolan, A. M. (2005). Application of a scaling model to establish and validate an interval level pain scale for assessment of acute pain in dogs. American Journal of Veterinary Research, 66(12), 2154–2166. https://doi.org/10.2460/ajvr.2005.66.2154
Nielsen, J. (2006, December 3). Progressive disclosure. Nielsen Norman Group. https://www.nngroup.com/articles/progressive-disclosure/
Padilla, L., Kay, M., & Hullman, J. (2021). Uncertainty visualization. In Wiley StatsRef: Statistics Reference Online (pp. 1–18). John Wiley & Sons. https://doi.org/10.1002/9781118445112.stat08296
Reid, J., Nolan, A. M., Hughes, J. M. L., Lascelles, D., Pawson, P., & Scott, E. M. (2007). Development of the short-form Glasgow Composite Measure Pain Scale (CMPS-SF) and derivation of an analgesic intervention score. Animal Welfare, 16(S1), 97–104. https://doi.org/10.1017/S096272860003178X
Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206–215. https://doi.org/10.1038/s42256-019-0048-x
Shneiderman, B. (1996). The eyes have it: A task by data type taxonomy for information visualizations. In Proceedings of the 1996 IEEE Symposium on Visual Languages (pp. 336–343). IEEE. https://doi.org/10.1109/VL.1996.545307
Spiegelhalter, D. (2017). Risk and uncertainty communication. Annual Review of Statistics and Its Application, 4, 31–60. https://doi.org/10.1146/annurev-statistics-010814-020148
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010). Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology, 21(1), 128–138. https://doi.org/10.1097/EDE.0b013e3181c30fb2
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1207/s15516709cog1202_4
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019). What clinicians want: Contextualizing explainable machine learning for clinical end use. In Proceedings of the 4th Machine Learning for Healthcare Conference (Vol. 106, pp. 359–380). PMLR. https://proceedings.mlr.press/v106/tonekaboni19a.html
Van Calster, B., McLernon, D. J., van Smeden, M., Wynants, L., & Steyerberg, E. W. (2019). Calibration: The Achilles heel of predictive analytics. BMC Medicine, 17, Article 230. https://doi.org/10.1186/s12916-019-1466-7
Varma, S., & Simon, R. (2006). Bias in error estimation when using cross-validation for model selection. BMC Bioinformatics, 7, Article 91. https://doi.org/10.1186/1471-2105-7-91
World Wide Web Consortium. (2024, December 12). Web content accessibility guidelines (WCAG) 2.2. https://www.w3.org/TR/2024/REC-WCAG22-20241212/
Zanusso, F., Contiero, B., Normando, S., Gottardo, F., & De Benedictis, G. M. (2024). Qualitative behavioral assessment of dogs with acute pain. PLOS ONE, 19(6), Article e0305925. https://doi.org/10.1371/journal.pone.0305925
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020). Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (pp. 295–305). Association for Computing Machinery. https://doi.org/10.1145/3351095.3372852
Zhong, Z. S., Ma, R., & Zhao, H. (2023). Human-uncertainty distillation for calibrated vision models on CIFAR-10H. Journal of Advanced Computing Systems, 3(2), 77–89. https://doi.org/10.69987/JACS.2023.30206
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Chole Zhao, Evelyn Liu, Zijian Cao
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
5.png)
