This study provides a machine learning-based modeling framework to predict animal rabies outcomes using commonly collected case history and clinical signs from animals that have potentially exposed people to rabies virus. We used the largest available IBCM-derived rabies surveillance dataset from LMICs, which offered standardized, electronically collected data. The data was comprehensive, with a high level of completeness, broad temporal span, and consistent reporting efficiency, resulting in a model with highly accurate and reliable predictions. We also provide a risk stratification framework to prioritize case investigations based on rabies probabilities in settings where resource limitations may prohibit the investigation of all suspected rabies exposures. The prediction approach could increase the usability of case investigation data, particularly in the absence of accessible laboratory services.In LMICs, a large proportion of animal rabies investigations result in inconclusive outcomes7. Often cited barriers for rabies testing and conclusive investigation outcomes include a preponderance for dogs to roam freely, non-compliance from owners for euthanasia, disposal of biting animals, lack of veterinary professionals for assessment and sample collection, lack of sample transportation networks, and lack of available diagnostic facilities6,27,28. Hence, in programs that only assess diagnostic results for epidemiologic monitoring, a majority of information collected during rabies surveillance remains largely unusable. Furthermore, there are often great financial costs to operating advanced rabies surveillance systems, making the data collected highly valuable (even those lacking confirmatory outcomes). Introducing a highly sensitive and specific machine-learning model to accurately characterize the risk of probable and suspected rabid animals can maximize the value of these advanced surveillance systems.Various strategies including monitoring dog bite events, clinically-confirmed rabies cases (based on WHO or project-specific IBCM case definition), and laboratory-confirmed cases have been used as a proxy for tracking rabies trends in regions where surveillance and testing capabilities are limited29,30,31,32. Reporting and tracking dog bites are an important part of the rabies surveillance system, serving as alerts to initiate field investigations. However, dog bites are poor indicators of rabies trends, as dog bites are a common occurrence and rarely due to a rabid dog15. Laboratory testing in many countries presents an incomplete epidemiologic depiction given the many challenges associated with collecting, submitting, and testing samples. Relying solely on laboratory confirmations ignores usable surveillance data and may miss important epidemiologic shifts. When testing capacities are low, risk stratification techniques using highly specific and sensitive rabies prediction models, as demonstrated here, result in a more meaningful epidemiologic understanding of rabies. Furthermore, WHO or IBCM definition based qualitative risk assessments, such as flow chart algorithms, can be complicated when trying to determine if a patient should receive PEP. As modeled risk becomes more commonplace in rabies control programs, quantitative values determining the need for PEP may help streamline rabies exposure treatment decisions.In our study, a large proportion of animals were classified as WHO-probable, yet only 20% of these aligned with the high and moderate model-derived risk categories, suggesting that this definition likely has low specificity for classifying true rabies cases. The WHO probable case definition is intentionally conservative, making it better suited for PEP recommendations in many settings. Conversely, the IBCM probable case definition is better aligned with model-derived risk categories, suggesting that this is a better option for epidemiologic analysis. The ORED utilizes definitive outcomes (e.g., laboratory confirmation and passing observation periods) followed by model-derived cut-offs when conclusive outcomes are lacking to maximize the usability of surveillance data. Furthermore, ORED increased the usable case data by 3.2-fold and identified 12% additional at-risk communities compared to the IBCM classification scheme. This enhanced understanding of rabies epidemiology could provide a clearer baseline to drive political and philanthropic support for rabies control activities and monitor the impact of nascent dog vaccination campaigns. Machine learning methods could complement epidemiologic monitoring, but as more countries implement WHO recommendations for surveillance, further validations focused on country and program-specific differences should be explored.Our study used a risk stratification framework with a flexible probability threshold to classify rabies cases into high, moderate, low, and negligible risk events, while also accounting for cases with known outcomes (definitive laboratory or observation outcomes). Such a framework could improve rabies control programs in numerous ways. For instance, with an estimated 3% of people bitten by a dog every year in many countries, there are far too many bite events for even wealthy rabies programs to investigate all offending animals. Based on minimal data typically provided at the time of a bite report, this model could be used to ensure that higher-risk cases are appropriately investigated. Additionally, a probability-based, quantitative risk value ascribed to cases may help convince owners to relinquish high-risk dogs for euthanasia and convince healthcare-averse bite victims to seek much-needed (and potentially costly) treatment. Rather than categorical definitions of “suspect” or “probable”, which have shown to be lacking in statistical accuracy, being able to provide an empirically-derived risk percentage that the animal may have rabies could be another tool for investigators to improve programmatic goals and bite-victim health outcomes.Despite the relatively low threshold that was used to maximize model sensitivity, a small number of outlier rabies cases were misclassified. Such classification error could be concerning especially when a rabid case is categorized as lower risk. In our analysis, only one confirmed rabid dog was classified as “negligible risk”. This dog had no noted rabies symptoms and had not bitten any people or animals. Similarly, of the five confirmed rabid dogs classified as “low risk”, four had no noted rabies symptoms and only one dog had any documented clinical signs (i.e., aggression). All of these animals were found dead and submitted for testing; WHO recommends PEP in all instances where an animal dies within 10 days of an exposure event. Under any WHO-aligned rabies risk assessment protocol, rabies-exposed victims in these situations would have received PEP. This closer examination of model “failures” is re-assuring in several ways. First, these animals were unlikely to have exposed other people or animals, and based on the model importance parameters, if they had symptoms and had bitten people or animals they would have been classified into a higher-risk category. Second, these findings underscore the importance of rabies programs having clear algorithms for PEP; a quantitative risk score is a supporting factor to consider but should not supersede WHO PEP recommendations.Our findings show that XGB-based techniques were better at both binary classification and probability estimation compared to LR counterparts. While there was no single best model suitable for all situations, the XBG with oversampling was better at maximizing the model SN while preserving high SP. Oversampling helps counter the unequal outcomes often found in advanced rabies surveillance systems by forcing the prediction models to simulate more rabies outcomes33. Hence, models trained with oversampled data typically have improved sensitivity compared to their imbalanced counterparts. However, balancing techniques have an intrinsic tendency to bias the posterior probabilities of classifiers, hindering the goal of having well-calibrated probabilistic classifications34. In situations like rabies risk stratification where reliable probabilities are desired, correcting these biases using calibration techniques such as isotonic regression, as performed in this study may be necessary.Despite rabies being preventable, it remains a neglected disease in most parts of the world in large part due to the lack of adequate surveillance systems to elucidate the true burden in people and animals. There is a wide disparity in the availability of surveillance, testing, and vaccination resources, especially in LMICs. WHO recommends that countries implement advanced surveillance systems to support their efforts to achieve rabies elimination, yet there is little guidance for how programs should utilize the data from these systems. Our study demonstrates the utility of advanced machine learning techniques for accurate and reliable predictions which can help support risk assessment and epidemiologic analysis. However, this approach is likely to be only applicable in settings where advanced surveillance systems have been implemented and the relevance of the specific model presented here to other countries needs further evaluation. Ending the cycle of neglected diseases begins with improved awareness of the burden of the disease. As countries embrace the WHO recommendations to implement advanced surveillance systems, the approach described here will be an integral component of ensuring that these programs maximize the value of their surveillance efforts.
