What the study evaluated
The study evaluated the External Device module of Carebot AI CXR, designed to detect and localize external medical devices on chest X-rays.
The AI model was tested for four device classes: catheter/drain, port, pacemaker, and endotracheal tube (ETT). A total of 18,798 images were prepared across training, validation, and test datasets. The independent test set included 348 chest X-rays, with 183 device-positive and 165 device-negative images.
Study results in clinical practice
Carebot AI CXR showed high reliability for detecting whether a chest X-ray contains at least one target external device. At the selected threshold of 0.32, the system achieved sensitivity 0.973, specificity 0.964, and balanced accuracy 0.968 at the image level.
Object-level localization was strongest for pacemakers and ports, both of which have distinct radiographic appearances. Catheters/drains and ETTs were more challenging because they can appear as subtle linear structures, often overlapping with ribs, mediastinum, soft tissue, or other anatomy.
In practice, this module can support chest X-ray workflows by identifying device presence, helping quality-control processes, and enabling structured outputs. The study evaluated device presence and localization, not detailed placement correctness such as tip position relative to anatomical landmarks.
Key numbers
Prepared dataset: 18,798 images
Test set: 348 chest X-rays
Target classes: catheter/drain, port, pacemaker, ETT
Sensitivity: 0.973
Specificity: 0.964
Best object-level AP: pacemaker 0.995, port 0.989
Abstract
External medical devices are frequently visible on chest X-rays acquired in intensive care, perioperative, and emergency settings. Rapid recognition of devices such as catheters, drains, ports, pacemakers, and endotracheal tubes can support clinical workflows, quality assurance, and structured reporting. This validation study evaluated the External Device module of Carebot AI CXR, an AI-based object detector designed to localize four device classes on chest X-rays: catheter/drain, port, pacemaker, and ETT. Devices were annotated using bounding boxes, and 18,798 images were prepared across training, validation, and test datasets. The operating threshold was selected on the validation set and then applied without modification to an independent test set of 348 images. At the image level, the AI system achieved sensitivity 0.973, specificity 0.964, balanced accuracy 0.968, PPV 0.967, and NPV 0.970 for detecting the presence or absence of any target device. At the object level, overall mAP@0.5 was 0.815, with highest performance for pacemaker and port detection. Catheter/drain and ETT detection remained more challenging due to their variable appearance, low contrast, and overlap with anatomical structures. These results support the use of Carebot AI CXR for reliable presence or absence detection of external devices on chest radiographs under the tested conditions.
