Uncertainty-Aware Semantic Segmentation via Decision Boundary Modeling and Confusion Zone Refinement

Abstract

Deep learning has significantly improved the performance of semantic segmentation in high-resolution imagery and computer vision tasks. However, standard segmentation models treat all pixel-wise predictions equally, despite the fact that predictions close to the decision boundary are inherently uncertain. This limitation often leads to unstable classifications and increased false positives or false negatives in complex scenes. In this paper, we propose a practical uncertainty-aware semantic segmentation refinement framework that explicitly handles decisionboundary ambiguity through an empirical routing mechanism through a Confusion Zone mechanism. Instead of forcing a binary decision for all pixels, predictions whose probability scores fall within a predefined interval [τ_low, τ_high] are marked as uncertain and redirected to a secondary refinement module that exploits additional spatial features and local context before assigning a final label. The proposed two-stage architecture allows the segmentation model to treat ambiguous predictions differently from confident ones, improving robustness near object boundaries and visually complex regions. Experiments on benchmark remote sensing segmentation datasets demonstrate that incorporating the Confusion Zone mechanism improves segmentation quality, achieving up to 0.851 IoU with SegFormer and consistent gains of 2-3 percentage points across U-Net, DeepLabV3+, and SegFormer backbones. These results suggest that explicitly routing decision-boundary predictions for local refinement is a simple and practical strategy for improving the reliability of deep semantic segmentation systems.