Multi-scale attention-edge interactive refinement network for salient object detection

Abstract

To improve salient object detection (SOD) based on fully convolutional neural networks (FCNs), attention and edge awareness have been used separately as a supporting mechanism for multi-level feature refinement. However, the performance advantages have not been found to be consistent across datasets, because each mechanism has its own limitations, meaning that the absence of attention or edge awareness during feature refinement leads to inaccurate predictions or blurred boundaries. This phenomenon has inspired the development of a new model to enable close interaction between attention and edges, resulting in the Multi-scale Attention-Edge Interactive Refinement Network (MAIRN) proposed in this paper. The proposed model consists of two interacting subnets to achieve not only SOD but also salient edge detection (SED), and each subnet consists of multiple interactive refinement modules cascaded in series with the Multi-scale Attention Refinement (MSAR) module proposed for the SOD subnet to provide edge-enhanced attention and the Edge Refinement (ER) module proposed for the SED subnet to provide attention-enhanced edges. Also proposed is a novel structure for Progressive Feature Concentration (PFC) to reduce information loss in feature fusion. From extensive quantitative and qualitative comparison against 24 state-of-the-art SOD models with and without incorporation of attention and edge awareness, the proposed model is shown to have the most accurate and robust SOD performance on 5 benchmark datasets. Furthermore, it stands out as one of the most computationally efficient networks in terms of the number of parameters and floating-point operations. The code and results of our method are available at https://github.com/LiangBoCheng/MAIRN.