Anisotropic neighborhoods of superpixels for thin structure segmentation

In the field of computer vision, image segmentation aims at decomposing an image into homogeneous regions. While usually an image is composed of a regular lattice of pixels, this manuscript proposes through the term of site a generic approach able to consider either pixels or superpixels. Robustness to noise in this challenging inverse problem is achieved by formulating the labels as a Markov Random Field, and finding an optimal segmentation under the prior that labels should be homogeneous inside the neighborhood of a site. However, this regularization of the solution introduces unwanted artifacts, such as the early loss of thin structures, defined as structures whose size is small in at least one dimension. Anisotropic neighborhood construction fitted to thin structures allows us to tackle the mentioned artifacts. Firstly, the orientations of the structures in the image are estimated from any of the three presented options: The minimization of an energy, Tensor Voting, and RORPO. Secondly, four methods for constructing the actual neighborhood from the orientation maps are proposed: Shape-based neighborhood, computed from the relative positioning of the sites, dictionary-based neighborhood, derived from the discretization to a finite number of configurations of neighbors for each site, and two path-based neighborhoods, namely target-based neighborhood with fixed extremities, and cardinal-based neighborhood with fixed path lengths. Finally, the results provided by the Maximum A Posteriori criterion (computed with graph cuts optimization) with these anisotropic neighborhoods are compared against isotropic ones on two applications: Thin structure detection and depth reconstruction in Shape From Focus. The different combinations of guidance map estimations and neighborhood constructions are illustrated and evaluated quantitatively and qualitatively in order to exhibit the benefits of the proposed approaches.