image processing - What is "semantic segmentation" compared to "segmentation" and "scene labeling"?

Question

Is semantic segmentation just a Pleonasm or is there a difference between "semantic segmentation" and "segmentation"? Is there a difference to "scene labeling" or "scene parsing"?

What is the difference between pixel-level and pixelwise segmentation?

(Side-question: When you have this kind of pixel-wise annotation, do you get object detection for free or is there still something to do?)

Please give a source for your definitions.

Sources which use "semantic segmentation"

• Jonathan Long, Evan Shelhamer, Trevor Darrell: Fully Convolutional Networks for Semantic Segmentation. CVPR, 2015 and PAMI, 2016
• Hong, Seunghoon, Hyeonwoo Noh, and Bohyung Han: "Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation." arXiv preprint arXiv:1506.04924, 2015.
• V. Lempitsky, A. Vedaldi, and A. Zisserman: A pylon model for semantic segmentation. In Advances in Neural Information Processing Systems, 2011.

Sources which use "scene labeling"

• Clement Farabet, Camille Couprie, Laurent Najman, Yann LeCun: Learning Hierarchical Features for Scene Labeling. In Pattern Analysis and Machine Intelligence, 2013.

Source which use "pixel-level"

• Pinheiro, Pedro O., and Ronan Collobert: "From Image-level to Pixel-level Labeling with Convolutional Networks." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015. (see http://arxiv.org/abs/1411.6228)

Source which use "pixelwise"

• Li, Hongsheng, Rui Zhao, and Xiaogang Wang: "Highly efficient forward and backward propagation of convolutional neural networks for pixelwise classification." arXiv preprint arXiv:1412.4526, 2014.

Google Ngrams

"Semantic segmentation" seems to be more used recently than "scene labeling"

Answer 1

"segmentation" is a partition of an image into several "coherent" parts, but without any attempt at understanding what these parts represent. One of the most famous works (but definitely not the first) is Shi and Malik "Normalized Cuts and Image Segmentation" PAMI 2000. These works attempt to define "coherence" in terms of low-level cues such as color, texture and smoothness of boundary. You can trace back these works to the Gestalt theory.

On the other hand "semantic segmentation" attempts to partition the image into semantically meaningful parts, and to classify each part into one of the pre-determined classes. You can also achieve the same goal by classifying each pixel (rather than the entire image/segment). In that case you are doing pixel-wise classification, which leads to the same end result but in a slightly different path...

So, I suppose you can say that "semantic segmentation", "scene labeling" and "pixelwise classification" are basically trying to achieve the same goal: semantically understanding the role of each pixel in the image. You can take many paths to reach that goal, and these paths lead to slight nuances in the terminology.