Question/Term

Description

Machine Learning

The Intellesis Segmentation module uses machine learning to automatically identify objects within an image according to a pre-defined set of rules (the model). This enables any microscopy user to perform image segmentation even on complex data sets without programming experience or advanced knowledge on how to set up an image segmentation.

What is a "Model" ?

A model is a collection of rules according to which the software attributes the pixels to a class. Such a class is mutually exclusive for a given pixel, i.e. a pixel can only belong to one class. The model is the result of (repeated) labeling and training a subset of the data. After the model is trained using the labels provided by the user, it can be applied to the full data set in image processing, or it can be used to create an image analysis setting (*.czias) to be used with the 2D Toolkit.

In image processing the trained model can be applied to an image or data set and perform segmentation automatically. As result you will get two images, the segmented image on the one hand and a confidence map on the other.

What is a "Class" ?

A class is a group of objects (consisting of individual pixels) with similar features. According to the selected model the pixels of the image will be attributed as belonging to a certain class, e.g. cell nuclei, inclusions in metals, etc.

Every model has by default already two classes built-in, because at least two classes are needed (e.g. cells and background or steel and inclusions). More classes can be defined if necessary.

What is "Labeling" ?

Instead of using a series of complex image processing steps in order to extract the features of the image, you can simply label some objects in the image that belong to the same class. Based on this manual labeling the software will attribute the pixels of the image as belonging to a certain class. In order to refine the result, you can re-label wrongly attributed pixels to assign them to another class.

What is "Training" ?

During the training process (in the Intellesis Segmentation training user interface) you can repeatedly label structures as belonging to one class, run the training, check if the result matches your expectation and if necessary, refine the labeling in order to improve the result. The result is a trained model (a set of rules) which produce the desired result when applied to the training data.

With the labeled pixels and their classes a classifier will be trained. The classifier will then try to automatically assign single pixels to classes.

Training UI
(User Interface)

The user interface for training is the starting point of the automatic image segmentation process. Here you import and label images, and train the model which you can later use for automatic image segmentation. Within this interface you can load the training data, define the classes of objects found in your data and train the classifier to assign the objects to the correct classes.

What is "Segmenting" or "Segmentation"?

Segmentation is the process of partitioning an image into segments, where each segment consists of pixels that share certain features. This involves assigning a class label to each pixel based on its features, such as color, texture, or intensity and grouping of pixels: pixels that are classified into the same class are grouped together to form distinct segments within the image. Before you can perform segmentation, the segmentation model has to be trained. Within the Training UI you train the software by labeling specific objects or structures that belong to different classes. A pseudo-segmentation is performed each time you train the model so that you see if the feature extractor works for your image.

One output of the Intellesis Segmentation processing is the fully segmented image using the trained model. The second output is the confidence map, helping you assess the reliability of the segmentation.

Confidence Map

The confidence map is one of two resulting images when you apply a trained model to an image by using the processing function Intellesis Segmentation.

The (resulting) grayscale image encodes the reliability of the segmentation. Areas which can be addressed to a certain class with a high confidence will appear bright, whereas areas which have a lower confidence to belong to a certain class will appear dark. The confidence is represented by a percentage value, where 0 means "Not confident at all" (dark) and 100 "Very confident" (bright).

What is a "Feature"?

A feature is a specific property of a pixel that is calculated using a predefined set of filters and processing functions. This process results in a "Feature Vector" for each pixel, which encapsulates various characteristics of the pixel.

What is a "Feature Extractor"?

A feature extractor is a pre-defined set of processing functions that is used to create the feature vector for every pixel. A specific layer of a pre-trained neuronal network can be used as feature extractor as well.

Prediction

When the model that was trained on example data is applied to a new unlabeled data set the result is called a prediction.

Multi-Channel Images

The Intellesis Segmentation module supports multi-channel data sets. It is important to understand that in case of multi-channel images every pixel can still only belong to one class, i. e. the classes are mutually exclusive.

The additional information of having more than one intensity value per pixel (e.g. one for every channel) is also used for classification.

Example: If you have overlapping regions A and B in the image you want to classify, consider labeling three independent classes:

  • Class 1: A
  • Class 2: B
  • Class 3: A overlapping with B