Key takeaways

Grayscale image of parallel wavy dark and light elongated bands with orange text 'Annotate regions/objects' at bottom — Overview of annotation process

It's possible to leave some image regions non-annotated.
The ML segmentation algorithm only uses annotated pixels for training.
Explicit annotation of the background is required.
Objects must be fully annotated without the use of "scribble" annotations.
Ensure that objects are separated by at least one pixel of background class for accurate instance segmentation.
The accuracy of annotations directly impacts the resulting model and segmentation.
Accurate annotation of borders between objects or classes is critical.

What is partial annotation?

To deal with microscopy datasets containing a large number of objects per image, we implemented partial annotation. It's not efficient to annotate all objects of a single image for ML training; it's more beneficial to include some objects from different images that span multiple experiments. This approach allows for training a more robust segmentation model, one that includes objects of all sizes, shapes, and colors that are expected to be segmented.
To distinguish between non-annotated regions and regions that don't contain any class that should be segmented, we introduced the background class. The background is a special class that tells the neural network that the pixels it covers aren't occupied by any of the classes the model should recognize. If there are regions you don't want to be segmented, it's essential to provide background annotations. Non-annotated pixels are completely ignored during model training, allowing you to partially annotate the dataset.

Annotating partially

We've also developed a way that allows you to automatically remove the background class wherever it overlaps with any other class, allowing you to annotate more efficiently. Here's how it works:

Guidelines for partial annotation on arivis AI on the arivis Cloud

Use the brush tool to annotate several objects in a region.
Draw the background class around the annotated objects with the Autofill holes option active, covering as much background as possible.
The background class is automatically cropped where other annotations exist. Make sure that the background class doesn't cover any non-annotated objects.
If you annotate all objects in your image, draw one big background annotation around the entire image to crop all objects at once (with the brush tool and Autofill holes active).
Select the image from the list of images (right panel) for training.

If there are any regions where you want to make sure that the algorithm interprets them as background, it can make sense to create additional background annotations. These are examples of regions where additional background annotations can help the algorithm to learn how to segment more precisely:

Narrow background lines between objects helps to separate them.
Background annotations on image features that look similar to your classes help to prevent them from getting (wrongly) segmented.

Requirements

To ensure robust training of an algorithm, it is recommended to provide at least 5000 pixels of annotations for each class. This can be achieved with just 5 annotations of size 10*100 pixels, but in more difficult cases, several hundred annotations may be necessary.
While most use cases require annotations of the background class, there are some instances where it may not be necessary. For example, when segmenting a microscopic image of a composite material comprising two phases, both phases must be annotated but there is no background to annotate. However, such cases are rare, and in most cases, background annotations are required.

Tips:

The arivis AI toolkit's segmentation algorithm is a powerful tool, but it requires a significant number of annotations to accurately learn the different sizes, shapes, and colors of the classes being segmented. The complexity of segmentation tasks can vary greatly, making it difficult to determine the exact number of objects that must be annotated for the desired level of accuracy and robustness. Based on our experience, it's best to plan on annotating several hundred objects, but it's important not to attempt to annotate the entire dataset at once. Instead, try training the algorithm on a smaller set of data after annotating around 50 objects per class.
It's likely that your first training won't produce a highly accurate and robust segmentation. However, it's important not to give up as developing a high-performance algorithm is an iterative process!
Once you've completed your first training run, identify areas of low performance and create additional annotations in those regions. By iteratively improving your model, you can achieve better results. (See "Improve Your Segmentation" for more guidance.)
It's critical to annotate accurately, as you are the teacher of the ML segmentation algorithm. If inaccurate annotations are provided, the algorithm will learn to segment inaccurately.

Have questions? We're here to help.