Very short, basic overview of how to create and run pipelines in ZEISS arivis Pro.
Image analysis in arivis Pro
Overview
Workflow Description
The video above shows the complete process of creating, modifying and executing a pipeline for tracking on a single dataset. Let's break down the various steps.
Opening the Analysis panel
All segmentation operations, and the operations needed to enhance or otherwise modify the segmentation, can be built into a pipeline within the analysis panel. When we first open the analysis panel we can create new pipelines from scratch or use existing sample pipelines.
Adding pipeline operations
Pipelines are built from basic building blocks called operations. The operations are sorted into categories:
- Image processing operations allow you to modify the pixel values to facilitate the segmentation of structures of interest. To understand more about image filtering, please check our other articles on the topic.
- Segmentation is the process of creating objects from the pixel values in the image. We can extract Features from these objects to quantify differences between objects in an image and between different samples.
- Use objects: because the segmentation on its own is rarely enough to extract all the information that we want from our images, and segmentation is rarely 100% accurate, it is often necessary to take segmented objects and modify them somehow. These operations can include:
- Labelling positive or negative objects.
- Tracking objects through time.
- Identifying inter-object relationships such as parent-child relationships or distance measurements.
- ...
- Export operations allow us to generate information that can be used in other packages, including numerical results of our analysis for data analytics, masks of segmentation results for analysis in 3rd party software, and 3D surfaces of the segmented objects
Adding operations to the pipeline is as easy as clicking the + Add operations... button, searching for the operation we need, and then placing it into our pipeline by either: double-clicking the operation we need, dragging-&-dropping the operation to correct position in our pipeline, or simply typing in the Enter key for the highlighted operation.
In our pipeline above, we use a range of these operations.
Blob Finder (Segmentation)
The Blob Finder is a popular segmentation operation that is available in arivis. It is a fairly powerful operation because it is quite robust in dealing with noise and uneven backgrounds without the need for additional image pre-processing. The exact details of how this operation works is covered in the help files which we can access by pressing the F1 key on our keyboard.
The first parameter that needs to be set in any segmentation operation is the channel from which the segmentation extracts objects. Most operation only use one channel for segmentation, but some, including the Machine Learning and Deep Learning segmenters can use multiple channel inputs.
When setting up any segmentation operation, we usually have a couple of parameters to set, and we can use the preview to help us set the correct values.
The exact values we used in this case aren't particularly important, by using the preview we can adjust the parameters until the segmentation seems optimal. If we use this pipeline with multiple images it is of course also important to use the same settings for all our images and therefore to also test on a variety of images.
Every segment operation, whether it creates de novo objects or modifies existing ones, uses tags to help us select the objects in downstream pipeline operations. The default tag is the name of the operation, which is fine as a 1st default value but likely not ideal if our pipeline contains several segmentations or object processing operations of the same type, and it is therefore recommended to change the tag to something more appropriate.
Useful tip with regards to naming: start with names that describe the process then narrow down the naming to specific object types. For example, we might start with a tag "DAPI Seg" and then use the tag "Nucleus" once all false positives have been removed.
Tracking (Using segmented objects)
The tracking operation is a typical case of an operation that uses existing objects to extract additional information. Tracking is covered more fully as a specific topic here, but for now let's look at how an operation like this works.
The first parameter in all these operations that use existing objects is the tag for the objects to be modified. This tag must belong to objects that were created or imported into the pipeline. So, for example, our Blob Finder operation created objects and tagged them "Vesicles" and so we can select this tag as an input for the objects we want to track:
Like the Segmentation operation that created the object applies a tag to every object it created, these Use Objects operations also apply a tag to every object they modify. In the case of tracking in particular, it applies two different to the segmented objects that it tracked, and the track type object that it creates from each group of tracked objects.
Store Objects
Finally, we have the Store Object feature.
As we create a pipeline we might create objects that need to be filtered out, objects that we modify into other objects, and in the end we may not need to save all of these objects. For example, we might start with a Blob Finder to segment the nuclei, but we may only want to keep the segmented objects that are large and/or round enough to be reliably identified as nuclei in the end. The Store Objects operation allows us to select which of the segmented objects we want to keep permanently beyond the scope of the pipeline.
Saving changes
By default, pipeline operations do not typically make any changes to the pixel values in the image data, but this is not the only piece of information that is stored with the SIS file. Along with the SIS file we also have a metadata file that includes several additional bits of information pertaining to the image, including the pipelines we used on that document. When we close the document we are prompted to save our changes, this includes any modification to the pipeline, and the objects we created with it. It is therefore important to save those changes so we don't need to rebuild the pipeline and run the segmentation every time we want to inspect the results.
Saving the document when we close it means we can review the pipelines and segmented objects when we next open the image, but we can also export the pipeline for use with other images and sharing with other users. Exporting a pipeline is done from the Analysis Panel's hamburger menu:
Conclusions
The general process of doing image analysis in arivis is fairly simple. We use the Analysis Panel to create Pipelines. Those pipelines are built from individual operations that work with each other to extract the information we need. Pipelines can then easily be re-used with other images as needed, including in batch mode to streamline the process.
Of course, the process we described here is only the basic principle of pipeline:
- Enhance images if needed
- Use images to segment objects
- Use the segmented objects to extract useful information
- Save the results
The full breadth of what can be done in a pipeline cannot be covered here. The inclusion of Machine Learning and Deep Learning makes it possible to segment objects that were previously impossible to segment, and the pipeline tools allow us to extract all sorts of useful information from these segmentation. Please check our pipeline examples, to find out more about the types of information arivis can extract from images, and to learn more about how individual operations work.
Finally, again since our Knowledge Base and sample pipelines couldn't hope to fully cover what can be achieved in a pipeline, don't hesitate to get in touch with your local ZEISS representative.