ZEISS Knowledge Base

Detecting spines with neurite/neuron tracer

This article covers the Spine Tracer pipeline operation and how it can be used in conjunction with Neurite Tracer and Neuron Tracer operations.

Overview

Start by using either the Neuron Tracer or Neurite Tracer operations to create neurites.
Use the Spine Tracer with existing neurites to detect and quantify spines.

Introduction

The Spine Tracer module was introduced in arivis Pro 4.3 as a complement to the Neuron Tracer functionality introduced in arivis Vision4D 4.0. It is designed to identify and quantify neuronal spines. It requires that neurites have already been segmented in the current pipeline as a prerequisite.

The Spine Tracer is then used to detect and quantify dendritic spines in the immediate vicinity of the detected neurites.

Adding Spine Detection to a Pipeline

As mentioned above, the first prerequisite for spine detection is having already detected neurites. We can use either the Neurite Tracer or Neuron Tracer for this task. The Spine Tracer operation can be added immediately after the Neurite Tracer in the pipeline, or after some filtering operations.

The Spine Tracer can use 3 different methods to detect spines:

AI-Assisted - This method uses a built in DL model for spine detection. This model automatically identifies spines around the neurites with little to no image pre-processing required and can be used in a majority of cases.
Use Segments - In this case another pipeline operations has already previously segmented the spines and the Spine Tracer is only responsible for assigning the spines to their respective neurites. The spines can be segmented in any suitable method.
Use Probability Map - Much like for the Use Segments option, we can use any suitable pipeline operation to isolate and enhance the spines from the rest of the image data, but now the segmentation itself is carried out by the Spine Tracer.

These options make it easy for the majority of users to carry out spine quantification with little additional effort while providing the flexibility to address more challenging images.

In every case below the first step is to select the input tag for the trace objects. Most pipelines of this type will only have one tag for the traces, but it is possible to have several tracing operations in the same pipeline, we therefore need to take care to select the correct tag for the traces at this stage.

AI Assisted Spine Detection

Assigning previously segmented objects as spines

Using the built in model for spine detection is clearly very easy and the results are generally very good, however, in some cases better results can be obtained through other segmentation methods. We can then use the Spine Tracer to assign these previously segmented objects to neurites if preferred.

How the spine head segments are segmented isn't important, but they must belong to the current pipeline. This can be done by either segmenting them de novo or by importing the results of previous pipelines. In this example we've segmented the spines using a custom DL model via the Deep Learning Segmenter:

Select the Use Segment method to show a couple of new options.
Select the Trace objects and the Spine segments that we want to link together. The selection is based on the tags created for these objects in the pipeline.
Note: Only the head of the spines need to be segmented. The neck of the spine is detected automatically from the Trace channel as selected in the operation parameters.
Set a maximum spine length to reduce occurrences of incorrectly segmented objects being included as spines. In this case, any object whose center is further than the selected value is not included in the final result.
Set a limit for the maximum percentage volume of the spine segment that is allowed to overlap the neurite. Because the neurite and spines are both segmented independently the segments could conceivably overlap and could also include portions of the trace segment.
Run the operation.

The segments are converted to spines and connected to the neurite:

Using a probability map

The integration of AI ML and DL algorithms in arivis has hugely facilitated the segmentation of complex structures from noisy images while requiring little by way of image processing experience for the users. Because of this, and thanks to significant improvement in GPU computing, it is now possible to use AI models that can be significantly better and overall faster than traditional algorithms in all aspects of the image analysis workflow. Indeed the AI Assisted method mentioned above uses a custom spine detection model to facilitate this type of analysis. However, it is also possible to use AI models for both neurite enhancement and spine detection if the included algorithms do not provide adequate results.

This complete pipeline uses a custom DL model to enhance both the neurites and the spine heads as separate classes, then uses the resulting probability maps to trace the neurites and segment the spines:

Note that the model must have been created in advance of the pipeline execution. These models can be created using arivis Cloud, and model creation is included in the arivis AI Toolkit module, but arivis supports any DL model that can be saved as an ONNX file for this purpose as well.

The first step in this case is to use the Deep Learning Reconstruction operation to create the probability maps. In our example the model has 2 classes, but it is also possible to run 2 separate DL reconstruction operations, each with its own single class to enhance the spines and neurites separately, or use DL to enhance the spines alone if the arivis tracing algorithms suffice for trace detection.

These probability maps are stored as temporary new channels that are available to the pipeline but automatically deleted upon pipeline completion. Consult this article on configuring the Result Storage operation to learn about storing the probability map permanently if needed.

In our example, the Neurite Tracer uses a threshold based algorithm on the Neurites probability map to detect the neurite. The Spine Tracer then uses both probability maps for the spine detection. The Trace channel is used to enhance the neck detection while the Probability map channel is used to detect the spines.

As before we still have the max. spine length parameter, but we also have a Head threshold parameter to optimise the spine detection. Since probability maps aren't necessarily binary and the results will depend on the quality of the model, some experimentation by the user with this parameter may be required to obtain the best results.