Background and integration of tracking systems in arivis Vision4D.
Tracking in Vision4D
Image quality
Because tracking is dependent on the ability to recognize objects over time, it is first, and foremost, dependent on the ability to recognize objects. This means that the image quality must be good enough for segmentation algorithms to work, or at the very least for manual identification of objects by the user.
Segmentation itself is the process of identifying which pixels in an image are part of objects, and also of identifying which pixels belong to which object so that touching objects can be separated. The simplest form of image segmentation is known as thresholding. This is where we set a rule that all pixels with an intensity either above or below a specific threshold are classified into the Objects pixel class, and the rest are classed into the Background class. Then we can just say that contiguous groups of pixels are individual objects.
Several aspects of images can make this process more or less difficult. For example, a noisy image will have a lot more punctual variation that could end up with a lot of single-pixel objects or very rough object outlines. Also, some modalities can be particularly better or worse suited for segmentation. Fluorescence images are generally easier to segment compared to phase contrast or DIC images.
This image, for example, is quite challenging for segmentation using thresholds. There are variations of intensity inside what might be considered objects that could cause the head and tail to be separated. The objects themselves are out of focus so their boundaries are imprecise which could lead to difficulties in separating objects in close proximity, and there are big variations in the background intensity which could make a simple threshold difficult with some pre-processing.
The same sample images under different conditions (e.g. darkfield illumination) could make the segmentation comparatively much easier.
Sampling resolution
Along with image quality being a very significant factor in segmentation, sampling resolution is also critical for correct tracking. If the movement is too great compared to the typical object separation, correct identification of movement can be challenging or even impossible.
If, however, we can increase the acquisition frequency so that the movement from one time-point to the next the confidence in the correct identification of tracks improves significantly.
As a rule of thumb, it is generally preferable to take images frequently enough that the typical movement of objects from one time-point to the next is no more than 20% of the typical distance between neighboring objects.
Similarly, in cases where we are looking to monitor intensity changes, it's important to take images frequently enough to measure those changes accurately. If those changes are rhythmical or affected by rhythms in the sample (e.g. heartbeats), the acquisition frequency for each time-point should be at least 4 times higher than the frequency of changes in the sample.
All this taken together means that in many cases tracking will not be possible and looking at macro before/after changes may be the most suitable type of analysis. Also, in many cases, the sampling frequency needs, coupled with the exposure time needed to capture images of sufficient quality may restrict the acquisitions to single planes thereby limiting the ability to measure changes in 4D.
Segmentation
Having acquired images for the purpose of tracking, the first task will be to establish an effective segmentation strategy. Several factors can complicate good segmentation throughout the time series. For example:
- The signal or background intensity may change throughout the time series.
- Objects coming into contact with other objects will need to be effectively separate.
- Object morphologies may change significantly throughout the time series resulting in the need for relatively permissive filters.
All of these factors can, to some extent, be handled by image and object processing algorithms.
Segmenting objects for tracking
Correct segmentation is crucial for successful tracking. An error of as little as 5% can mean that the mean correct track length would only be about 20 time-points. Of course, even incomplete tracks can still provide valuable information and some degree of error is inevitable, but generally, the better the segmentation, the better the tracking results.
Some common problems and solutions in segmentation include:
- Mean background or signal intensity changes over time - Use a normalization filter to even out the intensity range.
- Noisy signal - Use denoising filters (a median filter can provide a good compromise between speed of processing and edge preservation).
- Uneven background signal - The "Particle enhancement" denoising filter, or "Preserve bright objects" Morphology filter can both isolate bright features from their immediate background.
In many cases, the Blob Finder segmenter can over all of the above issues in one operation.
However, as stated above, the accuracy of the segmentation is particularly important for good tracking. Since tracking algorithms will typically try to find corresponding objects in timepoint n+1 for any objects in timepoint n, minimizing the number of incorrect or missing objects is particularly important. As such, there are typically two main problems with segmentation with regards to tracking:
- Objects that are segmented but shouldn't, or vice versa
- Objects that are incorrectly split, either splits that shouldn't exist or splits that didn't occur
Often the segmentation step will create objects that need not be tracked. In most cases, the objects to be tracked tend to have some common features that can be used to identify them from those that do not. We can use the Segment Feature Filter operation to tag only those objects we want:
Problems with splitting can usually be dealt with either by refining the segmentation method, either using different splitting parameters, or even a different segmentation operation, together with segment feature filters. the main aim is to avoid situations like the one here where, depending on the settings used, the tracking algorithm may need to decide what to do with two objects where there should only be one, or vice-versa.
Perfect segmentation is highly unlikely with non-perfect images, but we should strive to reduce the potential sources of error.
Reviewing Tracking results
Once the tracking parameters have been set as needed the operation can be run and the segmented images will be analysed to create tracks. The outputs are:
- A visual representation of the tracks on the image that allows the users to evaluate visually whether the results are correct
- Entries for each track in the Objects table where the users can review specific features of the tracks, including features of the tracks (displacement, duration, number of segments in the track, etc), and features of the segments in the tracks sorted by the track they belong to.
This enables the user to do a few things.
First, if the tracks appear wrong we can revert the tracking operation, change the parameters so as to try and improve the results, and run the operation again. If no good tracking parameters exist because the image doesn't provide good automatic tracking conditions, manual correction of the tracks is also possible.
Secondly, the objects table can be configured to display features of the tracks and their segments. Since the tracks are essentially groups of objects it is often best to switch the Objects table display to the Master/Detail view, which shows the tracks in the upper table and the segmented objects in those tracks in the lower table.
Each table can then be configured individually to show pertinent information for both tracks, and the segments in those tracks.
Finally, the results can be exported, either as a pipeline operation, or from the Object window's Im/Export... menu.
Summary
- Optimize your segmentation as much as possible. Tracking is dependent on segmentation results and the better these are, the more accurate the tracking will be.
- Try different parameters to see which work best for your data, especially with regards to the motion types.
- Only use what you need:
- Reduce the search radius to avoid big jumps in the tracks.
- Only use weighing if the segments don't change shape/intensity much over time.
- Don't allow fusions or divisions unless this is necessary.
- Avoid manual editing tracks to reduce the time required to do the analysis and reduce user biases.
- Consider using VR for manual track editing and creation is automated 4D tracking doesn't work for your images.