The Volume Fusion module for Vision4D has been available since 2017. This article explains how it can be used.
Introduction to volume fusion
Introduction
Volume Fusion was added to Vision4D as a tool to merge multiple image volumes that could not be merged otherwise. There are several reasons why two image volumes could not be merged:
- They may not have the same orientation - Stitching using the Tile Sorter only works if the volumes have the same Z-axis direction
- They may not have the same dimensions or resolution - this is also a pre-requisite for the Tile Sorter
Volume Fusion has been designed to work around these limitations.
This enables a few specific applications which would otherwise not be practically possible, including for example:
- Merging images of a sample that cannot be imaged in a single acquisition because it extends beyond the range of the field of view or stage movement. In this case, we might capture as much of the sample as possible, maybe including tiling, repositioning the sample, and imaging another part of it.
- Merging images of two parts of the same volume. This is an extension of the task above. If the sample is too large to fit in the imaging chamber, it may be possible to separate the sample into its constituent parts, image each of these individually, then merge them using Volume Fusion.
- Merging images taken in different modalities. Multiple modalities are available for imaging samples, from light microscopy, whether widefield or fluorescence, to electron microscopy and topical spectral imaging (e.g. IR microscopes). Such acquisitions have different outputs and sample preparation steps and will often result in images of different sizes, calibrations, and channel colors.
In all of these cases, and more, Volume Fusion can be used to generate a single volume from two separate volumes, and that fused volume can be used for analysis and visualisation like any other dataset.
Prerequisites
Volumes to be fused need to start as different image sets in the same SIS file
The volumes must be of the same bit depth
While volume fusion can work with images of different dimensions, the images must be of the same bit depth. Bearing in mind that higher bit depths lead to larger file sizes, it may be preferable to scale down higher bit depth images to match a lower bit depth volume than the other way around. Bit depth scaling is done through the Transformation Gallery (Data> Transformation Gallery).
If both images have the same bit depth no scaling is necessary.
Additional considerations
Multimodal imaging
As mentioned above, Volume Fusion works by taking one image set that we define as the Base image set and another that we define as the Moving image set, and transforming the Moving set to match the Base. If both the base and moving image set are similar in size and channels, which is the base and which is the moving set is relatively unimportant. However, if combining images of vastly different resolutions, as one might if combining electron microscopy and light microscopy image, it is worth bearing in mind that the moving set will be scaled to match the base set. This will usually result in the moving set being either scaled up to gigantic proportions (if it is much lower resolution) or scaled down significantly (if it is much higher resolution). Because of this, Volume Fusion is only really useful for multimodal imaging if the two imaging modes produce images of similar resolution.
Merging more than two sets
Each time we do a volume fusion we create a new image set. The typical destination for this new set is the same SIS file where the original sets currently reside. This means that when we fuse images, the size of the SIS file will grow significantly because it will now include both the original sets and the fused volume, which is usually about at large as the sum of the two original sets. So, if we merge a 1GB set, with another 1GB set, the resulting file will likely be in the region of 4GB (1GB for the base set + 1GB for the moving set + 2GB for the fused volume). Because of this, and the fact that the base and moving sets are generally only useful for the creation of the fused volume, many users will prefer to output the fused volume as a new SIS file and then delete the SIS file containing the original sets when finished.
However, as mentioned above, we can only fuse volumes that are in the same SIS file, so what would be the best way to fuse 3 or more sets together? Let us consider a case where we might merge 4 separate image sets into a single fused volume. We would start off by importing all of them as multiple sets in the same SIS file containing "set-1", "set-2", "set-3", and "set-4". We would then fuse "set-1" and "set-2" into our first fused volume, "fused-set-1". We can then do the fusion between "set-3" and "set-4" to create "fused-set-2". Finally, we would fuse "fused-set-1" and "fuse-set2" into a new "fused-set-all". The final result can be saved in a new SIS file, but all the intermediate steps must be created in the original SIS file.
Additionally, users should consider the point above concerning the size of the SIS file after fusion. If again we assume each set is 1GB, the SIS file will grow to around 8GB before we do the final fusion into a new SIS file that will only be around 4GB. So in total, we would need at least 16GB of spare disc space to store all the images, at least until we have finished and we can delete the original and interstitial images. Of course, the actual size of images users might wish to fuse will vary, but care should be taken to ensure there is enough storage space not just for the original images and the end results, but also for the intermediate steps.