Scripting and interaction with other programs

Image registration is the process of transforming different sets of data to overlay images from different experiments, taken at different times, positions or angles, or with different imaging modalities. The algorithm attempts to discover the matching areas and align them together. Image registration methods can be divided into two large groups: rigid and non-rigid transformations. For the multiplexing experiments, we will use the nuclei staining in each of the experiment cycles to register the subsequent experiments to the first image. This method is applicable both to 2D and 3D images.

The application workflow starts from importing the original image files (for example, CZI) in arivis Pro and registering the subsequent experimental cycles to the first one based on the indicated nuclei staining channel (i.e. DAPI or Hoechst). The images will be imported as a stack of individual tiles with all the channels from all the experiment cycles, each registered to the nuclei staining in the first experiment cycle. After the registration, the mosaic image should be stitched using the Tile Sorter tool.

The images used in this application note were published in https://www.sciencedirect.com/science/article/pii/S0165027022001807

Image registration is the process of transforming different sets of data to overlay images from different experiments, taken at different times, positions or angles, or with different imaging modalities. The algorithm attempts to discover the matching areas and align them together. Image registration methods can be divided into two large groups: rigid and non-rigid transformations. For the multiplexing experiments, we will use the nuclei staining in each of the experiment cycles to register the subsequent experiments to the first image. This method is applicable both to 2D and 3D images.

The application workflow starts from importing the original image files (for example, CZI) in arivis Pro and registering the subsequent experimental cycles to the first one based on the indicated nuclei staining channel (i.e. DAPI or Hoechst). The images will be imported as a stack of individual tiles with all the channels from all the experiment cycles, each registered to the nuclei staining in the first experiment cycle.  After the registration, the mosaic image should be stitched using the Tile Sorter tool.

arivis Pro runs multiple applications (image registration) using external and independent Python libraries and tools produced by third parties. These tools must be installed by the user under its responsibility, strictly following the instruction on this document. arivis has tested the setup protocol on several computers, however, due to the different and not predictable hardware and software configuration of each computer, the results can be different case by case. Therefore, arivis declines any responsibility concerning the correct tools installation and setup on the user computer. arivis cannot be blamed about any malfunctioning or failure of the deep learning environment setup. arivis will not give technical support on the setup task. Both activities are totally on the user charge. arivis also declines any responsibility about the scientific results gathered from this application.

To use the registration script for multiplexed data in arivis Pro, we need 3 configuration steps:

First, download the script file here: Cellpose_to_onnx.py

Copy and paste the py file in the main folder of your cellpose python solution.

The script requires that the ONNX python libraries are installed. If these libraries are not yet installed, or you just want to make sure you have the latest:

The script has two options:

• Convert all models: look for built-in and custom models where Cellpose saves them. Any custom change in Cellpose behaviour (where the models are saved, for example) is not guaranteed to work with this option.
• Convert individual model: convert single model at the time, the user must provide the model full path and a mean_diameter

Anaconda is a Python distribution that aims to simplify package management and deployment. Used in conjunction with the scripting module in arivis Vision4D it allows users to implement advanced image processing that may not be possible out of the box for Vision4D. Anaconda is necessary for a variety of scripts provided by arivis, including our StarDist and CellPose integrations, but is not normally needed except for advanced imaging workflows that use those libraries.

For all other users, the latest Anaconda release is recommended and can be found here.

Some Clahe algorithm options can be set to optimize the results:

• INPUT_CHANNEL : Set the source channel to be equalized. NOTE: Only one channel can be processed.
• OUTPUT_CH_CHANNEL : Set the output channel name.

The size used to compute the Clahe can be freely set.

• CLAHE_CV2 = True : Set the OpenCV2 algorithm – False the Skimage one.
• HISTO_STRETCH : With the CLAHE_CV2 = False sets the result stretching.
• BLOCK_NUMBER_X : Sets the number of horizontal patches (tiles) used to compute the local equlization.
• BLOCK_NUMBER_Y : Sets the number of vertical patches (tiles) used to compute the local equalization.
• CLIP_LIMIT_CV2 : Sets the clipping limit, normalized between 0 and 100 (higher values give more contrast). It is applied to the OpenCv2 algorithm.
• CLIP_LIMIT_SKIMAGE : Sets the clipping limit, normalized between 0 and 1 (higher values give more contrast). It is applied to the Skimage algorithm.
• ACTIVE_FRAME : If True, the Clahe is applied only to the active time point.
• ACTIVE_PLANE : If True, the Clahe is applied only to the active Z plane.

Identifying the contacts between adjacent objects belonging to the same TAG, is a complex task that requires to detect the space on which two objects edges are touching or overlapping each other. Objects can be in contact either sharing a single voxel rather than more volume.

The python script is available as stand alone code as well as operator.

The process to achive the contacts analysis is executed  as two separate processes:

• Identifying the overlapping object bounding boxes
• Verifying if the objects withing the touching bounding boxes are really in contact or overlapped.

Open the analysis panel (Pipeline workspace) using the "flask" icon on the main, top icon bar.

Add the Python Segment Modifier operator to the active pipeline. Chose it from the Add Operation list.

Select the ContactBetweenObjects_RevC(4_0)_OP.py python operator file from the folder on which it is stored. Click on the "3 dots" button to browser in your hard disk  and select the file.

Digit the objects TAG. The TAG must collect the objects to be analyzed.

NOTE: the TAG name is case sensitive.

Sets the «Refine_Contact» option ON

NOTE: A segmentation operator must come first the Python Segment Modifier one.

It creates the objects to be analyzing by contacts. If the script operator have to be executed on objects already segmented in a separated pipeline, it is necessary to import them in the active pipeline using the “Import Document Objects” operator.

Color or RGB images are common in histopathology where bright-field images capture white light from which the diagnostic dyes absorb a certain portion of the spectrum. These images can be segmented in a range of ways, but a better separation of the signal components from the red, green, and blue channels can significantly improve segmentation results. This script is an implementation in Vision4D of the method developed by Ruifrok, A.C. & Johnston, D.A. as described here. 

This method has been developed specifically for the unmixing of diaminobenzidine, hematoxylin and eosin dies.

The output of the script is the creation of three additional channels to the current image set containing the unmixed signal.

This script relies on the NumPy libraries that are installed automatically with version 3.4 or above of arivis Vision4D. Earlier versions may require additional components.

The script can be downloaded here. Once the ZIP file is decompressed the PY file can be loaded into the script editor.

Once opened the script will probably require some modifications for each specific usage.

The output of the script is three additional channels in the source image sets. These channels are named for the dye signal that has been extracted and can be used in any pipeline as any regular image channel for segmentation etc. In cases of dual staining (e.g. H&E), the third channel essentially contains only the signal that could not be reliably attached to either of the other two. this means that this third channel can essentially be used as some form of quality control for the output since in a perfect deconvolution this third channel would ideally be empty. If there is still a strong signal in the third channel this usually indicates that the deconvolution parameters do not perfectly match  the staining in the image. This could be caused by white balancing issues or extreme staining (either too weak or too strong). 

For more information on the process of the deconvolution, please refer to the published literature on which this is based, linked above.

Once the script has been run, pipelines can be created as per usual that can use the newly created channel for segmentation or any additional modification, just like any other image channel. 

Support for this script can be obtained by logging a support ticket using the link at the top of this page.

A more detailed description of the script functionality and editing can be found here.

Download the script for arivis Vision4D 3.3/3.4 here (may work with earlier versions but has been tested in 3.3 and 3.4).

Download the script for version 3.5 and above here.

In order to define the contiguous sub-regions (sampling volume) features, few parameters of the script should be adjusted to match your analysis needs. These parameters are located in the code area labeled as USER SETTING.

• SIZE_BOX_X = the X axis size of each box . The value is expressed in µm.
• SIZE_BOX_Y = the Y axis size of each box . The value is expressed in µm.
• SIZE_BOX_Z = the Z axis size of each box . The value is expressed in µm.

• PRESERVE_EDGES = This option sets the boxes touching / non touching feature. If TRUE, the boxes edges are not contiguous to each other (2 pixels space in between).
  Note: The volume sizes (XYZ) are divided by the single box XYZ size. The integer number of boxes fitting in the volume is computed. It may happen that the boxes don't cover the whole sample volume.

• TAG_SEGMENTS = defines the segments TAG to be used to compute the boxes. It is used as alternative of the standard method (whole volume). If empty, the default method (whole volume) is used.
  
• TAG_BOXES = defines the TAG assigned to the created boxes.
  
• TAG_SEGMENTS = “”
  
• TAG_SEGMENTS = “Manual”
  
  Note: If the TAG_SEGMENTS is set to an existing TAG, all the objects belonging to it are used to create the boxes matrix. Specifically, the bounding box volume of each object is taken as the reference volume that is divided into boxes.

In order to define the contigues sub-regions (sampling volume) features, some parameters of the script should be adjusted to match your analysis needs. These parameters are located in the code area labeled as USER SETTING.

SIZE_BOX_X : Set the X sub-volume size.

SIZE_BOX_Y : Set the Y sub-volume size.

SIZE_BOX_Z : Set the Z sub-volume size.

All the values are expressed in metric units (µm).

If one of the box dimensions is bigger than the correspondent volume size, the script will not be executed and an error message is issued.

Run the DivideScope RevE (3_4) Python Script by pressing the Run Script button or pressing the F5 key.

Note: Activate the Output Panel, if not already displayed. The status of the script execution (errors including) will be visualized here.

This is the script result, a 3D matrix of sub-volumes:

Results (segments and measurements) will be stored in the dataset only if the Store Objects operator has been correctly set. Tick appropriately the option as shown below before complete the pipeline execution.

Run the Free-Oriented Sub-volume Python Script by pressing the Run Script button or pressing the F5 key.

Note: Activate the Output Panel, if not already displayed. The status of the script execution (errors including) will be visualized here.

Run the Matryoshka_doll_revxx Python Script by pressing the Run Script button or by pressing the F5 key.

Run the Spiral Oriented Sub Volume Python Script by pressing the Run Script button or pressing the F5 key.

Note: Activate the Output Panel, if not already displayed. The status of the script execution (errors including) will be visualized here.

The pipeline can be executed step by step (back and forth). This method allows to run and undo a single Operation. Either the arrow buttons or the Operation list can be used to go through the operators list.

This icon, located on the right side of the operator title bar, shows the operator status.

Task running:

Task completed: