ZEISS Microscopy Knowledge Base

Help

ZEISS Microscopy Knowledge Base

Image Registration for Multiplexing Experiments in arivis Pro

How to: Align multiple 3D images based on the nuclei staining

This guides goal is to guide the user in the correct installation of the Anaconda3 Python package and the image registration workflow. This script will read multiple .czi image files (or SIS imagesets) in a given folder in a tile-wise manner and create the SIS files for each of the original files. During the registration process, the new SIS file will be created with each tile as a new imageset. The first cycle image will be copied from the original file and the image stacks from the following cycles will be registered to the first cycle image based on the indicated channel, typically, nuclei staining. In the last step, the Tile Sorter tool needs to be run to assemble the registered mosaic stack together. The resulting image can be used for the following image analysis.

Overview

To use image registration script in arivis Pro we need the following:

Install Anaconda Python for Vision4D
Import the registration environment in the Anaconda Navigator
Configure arivis Pro to install the arivis libraries into the registration environment
Define the required parameters and run the image registration script
Assemble the resulting image using the Tile Sorter tool in arivis Pro for the following analysis

Introduction

Image registration is the process of transforming different sets of data to overlay images from different experiments, taken at different times, positions or angles, or with different imaging modalities. The algorithm attempts to discover the matching areas and align them together. Image registration methods can be divided into two large groups: rigid and non-rigid transformations. For the multiplexing experiments, we will use the nuclei staining in each of the experiment cycles to register the subsequent experiments to the first image. This method is applicable both to 2D and 3D images.

The application workflow starts from importing the original image files (for example, CZI) in arivis Pro and registering the subsequent experimental cycles to the first one based on the indicated nuclei staining channel (i.e. DAPI or Hoechst). The images will be imported as a stack of individual tiles with all the channels from all the experiment cycles, each registered to the nuclei staining in the first experiment cycle. After the registration, the mosaic image should be stitched using the Tile Sorter tool.

Screenshot: file list of Cycle-02 to Cycle-09 .czi, Navigator dropdown Cycle_3_tile_0, TileSorter mosaic of fluorescent image

The images used in this application note were published in https://www.sciencedirect.com/science/article/pii/S0165027022001807

Image registration in a nutshell

Application Workflow

The application workflow starts by importing the original image files in arivis Pro. For each staining/hybridization cycle the new SIS file will be created. All Z planes from all the tiles will be imported separately into a single stack (single imageset). The import step is optional. Alternatively, the user can start the registration workflow by simply opening the already stored SIS files.

Import task diagram: Image file with 3x3 tiles T1-T9 imported as 3D stacks and saved to SIS

The registration workflow starts from creating the new SIS file to store the registration results. The first cycle image (fixed) will be identified automatically based on the filename. This image will remain untransformed. The 3D stack for each subsequent cycle (moving) will be transferred as new channels in the corresponding tile in the registered image file. The number of tiles should be consistent for all staining cycles.

Registration task diagram: SIS single 3D stacks per tile produce FIXED ImageSet or MOVING Image sets and REGISTERED SIS

Preliminary Remarks

arivis Pro runs multiple applications (image registration) using external and independent Python libraries and tools produced by third parties. These tools must be installed by the user under its responsibility, strictly following the instruction on this document. arivis has tested the setup protocol on several computers, however, due to the different and not predictable hardware and software configuration of each computer, the results can be different case by case. Therefore, arivis declines any responsibility concerning the correct tools installation and setup on the user computer. arivis cannot be blamed about any malfunctioning or failure of the deep learning environment setup. arivis will not give technical support on the setup task. Both activities are totally on the user charge. arivis also declines any responsibility about the scientific results gathered from this application.

Running image registration for multiplexing data in arivis Pro

Having set up all the pre-requisites, we can proceed to image import and registration. The script which we can download here.

In order to run the image registration script, arivis Pro window must be opened.

This could be a new window with no imageset open. In this case the new SIS file will be created in the same folder with the czi files are stored.
If the SIS file is already open/created, the new imagesets will be imported into the existing SIS file. If the existing imageset names are identical, they will be overwritten.

To run the image registration script, the Script Editor must be open in arivis Pro:
Arivis Vision4D 3.6.2 2D Viewer with Extras menu open and Script Editor highlighted

The Registration_for_omics_v7.py file can be opened in the arivis Pro Script Editor:
Script Editor File menu open showing Open... option highlighted with shortcut Ctrl+O

Several script variables must be customized by the customer, and are explained in the text below.

Script Editor code showing RUN_SIS_IMPORT = True, REGISTRATION_METHOD = 'affine', and ORIG_IMAGE_PATH and SIS_FILE_PATH

1. Path to the original files must be provided:
Code line showing ORIG_IMAGE_PATH = 'D:/data/orig_files/'
File explorer listing cycle-02.czi and cycle-04.czi in orig_files folder on D: drive

Note:Each image file must have ’-00’ (a dash symbol and a number) and the end of the filename. The file with the lowest number will be the fixed imageset and all the following images will be registered to this first imageset. The folder should have at least two images to run the registration on.

2. We need to provide the path to the folder where the imported and the resulting SIS file will be stored:
Code line showing SIS_FILE_PATH = 'D:/data/registration_result/'

Alternatively, the file import can be run earlier. The SIS files for each of the cycles must be opened in separate arivis Pro windows at the same time. If the imports were run earlier, we need to indicate this:
Code line showing RUN_SIS_IMPORT = False

If the import is run from the script as the first part of the workflow, the empty arivis Pro window will be opened. The Import Dialog window will open. We need to run the import using the Tile sorter option, but do not start the stitching process:
Arivis Vision4D 3.6.2 2D Viewer window with empty canvas and Color Settings panel visible

$Import dialog showing Tile Sorter selected, Folder D:\data\registration_result\, File Name Cycle-02.czi.sis, GZIP compression checked$

Important: if the SIS file conversion is run earlier, the import should be run the exact same way using the Tile Sorter import scenario.

We need to provide the channel with the nuclei staining (i.e. DAPI, Hoechst) for each of the images/cycles in the experiment. The channel count starts from one. The number of the image files in the import folder or open SIS files must be identical or shorter than the number of channels in the IMAGESET_DAPI_CHANNEL list.
Code line showing IMGSET_DAPI_CHANNEL = [1, 1, 2, 1, 1, 2]
The number of the channel with the nuclei staining (for registration) can be checked in the working panel:
Channel Visibility panel showing checked channels H3342-T4, AF568-T4, AF4882-T5, AF647-T5

The tile number must be identical for every image stack in the experiment, as well as the Z spacing. The number of planes in each cycle image may differ. The registration result will have the same number of planes as the fixed ImageSet (cycle_01). Code snippet showing TILE_NUMBER = 8

The workflow is applicable both on 2D and 3D image stacks. For the 3D images, the thickness of the single tile stack is computed based on the total number of planes in the stack divided by the number of tiles, provided by the customer. For the 2D images, please indicate: Code snippet showing TWO_D = True

Next, we need to select the registration method:
Code showing REGISTRATION_METHOD = 'affine' and comment listing options 'translation' 'rigid' 'affine' 'bspline' 'spline'
as well as the name of the resulting SIS file and the registered channel names:

Code showing NEW_IMAGESET_NAME = 'Registered_affine' and OUTPUT_CH_APPEND = '_cycle_'

We may choose to exclude all the subsequent nuclei staining channels from the final imported file. If KEEP_DAPI_CHANNEL is set to False, we will only import the nuclei staining channel from the first image/cycle:
Code snippet showing KEEP_DAPI_CHANNEL = True and comment about excluding all the DAPI channels

Once the registration process is finished, there will be an imageset created for each tile separately containing all the channels from all experimental cycles:

Navigator panel dropdown showing Image Set 'Cycle_3_tile_0 (default)' and tiles Cycle_3_tile_1 to Cycle_3_tile_8

Info panel listing Image Data 'Registered_all_tiles_no_Dapi.sis', Image Sets 9, Size (X,Y) 1884,1884, Planes 33, Channels 8

Each channel will be labeled according to the cycle in the experiment. The DAPI channels, if we decide to import them, will the renamed accordingly.
Channel list with checked entries labeled '_cycle_3', '_cycle_3', '_cycle_3', '_cycle_4_DAPI', '_cycle_4'
To set the same color settings for all imagesets prior to stitching them in the Tile Sorter, we need to set the preferences for one imageset and apply them on the rest of the images:
Channel Visibility panel showing checked channels and a context menu with 'Transfer to all image sets...' option

Stitch the mosaic image in Tile Sorter.
Image stacks will all the registered channels from the imaging cycles are imported separately and need stitching in the following step. The final image will be created from the individual imagesets all imported into one resulting SIS file.

First, we need to call the Tile Sorter:
Software Data menu listing options with Tile Sorter... highlighted

The following massage will appear.
Dialog titled 'No tile sorter dataset found.' with message about dataset not marked and Yes and No buttons

We need to confirm and press Yes.

The following massage will appear:
Dialog 'Source Tiles Structure' asking 'Select how the image sets should be used' with 'Tiles stored as multiple image sets' selected

In the Tile Sorter menu, we can adjust the number of rows and columns, or their order:
TileSorter window showing Columns 3 Rows 3, Overlap x=160 y=160 and tile count 297

In the Tile Sorter menu, we can first arrange the tile positions as a grid:
TileSorter Show panel with Grid, Manual, Alignment tabs; Single Plane and Projection toggles; Time Point 1 Plane 15
Note: In addition, it is useful to go to the middle plane of the 3D stack or make a Maximal projection.

Next, the positions of the tiles can be adjusted manually:
Show panel with three buttons labeled Grid, Manual (selected), and Alignment

The resulting stitched image can be imported into the same imageset or as a separate (new) SIS file.
Output pane with 'Save as' New File selected, 'In same folder as dataset' checked, File Name 20220624_Registered-stitched (2).sis
Note: For the large file it is often worth first exporting and testing the alignment on a single plane and only stitching the entire 3D stack afterwards.
Region panel with 'Current plane' selected, 'All planes from current time point' unselected, Planes value 15

Application note

Download Full How to: Install and run predictions with Cellpose PDF