ZEISS Microscopy Knowledge Base

Help

ZEISS Microscopy Knowledge Base

Image Registration for Multiplexing Experiments in arivis Pro

How to: Align multiple 3D images based on the nuclei staining

This guides goal is to guide the user in the correct installation of the Anaconda3 Python package and the image registration workflow. This script will read multiple .czi image files (or SIS imagesets) in a given folder in a tile-wise manner and create the SIS files for each of the original files. During the registration process, the new SIS file will be created with each tile as a new imageset. The first cycle image will be copied from the original file and the image stacks from the following cycles will be registered to the first cycle image based on the indicated channel, typically, nuclei staining. In the last step, the Tile Sorter tool needs to be run to assemble the registered mosaic stack together. The resulting image can be used for the following image analysis.

Overview

To use image registration script in arivis Pro we need the following:

Install Anaconda Python for Vision4D
Import the registration environment in the Anaconda Navigator
Configure arivis Pro to install the arivis libraries into the registration environment
Define the required parameters and run the image registration script
Assemble the resulting image using the Tile Sorter tool in arivis Pro for the following analysis

Introduction

Image registration is the process of transforming different sets of data to overlay images from different experiments, taken at different times, positions or angles, or with different imaging modalities. The algorithm attempts to discover the matching areas and align them together. Image registration methods can be divided into two large groups: rigid and non-rigid transformations. For the multiplexing experiments, we will use the nuclei staining in each of the experiment cycles to register the subsequent experiments to the first image. This method is applicable both to 2D and 3D images.

The application workflow starts from importing the original image files (for example, CZI) in arivis Pro and registering the subsequent experimental cycles to the first one based on the indicated nuclei staining channel (i.e. DAPI or Hoechst). The images will be imported as a stack of individual tiles with all the channels from all the experiment cycles, each registered to the nuclei staining in the first experiment cycle. After the registration, the mosaic image should be stitched using the Tile Sorter tool.

Screenshot of file list (Cycle_02-09 .czi), Navigator showing Cycle_3_tile_0-8, and TileSorter multicolor microscopy mosaic

The images used in this application note were published in https://www.sciencedirect.com/science/article/pii/S0165027022001807

Image registration in a nutshell

Application Workflow

The application workflow starts by importing the original image files in arivis Pro. For each staining/hybridization cycle the new SIS file will be created. All Z planes from all the tiles will be imported separately into a single stack (single imageset). The import step is optional. Alternatively, the user can start the registration workflow by simply opening the already stored SIS files.

Import task diagram showing Image file to 3x3 tiles T1-T9 to 3D stack then SIS; rows labeled Cycle-00, Cycle-01, Cycle-02

The registration workflow starts from creating the new SIS file to store the registration results. The first cycle image (fixed) will be identified automatically based on the filename. This image will remain untransformed. The 3D stack for each subsequent cycle (moving) will be transferred as new channels in the corresponding tile in the registered image file. The number of tiles should be consistent for all staining cycles.

Registration task diagram showing SIS to single 3D stack to FIXED ImageSet or MOVING Image to REGISTERED SIS; rows labeled Cycle-00, Cycle-01, Cycle-02

Preliminary Remarks

arivis Pro runs multiple applications (image registration) using external and independent Python libraries and tools produced by third parties. These tools must be installed by the user under its responsibility, strictly following the instruction on this document. arivis has tested the setup protocol on several computers, however, due to the different and not predictable hardware and software configuration of each computer, the results can be different case by case. Therefore, arivis declines any responsibility concerning the correct tools installation and setup on the user computer. arivis cannot be blamed about any malfunctioning or failure of the deep learning environment setup. arivis will not give technical support on the setup task. Both activities are totally on the user charge. arivis also declines any responsibility about the scientific results gathered from this application.

Running image registration for multiplexing data in arivis Pro

Having set up all the pre-requisites, we can proceed to image import and registration. The script which we can download here.

In order to run the image registration script, arivis Pro window must be opened.

This could be a new window with no imageset open. In this case the new SIS file will be created in the same folder with the czi files are stored.
If the SIS file is already open/created, the new imagesets will be imported into the existing SIS file. If the existing imageset names are identical, they will be overwritten.

To run the image registration script, the Script Editor must be open in arivis Pro:
arivis Vision4D window titled '2D Viewer 1 - arivis Vision4D 3.6.2' with Extras menu open and Script Editor selected

The Registration_for_omics_v7.py file can be opened in the arivis Pro Script Editor:
Script Editor File menu open with 'Open... Ctrl+O' option highlighted

Several script variables must be customized by the customer, and are explained in the text below.

Python script showing variables including REGISTRATION_METHOD='affine' and ORIG_IMAGE_PATH='D:/data/orig_files/'

1. Path to the original files must be provided:
Code close-up showing ORIG_IMAGE_PATH = 'D:/data/orig_files/'
File explorer showing folder orig_files with files cycle-02.czi and cycle-04.czi

Note:Each image file must have ’-00’ (a dash symbol and a number) and the end of the filename. The file with the lowest number will be the fixed imageset and all the following images will be registered to this first imageset. The folder should have at least two images to run the registration on.

2. We need to provide the path to the folder where the imported and the resulting SIS file will be stored:
Code line showing SIS_FILE_PATH = 'D:/data/registration_result/'

Alternatively, the file import can be run earlier. The SIS files for each of the cycles must be opened in separate arivis Pro windows at the same time. If the imports were run earlier, we need to indicate this:
Code line showing RUN_SIS_IMPORT = False

If the import is run from the script as the first part of the workflow, the empty arivis Pro window will be opened. The Import Dialog window will open. We need to run the import using the Tile sorter option, but do not start the stitching process:
Arivis Vision4D 3.6.2 2D Viewer window showing Navigator, Color Settings, Info panel and empty image area

$Import files dialog with Tile Sorter selected, Folder D:\data\registration_result\, File Name Cycle-02.czi.sis and Save with GZIP checked$

Important: if the SIS file conversion is run earlier, the import should be run the exact same way using the Tile Sorter import scenario.

We need to provide the channel with the nuclei staining (i.e. DAPI, Hoechst) for each of the images/cycles in the experiment. The channel count starts from one. The number of the image files in the import folder or open SIS files must be identical or shorter than the number of channels in the IMAGESET_DAPI_CHANNEL list.
Code line showing IMAGESET_DAPI_CHANNEL = [1, 1, 2, 1, 1, 2]
The number of the channel with the nuclei staining (for registration) can be checked in the working panel:
Channel Visibility panel listing four checked channels: H3342-T4, AF568-T4, AF4882-T5, AF647-T5

The tile number must be identical for every image stack in the experiment, as well as the Z spacing. The number of planes in each cycle image may differ. The registration result will have the same number of planes as the fixed ImageSet (cycle_01). Code snippet showing TILE_NUMBER = 8

The workflow is applicable both on 2D and 3D image stacks. For the 3D images, the thickness of the single tile stack is computed based on the total number of planes in the stack divided by the number of tiles, provided by the customer. For the 2D images, please indicate: Code snippet showing TWO_D = True

Next, we need to select the registration method:
Code snippet showing REGISTRATION_METHOD = 'affine'
as well as the name of the resulting SIS file and the registered channel names:

Code lines NEW_IMAGESET_NAME = 'Registered_affine' and OUTPUT_CH_APPEND = '_cycle_'

We may choose to exclude all the subsequent nuclei staining channels from the final imported file. If KEEP_DAPI_CHANNEL is set to False, we will only import the nuclei staining channel from the first image/cycle:
Code: KEEP_DAPI_CHANNEL = True; comment '# False will exclude all the DAPI channels from the resulting imageset'

Once the registration process is finished, there will be an imageset created for each tile separately containing all the channels from all experimental cycles:

Navigator panel with Image Set 'Cycle_3_tile_0 (default)' and list Cycle_3_tile_1..Cycle_3_tile_8

Info panel showing Registered_all_tiles_no_Dapi.sis; Image Sets 9; Size (X,Y) 1884,1884; Planes 33; Channels 8

Each channel will be labeled according to the cycle in the experiment. The DAPI channels, if we decide to import them, will the renamed accordingly.
Channel list with checked entries '_cycle_3', '_cycle_3', '_cycle_3', '_cycle_4_DAPI', '_cycle_4'
To set the same color settings for all imagesets prior to stitching them in the Tile Sorter, we need to set the preferences for one imageset and apply them on the rest of the images:
Channel Visibility menu with 'Transfer to all image sets...' highlighted and options 'Scale mode' and 'Auto color setup...'

Stitch the mosaic image in Tile Sorter.
Image stacks will all the registered channels from the imaging cycles are imported separately and need stitching in the following step. The final image will be created from the individual imagesets all imported into one resulting SIS file.

First, we need to call the Tile Sorter:
Data menu list with 'Tile Sorter...' highlighted and options like Pixel Size... and Background Correction...

The following massage will appear.
'No tile sorter dataset found.' dialog stating the current dataset has not been marked as a tile sorter and Yes and No buttons

We need to confirm and press Yes.

The following massage will appear:
Source Tiles Structure dialog with 'Tiles stored as multiple image sets' option selected and OK button

In the Tile Sorter menu, we can adjust the number of rows and columns, or their order:
TileSorter window titled 'TileSorter (297 tiles)' showing Columns 3, Rows 3 and Overlap (Pixel) x=160 y=160

In the Tile Sorter menu, we can first arrange the tile positions as a grid:
Show panel with 'Grid', 'Manual', 'Alignment' buttons, 'Single Plane' and 'Projection', Time Point 1 and Plane 15
Note: In addition, it is useful to go to the middle plane of the 3D stack or make a Maximal projection.

Next, the positions of the tiles can be adjusted manually:
Panel titled Show with three buttons labeled Grid, Manual, Alignment; Manual appears selected

The resulting stitched image can be imported into the same imageset or as a separate (new) SIS file.
Output dialog: Save as New File or New Image Set; In same folder checked; File 20220624_Registered-stitched (2).sis
Note: For the large file it is often worth first exporting and testing the alignment on a single plane and only stitching the entire 3D stack afterwards.
Region panel with radio option Current plane selected, All planes from current time point unselected; Planes set to 15

Application note

Download Full How to: Install and run predictions with Cellpose PDF