Welcome to JUNN’s documentation!¶

See JUNN Readme for information.

Indices and tables¶

Contents:¶

JUNN Readme¶

Important: The repository and accompanying manuscript are currently in the process of being finalized. During this period, not everything in the repository might be completely finished or in complete working order yet.

The Jülich U-Net Neural Network Toolkit. JUNN is a neural network training tool, aimed at allowing the easy training of configurable U-Nets or other pixel to pixel segmentation networks, with the resulting networks usable in a standalone manner.

Citation¶

The manuscript for JUNN is currently in preparation, if you use JUNN, please cite the upcoming publication.

Prerequisites¶

JUNN is split into two components: a training and a prediction component. The training component runs the computations locally, the usage of a CUDA-compatible GPU is highly advisable. The prediction component can either run the predictions locally, or offload the computations to remote machines, and in general prediction is, albeit slower, still somewhat feasible with CPU-only processing speed. Via the remote prediction functionality, a server with GPUs may be shared for multiple non-GPU prediction clients, see the documentation how such a scenario may be set up.

Installation/Usage¶

Docker¶

The simplest way to run JUNN is to use the Docker images provided (note that to use a GPU inside a container, the necessary runtime needs to be installed) :

# training
> docker run --gpus all --ipc=host --tty --interactive --rm -v `pwd`:/data modsim/junn train --NeuralNetwork Unet --model model_name training_input.tif
# prediction
> docker run --gpus all --ipc=host --tty --interactive --rm -v `pwd`:/data modsim/junn predict --model model_name data.tif

Anaconda¶

To install JUNN locally, we recommend using the Anaconda package manager.

Since Anaconda TensorFlow packages tend to lag a bit behind, it is necessary to install the official TensorFlow binaries from PyPI.

> conda create -n junn
> conda activate junn
> conda install -c modsim junn
> pip install tensorflow tensorflow-addons tensorflow-serving-api
# usage
> python -m junn train --NeuralNetwork Unet --model model_name training_input.tif
> python -m junn predict --model model_name data.tif

Github¶

Installing JUNN from Github is only recommended for users familiar with Python development. Either conda packages can be built locally and installed, or the pip packages can be built and installed, or the necessary dependencies can be installed and the source run in-place.

# let conda build and install the packages
> conda create -n junn
> conda activate junn
> conda build junn-predict/recipe recipe
> conda install -c local junn-predict junn

# ... or let pip build and install the packages
> pip install junn-predict
> pip install .

# always install TensorFlow via pip
> pip install tensorflow tensorflow-addons tensorflow-serving-api

Documentation¶

The documentation can be built using sphinx, or will be available at readthedocs.

Quick Start¶

Note: In the next two sections, junn’s call will just be abbreviated junn. Depending on whether Docker or a local Python installation should be used, this would mean either docker run --gpus all --ipc=host --tty --interactive --rm -v `pwd`:/data modsim/junn or python -m junn.

Training¶

For training a new neural network, first the network structure needs to be chosen, therefore the --NeuralNetwork parameter is used. Most commonly the Unet neural network will be chosen here, with various parameters being available defining the network structure (e.g. levels=4, filters=64, activation=relu to name a few). The model save path is passed via the --model parameter, which will contain a TensorFlow formatted model after training (which is itself a directory structure). Finally, an input of ground truth data is needed, for example an ImageJ TIFF file with ROIs denoting the desired structures, such as cells. Various tunable parameters can be set via the -t switch; a list of available tunables can be output by using the --tunables-show argument.

> junn train --NeuralNetwork "Unet(optionA=1,optionB=2)" --model model_name -t SomeTunable=value training_data.tif

Upon call, the training will start, outputting the configured metrics at each epoch. If configured, outputs for TensorBoard will be written. Once the training is finished, or was interrupted by the user, e.g. because the achieved quality is good enough, the model is ready for prediction:

Prediction¶

JUNN can take various input formats, such as Nikon ND2, Zeiss CZI, or OME-TIFF, and predict the image data, outputting either the raw probability masks from the neural networks, or detected objects as ImageJ ROIs.

> junn predict --model model_name file_to_predict.tif --output result.tif --output-type roi

License¶

JUNN is licensed under the 2-clause BSD License, see License.

Concepts¶

This page’s aim is to convey some of JUNN’s concepts, which might go amiss if the merely the API documentation is observed.

Network and training pipeline assembly by OOP & Mix-ins¶

The core structure defining a network and training pipeline within JUNN is the NeuralNetwork class, it has various member methods which can be overridden in order to quickly create a new type of network or altered training methodology.

To this extend, various steps, such as defining the network, defining the input augmentations, etc. are done by individual methods.

Furthermore, concepts, such as ‘tile-based training’ or ‘augmentation’ are done in individual mix-ins, which can just be inherited into a desired training class.

E.g. to implement a novel network following a tile-based prediction approach with the standard JUNN augmentation functionality, one can just subclass the NeuralNetwork class as follows:

TODO: Example

Strictly building a compute graph using TensorFlow primitives¶

Other tools often follow a mixed approach, where data is pre-processed within Python to specifically match a TensorFlow compute graph generated, which has the downside that (possibly slower) Python computation is used, and the resulting model remains highly dependent on the Python-based pre/postprocessing.

To avoid those potential downsides, JUNN tries to model all steps necessary for processing the data in pure TensorFlow: Once data is transferred into a .tfrecord file, processing is done solely by (performant C++/GPU-based) TensorFlow primitives: The data augmentation pipeline, as well as the training. As an additional gimmick, the prediction routines (such as tiling the image, performing normalization, etc.) are encoded via TensorFlow operations as well, yielding to a completely standalone usable model, which can readily be used in completely distinct tools, such as TensorFlow Serving or DeepImageJ.

To this extend, JUNN expects the programmer to define @tf.function decorated functions for various purposes, which can be completely transformed into a TensorFlow compute graph.

TODO: Example

Furthermore, the prediction part of JUNN makes use of these properties: Instead of completely reinstantiating the model, only the graph is loaded in lightweight manner and execution is left to TensorFlow. By this unified approach, the various prediction backends (direct, via GRPC/HTTP and TensorFlow Serving) were easily implementable.

junn package¶

Subpackages¶

junn.common package¶

Subpackages¶

junn.common.functions package¶

Submodules¶

junn.common.functions.affine module¶

Additional affine transformation functions written as TensorFlow-tf.functions.

junn.common.functions.affine.radians(degrees)¶

Convert degrees to radians.

Parameters: degrees – Degrees to convert
Returns: Resulting radians

junn.common.functions.affine.shape_to_h_w(shape)¶

Extract height and width from a shape tuple with between 1 and 4 dimensions.

Parameters: shape – The input shape
Returns: A tuple of height and width.

junn.common.functions.affine.tfm_identity(shape=(0, 0))¶

Generate an identity matrix for use with the affine transformation matrices.

Parameters: shape – The parameter is ignored and only present for call-compatibility with the other functions.
Returns: The affine transformation matrix

junn.common.functions.affine.tfm_reflect(x=0.0, y=0.0, shape=(0, 0))¶

Generate a reflection affine transformation matrix.

Parameters

x – Whether to reflect in horizontal direction, a value of 1 leads to a reflection, any other value not.
y – Whether to reflect in vertical direction, a value of 1 leads to a reflection, any other value not.
shape – The shape of the image to be transformed

Returns