Reading notes: Degenerative Adversarial NeuroImage Nets: Generating Images that Mimic Disease Progression

This blog is the reading note for the paper "Degenerative Adversarial NeuroImage Nets: Generating Images that Mimic Disease Progression" by Ravi, Daniele, et al. MICCAI 2019. Broadly speaking, the authors try to simulating images representative of neurodegenerative diseases. Specifically, they designed a novel network named Degenerative Adversarial NeuroImage Net (DaniNet) to produce accurate and convincing synthetic images that emulate disease progression.

Introduction 

Disease progression modelling help to map out longitudinal change during chronic diseases. However, modelling temporal neurodegeneration of full resolution MRI is still a challenging problem. Several traditional simulators proposed in the literature are extremely resource-demanding and is not scalable to high resolution image. [1] proposed a deep learning framework based on GAN to manipulate MRI, but their assumption is over-simplified: 1. disease progression is modelled linearly and 2. morphological changes are the same across all patients.

To overcame this limitation, inspired by [2], authors proposed DaniNet. Here are the major contributions: 1. They employ adversarial training to ensure realism in the synthetic MRI. 2. They implement biological constraints learned directly from the data instead of imposing predefined atrophy patterns. 3. They also embed non-imaging features such as age and diagnosis in MRI generation. The experiment shows their DaniNet can produce accurate and realistic synthetic images that emulate disease progression.

Methods

Figure 1 shows the workflow of DaniNet framework. The first component (blue) extracts a normalized slice from MRI. The second component (gray) is a Conditional Deep Autoencoder (CDA) and the third component (orange) is the biological constraints.

Figure 1. DaniNet framework

As shown in Figure 1, the major generation component is the Conditional Deep Autoencoder (conditional on age and diagnosis) with the following 3 groups of objective functions to address the challenges.

1. Adversarial Training

The first discriminator D_z guides Encoder(E) to generate latent z with a uniform distribution U to ensure temporal smoothness,

The second discriminator D_b guides the generator G to produce realistic brain images.

2. Biological Constraints

Authors define these two loss functions to capture the patterns of image intensity changes that accompany disease progression based on biological prior knowledge :

The first loss function ensure the later image have lower intensity values.

The second loss improve spatial consistency across neighboring voxels.

3. Deformation Loss
The final loss function ensures consistency with the individual subject over time by minimizing the difference between individual input image sequences and the corresponding output sequences.

It minimizes the difference between each input image and a weighted average of two outputs from the nearest age bins.

In summary, L_reg and L_vox ensure monotonic intensity change to mimic neurodegeneration and L_def together with D_b ensure realistic brain morphology.

Experiments and Results

The data used in the paper were obtained from the ADNI database, the training dataset has 9852 MRI slices from 876 patients while the testing dataset contains 1283 MRI slices from 197 patients.

Figure 2. Visual results obtained by different configurations of DaniNet

Table 1. Quantitative results based on Structural Similarity Index Matrices (SSIM) 

The visual results and quantitative results are shown in Figure 2 and Table 1. The results are reported with different loss functions mentioned above. Clearly the best result is obtained when DaniNet include all three group of loss.


Summary and Reviews

Authors propose a method DaniNet to simulate magnetic resonance Imaging (MRI) data representative of neurodegenerative diseases. Authors experiments demonstrate that their proposed method shows the ability to produce accurate and realistic synthetic MRI images. They achieve a better performance compared to prior approaches.

Pros:

• Based on [2], the authors carefully tailor the framework and incorporate their domain knowledge to address the challenge in this task.
• Three new designed loss function together with personalization trick significantly increasing the overall performance

Cons:

• The model is tailored for this task only, the biological constraints (keep progression along time) needs to be changed in other tasks.
• The complicated framework and multiple loss functions make the training unstable and hard to tune. 
• The major improvement come from the personalization trick, the improvement from the new components are limited.
• Due to the page limits, many experiment details are missing, hard to follow.

Reference

[1] Bowles, Christopher, et al. "Modelling the progression of Alzheimer's disease in MRI using generative adversarial networks." Medical Imaging 2018: Image Processing. Vol. 10574. International Society for Optics and Photonics, 2018.

[2] Zhang, Zhifei, Yang Song, and Hairong Qi. "Age progression/regression by conditional adversarial autoencoder." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.