PUBLICATION
3D Domain Adaptive Instance Segmentation Via Cyclic Segmentation GANs
- Authors
- Lauenburg, L., Lin, Z., Zhang, R., Santos, M.D., Huang, S., Arganda-Carreras, I., Boyden, E.S., Pfister, H., Wei, D.
- ID
- ZDB-PUB-230531-34
- Date
- 2023
- Source
- IEEE journal of biomedical and health informatics 27(8): 4018-4027 (Journal)
- Registered Authors
- Boyden, Edward, Zhang, Ruihan
- Keywords
- none
- MeSH Terms
-
- Animals
- Benchmarking*
- Image Processing, Computer-Assisted
- Microscopy
- Zebrafish*
- PubMed
- 37252868 Full text @ IEEE J Biomed Health Inform
Citation
Lauenburg, L., Lin, Z., Zhang, R., Santos, M.D., Huang, S., Arganda-Carreras, I., Boyden, E.S., Pfister, H., Wei, D. (2023) 3D Domain Adaptive Instance Segmentation Via Cyclic Segmentation GANs. IEEE journal of biomedical and health informatics. 27(8):4018-4027.
Abstract
3D instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming. Existing works segment a new modality by either deploying pre-trained models optimized on diverse training data or sequentially conducting image translation and segmentation with two relatively independent networks. In this work, we propose a novel Cyclic Segmentation Generative Adversarial Network (CySGAN) that conducts image translation and instance segmentation simultaneously using a unified network with weight sharing. Since the image translation layer can be removed at inference time, our proposed model does not introduce additional computational cost upon a standard segmentation model. For optimizing CySGAN, besides the CycleGAN losses for image translation and supervised losses for the annotated source domain, we also utilize self-supervised and segmentation-based adversarial objectives to enhance the model performance by leveraging unlabeled target domain images. We benchmark our approach on the task of 3D neuronal nuclei segmentation with annotated electron microscopy (EM) images and unlabeled expansion microscopy (ExM) data. The proposed CySGAN outperforms pre-trained generalist models, feature-level domain adaptation models, and the baselines that conduct image translation and segmentation sequentially. Our implementation and the newly collected, densely annotated ExM zebrafish brain nuclei dataset, named NucExM, are publicly available at https://connectomics-bazaar.github.io/proj/CySGAN/index.html.
Genes / Markers
Expression
Phenotype
Mutations / Transgenics
Human Disease / Model
Sequence Targeting Reagents
Fish
Orthology
Engineered Foreign Genes
Mapping