Deep learning facilitated whole live cell fast super-resolution imaging

doi:10.1088/1674-1056/ac1b93

中国物理B ›› 2022, Vol. 31 ›› Issue (4): 48705-048705.doi: 10.1088/1674-1056/ac1b93

Deep learning facilitated whole live cell fast super-resolution imaging

Yun-Qing Tang(唐云青)^1,†,‡, Cai-Wei Zhou(周才微)^2,3,†, Hui-Wen Hao(蒿慧文)^3,4,5,†, and Yu-Jie Sun(孙育杰)^3,4,5,§

1 Institute of Systems Biomedicine, School of Basic Medical Sciences, Peking University Health Science Center, Beijing 100191, China;
2 Academy for Advanced Interdisciplinary Studies, Peking University, Beijing 100871, China;
3 State Key Laboratory of Membrane Biology, Biomedical Pioneer Innovation Center(BIOPIC), Beijing 100871, China;
4 School of Life Sciences, Peking University, Beijing 100871, China;
5 School of Future Technology, Peking University, Beijing 100871, China

收稿日期:2021-07-11 修回日期:2021-08-04 接受日期:2021-08-07 出版日期:2022-03-16 发布日期:2022-03-21
通讯作者: Yun-Qing Tang, Yu-Jie Sun E-mail:tang@ucas.ac.cn;sun_yujie@pku.edu.cn
基金资助:
Project supported by the China Postdoctoral Science Foundation, the National Key Research and Development Program of China for Y.S. (Grant No. 2017YFA0505300), and the National Science Foundation of China for Y.S. (Grant No. 21825401). The authors thank the Olympus engineer Mrs Shaoling Qi for assistance with the microscopy, Professor Wei Guo (University of Pennsylvania) for providing the human retinal pigment epithelium cells and Professor Xiaowei Chen (School of Future Technology, Peking University) for providing the Media-Golgi marker. Thanks to the High-performance Computing Platform of Peking University for providing computing resources and platforms.

Deep learning facilitated whole live cell fast super-resolution imaging

Yun-Qing Tang(唐云青)^1,†,‡, Cai-Wei Zhou(周才微)^2,3,†, Hui-Wen Hao(蒿慧文)^3,4,5,†, and Yu-Jie Sun(孙育杰)^3,4,5,§

1 Institute of Systems Biomedicine, School of Basic Medical Sciences, Peking University Health Science Center, Beijing 100191, China;
2 Academy for Advanced Interdisciplinary Studies, Peking University, Beijing 100871, China;
3 State Key Laboratory of Membrane Biology, Biomedical Pioneer Innovation Center(BIOPIC), Beijing 100871, China;
4 School of Life Sciences, Peking University, Beijing 100871, China;
5 School of Future Technology, Peking University, Beijing 100871, China

Received:2021-07-11 Revised:2021-08-04 Accepted:2021-08-07 Online:2022-03-16 Published:2022-03-21
Contact: Yun-Qing Tang, Yu-Jie Sun E-mail:tang@ucas.ac.cn;sun_yujie@pku.edu.cn
Supported by:
Project supported by the China Postdoctoral Science Foundation, the National Key Research and Development Program of China for Y.S. (Grant No. 2017YFA0505300), and the National Science Foundation of China for Y.S. (Grant No. 21825401). The authors thank the Olympus engineer Mrs Shaoling Qi for assistance with the microscopy, Professor Wei Guo (University of Pennsylvania) for providing the human retinal pigment epithelium cells and Professor Xiaowei Chen (School of Future Technology, Peking University) for providing the Media-Golgi marker. Thanks to the High-performance Computing Platform of Peking University for providing computing resources and platforms.

摘要/Abstract

摘要： A fully convolutional encoder-decoder network (FCEDN), a deep learning model, was developed and applied to image scanning microscopy (ISM). Super-resolution imaging was achieved with a 78 μm×78 μm field of view and 12.5 Hz-40 Hz imaging frequency. Mono and dual-color continuous super-resolution images of microtubules and cargo in cells were obtained by ISM. The signal-to-noise ratio of the obtained images was improved from 3.94 to 22.81 and the positioning accuracy of cargoes was enhanced by FCEDN from 15.83±2.79 nm to 2.83±0.83 nm. As a general image enhancement method, FCEDN can be applied to various types of microscopy systems. Application with conventional spinning disk confocal microscopy was demonstrated and significantly improved images were obtained.

关键词: optical microscopy, imaging and optical processing, image processing

Abstract: A fully convolutional encoder-decoder network (FCEDN), a deep learning model, was developed and applied to image scanning microscopy (ISM). Super-resolution imaging was achieved with a 78 μm×78 μm field of view and 12.5 Hz-40 Hz imaging frequency. Mono and dual-color continuous super-resolution images of microtubules and cargo in cells were obtained by ISM. The signal-to-noise ratio of the obtained images was improved from 3.94 to 22.81 and the positioning accuracy of cargoes was enhanced by FCEDN from 15.83±2.79 nm to 2.83±0.83 nm. As a general image enhancement method, FCEDN can be applied to various types of microscopy systems. Application with conventional spinning disk confocal microscopy was demonstrated and significantly improved images were obtained.

Key words: optical microscopy, imaging and optical processing, image processing

中图分类号: (Optical microscopy)

87.64.M-

42.30.-d (Imaging and optical processing) 95.75.Mn (Image processing (including source extraction))

Yun-Qing Tang(唐云青), Cai-Wei Zhou(周才微), Hui-Wen Hao(蒿慧文), and Yu-Jie Sun(孙育杰). Deep learning facilitated whole live cell fast super-resolution imaging[J]. 中国物理B, 2022, 31(4): 48705-048705.

Yun-Qing Tang(唐云青), Cai-Wei Zhou(周才微), Hui-Wen Hao(蒿慧文), and Yu-Jie Sun(孙育杰). Deep learning facilitated whole live cell fast super-resolution imaging[J]. Chin. Phys. B, 2022, 31(4): 48705-048705.

参考文献

[1] Valli J, Garcia-Burgos A, Rooney L M, e Oliveira B V D M, Duncan R R and Rickman C 2021 J. Biol. Chem. 297 100791
[2] Hell S W and Wichmann J 1994 Opt. Lett. 19 780
[3] Klar T A, Jakobs S, Dyba M, Egner A and Hell S W 2000 Proc. Natl. Acad. Sci. USA 97 8206
[4] Schneider J, Zahn J, Maglione M, Sigrist S J, Marquard J, Chojnacki J, Kräusslich H, Sahl S J, Engelhardt J and Hell S W 2015 Nat. Methods 12 827
[5] Danzl J G, Sidenstein S C, Gregor C, Urban N T, Ilgen P, Jakobs S and Hell S W 2016 Nat. Photonics 10 122
[6] Göttferta F, Pleiner T, Heine J, Westphal V, Görlich D, Sahl S J and Hell S W 2017 Proc. Natl. Acad. Sci. USA 114 2125
[7] Heine J, Reuss M, Harke B, D'Este E, Sahl S J and Hell S W 2017 Proc. Natl. Acad. Sci. USA 114 9797
[8] Betzig E, Patterson G H, Sougrat R, Lindwasser O W, Olenych S, Bonifacino J S, Davidson M W, Lippincott-Schwartz J and Hess H F 2006 Science 313 1642
[9] Hess S T, Girirajan T P K and Mason M D 2006 Biophys. J. 91 4258
[10] Rust M J, Bates M and Zhuang X 2006 Nat. Methods 3 793
[11] Schüttpelz M, Wolter S, van de Linde S, Heilemann M and Sauer M 2010 Proc. SPIE 7571 75710V
[12] Gustafsson M G L 2000 J. Microsc. 198 82
[13] Gustafsson M G L, Shao L, Carlton P M, Wang C J R, Golubovskaya I N, Cande W Z, Agard D A and Sedat J W 2008 Biophys. J. 94 4957
[14] Huang X, Fan J, Li L, Liu H, Wu R, Wu Y, Wei L, Mao H, Lal A, Xi P, Tang L, Zhang Y, Liu Y, Tan S and Chen L Y 2018 Nat. Biotech. 36 451
[15] Jost A and Heintzmann R 2013 Annu. Rev. Mater. Res. 43 261
[16] Heilemann M 2010 J. Biotechnol. 149 243
[17] Sheppard C J R 1988 Optik 80 53
[18] Mueller C B and Enderlein J 2009 Phys. Rev. Lett. 104 198101
[19] Huff J 2015 Nat. Methods 12 i
[20] Huff J, Bergter A Birkenbeil J Kleppe I Engelmann R and Krzic U 2017 Nat. Methods 14 1223
[21] De Luca G M R, Breedijk R M P, Brandt R A J, Zeelenberg C H C, Jong B E D, Timmermans W, Azar L N, Hoebe R A, Stallinga S and Manders E M M 2013 Biomed. Opt. Express 4 2644
[22] De Luca G M R, Breedijk R, Hoebe R, Stallinga S and Manders E 2017 Methods Appl. Fluores. 5 015002
[23] De luca G M R, Desclos E, Breedijk R M P, Dolz-edo L, Smits G J, Nahidiazar L, Bielefeld P, Picavet L, Fitzsimons C P, Hoebe R and Manders E M M 2017 J. Microsc. 266 166
[24] Sheppard C J R, Castello M, Tortarolo G, Deguchi T, Koho S V, Vicidomini G and Diaspro A 2020 J. Opt. Soc. Am. A 37 154
[25] DuBose T B, LaRocca F, Farsiu S and Izatt J A 2019 Nat. Photonics 13 257
[26] Roth S, Sheppard C J R, Wicker K and Heintzmann R 2013 Opt. Nanoscopy 2 1
[27] York A G, Chandris P, Nogare D D, Head J, Wawrzusin P, Fischer R S, Chitnis A and Shroff H 2013 Nat. Methods 10 1122
[28] Curd A, Cleasby A, Makowska K, York A, Shroff H and Peckham M 2015 Methods 88 37
[29] Schulz O, Pieper C, Clever M, Pfaff J, Ruhlandt A, Kehlenbach R H, Wouters F S, Grobhans J, Bunt G and Enderlein J 2015 Proc. Natl. Acad. Sci. USA 110 21000
[30] Hayashi S 2016 Jpn. J. Appl. Phys. 55 082501
[31] Hayashi S and Okada Y 2015 Mol. Biol. Cell 26 1743
[32] Azuma T and Kei T 2015 Opt. Express 23 15003
[33] Weigert M, Schmidt U, Boothe T, Müller A, Dibrov A, Jain A, Wilhelm B, Schmidt D, Broaddus C, Culley S, Rocha-Martins M, Segovia-Miranda F, Norden C, Henriques R, Zerial M, Solimena M, Rink J, Tomancak P, Royer L, Jug F and Myers E W 2018 Nat. Methods 15 1090
[34] Wang H, Rivenson Y, Jin Y, Wei Z, Gao R, Günaydın H, Bentolila L A, Kural C and Ozcan A 2019 Nat. Methods 16 103
[35] Ouyang W, Aristov A, Lelek M, Hao X and Zimmer C 2018 Nat. Biotech 36 460
[36] Nehme E, Weiss L E, Michaeli T and Shechtman Y 2018 Optica 5 458
[37] Wu Y, Rivenson Y, Wang H, Luo Y, Ben-David E, Bentolila, L, Pritz C and Ozcan A 2019 Nat. Methods 16 1323
[38] Bai C, Yu X, Peng T, Liu C, Min J, Dan D and Yao B 2020 IEEE Photon. Technol. Lett. 32 1131
[39] Wang Z, Zhu L, Zhang H, Li G, Yi C, Li Y, Yang Y, Ding Y, Zhen M, Gao S, Hsiai T and Fei P 2021 Nat. Methods 18 551
[40] Loffe S and Szegedy C 2015 Proceedings of the 32nd International Conference on International Conference on Machine Learning, July 7-9, 2015, Lille, France, p. 448
[41] Zhao H, Gallo O, Frosio I and Kautz J 2016 IEEE Trans. Comput. IMAGING 3 47
[42] Kingma D and Ba J 2014 arXiv:1412.6980
[43] Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, Kudlur M, Levenberg J, Monga R, Moore S G, Murray D, Steiner B, Tucker P, Vasudevan V, Warden P, Wicke M, Yu Y and Zheng X 2016 Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation, November 2-4,2016, Savannah, USA, p. 256
[44] Qiao C, Li D, Guo Y, Liu C, Jiang T, Dai Q H and Li D 2021 Nat. Methods 18 194
[45] Bialkowski S E 1996 Crit. Rev. Anal. Chem. 26 101
[46] Otsu N 1979 IEEE Trans. Sys. Man. Cyber. 9 62
[47] Hao H, Niu J, Xue B, Su Q P, Liu M, Yang J, Qin J, Zhao S, Wu C and Sun Y 2020 EMBO Rep. 21 e48385
[48] Belthangady C and Royer L A 2019 Nat. Methods 16 1215
[49] Cascarano P, Piccolomini E L, Morotti E and Sebastiani A 2021 arXiv:2102.07510
[50] Ruhnow F, Zwicker D and Diez S 2011 Biophys. J. 100 2820

Deep learning facilitated whole live cell fast super-resolution imaging

Deep learning facilitated whole live cell fast super-resolution imaging

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 10

Metrics

本文评价

推荐阅读 0

[1]	Ying Wang(王莹), Feng Qi(祁峰), Zi-Xu Zhang(张子旭), and Jin-Kuan Wang(汪晋宽). Super-resolution reconstruction algorithm for terahertz imaging below diffraction limit[J]. 中国物理B, 2023, 32(3): 38702-038702.
[2]	Hai-Zhu Pan(潘海珠), Mo-Qi Liu(刘沫岐), Hai-Miao Ge(葛海淼), and Qi Yuan(袁琪). Learnable three-dimensional Gabor convolutional network with global affinity attention for hyperspectral image classification[J]. 中国物理B, 2022, 31(12): 120701-120701.
[3]	Xingbing Chao(潮兴兵), Yuan Gao(高源), Jianping Ding(丁剑平), and Hui-Tian Wang(王慧田). Impact of the spatial coherence on self-interference digital holography[J]. 中国物理B, 2021, 30(8): 84212-084212.
[4]	罗高峰, 周日贵, 胡文文. Novel quantum secret image sharing scheme[J]. 中国物理B, 2019, 28(4): 40302-040302.
[5]	毕篆芳, 杨沐, 商广义. Optical polarization response at gold nanosheet edges probed by scanning near-field optical microscopy[J]. 中国物理B, 2018, 27(8): 87801-087801.
[6]	张星煦, 乔磊, 赵廷玉, 仇荣生. Passive ranging and a three-dimensional imaging system through wavefront coding[J]. 中国物理B, 2018, 27(5): 54205-054205.
[7]	刘小威, 孟超, 徐雪初, 汤明炜, 庞陈雷, 杨青. Applications of nanostructures in wide-field, label-free super resolution microscopy[J]. 中国物理B, 2018, 27(11): 118704-118704.
[8]	何小川, 杨建兵, 闫东航, 翁羽翔. New method for fast morphological characterization of organic polycrystalline films by polarized optical microscopy[J]. 中国物理B, 2015, 24(7): 76803-076803.
[9]	王立文, 娄淑琴, 陈卫国, 李宏雷. A novel method of rapidly modeling optical properties of actual photonic crystal fibres[J]. 中国物理B, 2010, 19(8): 84209-084209.
[10]	葛炳辉, 李方华, 李雪明, 王玉梅, 迟振华, 靳常青. Study of superstructure II in multiferroic BiMnO₃[J]. 中国物理B, 2008, 17(9): 3163-3169.