Image segmentation of exfoliated two-dimensional materials by generative adversarial network-based data augmentation

doi:10.1088/1674-1056/ad23d8

中国物理B ›› 2024, Vol. 33 ›› Issue (3): 30703-030703.doi: 10.1088/1674-1056/ad23d8

Image segmentation of exfoliated two-dimensional materials by generative adversarial network-based data augmentation

Xiaoyu Cheng(程晓昱)¹, Chenxue Xie(解晨雪)¹, Yulun Liu(刘宇伦)¹, Ruixue Bai(白瑞雪)¹, Nanhai Xiao(肖南海)¹, Yanbo Ren(任琰博)¹, Xilin Zhang(张喜林)¹, Hui Ma(马惠)^2,†, and Chongyun Jiang(蒋崇云)^1,‡

1 College of Electronic Information and Optical Engineering, Nankai University, Tianjin 300350, China;
2 School of Physical Science and Technology, Tiangong University, Tianjin 300387, China

收稿日期:2024-01-02 修回日期:2024-01-26 接受日期:2024-01-30 出版日期:2024-02-22 发布日期:2024-03-06
通讯作者: Hui Ma, Chongyun Jiang E-mail:mahuimoving@163.com;jiang.chongyun@nankai.edu.cn
基金资助:
Project supported by the National Key Research and Development Program of China (Grant No. 2022YFB2803900), the National Natural Science Foundation of China (Grant Nos. 61974075 and 61704121), the Natural Science Foundation of Tianjin Municipality (Grant Nos. 22JCZDJC00460 and 19JCQNJC00700), Tianjin Municipal Education Commission (Grant No. 2019KJ028), and Fundamental Research Funds for the Central Universities (Grant No. 22JCZDJC00460).

Image segmentation of exfoliated two-dimensional materials by generative adversarial network-based data augmentation

1 College of Electronic Information and Optical Engineering, Nankai University, Tianjin 300350, China;
2 School of Physical Science and Technology, Tiangong University, Tianjin 300387, China

Received:2024-01-02 Revised:2024-01-26 Accepted:2024-01-30 Online:2024-02-22 Published:2024-03-06
Contact: Hui Ma, Chongyun Jiang E-mail:mahuimoving@163.com;jiang.chongyun@nankai.edu.cn
Supported by:
Project supported by the National Key Research and Development Program of China (Grant No. 2022YFB2803900), the National Natural Science Foundation of China (Grant Nos. 61974075 and 61704121), the Natural Science Foundation of Tianjin Municipality (Grant Nos. 22JCZDJC00460 and 19JCQNJC00700), Tianjin Municipal Education Commission (Grant No. 2019KJ028), and Fundamental Research Funds for the Central Universities (Grant No. 22JCZDJC00460).

1. 2024-030703-SI.pdf(1121KB)

摘要/Abstract

摘要： Mechanically cleaved two-dimensional materials are random in size and thickness. Recognizing atomically thin flakes by human experts is inefficient and unsuitable for scalable production. Deep learning algorithms have been adopted as an alternative, nevertheless a major challenge is a lack of sufficient actual training images. Here we report the generation of synthetic two-dimensional materials images using StyleGAN3 to complement the dataset. DeepLabv3Plus network is trained with the synthetic images which reduces overfitting and improves recognition accuracy to over 90%. A semi-supervisory technique for labeling images is introduced to reduce manual efforts. The sharper edges recognized by this method facilitate material stacking with precise edge alignment, which benefits exploring novel properties of layered-material devices that crucially depend on the interlayer twist-angle. This feasible and efficient method allows for the rapid and high-quality manufacturing of atomically thin materials and devices.

关键词: two-dimensional materials, deep learning, data augmentation, generating adversarial networks

Abstract: Mechanically cleaved two-dimensional materials are random in size and thickness. Recognizing atomically thin flakes by human experts is inefficient and unsuitable for scalable production. Deep learning algorithms have been adopted as an alternative, nevertheless a major challenge is a lack of sufficient actual training images. Here we report the generation of synthetic two-dimensional materials images using StyleGAN3 to complement the dataset. DeepLabv3Plus network is trained with the synthetic images which reduces overfitting and improves recognition accuracy to over 90%. A semi-supervisory technique for labeling images is introduced to reduce manual efforts. The sharper edges recognized by this method facilitate material stacking with precise edge alignment, which benefits exploring novel properties of layered-material devices that crucially depend on the interlayer twist-angle. This feasible and efficient method allows for the rapid and high-quality manufacturing of atomically thin materials and devices.

Key words: two-dimensional materials, deep learning, data augmentation, generating adversarial networks

中图分类号: (Image processing)

07.05.Pj

68.65.-k (Low-dimensional, mesoscopic, nanoscale and other related systems: structure and nonelectronic properties) 84.35.+i (Neural networks) 87.64.M- (Optical microscopy)

Xiaoyu Cheng(程晓昱), Chenxue Xie(解晨雪), Yulun Liu(刘宇伦), Ruixue Bai(白瑞雪), Nanhai Xiao(肖南海), Yanbo Ren(任琰博), Xilin Zhang(张喜林), Hui Ma(马惠), and Chongyun Jiang(蒋崇云). Image segmentation of exfoliated two-dimensional materials by generative adversarial network-based data augmentation[J]. 中国物理B, 2024, 33(3): 30703-030703.

参考文献

[1] Tan Q, Rasmita A, Zhang Z, Cai H, Cai X, Dai X, Watanabe K, Taniguchi T, MacDonald A H and Gao W 2023 Nat. Mater. 22 605
[2] Sauer M O, Taghizadeh A, Petralanda U, Ovesen M, Thygesen K S, Olsen T, Cornean H and Pedersen T G 2023 NPJ Comput. Mater. 9 35
[3] Bertoldo F, Ali S, Manti S and Thygesen K S 2022 NPJ Comput. Mater. 8 56
[4] Zhou X, Zhang R W, Zhang Z, Feng W, Mokrousov Y and Yao Y 2021 NPJ Comput. Mater. 7 160
[5] Li M Y, Pu J, Huang J K, Miyauchi Y, Matsuda K, Takenobu T and Li L J 2018 Adv. Funct. Mater. 28 1706860
[6] Dong J, Zhang L and Ding F 2019 Adv. Mater. 31 e1801583
[7] Zhao B, Shen D, Zhang Z, Lu P, Hossain M, Li J, Li B and Duan X 2021 Adv. Funct. Mater. 31 2105132
[8] Zhang X W, D W He, He J Q, Zhao S Q, Hao S C, Wang Y S and Yi L X 2017 Chin. Phys. B 26 97202
[9] Qi X, Gao M, Ding C, Zhang W, Qu R, Guo Y, Gao H and Zhang Z 2021 Phys. Status Solidi (RRL) 15 2100052
[10] Hu B, Zhang T, Wang K, Wang L, Zhang Y, Gao S, Ye X, Zhou Q, Jiang S, Li X, Shi F and Chen C 2023 Small 19 e2207538
[11] Hu X, Qiu C and Liu D 2020 Nano Res. 14 840
[12] Masubuchi S, Morimoto M, Morikawa S, Onodera M, Asakawa Y, Watanabe K, Taniguchi T and Machida T 2018 Nat. Commun. 9 1413
[13] Masubuchi S and Machida T 2019 NPJ 2D Mater. Appl. 3 4
[14] Dong X, Yetisen A K, Tian H, Gueler I, Stier A V, Li Z, Koehler M H, Dong J, Jakobi M, Finley J J and Koch A W 2020 ACS Photonics 7 1216
[15] Li H, Wu J, Huang X, Lu G, Yang J, Lu X, Zhang Q and Zhang H 2013 ACS Nano 7 10344
[16] Nolen C M, Denina G, Teweldebrhan D, Bhanu B and Balandin A A 2011 ACS Nano 5 914
[17] Han B, Lin Y, Yang Y, Mao N, Li W, Wang H, Yasuda K, Wang X, Fatemi V, Zhou L, Wang J I, Ma Q, Cao Y, Rodan-Legrain D, Bie Y Q, Navarro-Moratalla E, Klein D, MacNeill D, Wu S, Kitadai H, Ling X, Jarillo-Herrero P, Kong J, Yin J and Palacios T 2020 Adv. Mater. 32 e2000953
[18] Saito Y, Shin K, Terayama K, Desai S, Onga M, Nakagawa Y, Itahashi Y M, Iwasa Y, Yamada M and Tsuda K 2019 NPJ Comput. Mater. 5 124
[19] Masubuchi S, Watanabe E, Seo Y, Okazaki S, Sasagawa T, Watanabe K, Taniguchi T and Machida T 2020 NPJ 2D Mater. Appl. 4 3
[20] Talaei Khoei T, Ould Slimane H and Kaabouch N 2023 Neural. Comput. Appl. 35 23103
[21] Whang S E, Roh Y, Song H and Lee J G 2023 VLDB J 32 791
[22] Yang R, Mei L, Zhang Q, Fan Y, Shin H S, Voiry D and Zeng Z 2022 Nat. Protoc. 17 358
[23] Miao Q, Lin H, Wang X and Hassan M M 2021 Comput. Netw. 197 108327
[24] Weiss K R and Khoshgoftaar T M 2016 Proceedings of the 15th IEEE International Conference on Machine Learning and Applications, December 18–20, 2016, Anaheim, CA, USA, pp. 207–213
[25] Wang K Z, Zhang D Y, Li Y, Zhang R M and Lin L 2017 IEEE Trans. Circuits Syst. Video Technol. 27 2591
[26] Karras T, Aittala M, Laine S, Harkonen E, Hellsten J, Lehtinen J and Aila T 2021 Proceedings of the Conference on Neural Information Processing Systems, December 06-14, 2021, Electr Network, pp. 852-863
[27] Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S and Soc I C 2019 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 16-20, 2019, Long Beach, CA, pp. 658-666
[28] Jin C, Tao Z, Li T, Xu Y, Tang Y, Zhu J, Liu S, Watanabe K, Taniguchi T, Hone J C, Fu L, Shan J and Mak K F 2021 Nat. Mater. 20 940
[29] Yang Y, Li J, Yin J, Xu S, Mullan C, Taniguchi T, Watanabe K, Geim A K, Novoselov K S and Mishchenko A 2020 Sci. Adv. 6 eabd3655
[30] Liu Y, Dini K, Tan Q, Liew T, Novoselov K S and Gao W 2020 Sci. Adv. 6 eaba1830
[31] Liu F, Wu W J, Bai Y S, Chae S H, Li Q Y, Wang J, Hone J and Zhu X Y 2020 Science 367 903
[32] Tang R, Chen W, Wu Y, Xiong H and Yan B 2023 Sensors 23 3834
[33] Nhat-Duc H, Quoc-Lam N and Van-Duc T 2018 Autom. Constr. 94 203
[34] Borji A 2022 Comput. Vision Image Understanding 215 103329

Image segmentation of exfoliated two-dimensional materials by generative adversarial network-based data augmentation

Image segmentation of exfoliated two-dimensional materials by generative adversarial network-based data augmentation

RichHTML

PDF (PC)

赞

补充材料

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	Tianjun Cao(曹天俊), Song Hao(郝松), Chenchen Wu(吴晨晨), Chen Pan(潘晨), Yudi Dai(戴玉頔), Bin Cheng(程斌), Shijun Liang(梁世军), and Feng Miao(缪峰). Improving the electrical performances of InSe transistors by interface engineering[J]. 中国物理B, 2024, 33(4): 47302-047302.
[2]	Zhi-Gang Zheng(郑志刚), Fei-Fei Han(韩菲菲), Le Wang(王乐), and Sheng-Mei Zhao(赵生妹). Generation of orbital angular momentum hologram using a modified U-net[J]. 中国物理B, 2024, 33(3): 34207-034207.
[3]	Zhi-Yuan Li(李志远) and Jianfeng Chen(陈剑锋). Corrigendum to “Atomic-scale electromagnetic theory bridging optics in microscopic world and macroscopic world”[J]. 中国物理B, 2024, 33(2): 29901-029901.
[4]	Ping Li(李平), Bang Liu(刘邦), Shuai Chen(陈帅), Wei-Xi Zhang(张蔚曦), and Zhi-Xin Guo(郭志新). Progress on two-dimensional ferrovalley materials[J]. 中国物理B, 2024, 33(1): 17505-17505.
[5]	Denghui Peng(彭登辉), Shenlong Wang(王神龙), and Yuanchen Huang(黄元辰). A deep learning method based on prior knowledge with dual training for solving FPK equation[J]. 中国物理B, 2024, 33(1): 10202-10202.
[6]	Tian Wang(王田), Jiahui Chen(陈家辉), Jing Teng(滕婧), Jingang Shi(史金钢),Xinhua Zeng(曾新华), and Hichem Snoussi. Crysformer: An attention-based graph neural network for properties prediction of crystals[J]. 中国物理B, 2023, 32(9): 90703-090703.
[7]	Jiamei Cui(崔佳梅), Yunjie Li(李韵洁), Cai Zhao(赵偲), and Wen Zheng(郑文). Classification and structural characteristics of amorphous materials based on interpretable deep learning[J]. 中国物理B, 2023, 32(9): 96101-096101.
[8]	Wei Zheng(郑玮), Fengming Xue(薛凤鸣), Zhongyong Chen(陈忠勇), Chengshuo Shen(沈呈硕), Xinkun Ai(艾鑫坤), Yu Zhong(钟昱), Nengchao Wang(王能超), Ming Zhang(张明),Yonghua Ding(丁永华), Zhipeng Chen(陈志鹏), Zhoujun Yang(杨州军), and Yuan Pan(潘垣). Disruption prediction based on fusion feature extractor on J-TEXT[J]. 中国物理B, 2023, 32(7): 75203-075203.
[9]	Hai-Jian Bai(柏海舰), Chen-Chen Guo(过晨晨), Heng Ding(丁恒), Li-Yang Wei(卫立阳), Ting Sun(孙婷), and Xing-Yu Chen(陈星宇). Modeling differential car-following behavior under normal and rainy conditions: A memory-based deep learning method with attention mechanism[J]. 中国物理B, 2023, 32(6): 60507-060507.
[10]	Ming Yan(闫明), Zhi-Yuan Xie(谢志远), and Miao Gao(高淼). High-temperature ferromagnetism and strong π-conjugation feature in two-dimensional manganese tetranitride[J]. 中国物理B, 2023, 32(3): 37104-037104.
[11]	Ming Mei(梅明), Zheng Chen(陈正), Yong Nie(聂勇), Yuanyuan Wang(王园园), Xiangde Zhu(朱相德), Wei Ning(宁伟), and Mingliang Tian(田明亮). Magnetic and magnetotransport properties of layered TaCoTe₂ single crystals[J]. 中国物理B, 2023, 32(12): 127303-127303.
[12]	Zhiwei Fan(范芷薇), Jingyuan Qu(渠靖媛), Tao Wang(王涛), Yan Wen(温滟), Ziwen An(安子文), Qitao Jiang(姜琦涛), Wuhong Xue(薛武红), Peng Zhou(周鹏), and Xiaohong Xu(许小红). Recent progress on two-dimensional ferroelectrics: Material systems and device applications[J]. 中国物理B, 2023, 32(12): 128508-128508.
[13]	Yue Hou(侯越), Di Zhang(张迪), Da Li(李达), and Ping Yang(杨萍). Inatorial forecasting method considering macro and micro characteristics of chaotic traffic flow[J]. 中国物理B, 2023, 32(10): 100508-100508.
[14]	Shuhang Qian(钱树航), Kai Wang(王凯), Jiaxing Yang(杨加兴), Chao Guan(关超), Hua Long(龙华), and Peixiang Lu(陆培祥). Multifunctional light-field modulation based on hybrid nonlinear metasurfaces[J]. 中国物理B, 2023, 32(10): 107803-107803.
[15]	Zhi-Yuan Li(李志远) and Jian-Feng Chen(陈剑锋). Atomic-scale electromagnetic theory bridging optics in microscopic world and macroscopic world[J]. 中国物理B, 2023, 32(10): 104211-104211.