Image-free single-pixel semantic segmentation for complex scene based on multi-scale U-Net

doi:10.1088/1674-1056/ade4b0

Abstract Single-pixel imaging (SPI) receives widespread attention due to its superior anti-interference capabilities, and image segmentation technology can effectively facilitate its recognition and information extraction. However, the complexity of the target scene and plenty of imaging time in SPI make it challenging to achieve high-quality and concise segmentation. In this paper, we investigate the image-free intricate scene semantic segmentation in SPI. Using "learned" illumination patterns allows for the full extraction of the object's spatial information, thereby enabling pixel-level segmentation results through the decoding of the received measurements. Simulation and experimentation show that, in the absence of image reconstruction, the mean intersection over union (MIoU) of segmented image can reach higher than 85%, and the Dice coefficient (DICE) close to 90% even at the sampling ratio of 5%. Our approach may be favorable to applications in medical image segmentation and autonomous driving field.

Keywords: single-pixel imaging image-free deep learning complex scene

Received: 29 March 2025
Revised: 12 May 2025
Accepted manuscript online: 16 June 2025

PACS:	42.30.-d	(Imaging and optical processing)
	42.30.Va	(Image forming and processing)
	42.30.Tz	(Computer vision; robotic vision)

Fund: Project supported by the Fundamental Research Funds for the Central Universities of China (Grant No. 531118010757).

Corresponding Authors: Yanfeng Bai, Xiquan Fu
E-mail: yfbai@hnu.edu.cn;fuxq@hnu.edu.cn

Cite this article:

Tengfei Liu(刘腾飞), Yanfeng Bai(白艳锋), Jianxia Chen(陈健霞), Jintao Zhai(翟锦涛), Siqing Xiang(向思卿), Xianwei Huang(黄贤伟), and Xiquan Fu(傅喜泉) Image-free single-pixel semantic segmentation for complex scene based on multi-scale U-Net 2026 Chin. Phys. B 35 014202

[1] Saleh K, Szénási S and Vámossy Z 2021 IEEE 19th World Symposium on Applied Machine Intelligence and Informatics (SAMI) 000477
[2] Zaitoun N M and Aqel M J 2015 Procedia Comput. Sci. 65 797
[3] 1978 IEEE Trans. Syst. Man Cybern. 8 630
[4] Canny J 1986 IEEE Trans. Pattern Anal. Mach. Intell. PAMI- 8 679
[5] Adams R and Bischof L 1994 IEEE Trans. Pattern Anal. Mach. Intell. 16 641
[6] Ronneberger O, Fischer P and Brox T 2015 Medical image computing and computer-assisted intervention, MICCAI 2015: 18th international conference, Munich, Germany, October 5–9, 2015, proceedings, part III 18, pp. 234–241
[7] Badrinarayanan V, Kendall A and Cipolla R 2017 IEEE Trans. Pattern Anal. Mach. Intell. 39 2481
[8] He K, Zhang X, Ren S and Sun J 2016 Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778
[9] He K, Gkioxari G, Dollár P and Girshick R 2017 Proceedings of the IEEE international conference on computer vision, pp. 2961–2969
[10] Zhou Y, Guo S X, Zhong F and Zhang T 2019 Chin. Phys. B 28 84204
[11] Hou H Y, Zhao Y N, Han J C, Cao D Z, Zhang S H, Liu H C and Liang B L 2023 Chin. Phys. B 32 064201
[12] Bie S H, Wang C H, Lv R B, Bao Q Q, Fu Q, Meng S Y and Chen X H 2023 Chin. Phys. B 32 128702
[13] Hardy N D and Shapiro J H 2013 Phys. Rev. A 87 023820
[14] Hardy N D and Shapiro J H 2011 Phys. Rev. A 84 063824
[15] Liu J J, Bai Y F, Huang X W, Tan W, Nan S Q, Zou X P F and Fu X Q 2021 Appl. Sci.-Basel 11 4115
[16] Zou X, Huang X, Liu C, Tan W, Bai Y and Fu X 2023 Opt. Laser Technol. 167 109807
[17] Zhou L Y, Huang X W, Fu Q, Zou X P F, Bai Y F and Fu X Q 2021 Chin. Opt. Lett. 19 121101
[18] Pan L, Deng C J, Yu C P, Yue S, Gong W L and Han S S 2021 Chin. Opt. Lett. 19 041103
[19] Lin L X, Cao J, Zhou D, Cui H and Hao Q 2022 Opt. Express 30 11243
[20] Fan Y R, Bai Y F, Fu Q, Zhang R, Zhou L Y, Zhu X H, Zou X P F and Fu X Q 2024 Opt. Commun. 566 130684
[21] Cheng J and Han S S 2004 Phys. Rev. Lett. 92 093903
[22] Liu H C and Zhang S 2017 Appl. Phys. Lett. 111 031110
[23] Wang C L, Mei X D, Pan L, Wang P W, Li W, Gao X, Bo Z W, Chen M L, Gong W L and Han S S 2018 Remote Sens. 10 732
[24] Liu H, Bian L and Zhang J 2023 Opt. Laser Technol. 157 108600
[25] Liu X, Han T, Zhou C, Huang J, Ju M, Xu B and Song L 2023 Opt. Express 31 9945
[26] Gibson G M, Johnson S D and PadgettMJ 2020 Opt. Express 28 28190
[27] He Z Q and Dai S S 2024 AIP Adv. 14 045316
[28] Zhao H, Shi J, Qi X, Wang X and Jia J 2017 Proceedings of the IEEE conference on computer vision and pattern recognition 2881
[29] Cohen G, Afshar S, Tapson J and Van Schaik A 2017 International joint conference on neural networks (IJCNN) 2921
[30] Dong C, Loy C C and Tang X 2016 Computer Vision, ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part II 14 391
[31] Zhang Z B, Li X, Zheng S J, YaoMH, Zheng G A and Zhong J G 2020 Opt. Express 28 13269
[32] Hou H and Andrews H 1978 IEEE Trans. Acoust. Speech Signal Process. 26 508
[33] Floyd R W 1976 Proc. Soc. Inf. Disp. 17 75

[1]	Predicting the synthesizability of inorganic crystals by bridging crystal graphs and phonon dynamics Mei Ma(马梅), Wei Ma(马薇), Le Gao(高乐), Zong-Guo Wang(王宗国), and Hao Liu(刘昊). Chin. Phys. B, 2026, 35(1): 016101.
[2]	Color Fourier single-pixel imaging with random color filter array Jialiang Chen(陈佳亮), Wei Zhu(朱维), Le Wang(王乐), and Shengmei Zhao(赵生妹). Chin. Phys. B, 2025, 34(9): 094210.
[3]	RLsite: Integrating 3D-CNN and BiLSTM for RNA-ligand binding site prediction Yan Zou(邹艳), Lang Yang(杨浪), Yanhui Liu(刘艳辉), and Yuyu Feng(冯玉宇). Chin. Phys. B, 2025, 34(8): 088709.
[4]	Deep learning-enabled inverse design of polarization-selective structural color based on coding metasurface Haolin Yang(杨昊霖), Bo Ni(倪波), Junhong Guo(郭俊宏), Hua Zhou(周华), and Jianhua Chang(常建华). Chin. Phys. B, 2025, 34(5): 050702.
[5]	Trajectory tracking on the optimal path of two-dimensional quadratic barrier escaping Zengxuan Zhao(赵曾轩), Xiuying Zhang(张秀颖), Pengchen Zhao(赵鹏琛), Chunyang Wang(王春阳), Chunlei Xia(夏春雷), Mushtaq Rana Imran, and Joelous Malamula Nyasulu. Chin. Phys. B, 2025, 34(5): 050205.
[6]	Learning complex nonlinear physical systems using wavelet neural operators Yanan Guo(郭亚楠), Xiaoqun Cao(曹小群), Hongze Leng(冷洪泽), and Junqiang Song(宋君强). Chin. Phys. B, 2025, 34(3): 034702.
[7]	Accurate prediction of essential proteins using ensemble machine learning Dezhi Lu(鲁德志), Hao Wu(吴淏), Yutong Hou(侯俞彤), Yuncheng Wu(吴云成), Yuanyuan Liu(刘媛媛), and Jinwu Wang(王金武). Chin. Phys. B, 2025, 34(1): 018901.
[8]	A large language model-powered literature review for high-angle annular dark field imaging Wenhao Yuan(袁文浩), Cheng Peng(彭程), and Qian He(何迁). Chin. Phys. B, 2024, 33(9): 098703.
[9]	High-quality ghost imaging based on undersampled natural-order Hadamard source Kang Liu(刘炕), Cheng Zhou(周成), Jipeng Huang(黄继鹏), Hongwu Qin(秦宏伍), Xuan Liu(刘轩), Xinwei Li(李鑫伟), and Lijun Song(宋立军). Chin. Phys. B, 2024, 33(9): 094204.
[10]	Properties of radiation defects and threshold energy of displacement in zirconium hydride obtained by new deep-learning potential Xi Wang(王玺), Meng Tang(唐孟), Ming-Xuan Jiang(蒋明璇), Yang-Chun Chen(陈阳春), Zhi-Xiao Liu(刘智骁), and Hui-Qiu Deng(邓辉球). Chin. Phys. B, 2024, 33(7): 076103.
[11]	Efficient single-pixel imaging encrypted transmission based on 3D Arnold transformation Zhen-Yu Liang(梁振宇), Chao-Jin Wang(王朝瑾), Yang-Yang Wang(王阳阳), Hao-Qi Gao(高皓琪), Dong-Tao Zhu(朱东涛), Hao-Li Xu(许颢砾), and Xing Yang(杨星). Chin. Phys. B, 2024, 33(3): 034204.
[12]	Image segmentation of exfoliated two-dimensional materials by generative adversarial network-based data augmentation Xiaoyu Cheng(程晓昱), Chenxue Xie(解晨雪), Yulun Liu(刘宇伦), Ruixue Bai(白瑞雪), Nanhai Xiao(肖南海), Yanbo Ren(任琰博), Xilin Zhang(张喜林), Hui Ma(马惠), and Chongyun Jiang(蒋崇云). Chin. Phys. B, 2024, 33(3): 030703.
[13]	Generation of orbital angular momentum hologram using a modified U-net Zhi-Gang Zheng(郑志刚), Fei-Fei Han(韩菲菲), Le Wang(王乐), and Sheng-Mei Zhao(赵生妹). Chin. Phys. B, 2024, 33(3): 034207.
[14]	Quantum state estimation based on deep learning Haowen Xiao(肖皓文) and Zhiguang Han(韩枝光). Chin. Phys. B, 2024, 33(12): 120307.
[15]	Reconstruction algorithm for cross-waveband optical computing imaging Jin-Tao Xie(谢锦涛), Shu-Hang Bie(别书航), Ming-Fei Li(李明飞), Yuan-Jin Yu(余远金), Yi-Fei Li(李毅飞), Jin-Guang Wang(王进光), Bao-Gang Quan(全保刚), and Ling-An Wu(吴令安). Chin. Phys. B, 2024, 33(10): 104203.

No Suggested Reading articles found!

Viewed

Full text

Abstract

Cited

Metrics
Related Articles

Image-free single-pixel semantic segmentation for complex scene based on multi-scale U-Net

Cite this article:

Online attention