Please wait a minute...
Chin. Phys. B, 2024, Vol. 33(11): 116804    DOI: 10.1088/1674-1056/ad8a4a
SPECIAL TOPIC — Stephen J. Pennycook: A research life in atomic-resolution STEM and EELS Prev   Next  

Real-time four-dimensional scanning transmission electron microscopy through sparse sampling

A W Robinson1,†, J Wells1, A Moshtaghpour3, D Nicholls1, C Huang3, A Velazco-Torrejon3, G Nicotra4, A I Kirkland3, and N D Browning1,6
1 SenseAI Innovations Ltd., Liverpool, UK;
2 Centre for Doctoral Training, University of Liverpool, Liverpool, UK;
3 Correlative Imaging Group, Rosalind Franklin Institute, Oxford, UK;
4 CNR-IMM, Z. I. VIII Strada 5, Catania, 95121, Italy;
5 Department of Mechanical, Materials, and Aerospace Engineering, University of Liverpool, Liverpool, UK
Abstract  Four-dimensional scanning transmission electron microscopy (4-D STEM) is a state-of-the-art image acquisition mode used to reveal high and low mass elements at atomic resolution. The acquisition of the electron momenta at each real space probe location allows for various analyses to be performed from a single dataset, including virtual imaging, electric field analysis, as well as analytical or iterative extraction of the object induced phase shift. However, the limiting factor in 4-D STEM is the speed of acquisition which is bottlenecked by the read-out speed of the camera, which must capture a convergent beam electron diffraction (CBED) pattern at each probe position in the scan. Recent developments in sparse sampling and image inpainting (a branch of compressive sensing) for STEM have allowed for real-time recovery of sparsely acquired data from fixed monolithic detectors, Further developments in compressive sensing for 4-D STEM have also demonstrated that acquisition speeds can be increased, i.e., live video rate 4-D imaging is now possible. In this work, we demonstrate the first practical implementations of compressive 4-D STEM for real-time inference on two different scanning transmission electron microscopes.
Keywords:  compressive sensing      4-D STEM      inpainting  
Received:  23 August 2024      Revised:  16 October 2024      Accepted manuscript online:  23 October 2024
PACS:  68.37.Ma (Scanning transmission electron microscopy (STEM))  
  61.72.uf (Ge and Si)  
  07.05.Pj (Image processing)  
  07.05.Hd (Data acquisition: hardware and software)  
Fund: The authors would like to thank the Rosalind Franklin Institute for providing access to the JEOL GrandARM 2 “Ruska” and National Research Council of Italy’s Institute for Microelectronics and Microsystems at Catania for providing access to the JEOL JEM-ARM 200F to gather results for this work. We thank the Royal Society for providing funding under grant number EGR10965.
Corresponding Authors:  A W Robinson     E-mail:  alex.robinson@senseai.vision

Cite this article: 

A W Robinson, J Wells, A Moshtaghpour, D Nicholls, C Huang, A Velazco-Torrejon, G Nicotra, A I Kirkland, and N D Browning Real-time four-dimensional scanning transmission electron microscopy through sparse sampling 2024 Chin. Phys. B 33 116804

[1] Krivanek O L, Dellby N and Lupini A R 1999 Ultramicroscopy 78 1
[2] Batson P E, Dellby N and Krivanek O L 2002 Nature 418 617
[3] Faruqi A R and McMullan G 2018 Nucl. Instrum. Methods Phys. Res. A 878 180
[4] Faruqi A R, Henderson R, Pryddetch M, Allport P and Evans A 2005 Nucl. Instrum. Methods Phys. Res. A 546 170
[5] Ciston J, Johnson I J, Draney B R, Ercius P, Fong E, Goldschmidt A, Joseph J M, Lee J R, Mueller A, Ophus C, Selvarajan A, Skinner D E, Stezelberger T, Tindall C S, Minor A M and Denes P 2019 Microscopy and Microanalysis 25 1930
[6] Ryll H, Simson M, Hartmann R, Holl P, Huth M, Ihle S, Kondo Y, Kotula P, Liebel A, Müller-Caspary K, Rosenauer A, Sagawa R, Schmidt J, Soltau H and Strüder L 2016 J. Instrument. 11 P04006
[7] Yang H, Jones L, Ryll H, Simson M, Soltau H, Kondo Y, Sagawa R, Banba H, MacLaren I and Nellist P D 2015 J. Phys. Conf. Ser. 644 012032
[8] Zheng Q, Feng T, Hachtel J A, Ishikawa R, Cheng Y, Daemen L, Xing J, Idrobo J C, Yan J, Shibata N, Ikuhara Y, Sales B C, Pantelides S T and Chi M 2021 Sci. Adv. 7 eabe6819
[9] Mas-Ballesté R, Gómez-Navarro C, Gómez-Herrero J and Zamora F 2011 Nanoscale 3 20
[10] Novoselov K S, Mishchenko A, Carvalho A and Castro Neto A H 2016 Science 353
[11] Mayoral A, Mahugo R, SánchezSánchez M and Díaz I 2017 ChemCatChem 9 3497
[12] Shen B, Chen X, Shen K, Xiong H and Wei F 2020 Nat. Commun. 11 2692
[13] Pennycook S J, McGibbon A J, McGibbon M M, Browning N D, Chisholm M F and Jesson D E 1994 Determination of interface structure and bonding at atomic resolution in the STEM, in Proceedings of the 13th International Congress on Electron Microscopy, Paris (France), 17-22 July, 1994
[14] Benthem K and Pennycook S J 2009 Appl. Phys. A 96 161
[15] Chisholm M F and Pennycook S J 2006 Philosoph. Mag. 86 4699
[16] Chejarla V S, Ahmed S, Belz J, Scheunert J, Beyer A and Volz K 2023 Small Methods 7
[17] Pennycook S J, Rafferty B and Nellist P D 2000 Microscopy and Microanalysis 6 343
[18] Pennycook S J and Jesson D E 1991 Ultramicroscopy 37 14
[19] Kim Y M, Pennycook S J and Borisevich A Y 2017 Ultramicroscopy 181 1
[20] Egerton R F 2011 Electron energy-loss spectroscopy in the electron microscope (Springer Science & Business Media)
[21] Chang T Y, Tanaka Y, Ishikawa R, Toyoura K, Matsunaga K, Ikuhara Y and Shibata N 2014 Nano Lett. 14 134
[22] Ophus C 2019 Microscopy and Microanalysis 25 563
[23] Bustillo K C, Zeltmann S E, Chen M, Donohue J, Ciston J, Ophus C and Minor A M 2021 Acc. Chem. Res. 54 2543
[24] Wen Y, Ophus C, Allen C S, Fang S, Chen J, Kaxiras E, Kirkland A I and Warner J H 2019 Nano Lett. 19 6482
[25] O’Leary C M, Allen C S, Huang C, Kim J S, Liberti E, Nellist P D and Kirkland A I 2020 Appl. Phys. Lett. 116 124101
[26] Tan J A, Dull J T, Zeltmann S E, Tulyagankhodjaev J A, Johnson H M, LiebmanPeláez A, Folie B D, Dönges S A, Khatib O, Raybin J G, Roberts T D, Hamerlynck L M, Tanner C P N, Lee J, Ophus C, Bustillo K C, Raschke M B, Ohldag H, Minor A M, Rand B P and Ginsberg N S 2023 Adv. Funct. Mater. 33
[27] Nicholls D, Robinson A, Wells J, Moshtaghpour A, Bahri M, Kirkland A and Browning N 2022 Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 1586-90
[28] Nicholls D, Wells J, Robinson A W, Moshtaghpour A, Kobylynska M, Fleck R A, Kirkland A I and Browning N D 2022 arXiv: 2211.03494
[29] Robinson A W, Wells J, Nicholls D, Moshtaghpour A, Chi M, Kirkland A I and Browning N D 2023 J. Microsc. 290 53
[30] Robinson A, Nicholls D, Wells J, Moshtaghpour A, Kirkland A and Browning N D 2022 Ultramicroscopy 242 113625
[31] Robinson A W, Moshtaghpour A, Wells J, Nicholls D, Chi M, MacLaren I, Kirkland A I and Browning N D 2024 J. Microsc. 295 278
[32] Sertoglu S and Paisley J 2015 2015 23rd European Signal Processing Conference (EUSIPCO) p. 2771
[33] Zhou M, Chen H, Paisley J W, Ren L, Sapiro G and Carin L 2009 NIPS 9 2295
[34] SenseAI Innovations Ltd. Live Subsampled 4D-STEM with SenseAI
[1] Dynamics analysis and cryptographic implementation of a fractional-order memristive cellular neural network model
Xinwei Zhou(周新卫), Donghua Jiang(蒋东华), Jean De Dieu Nkapkop, Musheer Ahmad, Jules Tagne Fossi, Nestor Tsafack, and Jianhua Wu(吴建华). Chin. Phys. B, 2024, 33(4): 040506.
[2] Enhancing visual security: An image encryption scheme based on parallel compressive sensing and edge detection embedding
Yiming Wang(王一铭), Shufeng Huang(黄树锋), Huang Chen(陈煌), Jian Yang(杨健), and Shuting Cai(蔡述庭). Chin. Phys. B, 2024, 33(1): 010502.
[3] Lossless embedding: A visually meaningful image encryption algorithm based on hyperchaos and compressive sensing
Xing-Yuan Wang(王兴元), Xiao-Li Wang(王哓丽), Lin Teng(滕琳), Dong-Hua Jiang(蒋东华), and Yongjin Xian(咸永锦). Chin. Phys. B, 2023, 32(2): 020503.
[4] Efficient implementation of x-ray ghost imaging based on a modified compressive sensing algorithm
Haipeng Zhang(张海鹏), Ke Li(李可), Changzhe Zhao(赵昌哲), Jie Tang(汤杰), and Tiqiao Xiao(肖体乔). Chin. Phys. B, 2022, 31(6): 064202.
[5] Multiple-image encryption by two-step phase-shifting interferometry and spatial multiplexing of smooth compressed signal
Xue Zhang(张学), Xiangfeng Meng(孟祥锋), Yurong Wang(王玉荣), Xiulun Yang(杨修伦), Yongkai Yin(殷永凯). Chin. Phys. B, 2018, 27(7): 074205.
[6] Optical encryption of multiple three-dimensional objects based on multiple interferences and single-pixel digital holography
Ying Wang(王莹), Qi Liu(刘琦), Jun Wang(王君), Qiong-Hua Wang(王琼华). Chin. Phys. B, 2018, 27(3): 034202.
[7] Piecewise spectrally band-pass for compressive coded aperture spectral imaging
Qian Lu-Lu (钱路路), Lü Qun-Bo (吕群波), Huang Min (黄旻), Xiang Li-Bin (相里斌). Chin. Phys. B, 2015, 24(8): 080703.
[8] A joint image encryption and watermarking algorithm based on compressive sensing and chaotic map
Xiao Di (肖迪), Cai Hong-Kun (蔡洪坤), Zheng Hong-Ying (郑洪英). Chin. Phys. B, 2015, 24(6): 060505.
[9] Correspondence normalized ghost imaging on compressive sensing
Zhao Sheng-Mei (赵生妹), Zhuang Peng (庄鹏). Chin. Phys. B, 2014, 23(5): 054203.
No Suggested Reading articles found!