Optical Distortion Correction of Convergent Beam Electron Diffraction Disks Using Deep Learning

Recent technological developments in hybrid-type pixelated direct electron detectors have sparked growing interests in 4-dimensional scanning transmission electron microscopy (4D-STEM) techniques. These techniques can be used to determine for instance orientation and structural information in crystals, as well as phase information when combined with ptychography. However, the accuracy of 4D-STEM techniques are often limited by the presence of optical distortion in the collected convergent beam electron diffraction (CBED) patterns, caused by aberrations of the electromagnetic lenses. Existing distortion characterization techniques typically involve estimating the centers of the CBED disks in a given pattern, using e.g. the radial gradient maximization (RGM) technique [1], and then performing some kind of least-squares optimization procedure according to an expected or assumed reciprocal lattice system. In cases where the CBED patterns are subject to appreciable degrees of elliptical and pincushion distortion, methods like RGM can potentially breakdown as they rely on the CBED disks being more or less circular or elliptic. To overcome the aforementioned limitations of existing distortion characterization techniques, we develop a deep learning framework for estimating distortion. We train a modified ResNeXt model [2] to return as output the common undistorted CBED disk radius, and the parameters of the distortion model, which we assume to be a generic trigonometric series following [3]. After obtaining the estimated distortion model, we perform inverse mapping to generate a distortion-corrected CBED pattern. Figure 1 presents an example of the outcome of the distortion correction procedure applied to a mutli-slice simulated CBED pattern of 5-layer MoS2 on amorphous C. We trained our neural network using artificially generated CBED patterns that are subject to a variety of different distortion transformations, dosage levels, and disk position distributions to maximize the diversity of images to which our model can be applied. We generated 28800 images of dimensions 512x512, of which 80% were used for training while the remainder were used for validation/testing. Using 15 computing nodes, each equipped with an Intel E5-2683 v4 Broadwell processor with 16 CPUs, we generated the entire dataset in under 6 hours. Our neural network was trained on a single 32 GB NVIDIA V100 Volta GPU in under 12 hours. Each output variable in our model is normalized to lie in the range of [0, 1], where 0 and 1 correspond to the lowest and highest values of the output variable in the entire dataset. Table 1 shows the final root-mean squared (RMS) testing error of each output variable, where we see that the normalized spiral distortion yields the highest RMS error with a value of 0.076. Preliminary results suggest that these errors can be further reduced with more training data, data balancing, and hyperparameter optimization. The advantages of our deep learning approach are that the model requires no prior knowledge of the sample in order to estimate the distortion, and that the model can handle CBED patterns with overlapping disks and distorted disk shapes that are far from circular. [4]. An example of distortion correction performed on a multi-slice simulated CBED pattern of 5-layer MoS2 on amorphous C, subject to elliptical, pincushion, and spiral distortion. (a) The undistorted image; (b) The distorted image; (c) The distortion-corrected image. Note that all images were cropped to the same reduced dimensions such that the zero-valued pixel areas introduced by distortion and distortion correction in (b) and (c) respectively were removed. The final root-mean-squared testing error of each output variable of our deep learning model. Each output variable is normalized to lie in the range of [0, 1], where 0 and 1 correspond to the lowest and highest values of the output variable in the entire dataset used for training and testing. Note that the norm and direction of the elliptical distortion vector encode the amplitude and the direction of the corresponding distortion. The final root-mean-squared testing error of each output variable of our deep learning model. Each output variable is normalized to lie in the range of [0, 1], where 0 and 1 correspond to the lowest and highest values of the output variable in the entire dataset used for training and testing. Note that the norm and direction of the elliptical distortion vector encode the amplitude and the direction of the corresponding distortion.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.