Framework for Generative Artificial Intelligence Enhanced Microscopy Image Analysis Automation of Metallic Materials: a Case Study

The characterization of the process-structure-property relationship is essential to optimize processes in manufacturing. However, in practice it often suffers from a lack of quantity and quality in datasets. In an industrial setting subject to time-sensitive considerations, such as troubleshooting by investigating defective parts through microscopy characterization, data are often obtained with suboptimal sample preparation and imaging conditions with limited sampling. Furthermore, the difficulty in quantitatively describing complex microstructures can be compounded by a lack of specialized expertise in industry. To address these challenges, automated extraction of quantitative microstructure data has become a main objective. In recent years, effective approaches have been developed leveraging advanced digital image processing and deep learning techniques [1–3]. Specifically, models employing a U-Net architecture [4] have demonstrated proficiency in performing semantic segmentation of microstructure images with complex contrasts [5–9], a task that traditional histogram-based methods struggle with. However, it is notable that these models, when trained on a specific dataset, may underperform on new images that slightly differ in sample preparation or imaging conditions [10]. These variations, often subtle and challenging to control, can significantly impact the model's accuracy to provide reliable microstructure quantification results. In this study, a framework supported by generative artificial intelligence is introduced for an automatic quantification of microstructures, focusing specifically on aluminum alloy manufactured by cold spray processing. These complex microstructures feature primary alpha and eutectic phases from the initial powder feedstock as well as the boundaries of the deformed and consolidated particles. A significant challenge in this context is an inconsistency in etching response among samples processed with the same nominal etching conditions. Our innovative solution involves the application of generative AI models, including a conditional generative adversarial network (CGAN) [11] and an Image-to-Image translation model (Pix2Pix) [12], to adjust images of under etched samples (class A) to resemble suitably etched ones (class B). This enhancement enables a U-Net model, primarily trained on class B images, to achieve robust segmentation on class A images as well, which addresses the issue of variability in sample preparation and applicability of the U-Net model across a broader range of samples. Test coupons of Al6061 alloy, produced using various cold spray process parameters, were cross-sectioned, mounted, polished and etched, as described in [13]. The optical images were then with an Olympus BX51 optical microscope and Clemex Vision PE software. Images of both class A and class B were selected from the acquired images for training of the CGAN, as shown in Fig.1. The CGAN is able to generate realistic micrographs similar to images acquired by microscope. Moreover, it is capable of forming pairs of images for both classes at the same virtual imaging spots. These paired images, sharing identical microstructure content but different etching level, were employed to train a Pix2Pix model, enabling the conversion of images from class A to class B, a process illustrated in Figure 1. Subsequently, a U-Net model previously developed in our research [13] is employed to highlight the improvement in segmentation outcomes when comparing the original under etched images with their enhanced counterparts. The predicted area of particles from U-Net tend to be systematically larger than annotated areas, while the predicted AR is systematically smaller, as discussed in our previous study [13]. However, images acquired from under etched samples were unable to be segmented using the same U-Net, images as shown in Figure 2A with incomplete U-Net predicted mask of four class A images. These images were also utilized to assess how the Pix2Pix model improves particle boundary delineation and particle size evaluations. Figure 2A demonstrates that the masks generated from the Pix2Pix processed images are visually more aligned with manual annotations, showcasing enhanced boundary detection and closure. This is in agreement with the particle area and aspect ratio (AR) measurements as ratio of U-Net prediction over annotation, shown in Figure 2B. Applying the Pix2Pix model twice to these images reduces the overestimation of area measurements as well as the standard deviation. Regarding AR measurements, accuracy remains similar to the reference, while no significant change in the standard deviation was observed. Crucially, the standard deviation in average particle area and average AR across images are also relatively small (5.9% for area and 0.7% for AR compared to 2.3% and 0.7% for reference, respectively), suggesting reliable measurements and manageable systematic errors. In conclusion, the study effectively utilized CGAN and Pix2Pix for enhancing images with virtual metallography etching control. Images that were initially less etched were transformed to enable more precise and consistent segmentation results. This AI-driven automatic segmentation approach shows promise for further applications in quantifying microstructures in other downstream tasks [14]. The workflow of GAN based image enhancement for better automatic quantification reliability in the case of aluminum alloy cold spray particle measurements. The Pix2Pix enhanced micrograph transformed from a under etched one shows better contrast at the edge of particles and provide automatic segmentation much closer to human annotation compared to the mask from under etched one. (A) Visualization of pre-trained U-Net segmentation difficulties on under etched images compared to suitably etched image and the improvement offered by Pix2Pix. (B) Comparison between Pix2Pix enhanced images and reference class B images for particle area and aspect ratio obtained from U-Net predicted masks with respect to annotations.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.