Fashioning the Future: Could <scp>AI</scp> Enhanced <scp>MRI</scp> Put <scp>PET</scp> Out of Style?

Deep learning image synthesis models, such as StyleGAN, DALLE-2, and Stable Diffusion, have captivated the public attention by performing tasks from winning art competitions to semantically segmenting city streets.1-3 While less visually dramatic, the biomedical imaging and radiology AI communities are experiencing equally significant advancements. Generative image models are beginning to reshape diagnostic processes, paving the way for improved patient care and advanced medical research. The two articles in this issue, focus on the unconventional task of utilizing a technique known as "style transfer" to perform the seemingly impossible task of image translation from MRI to positron emission tomography (PET). The first study, "Image Translation for Estimating Two-Dimensional Axial Amyloid Beta PET from Structural MRI,"4 employed a conditional generative adversarial network to generate amyloid-beta PET images from structural MRI. The researchers used the Open Access Series of Imaging studies, a public dataset of paired MRI and PET scans from Washington University, which includes both cognitively normal subjects and those at various stages of cognitive decline. Of the total 552 paired scans used for training, 331 were designated for internal testing/validation, notably pairing 11C-PiB PET images with only T1w MRI sequences. The impressive results, with an SSIM of 0.905 and PSNR of 22.685, along with compelling positive and negative synthetic amyloid PET images are both startling and exciting, made more so when one tries to closely examine the original T1w images and finds nothing uniquely distinguishing between cases and controls. The second study, "Predicting FDG-PET Images from Multi-contrast MRI using Deep Learning in Patients with Brain Neoplasms,"5 synthesized FDG PET scans using simultaneous 18F-FDG PET and MRI images of brain tumors. A combination of 3T MRI T1w pre/post, T2 FLAIR, and arterial spin labeled images were used as inputs. Image quality was gauged using both objective metrics and subjective physician evaluations of image quality and lesion review. Notably, the synthesized PET images achieved an accuracy of 87% for tumor versus no viable tumor classification, persuasively arguing the concept the authors have termed "zero-dose" FDG PET imaging. Both studies used similar strategies, including generative adversarial designs for image synthesis and an emphasis on careful pre-processing and close cross-registration of input and output images. Such approaches push the extreme boundaries of style transfer capabilities crossing the boundaries of modalities, physical signals, and indeed medical subspecialties and allude to future post-processed sequences where functional imaging techniques are performed synthetically and at greater scale. Generative systems are not without their weaknesses, and concerns have been raised about potential systemic biases and sensitivities. Cohen et al6 warned about the potential for generative systems to be artificially manipulated toward hallucinating lesions, or omitting lesions altogether from synthesized images, distortions due to biases from the prevalences of disease or normality introduced from training datasets. Some researchers have proposed that GAN systems are susceptible to adversarial attack, resulting in both security and image tampering implications.7 These criticisms and their novelty raise the stakes for thorough real-world validation, expert supervision, and post-market surveillance. In a modern era challenged by AI-generated deep fakes and synthetic images, it is remarkable to see this at times controversial technology being harnessed for practical purposes in medicine. As we delve deeper into these methodologies, we must be vigilant about potential pitfalls and challenges, ensuring the responsible use and safety of technology in enhancing patient care and medical research. These techniques are still nascent, but they provide a glimpse into a future of medical imaging where the acquired image is not necessarily the final image for interpretation, and just as easily as one adjusts a window level today, you might soon be able to restyle an MRI examination as something entirely different—a PET born of an MRI. The author declares no conflicts of interest.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.