Analysis of Serum Proteomic Patterns for Early Cancer Diagnosis: Drawing Attention to Potential Problems

In a recent update (1) of already impressive data (2), it was reported that the use of proteomic patterns in serum to diagnose ovarian and prostate cancers can achieve perfect diagnostic sensitivity and specificity. A diagnostic sensitivity and specificity of 100% is unprecedented for any tumor marker known to date and, if reproducible, this finding could have a major impact on the way we diagnose cancer in the future. Over the last 2 years, results reported by several groups (2–6) have suggested that such proteomic patterns, particularly those generated by surface-enhanced laser desorption/ionization time-of-flight (SELDI-TOF) mass spectrometry, may facilitate the early diagnosis of various cancers, including those of the ovary, prostate, breast, and bladder. SELDI-TOF proteomic profiling technology has been reviewed (7,8). The impressive results reported with this new technology were welcomed by scientists, the popular press, the public, and even by politicians (9). Although there has been little published criticism of this methodology (10–12), serious skepticism about its utility has been expressed publicly at various scientific meetings. Many investigators and clinicians have adopted a wait-and-see approach pending the outcomes of prospective clinical studies using this technology which are starting now but will require years to complete. Here, I summarize some shortcomings of this technology for the purpose of stimulating further discussion and research. Considering what is known about tumor markers, the mechanisms by which they are released into the circulation, their abundance in biologic fluids, their metabolism and excretion, and their dynamic relationships within the host, it is unlikely that proteomic profiling by SELDI-TOF methods will be a useful approach for the diagnosis of cancer. Moreover, it is conceivable that published data may, in fact, be biased by artifacts related to the nature of the clinical samples used, the mass spectrometry instrument, and/or the bioinformatic analysis. In a recent meta-analysis (12) of prostate cancer proteomic data from four papers by three different research groups, I pointed out that the discriminatory peaks (i.e., peaks representing molecules that appear or disappear during cancer progression, or whose amounts differ in cancerous versus noncancerous tissue) identified in the four papers were very different, even in the two papers published by the same group using the same experimental data but different bioinformatic tools (12,13). These data are summarized in Table 1. These discrepancies suggest that serum proteomic patterns obtained by the SELDI-TOF technique may not be reproducible and that the discriminatory peaks are not consistent either within a group or among groups of investigators for the same type of cancer, even when the general analytical methods or datasets are the same. Furthermore, the reported diagnostic sensitivities and specificities of prostate cancer diagnosis based on SELDI-TOF technology differ substantially among the four reports. Another rather surprising phenomenon associated with these data is that serum proteins that are known to distinguish patients with benign conditions from patients with malignancies (e.g., prostate-specific antigen in prostate cancer) were not identified by this new technology, raising serious questions about its analytic sensitivity. After reviewing the serum levels of known tumor markers for various malignancies and the current sensitivity of mass spectrometers, I have concluded that the SELDI-TOF technology that is currently used for serum analysis is not capable of detecting any serum component at concentrations of less than 1 μg/mL (12). This range of concentrations is approximately 1000-fold higher than the concentrations of known tumor markers in the circulation (12). This analysis led me to conclude that the serum discriminatory peaks identified by mass spectrometry very likely represent high-abundance molecules that were unlikely to have been released into the circulation by very small tumors or their microenvironments. I suggested that the discriminatory peaks may instead represent acute-phase reactants (i.e., molecules whose serum concentrations are increased in patients with acute or chronic inflammatory conditions) or other proteins or protein fragments that are released into the circulation by large organs, such as the liver, in response to the presence of a tumor or cancer epiphenomena, such as infection, inflammation, or malnutrition (10,11). Alternatively, some of these proteomic changes may represent artifacts of sample collection, storage, or pretreatment, patient selection, or other idiosyncrasies. Little effort has been made to positively identify at least some of the molecules that comprise the discriminatory peaks to understand their origins and why their levels are altered. Indeed, in the few cases where such peaks were identified, some were composed of high-abundance molecules released by the liver and others were composed of acute-phase reactants (Table 2) (16–18). At the 2003 annual meeting of the American Association for Cancer Research, Zhang et al. (16) reported the identities of three discriminatory peaks in ovarian cancer: apolipoprotein A1, transthyretin (pre-albumin) fragment, and inter-alpha-trypsin inhibitor. Others identified haptoglobin-α subunit for ovarian cancer (17) and vitamin D-binding protein for prostate cancer (18). Table 2 presents the serum concentrations of these putative tumor markers and of classical tumors markers, such as alpha-fetoprotein and prostate-specific antigen. Some of the “new” tumor biomarkers identified with the use of SELDI-TOF technology were in fact originally identified more than 30 years ago by classical techniques as putative tumor markers (e.g., elevated serum haptoglobin-α subunit in ovarian cancer) (19–21) but were never used for clinical diagnosis because of their low sensitivity and specificity. For example, in a recent MEDLINE search using the keywords “haptoglobin” and “cancer,” I identified 571 papers published from 1965 through 2003. A review of the titles confirmed that haptoglobin was reported as early as 1966 to be elevated in the following malignancies: leukemias, Hodgkin's disease, Burkitt's lymphoma, multiple myeloma, neuroblastoma, melanoma, glioma, and cancers of the cervix, genitals, stomach, breast, liver, kidney, ovaries, lung, endometrium, colon, prostate, gallbladder, bladder, head and neck, brain, and larynx. Thus, the notion that haptoglobin-α subunit or other acute-phase reactants may represent new cancer biomarkers is not accurate. If indeed the proteins or fragments identified by SELDI-TOF technology do not originate from cancer cells or their microenvironment but instead represent epiphenomena, it is useful to ask how such profound changes in serum proteomic patterns could be induced by small tumors localized within the original organs, as is usually the case with early disease. In my opinion, it is highly unlikely that a small, localized tumor within an organ would be able to stimulate other organs to produce and secrete vast amounts of proteins into the circulation, thus affecting a change in the proteomic pattern that would allow an early cancer diagnosis. Despite the impressive results reported by several groups (2–8) and the implications of those results, we still do not have answers to critical questions related to sample collection and processing. For example, we do not as yet know whether proteomic patterns differ between plasma and serum, or how they are affected by lipemia, icterus, the number of freeze/thaw cycles the sample underwent or its length of storage, or the subject's menstrual cycle, nutritional status, or drug use. It is possible that the different proteomic patterns observed for case patients and control subjects may be due to such differences and not to the presence of cancer. Furthermore, the use of “black box” bioinformatic tools to analyze these data increases the likelihood of data overfitting. Data overfitting in one study can produce test sensitivities and specificities that may not be reproduced in subsequent studies. For example, data overfitting may account for the inconsistent results reported by Rogers et al. (22). In that paper, the sensitivity values for a proteomic profiling test to discriminate between renal cell carcinoma patients and control subjects by SELDI-TOF technology were initially 98%–100%. However, when the authors used the same procedure 10 months later to test a new set of patients, the sensitivity dropped to 41%. The authors speculated that this dramatic decline in test performance may have been due to sample instability, laser performance, or protein chip variability. These data call into question the long-term robustness of this method. Regarding the reproducibility of bioinformatics, Sorace and Zhan (23) recently reanalyzed the raw ovarian cancer proteomic dataset originally used by Petricoin et al. (2). Although they identified several peaks that contributed decisively to the discrimination between control subjects and case patients, those peaks had m/z values of less than 2000, and peaks with such low m/z values are usually discarded by other investigators as “experimental noise” due to matrix effects (e.g., one highly discriminatory peak had an m/z value of 2.79) (23). Sorace and Zhan (23) concluded that there was a substantial, nonbiologic experimental bias between the cancer patients and control subjects that cast doubts on the validity of the discriminatory peaks with m/z values greater than 2000. These authors further concluded that data-mining algorithms for mass spectrometric data should be carefully reviewed to avoid similar false discovery. In response to my questions about the identity and the relative abundance of molecules in the discriminatory peaks in the circulation (12), Petricoin and Liotta (24) have suggested that the discriminatory peaks likely represent fragments generated by the proteolytic digestion of high-abundance proteins within the tumor microenvironment, which ultimately leads to high levels in the serum. This hypothesis merits further investigation. However, it would be surprising to see a high abundance of such peptides in the circulation, given their low molecular weights and their expected effective clearance by the kidney. Another hypothesis proposed by Petricoin and Liotta (24) is that such peptides may escape clearance by the kidney because they are strongly bound to albumin. There is, as yet, no published experimental evidence to support this hypothesis. However, it is also important to consider whether these peptides originate in serum or plasma before (i.e., in vivo) or after (i.e., ex vivo) blood is drawn. Marshall et al. (25) recently demonstrated that peptides identified by matrix-assisted laser desorption/ionization time-of-flight (MALDI-TOF) mass spectrometry analysis that were diagnostic for myocardial infarction were generated ex vivo by the action of proteases and that these peptide patterns could be substantially altered by the addition of protease inhibitors, such as phenylmethylsulfonyl fluoride, to the blood samples. The reproducibility of protein patterns obtained by SELDI-TOF remains in doubt. For example, there are no published reports that similar data can be obtained by using different batches of SELDI chips, different technologists, different instruments, or by using the same conditions at a later time. Similar concerns about the reproducibility of proteomic data for prostate cancer diagnosis were raised by Walsh (26). One working hypothesis for the published data (2–8) is that the differences in serum proteomic patterns between control subjects and case patients are due to the presence of cancer in the latter group. The alternative hypothesis is that these differences are not due to the presence of cancer but to something else. Possible confounders could include 1) variability in sample collection, processing, and storage; 2) baseline characteristics of study subjects (e.g., sex, age, ethnicity, level of exercise, menopausal status, nutritional habits, or drug use); 3) inappropriate statistical design; and 4) variations in mass spectrometer stability and protein chip performance. Interestingly, the effects of all these parameters on the serum proteomic patterns are unknown. Thus it will be interesting to see if proteomic profiling by SELDI-TOF technology can effectively distinguish between premenopausal and postmenopausal women without cancer or among different ethnic groups. Clinicians usually use numeric cutoff points to evaluate tumor markers. Until now, all studies using serum proteomic patterns have compared “disease” patterns with “normal” patterns. In practice, the method will need to adopt a “normal” pattern with which the pattern for the patient will be compared. I believe that it will be very difficult to choose a single “normal” pattern that will be applicable to patients of different ages, sex, ethnicity, menopausal status, or nutritional status. In other words, it seems that the “normal” pattern will be highly influenced by numerous parameters which will make interpretation of the data very difficult. In conclusion, serious questions about SELDI-TOF technology for protein profiling remain unanswered. It is possible that the data obtained using this technology are biased by artifacts of sample collection, storage and processing; patient selection; or inappropriate analytical and statistical methods. At this point, it would be prudent to invest in efforts to identify the discriminatory peaks obtained with this method so that any resulting proteomic patterns will have a solid scientific basis. Furthermore, it will be important to standardize the methodology for at least one or two cancers and then conduct blinded studies that use the finalized algorithms to establish whether this technology can efficiently diagnose early-stage cancer with near-perfect sensitivity and specificity. Until these data are generated, the use of the method in clinical practice should be discouraged. As a final note, I wish to emphasize that the limitations outlined above concern the specific use of serum proteomic patterns obtained using SELDI-TOF technology to diagnose human disease. I do not wish to imply that these limitations apply to the use of mass spectrometry as an analytic tool or to other proteomic approaches used to identify proteins in complex mixtures such as serum and other biologic fluids. For example, the ongoing Human Plasma Proteome Project, which is sponsored by the Human Proteome Organization (HUPO), will likely positively identify numerous potential biomarkers that could be used either alone or in combination to better diagnose human disease. Comparison of data from four reports of prostate cancer diagnosis based on SELDI-TOF technology* SELDI-TOF = surface-enhanced laser desorption/ionization time-of-flight; IMAC-Cu = immobilized metal affinity capture array with copper surface; BPH = benign prostatic hyperplasia; WCX2 = weak cation exchange array. m/z values were rounded to whole numbers for simplicity. m/z values in bold represent peaks identified by Adam et al. (4) and Qu et al. (6) that discriminated between patients with and without cancer. The underlined m/z value represents a peak that discriminated between patients with and without cancer (4) and between healthy individuals and patients with benign prostatic hyperplasia (6). Comparison of data from four reports of prostate cancer diagnosis based on SELDI-TOF technology* SELDI-TOF = surface-enhanced laser desorption/ionization time-of-flight; IMAC-Cu = immobilized metal affinity capture array with copper surface; BPH = benign prostatic hyperplasia; WCX2 = weak cation exchange array. m/z values were rounded to whole numbers for simplicity. m/z values in bold represent peaks identified by Adam et al. (4) and Qu et al. (6) that discriminated between patients with and without cancer. The underlined m/z value represents a peak that discriminated between patients with and without cancer (4) and between healthy individuals and patients with benign prostatic hyperplasia (6). Serum concentration of selected abundant proteins, putative cancer biomarkers identified by SELDI-TOF, and classical cancer biomarkers* SELDI-TOF = surface-enhanced laser desorption/ionization time-of-flight; — = not applicable. Serum concentration of selected abundant proteins, putative cancer biomarkers identified by SELDI-TOF, and classical cancer biomarkers* SELDI-TOF = surface-enhanced laser desorption/ionization time-of-flight; — = not applicable.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.