CardioExchange Archive

In a recent Viewpoint in JACC: Cardiovascular Interventions, Marso and colleagues expressed grave reservations about the application of appropriate use criteria for PCI in a controversial study published last year in JAMA by Chan and colleagues, which found that only half of PCIs performed for nonacute indications were classified as appropriate. Interventional cardiology editors Rick Lange and David Hillis asked CardioExchange members for their opinions on this topic. In this series of posts, Lange and Hillis question the main protagonists in the debate, Steven Marso and Aaron Grantham, on the one side, and Paul Chan and John Spertus on the other side. All the authors are affiliated with the Saint Luke’s Mid America Heart Institute in Kansas City, Missouri. Click for Part 1, Part 2, Part 3, and Part 4 of this debate. This is the final installment of this series.

Lange & Hillis: What is the acceptable threshold for inappropriate PCI for non-acute patients?

Chan & Spertus: We do not currently have an absolute threshold that is ‘acceptable’. However, the variability across centers for non-acute PCIs, ranging from 0 to >50%, is clearly unacceptable and suggests that some patients are being treated because of who their doctor is, rather than the severity of their disease and their potential to benefit from revascularization.

Marso & Grantham: In an ideal world, the inappropriate rate should approach 0%. However, given the complexities of measuring appropriateness, the writing committee rightly established the inappropriate rate at “national norms”, which unfortunately remains elusive. Revealing the true norm will be a challenge and will require a rework of the AUC methodology.

In a recent Viewpoint in JACC: Cardiovascular Interventions, Marso and colleagues expressed grave reservations about the application of appropriate use criteria for PCI in a controversial study published last year in JAMA by Chan and colleagues, which found that only half of PCIs performed for nonacute indications were classified as appropriate. Interventional cardiology editors Rick Lange and David Hillis asked CardioExchange members for their opinions on this topic. In this series of posts, Lange and Hillis question the main protagonists in the debate, Steven Marso and Aaron Grantham, on the one side, and Paul Chan and John Spertus on the other side. All the authors are affiliated with the Saint Luke’s Mid America Heart Institute in Kansas City, Missouri. Click for Part 1 and Part 2 of this debate.

Lange & Hillis: A recently published study that garnered national attention relied on data collected from the National Cardiovascular Disease Registry (NCDR) to assess the appropriateness of PCI in a large contemporary United States patient cohort. Would you comment on the reliability of these data? 

Chan & Spertus: Extensive efforts were undertaken to assure the accuracy of the mapping and there is extensive site training provided to NCDR data collectors to insure the best possible abstraction of data. An internal audit among 11 sites also found exceedingly high accuracy rates (>95%) in coding for their 127 inappropriate cases, and none of the divergent cases suggested that reforms were needed prior to publishing the data (e.g. one site mentioned that ‘they do not believe their stress test results’, which is hardly a compelling reason to change the AUC or its reliance on the magnitude of ischemia as defining patients more appropriate for revascularization). We do recognize that there may be difficulties in the data collection due to a) poor documentation in the medical record surrounding patient characteristics and the indications of the procedure (including symptom status); b) non-uniform reporting of stress testing that warrants improvement in the accuracy of interpretation and reporting; c) variability in the interpretability of angiographic stenosis; etc. However, we believe that the AUC are serving as an important stimulus to improve all of these aspects and will lead to continued improvements in the accuracy and interpretability of the AUC data.

Marso & Grantham: The NCDR is an important vehicle for research, quality improvement, and performance feedback in the United States. Like any registry it has strengths and weaknesses. This registry is voluntary, has limited auditing and no ongoing adjudication. These pragmatic design features enable broad participation, the findings from ongoing quality improvement projects are perhaps even more generalizeable. Furthermore, this approach helps to restrain the cost of maintaining this registry. We suspect the collection of routine data elements including age, sex, stent type and binary covariates such as death are likely recorded reliably.

However, it is problematic to distill complex information into discreet data points for use in the NCDR Registry. The case example for this is the need to assess future risk based upon stress test results. Is the test normal or abnormal and if abnormal low, intermediate or high risk? Unfortunately stress test result reports are not standardized and many do not comment specifically upon the risk of the result. In order to categorize a PCI as appropriate, uncertain or inappropriate it is required that data abstractors often “interpret the interpretation”. This level of data collection is usually performed by highly trained personnel in core laboratories in high cost, randomized controlled clinical trials. This data collection requirement is problematic for a number of reasons. Firstly, the quality of stress testing in the United States is highly variable. This is perhaps an insurmountable problem. Secondly the AUC’s guidance on whether a stress test is high risk or not is far too vague. Thirdly, there is no systematic method in which clinicians summarize the interpretation of the stress test. This leads to substantial challenges when attempting to record this information. Finally there is no requirement for training in order to enter data into a case report form. At some institution it is done by first year fellows while others hire professional abstractors or require interventional cardiologists to enter the data. We believe this leads to substantial variability in categorizing a PCI as A, U or I. This is the crux of the problem with the JAMA paper. It is just as likely that the inappropriate (I) rate of 11.6% among stable patients and the variability found between hospitals is related to variable quality in these interpretations as it is to do with case selection.

In a recent Viewpoint in JACC: Cardiovascular Interventions, Marso and colleagues expressed grave reservations about the application of appropriate use criteria for PCI in a controversial study published last year in JAMA by Chan and colleagues, which found that only half of PCIs performed for nonacute indications were classified as appropriate. Interventional cardiology editors Rick Lange and David Hillis asked CardioExchange members for their opinions on this topic. In this series of posts, Lange and Hillis question the main protagonists in the debate, Steven Marso and Aaron Grantham, on the one side, and Paul Chan and John Spertus on the other side. All the authors are affiliated with the Saint Luke’s Mid America Heart Institute in Kansas City, Missouri. Click here for Part 1.

Lange & Hillis: The writing group and technical panel for the 2009 AUC and the 2012 update were almost identical (only 1 exception), and only four interventional cardiologists participated in the writing group. Should more (or different) interventional cardiologists be involved in the AUC process?

Chan & Spertus: We believe that the inclusion of a broad range of perspectives in adjudicating the strength of revascularization appropriateness is an asset — particularly because the evidence of benefit should be transparent to all physicians (interventionalists and noninvasive cardiologists alike) caring for CAD patients. In fact, 12 of the 17 panel members are practicing cardiologists who either refer patients for or perform PCI. This is important because the responsibility for a potentially inappropriate PCI should be shared not only by interventionalists performing the procedure, but also by referring physicians.

Contrary to the assertion of Marso and colleagues, the ratings of 85 practicing cardiologists in our paper in JACC correlated exceedingly well with the expert panel, especially given that the latter were able to meet face-to-face to resolve differences of opinion. The interventional members of the AUC technical panel were recommended by SCAI and ACC, and we believe that they should have represented the mainstream perspective of that community; furthermore, the iterative process of determining AUC ratings within the expert panel gave the interventionalist members the opportunity to persuade the rest of the panel if there were indeed compelling data and experiences that should have swayed the appropriateness ratings.

Marso & Grantham: We feel strongly there needs to be broader inclusion of interventional cardiologists in the AUC process. Four is just not enough. We acknowledge that the ACCs current methodology purposefully limits the absolute number of specialty-specific experts involved in the technical panel when creating appropriate use criteria. However, we believe the process would be improved with better representation from interventional cardiologists. Perhaps a way to accomplish this would be for the AUC technical committee to commission a working group of interventional cardiologists to provide direct feedback to the AUC Technical Committee. Certainly, the FDA uses advisory panels when considering drug or device approval decisions. We believe this would be a feasible strategy to broaden participation while limiting the Technical Panel to the RAND criteria. Perhaps it is also time for the ACC to review the rationale for using RAND methodology in establishing AUC documents. The RAND methodology has been around since the 1950s. It is certainly commonly employed in many complex decision-making situations. One wonders, has RAND methodology been consistent and successful in changing physician behavior over the years? Is this the best methodology for the ACC to employ moving forward? To those of us non-RAND experts, it does seem to be rather exclusionary, and its decision-making process less than transparent to outsiders.