Building Real-World Evidence Infrastructure to Improve Health and Healthcare in the United States: Part II—How Coordinated Registry Networks Operate Like Systemic Coordinated Inter-Organizational Networks

SCIONs Criteria Applied to CRN: Characteristics, Coordination Mechanisms, and Performance Evaluation

The criteria to identify SCIONs include characteristics, modes of coordination, and performance evaluation. We illustrate these criteria with examples from existing CRNs. This section examines the structure and function of a diverse CRN community of practice, while also documenting how CRNs have improved health outcomes, quality, safety, and reduced unnecessary care. Following sections provide specific examples of performance evaluation in this context.

Characteristics

SCIONs are defined by the following characteristics: (1) a network of organizations, (2) pursuing or contributing to a common desired goal, and (3) with an autonomous unit coordinating their related activities, with (4) technical staff and methods of measuring and evaluating performance [Table 1]. CRNs demonstrate these characteristics in the following ways:

Systemic. CRNs are voluntary networks of between 30 and 1,000 clinical facilities that may be hospitals, freestanding clinics, or group or solo practices that, as a network, come together around shared goals for improving patient outcomes and efficiency of care in a specific clinical specialty. They address institutional and systemic problems. Mature CRNs support additional goals of stakeholders, including public health preparedness, clinical trial conduct, medical product development, and regulatory decision-making.^[2,3,4] Mature CRNs typically partner with industry (e.g., for medical products and information technology) and government agencies to coordinate data standards and data needs as partners.^[5,6] Mature CRNs also have formal interfaces with patient organizations and patient advisors.^[7,8] These organizations unite as part of the US healthcare system to produce health and healthcare for the US population.

Organizational. CRNs are typically run by a network-coordinating organization (NCO) that operates as a 501(c) (3) organization to promote public health or science or as a 501(c)(6) organization that promotes common business interests. These organizational relations are distinct from the personal relationships between individuals that may develop as a result of their work. The NCO is an autonomous organization with its own CEO and board, which is likely composed of representatives from its members.

Coordination. Each operating CRN has an independent CEO affiliated with one or more clinical societies that coordinate activities around shared goals.^[9] Coordination includes management of various data types involving extraction,^[10–12] curation,^[13] standardization,^[14] interoperability,^[15–16] and harmonization^[14] processes to ensure seamless integration into a multipurpose, accessible data source. The NCOs of CRNs have brought together legacy activities (e.g., quality improvement [QI] efforts and registries) and combined them into robust organizations.^[17–18]

Technical staff. NCOs’ full-time technical staff collaborates with network members in a supportive and supervisory manner on clinical and data management activities. The NCO staff facilitates committees that support ongoing clinical efforts such as improving guidelines, developing training programs, and coordinating peer-to-peer support for QI. NCO staff lead multiple committees and information technology contractors that sustain data management. The clinical and other unique skills of the staff and committees make this work possible.

Physician-scientists^[19] are critical participants in technical coordination activities. The role of the physician-scientist in a CRN can be described as a “boundary spanner,” a term for an individual who crosses the group boundaries to enable knowledge exchange, translate language, and share values among various groups.^[20] Their scientific training and clinical skills uniquely position them to bring together clinical and investigational efforts.^[21]

Coordination Mechanisms

CRNs use three coordination mechanisms described for SCIONs: allocating, controlling, and stimulating cooperation.

Allocating. The NCO allocates resources across the network partners and pays support contracts. Depending on the CRN’s financial model, resources may be in the form of services or payments flowing from the NCO to the CRN network clinical partners or in reverse. Each stakeholder can contribute to and benefit from the CRN in unique ways.^[22] In some CRNs, funds flow from the NCO to members for services. Overall, NCOs allocate a mixture of funds, services, and shared tools to support the data activities and processes that help clinical entities change their processes to improve quality.

Controlling. The NCO exercises control over data management procedures to maintain the registries. Control of the data processes of the CRN is critical to ensuring the quality and utility of the data. Data quality reporting requirements are conditions of membership. According to the SCION literature, the NCO has the authority to remove organizations from the network due to poor performance.^[1] Although this is formally true for CRNs, the authors are unaware of any instances where a clinical unit’s inability to contribute data of sufficient quality led to its removal from the network. Voluntary exit from the CRN is known but rare.

Stimulating Cooperation. CRNs use allocation and control as mechanisms of coordination, but what distinguishes their work is the cooperation between clinical entities. The NCO stimulates cooperation by encouraging the sharing of information between members about clinical and data-related activities in ways that are more effective and faster than other communication mechanisms (e.g., peer-reviewed publications). The network stimulates cooperation by coordinating joint scholarly activities, facilitating peer-to-peer consultation, sharing best practices, and promoting shared issues. Cooperation is facilitated by many committees of volunteers from different clinical entities and partners who represent human capital across the network.^[3,23]

Performance Evaluation

Performance evaluation as a core function of CRNs represents another way in which CRNs operate like SCIONs. The CRN’s foundational purpose is to measure the quality of clinical care. As they mature, CRNs function to expand these activities to produce three types of organizational change evaluated in the SCIONs literature: learning, innovation, and adaptation.^[3]

Learning. Organizations learn through CRNs by diffusing knowledge through the network. In clinical medicine broadly, the slow pace of diffusion of new knowledge and implementation into practice (from bench to bedside) is well documented.^[24] CRNs more effectively promote the implementation of innovations in many clinical areas (interventional cardiology,^[25] medical cardiology,^[26] vascular surgery,^[27] hernia repair,^[28] oncology,^[29] orthopedics,^[30] and breast surgery^[31]). In certain clinical areas, CRNs have measured the rate of new knowledge diffusion and the adoption of clinical guidelines.^[26]

Organizational learning in CRNs is observed through the implementation of care standards across their networks. Training, monitoring,^[32] and sharing of experiences and data across the network further enhance social integration in different clinical settings. As described in Part I, CRN learning that improves the quality of care represents a paradigm shift, a radical change, when contrasted to traditional top-down process management models, which have often shown limited impact.^[33–35]

Innovation. CRNs have made new knowledge, procedures, clinical guidelines, and the development of medical products possible. The unique collaborative relationships between the CRN clinical members who share experience and data speed innovation. For example, CRNs have used data to produce new knowledge about racial, sexual, and socioeconomic differences in healthcare.^[36–46] Registry data suggests that minority racial groups are underrepresented among patients undergoing transcatheter aortic valve replacement (TAVR) in the US. However, once engaged in care, their adjusted 30-day and 1-year clinical outcomes are comparable with those of white patients.^[40]

New procedures and guidelines have been developed in CRNs, as well as new indications for existing products through Food and Drug Administration label modifications supported by CRN data.^[47] Procedural innovations that improve safety and treatment outcomes are especially well addressed by large information-sharing networks.^[48] For instance, the use of the National Cardiovascular Disease Registry linked to CMS claims data supported expanding the indication for transcatheter valve therapy (TVT) to include older patients.^[40] CRN-funded epidemiological studies have led to refinements of clinical guidelines to address differences in patient and disease characteristics.^[49] Evaluation of different clinical definitions of disease between clinical centers has led to refinements that support patient selection for treatment and improved outcomes.^[26]

The iterative nature of medical product development may make CRNs a useful context for product innovation.^[50] The National Cardiovascular Data Registry (NCDR®) supported innovations in TAVR across three generations for two companies’ products, iteratively addressing challenges identified through the TVT and other large registries.^[51] The highly competitive nature of medical product development places limitations on what CRNs can contribute, a topic further discussed in Part III.

While CRNs have promoted innovation, increased maturity will enhance their potential through greater systematic gap analysis, goal setting for communities of clinicians, and more systematic collaboration with entrepreneurs that increase the value created in a given CRN.

Adaptation. A CRN demonstrates SCION-like adaptation by solving problems and coping with change, offering radical, data-driven solutions. For example, projections from an active surveillance tool collecting data on cardiac pacemakers suggested that evidence of adverse events associated with a particular product could have been detected over two years earlier if the full CRN had been used.^[52] Recalls of hernia mesh and breast implants are other examples of the adaptive utility of registries.

CRN registries have also been adapted for national monitoring. The American College of Cardiology data resources support the national Healthy People initiative of the US Department of Health and Human Services^[53] that addresses racial and ethnic differentials in cardiovascular disease.^[54] Other CRNs may offer unique data that may be adapted to national efforts for disease monitoring, particularly in clinical areas where data is not readily available to Healthy People.

SCION characteristics of the VQI/VISION CRN

SCION criteria are used to describe the VQI/VISION CRN, including the identification of the following components: system, organization, coordination, and technical staff.

Systemic. The VQI/VISION is the oldest and most mature of CRNs, whose components function as a system to achieve shared goals. Established in 2011, the VQI created a distributed network of pre-existing regional QI groups. The Society for Vascular Surgery Patient Safety Organization (SVS PSO) is structured as a Patient Safety Organization (PSO) and listed by the Agency for Healthcare Research and Quality (AHRQ) as described in the Patient Safety and QI Act of 2005 (Patient Safety Act).^[55] The mission of the SVS VQI is “to improve the quality, safety, effectiveness, and cost of vascular healthcare.”^[56]

Organizational. As of March 2024, there were 1,035 participating institutions^[57] and 18 regional QI groups in the US and Canada participating in the network.^[58] With over 8,000 participating physicians from multiple disciplines, the VQI covers approximately one-third of all vascular procedures in the United States, drawing from a variety of clinical settings, including those in urban, rural, and academic environments.^[57]

The SVS VQI network of regional QI groups provides the structure needed to translate data into changes in clinical practice. Its central methodology is to aggregate information from multiple procedures and analyze patient outcomes to drive QI. To date, the SVS PSO VQI has participated in the data collection of over one million patient encounters.

Coordination. The SVS PSO operates as a nonprofit according to Section 501(c)(6) of the US Internal Revenue Code. Other CRNs are 501(c)(3) nonprofits^[59] with different rules about accepting money from other organizations and a somewhat broader ability to participate in political activities. Despite these differences, both types of nonprofits operate similarly when viewed as CRNs. The SVS PSO can be identified as the autonomous NCO that coordinates the network. The SVS PSO operates through the efforts of an executive director and a governing board, who oversee a series of committees and programs. An SVS PSO staff composed of physicians, nurses, analysts, and administrative personnel provides clinical expertise, statistical analyses, and oversight of QI activities conducted through the PSO. Much of the coordination is decentralized in the regional groups, which act as sub-SCIONs.

The organizational structure includes a representative governing council that provides supportive supervision through several committees and programs. The Quality Committee supervises the analyses, reporting, and quality initiatives of the SVS PSO. The Research Advisory Committee, which has a similar committee structure, is responsible for reviewing member applications for the release of aggregated, de-identified data sets under select rules to participants.

Technical staff. The PSO has a full-time technical staff of 12 people. It acts as the NCO of the VQI/VISION CRN, coordinating the network. The PSO staff also works with contractors,^[60] volunteer committees, and regional bodies. They coordinate activities between partners in a collaborative manner, providing technical assistance and supportive supervision. Their role is akin to that of middle managers, who facilitate organizational processes that create more effective working environments.^[61]

With the assistance of the PSO staff, a volunteer physician lead, an associate physician lead, and a lead data manager oversee the 18 regional groups. The governance policy and organizational chart can be seen in SVS PSO Governance Policies.^[62] A steering committee for industry studies consists of PSO members and the manufacturer who requests data from the VQI.

SCION Coordinating Mechanisms of the VQI/VISION CRN

The VQI/VISION applies SCION coordination mechanisms: allocating, controlling, and stimulating cooperation.

Allocating. Members of the network, hospitals, and other clinical entities pay membership dues. The PSO administers these funds, along with other proceeds from grants and services to industry. The PSO also allocates other resources, equipment, appropriate skills, and technologies across the network. VISION manages resources around the linked VQI and CMS claims data.

Controlling. Control of data quality, analysis, and reporting is critical to the network’s operation. VQI’s suite of 14 procedure-or disease-specific registries contains demographic, clinical, procedural, and outcomes data performed across the US, Canada, Puerto Rico, and Singapore. Biannual regional meetings allow physicians, nurses, data managers, quality officers, and others to meet, share information and ideas, and learn from each other in a collegial and supportive environment. While an NCO has formal authority to remove organizations from the network due to poor performance, VQI/VISION PSO has never dismissed a member.

Stimulating Cooperation. The VQI/VISION CRN fosters cooperation among various members through regional meetings, data analysis, and QI projects. The network shares data and experience, and regional groups convene twice a year to collectively review results. Figure 1 provides a list of the committees through which the VQI operates. To facilitate this cooperation, the VQI provides web-based tools for the collection, aggregation, and analysis of clinical data submitted by those providing vascular care. VQI collaborates with multiple organizations, including the American College of Cardiology, the American Heart Association, the Society of Vascular Medicine, the Society of Interventional Radiology, the American Venous Forum, governmental regulatory agencies, device manufacturers, and payers.

Close

Figure 1:

Structure of the VQI/VISION CRN. VQI is a collaboration of the SVS PSO, 18 regional quality improvement groups organized under the SVS PSO, and FIVOS (a healthcare performance management solutions company), its commercial technology partner. VQI functions as a collecting and analyzing data voluntarily reported by healthcare providers to help improve patient safety and healthcare quality. The SVS VQI Vascular Implant Surveillance and Interventional Outcomes Network (VISION) is an independent university-based project that links Medicare claims data to the registry. VQI: Vascular quality initiative, SVS: Society for vascular surgery, PSO: Patient safety organization.

Export to PPT

Biannual reports to clinical centers provide information on volume, outcomes, benchmarks, and process measures (such as infection rates, number of patients on statins, number of patients who stop smoking, and number of patients on antiplatelet agents). The clinical centers may initiate requests for guidance and assistance from the PSO, which is readily provided via webinars, group conference calls, and individual consultations.

SCION Performance Evaluation of VQI/VISION

VQI/VISION has documented radical organizational change in learning, innovation, and adaptation, playing a key role in identifying and disseminating QI incentives that have transformed practice patterns, enhanced quality, and reduced healthcare costs, as evidenced in over 800 peer-reviewed publications.^[63]

Learning. The registries in vascular surgery have been instrumental in promoting and monitoring adherence to clinical practice guidelines developed through SVS research and expert consensus.^[64,65] VQI provides opportunities for real-time monitoring of clinical practice against societal guidelines, using the data to identify needs for either education or change in practice. These activities exemplify the human capital that SCIONs have produced. Ongoing QI efforts have led to documented improvements in patient outcomes.^[66] Clinical entities have successfully used the VQI/VISION CRN to monitor alignment with new clinical guidelines.^[64,65] For instance, the SVS guidelines on Peripheral Vascular Intervention for claudication were released in 2015 and included optimal medical management and procedural and post-procedure recommendations. A study of 93,654 cases of claudication treatment from 2010 to 2022 documented the uptake of guidelines and changes to clinical practice.^[67] Identification of groups of patients who did not receive guideline-aligned treatment, particularly those living in low-social and economic neighborhoods, will be a subject for future work using VQI data.

Innovation. Innovation in medical products and procedures is well documented through the work of the VQI/VISION CRN.

The VQI/VISION CRN facilitated improvements in care delivery and the monitoring of new products. The VQI has documented reductions in the length of hospital stay after endovascular aneurysm repair and contrast volume usage during endovascular aneurysm repair.^[66] Other innovations generated medium-term outcomes data on peripheral vascular interventions and smoking cessation strategies.^[66] VQI/VISION data contributed to six label changes that expanded the use of vascular product labels through post-approval studies and provided comparative data for propensity matching with existing device data.^[68–69] Over the course of the past decade, evidence-based medicine has advanced in vascular surgery through the VQI/VISION.^[70] Randomized clinical trials that document effective interventions have been developed in the network, and evidence-based interventions have been spread across the nation through the professional society and its many academic efforts and professional activities. Participation in the VQI registry has grown strongly, providing benchmarking and feedback to clinicians that further improved quality and outcomes. The publication of clinical practice guidelines by the SVS, while lagging behind much of the implementation of Evidence-based Medicine (EBM) into practice, has further solidified a culture of quality in vascular surgery.^[71] Further evaluation is necessary to better understand the role of the VQI in the decline in peripheral arterial disease patient deaths (over 50% from the year 2000 to 2016) that has been attributed to medical care.^[72]

Adaptation. The evolution of VQI/VISION CRN began in 2015, reaching full maturity by integrating longitudinal claims data into the VQI registry, enhancing utility and efficiency. VQI recognized the importance of follow-up of vascular procedures and began with a one-year follow-up. The ensuing 5–10 years of follow-up of registry data with traditional methods are expensive and often incomplete due to patients frequently changing providers, insurance, and residence. The SVS VQI/VISION^[73–75] efficiently linked the patient data in VQI with Medicare claims, creating unique datasets for developing clinical details based on the availability of long-term patient outcomes. Using VISION, VQI patient follow-up can be obtained for any Medicare patient who receives care from any provider at any location (given it is a billable event).

Key Messages

1) Coordinated Registry Networks (CRNs) are networks of hospitals, doctors, researchers, and others who share patient data to find better treatments and improve care. The criteria and functions used to define SCIONs accurately describe the CRN. 2) The Vascular Quality Initiative (VQI) serves as a case study to illustrate how a CRN helped to improve surgical practices, provide better treatments, and track the safety of devices, e.g., stents. 3) A CRN requires an independent Network-Coordinating Organization (NCO) with dedicated and skilled staff to balance stakeholders’ needs and optimize network participants’ contributions. 4) CRNs unite different groups around a common goal through coordination mechanisms, critically including cooperation. In the VQI, each regional group works on local issues while sharing ideas and solutions with the entire organization. This contrasts with market-driven industry and the rule-driven structures of government bodies.