Can global co‐operation enhance quality in medical education? Some lessons from an international assessment consortium

Globalism has been one of the undoubted hallmarks of the late 20th and early 21st centuries. E-mail, the internet and improved telecommunications have brought us all closer together. Medical education has not been immune from such developments. Groups such as the World Federation for Medical Education (WFME), the Institute for International Medical Education (IIME) and the International Virtual Medical School (IVIMEDS) have actively promoted the concept of global standards and co-operation in medical education.1–3 It is important, however, that medical education avoids some of the worst excesses of economic globalization. In the High Streets/Main Streets of cities, towns and villages all over the world we find the same shops, with the same logos selling the same uniform products. While this may provide some comfort for the tourist abroad, absolute uniformity is not desirable in medical education. Our challenge is to recognize those educational outcomes that should be the same all over the world, and those that need to vary in response to local need. It has been argued that contemporary curricula should be built upon a 'symbiosis' with health services where education enhances clinical practice and clinical practice enhances education.4 International standards must be capable of being applied in local contexts. Nowhere is this more important than in establishing international standards in assessment, given the well-recognized propensity of assessment to 'drive student learning'. The construction of valid and reliable instruments to test students' progress remains a time-consuming challenge in time-poor medical schools. Given the importance of assessing students for generic and specific knowledge and skills, globalization in medical education presents an opportunity to develop and maintain a bank of high quality test items, using a variety of instruments and without reinventing the assessment wheel. These issues provided the backdrop for the establishment of an international consortium for the development of such an assessment bank. The consortium was formally established at a meeting in Hong Kong in December 2001, driven by the energy and commitment of Professor Clarke Hazlett of the Chinese University of Hong Kong (CUHK). Current consortium members include the CUHK, the University of Hong Kong, the Universities of Alberta and British Columbia in Canada, Flinders University and the University of Sydney in Australia, the University of Otago in New Zealand and the University of Witswatersrand in South Africa. Establishing a consortium of this nature is not an easy task, especially if its focus is to go beyond considerations of 'High Street uniformity' to address quality standards in assessment that can be applied in local contexts. This consortium had a head start in the form of a sophisticated database developed through the work of Professor Hazlett at the University of Alberta and Chinese University of Hong Kong. Funds for the establishment of the consortium were obtained through a joint Teaching Development Grant awarded to the two Hong Kong Universities in the consortium. While this provided a firm foundation for the consortium, its subsequent development and ultimate success will depend on the further actions of consortium members. These will be guided by a set of principles established at the initial meeting in December. The first principle is that the members of the consortium volunteer for membership. Medical schools registered with their national authorities are eligible to apply for membership. At this stage the bank contains items written in English only. Potential members must also demonstrate commitment to the work of the consortium through signing a Memorandum of Agreement, making a financial contribution to running costs and agreeing to contribute and review a set number of items per year. The financial contribution will decrease as more members join. The second principle is that decision-making about the workings of the consortium will be participatory. For the initial discussions Deans' representatives worked on policy issues while item bank administrators met to consider implementation. The former group was chaired by an expert facilitator who was not a member of the consortium, to ensure that all members could participate fully in discussions. Respecting local issues is also important. There was considerable discussion in the initial meeting about security of items in the bank. Some universities emphasise test security by withholding specific feedback on students' performance in each item of an assessment, while other universities operate on the principle that all assessments should have a formative component. The latter allow students to review their papers under supervision. In the end a decision was made to designate parts of the bank as 'open' or 'closed' with the latter requiring high levels of security. Items from the open part of the bank will be available for formative assessment or in universities where feedback on items is provided. Following on from this was the commitment to create a large multiformat bank. This would provide for effective choice of items by the participating universities. This highlighted the need for a comprehensive and inclusive classification system which would address the needs of member universities in responding to their test blueprints, searching the bank for appropriate questions and devising rigorous assessments. The last, and most important, principle was that member universities would need to share a vision about the fundamental purposes of the item bank. Clearly, there was a commitment by all members to produce quality items for assessment. While quantity is important for effective choice it was agreed that it should not be achieved at the expense of quality. All items to be entered into the bank are to be subjected to trial and quality review preferably involving the calculation of relevant item statistics to indicate the validity and reliability of the test items. The members of the consortium endorsed the use of appropriate formats for MCQ items and undertook to contribute across the range of credible item formats for assessment. It is this vision for quality items and quality formats that binds the members together. However, while these principles have been established, they are as yet untested. Furthermore, there is not a great deal written about the characteristics of successful and unsuccessful consortia.5 A review of successful educational consortia in the Department of Health Resources and Services Administration under the auspices of the United States Department of Health and Human Services emphasised the key importance of benefit to every consortium member.6 This needs to be carefully considered by existing and new members. The establishment of the principles does point to the need for careful thought and critique of global and international co-operation proposals. International alliances can save time, money and effort but, at the same time, their outputs need to be carefully designed so that medical educators can apply them to the needs of their programmes and the health contexts in which they operate. Important principles for cooperation include participatory decision-making, local sensitivity, tangible benefits for all members and shared vision The authors gratefully acknowledge the work of Professor Clarke Hazlett of the Chinese University of History for the inspiration of this project and the development of the data base. Acknowledgement is also given to Professor Hazlett and Dr John Nicholls of the Hong Kong University for obtaining grant funds from the University Grants Committee of Hong Kong.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.