MétaCan
Menu
Back to cohort
Record W240041370

High-Performance Research Organizations: Here Are Ten Attributes That Help Managements Do the Right Things to Turn Their Visions into Reality

2001· article· en· W240041370 on OpenAlexaboutno aff
George A. Neufeld, Peter A. Simeoni, Marilyn Taylor

Bibliographic record

VenueResearch-Technology Management · 2001
Typearticle
Languageen
FieldDecision Sciences
TopicResearch, Science, and Academia
Canadian institutionsnot available
Fundersnot available
KeywordsBest practiceSet (abstract data type)VisionAuditBusinessQuality (philosophy)Government (linguistics)Work (physics)Perspective (graphical)Public relationsKnowledge managementProcess managementComputer sciencePolitical scienceEngineeringSociologyAccounting
DOInot available

Abstract

fetched live from OpenAlex

This article has two messages. First, it is both useful and possible to develop a set of performance ideals or attributes for a notoriously difficult area to manage--research organizations. Second, it would be both useful and practical to apply the same approach to developing attributes for other types of organizations, such as software firms, manufacturers, insurance companies, government agencies, and universities. The concept of attributes comes from the question, How can you tell if an organization is well-managed? In other words, is there a set of performance ideals that could be used to assess the quality of management of any organization? To be most useful, these attributes would have to be end-results-oriented questions that go beyond the development and implementation of best practices. From a CEO's perspective, best practices are means to the end in mind. So, our initial question leads to several others: What are the intended results of the best practices? What should Boards and CEOs be looking for to be sure that best practices are in fact moving the organization toward the intended results? Can these results be stated in terms that are useful, observable and, preferably, measurable? The development of attributes of high-performance research organizations serves several purposes: * Over a six-year period, the Office of the Auditor General of Canada undertook work that pointed to the need for a description of what a well-managed research organization looks like. We used the guidance provided in the federal government's and Technology Strategy and Framework for the Human Resources Management of the Federal Science and Technology Community, and other sources to create a set of ideal outcomes of research management. We call these ideal outcomes attributes. The extent to which an attribute is demonstrated by an organization is an indication of the quality of management. * Companies are increasingly dependent upon the results from research for new and improved products in order to maintain competitive advantage. Governments view industry-driven science and technology as economic engines, and are increasingly dependent upon their own science and technology program for dealing with public policy issues such as climate change and the impacts of toxic substances. Furthermore, governments are placing more emphasis on achieving results, e.g., Results for Canadians, an initiative of the federal government, and the Government Performance and Results Act in the United States. * Assessing the performance of, and return on investment (ROI) from, research is a challenge faced by private and public sector executives as well as politicians. Research is a risky activity; not all research activity leads to expected results. Furthermore, the benefits from research sometimes take years to materialize. * The information available to Boards and CEOs for assessing the performance of research organizations is inadequate. The information tends to focus on past performance (e.g., published papers and patents) and on process and activities. However, high past performance does not guarantee the same in the future; today's executives need better and more current information. Furthermore, performance assessments that focus on process beg the question: What has happened as a result of having implemented good practices? Approaches to assessing the performance and ROI of research organizations generally fall into three categories: (1) Retrospective evaluation (examining the relevance and impact of research completed in the past); (2) Current evaluation (examining the organization's vision, strategies, target clients, practices, and people); and (3) Future evaluation (examining planned research, its relevance, potential benefits, and likelihood of success). Methodologies are most advanced for retrospective evaluations, and tend to involve costly studies that are conducted by third parties. …

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame machine prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.

metaresearch head score (Codex)0.059
metaresearch head score (Gemma)0.054
Version: metacan-v3-hybrid-931329e0061cValidation status: machine_predicted_unvalidated
Candidate categoriesMetaresearch
Consensus categoriesnone
DomainCandidate signal: Incentives · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: none
GenreCandidate signal: Empirical · Consensus signal: none
Teacher disagreement score0.941
Threshold uncertainty score0.311

Distilled classifier scores by category (both heads)

CategoryCodexGemma
Metaresearch0.0590.054
Meta-epidemiology (narrow)0.0010.001
Meta-epidemiology (broad)0.0010.001
Bibliometrics0.0040.005
Science and technology studies0.0110.023
Scholarly communication0.0400.038
Open science0.0020.014
Research integrity0.0060.011
Insufficient payload (model declined to judge)0.0060.004

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.157
GPT teacher head0.449
Teacher spread0.292 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.

Study designNot applicable
DomainIncentives
GenreEmpirical

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations13
Published2001
Admission routes1
Has abstractyes

Explore more

Same venueResearch-Technology ManagementSame topicResearch, Science, and AcademiaFrench-language works237,207