Defining KPIs to assess the impact of innovation in construction: Insights from Public Project Owners
Bibliographic record
Abstract
This thesis aims to define key performance indicators (KPIs) for evaluating the impact of innovations in the AECO industry. The work began by clarifying core concepts - innovation, performance measurement, and impact assessment - to build a shared vocabulary and an analytical baseline. With these foundations, an investigation of current practice was carried out with public project owners (PPOs) in Québec to understand what is measured, how it is measured, and why. A Design Science Research (DSR) approach was used. First, a literature review assembled 42 KPIs grouped into nine categories: Cost, Time, Productivity, Quality, Safety, Scope (changes/claims/RFIs), Risk, Sustainability, and Innovation. Second, semi-structured interviews with five Québec PPOs documented current KPI practice, data workflows, and difficulties. Third, prioritization workshops with the same four PPOs plus one additional owner positioned the 42 KPIs on a usefulness–measurability matrix. Finally, a post-workshop validation questionnaire (seven responses from six PPOs) confirmed priorities for implementation. Findings show a narrow core in current dashboards - cost and time - together with implementation barriers: data-quality issues, fragmented systems, and cultural factors affecting adoption. At the same time, owners expressed a clear interest in moving beyond the “iron triangle” toward scope discipline, risk management, productivity, quality and stakeholder outcomes, safety, sustainability, and innovation. Workshop inputs were ordered by combining the average scores for usefulness and ease of measurement with agreement weights based on the interquartile range (IQR). Usefulness was emphasized (80%), while ease of measurement contributed (20%) to reflect feasibility without letting it dominate. This procedure produced an owner-endorsed shortlist of 17 KPIs that balances decision value, consensus, and practicality. The prioritized set comprises the following metrics: cost predictability (design/construction), cost per unit, time predictability (design/construction), time per unit, labor productivity expressed as dollars per unit, stakeholder satisfaction, lost-time incidents, change order (cost) – client-initiated, change order (cost) – contractor-initiated, number of change orders – clientinitiated, number of change orders – contractor-initiated, requests for information (RFI), waste generation, carbon footprint, energy consumption, scope of risks (risk-exposure score), and cost of risk mitigation. Together, these cover the main control needs of public owners while opening space for sustainability and risk-informed decisions as data pipelines mature. The main contribution is an owner-validated KPI set spanning nine categories and a replicable prioritization procedure that integrates perceived value, stakeholder agreement, and measurability. Practically, owners can begin routine reporting with the validated shortlist, while planning staged adoption of sustainability and future innovation-specific indicators as data governance improves. Limitations include a small, Québec-based sample and a crosssectional snapshot of the current practices of public project owners. Future work will pilot the prioritized KPIs on live projects, strengthen data governance and integration, and run longitudinal studies linking project-phase indicators to post-handover outcomes, enabling a more complete evaluation of innovation impacts across the asset life cycle. With these results, the impact of innovation can be identified across a wider set of dimensions than time and cost, and assessed beyond immediate outputs to include longer-term outcomes.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.027 | 0.044 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.004 | 0.005 |
| Science and technology studies | 0.004 | 0.004 |
| Scholarly communication | 0.010 | 0.006 |
| Open science | 0.002 | 0.005 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".