MétaCan
Menu
Back to cohort
Record W4255224140 · doi:10.1080/17508487.2004.9525886

PhD Theses at the Margin: Examiner Comment on Re-examined Theses

2004· article· en· W4255224140 on OpenAlex
Allyson Holbrook, Sid Bourke, Kerry Dally

Why this work is in the frame

A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.

aboutThe title or abstract carries a Canadian signal from the geographic lexicon.
no affNo Canadian affiliation: this work is invisible to an affiliation-only frame.
No Canadian affiliation. An affiliation-only frame, the usual design, would never have seen this work. It is one of the works that make the case for inverting the frame.

Bibliographic record

VenueCritical Studies in Education · 2004
Typearticle
Languageen
FieldDecision Sciences
Topicscientometrics and bibliometrics research
Canadian institutionsnot available
Fundersnot available
KeywordsMargin (machine learning)SociologyEpistemologyPhilosophyComputer science

Abstract

fetched live from OpenAlex

It is rare for a PhD candidate rvho submits a rhesis for cxemin:rrion to fail outright..lf a thesis exhibits significar.rtflaws thc candidate may bc rcquited to make major rcvisions and rc- submit the work io, r..-.xrn-ti.ation'The writtcn comments of examincrs before and after rcsubmission can provide important insights into the Process of examination and the tlualities .*"fii,.r.r, identilyin a marginal thesis' Drawing on 101 of the most rcccnt, completccl thcses "cross fields in one Australian university, this ariicle investigates the differencc's in examiner ..r--"n,.on the qualities of theset by the same candidatcs before 'and after m"lor revision and rc-submission (N = 6), and benveen these thcses and those that n'ere 'passed' at the first exatnination (N=95).Crirical comments about the literaturc revieu' and the j"gr..',o which the examincr moved into a supervisory role *."r.fo.r.td to be strong indicators of theses'at the margin" Since the 1980s there has been a growing interest in the'visibiliry'of doctorai processes, particularly with respect to supervision, but more recenrly *i,h ,.rp"., to examination.Questions are being asked that ..r.o-p"r, ..".rg. of issues from examiner selection through to the rigour an,1 cr.difility of assessment procedures (Lawson et al' 2003, pJwell and Green 2003).Many commenrarors have pointed out that doctoral examination, and doctoral study generally, is an exceedingly complex phenomenon that has yet-to be subiected to sustained and ,yrt.-"ti.research.How students achieve success, the role the supervisor plays in getting a candidate's thesis to submission stage' or through an or"i d"f."n.", ""rrd *h", consrirures quality in postgraduate research are ail areas that are receiving attention in the field of research training in higher education.Alltlsott Holbrook cl nl.There are ferv empirical studies addressing the written examination of docroral theses or dissertations in rhe Iiteratu|e.As Morley et al. (.2002) have indicated, studies of the assessment Process atld its consistency rencl ro be rare because :rccess to exatninatiot.td.lcumentation is difficult' In addition, many universiries do nor call for extensive documentation of process.Jackson and Tinklcr (200 l) investigated examination pro..dur., a-nd student and staff resPonses to examination in the UK Th.1','1-'1"1ned documentation from 20 universities (b1sed on a stfatifieci sample of old and neu, insritutions) ancl clrerv on quesrio^nnaire.re.spo-nses fro,-,r r,,,t" 100 examiners ancl candidatcs fronr two of the'old'.With fespccr ro rhe viva (oral cxanrination) rhel' Founcl rhere w:rs 'no consensus "b.r.rr tlr"'roles'plaved by the viv:r and there rvere itrconsistencies ancl contradictions at the levels of policy and Practice (p.364) ln Atrstralia ,r compulsory oral examination is n()t the norm, rather examinltior-r hingcs on the written cxaminer rePorts on the thesis.In an attempt to .*pl.,r.this process lvlullins ancl Kilel' Q002) collectetl inr"rvi.*cl:rta-fr-ont 30 experienccd examiners about examination in Austrllia.Johnston (1997) ,-rndertc,ok a conte nt anlly.sis of t[e text of 5I exanrincr rep()rts flc,nt one Australian r-rniversity across five fucultiet over so'eral years.Pitkethl,v and Prosser (1995) utilised the reports of 74 rhesis candidares ar one Australian university.The findings include general agreement among examiners and abt)r'rt the core exPect:r'tionsJ i.,"-.lvrhat th.y expecr the thesis will derrionstrate oriqinaliry and makc a cont;ibution to the field.I'.''idenc,e From a colnParative cross-national survcy by Kouptsov (1994) further bears rlut general widespreacl agreenlenr on thls poinr.However, some polarisation occurs arour-rd the issue .,frvh,rt in -,ri" important -the cor.rtribution, or the training (Powell and Green 2003).Johnston (1997) found examiners tended to follow universitl' guidelines or recommendations about how to rePort on a thesil, whereas Mullin, and Kile1, Q0o2) reporrecl the opposite on rhe basis of interview data.Tl-rey fbund examiners had established their orvn criteria, and that thev noted, but did not use, guidelines providecl.It would seem rhar those rvho examine are inhe rently intercsred in doing so and approach tlie task in a positive light (Johnston 1997, Jackson "nd J'inkler 2001, Mullins and Kiley 2002).However, a poorll'rvritten thesis gencrally had a negative effect on the examit.rer (Johnston i997' Mulliris and Kiley 2002).A panel of 67 scholars from the USA, UK' Australia and Canada iclentified rvriting qualiry as one oF the most problerlatic issucs about PhD studr.(Noble 1994).N4ost researchers in rhe field have pointecl out th:lt editorial errors and Presentrtion issrtes attract a slLbsrantial proportion of examiner colnment.PhD Theses ot the Mnrgitt Mullins and Kiley (2002) nored that examiners appeared very clear in the clistinctions they made between Poor, acccptable.andoutstanding rheses, bur they also dete cted that examiners approache.dthe exami,nation process anticiparing rhar stuclcnts ruor-rld pass.It has been remarked by "*"-in.r.,f,"i th"y-rrr.lyfail a rhesis outright (Mullins and Krley 2002, Grabbe 2003), h,r*.t"i,thev mar.suggesr r.ajor revisions and re- examination.Becher's (1993 p. 134) similar comment re-infbrces this view.An exceprion is Johnstor.r's (1997) studt'comprising all i6 theses rhat hacl been examined in a'newef universiry" o[u,hich six theses were required ro be re-examined.Despire the latte r sn.rall stuclv, the expectation in iustr"lian universities is that a candidate will not Present their thesis for examin:rtion unless ir is ready (i.e. of pass sranclard).Reports on those theses rhat clo require re-examination Provide a rare and important opportunity to identiFy' the qualities that identil,v, and are used to .rrb itrat e,' rr'adi ncss' fo r .xarr r i nariotr.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

Full frame distilled prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.

metaresearch head score (Codex)0.008
metaresearch head score (Gemma)0.136
Version: codex-gemma-dda1882f352aValidation status: machine_predicted_unvalidated
Candidate categoriesMetaresearch, Bibliometrics
Consensus categoriesnone
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: none
GenreCandidate signal: Empirical · Consensus signal: Empirical
Teacher disagreement score0.356
Threshold uncertainty score0.990

Codex and Gemma teacher scores by category

CategoryCodexGemma
Metaresearch0.0080.136
Meta-epidemiology (narrow)0.0000.000
Meta-epidemiology (broad)0.0000.000
Bibliometrics0.0060.030
Science and technology studies0.0010.001
Scholarly communication0.0010.000
Open science0.0010.001
Research integrity0.0000.000
Insufficient payload (model declined to judge)0.0000.000

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.793
GPT teacher head0.650
Teacher spread0.143 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it