Assessing the Effectiveness of AI Tools (Elicit, SciSpace, and Consensus) in Literature Review and Research
Notice bibliographique
Résumé
The authors share the sentiments of other researchers that conducting a literature review and creating a matrix is a cumbersome task. The development of research tools that benefit researchers in their study conduct is always interesting and welcome. Thus, the objectives of this study are: (1) To evaluate the features and functionalities of Elicit, SciSpace, and Consensus in facilitating literature review and research and recommend the applicability of each tool for a type of research; (2) Assess and identify the strengths of these tools and align the impact with research workflows and efficiency; and (3) Utilize documented user feedback and opinions to ensure the appropriate AI tool is available for the researchers. This study adopts a mixed-methods approach. Firstly, a systematic literature review is conducted to gather relevant studies, user feedback, and expert opinions on the AI tools. These same studies will be used as the sample variables for this study. Secondly, a comparative evaluation of each AI tool against the original document and each other is conducted to assess each tool against the established evaluation criteria, which include search capabilities, document retrieval, summarization accuracy, citation analysis, and integration with existing research workflows. Lastly, the strengths and weaknesses of each tool are identified in relation to the criteria, and the effectiveness of the AI tool is determined based on the original content of the sample material. The findings of this study include (a) identified AI tools, such as Elicit, SciSpace, and Consensus, which offer valuable contributions to the research community through productivity-boosting features. (b) Each tool has strengths and weaknesses in search capabilities, document retrieval, summarization accuracy, citation analysis, and integration with existing research workflows; and (c) Documented user feedback indicates positive experiences with the usability and effectiveness of the tools, highlighting their potential to enhance research workflows. This study acknowledges potential limitations, including the reliance on user feedback and the subjective nature of user experiences. The evaluation is based on a specific set of criteria, and the results may vary depending on individual research needs and preferences. This study offers practical implications for researchers, students, and professionals seeking efficient and effective tools for conducting literature reviews. Elicit, SciSpace, and Consensus offer insights into their strengths, weaknesses, and potential applications. The findings contribute to informed decision-making regarding the adoption and utilization of these AI tools in research. This study examines AI tools designed explicitly for conducting literature reviews, distinguishing itself from existing research that predominantly emphasizes the application of AI in academic research settings. It offers an analysis of how various AI features can enhance the literature review process, thereby contributing a unique perspective to the ongoing discourse on the integration of AI in research methodologies.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction distillée sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Apprise à partir de 10 348 étiquettes directes de Codex et de 10 348 étiquettes directes de Gemma. Le mode candidate est l'union des têtes enseignantes seuillées; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont ni des étiquettes humaines ni des étiquettes directes de modèles de pointe.
Scores Codex et Gemma par catégorie
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,110 | 0,027 |
| Méta-épidémiologie (sens strict) | 0,000 | 0,000 |
| Méta-épidémiologie (sens large) | 0,001 | 0,000 |
| Bibliométrie | 0,002 | 0,004 |
| Études des sciences et des technologies | 0,000 | 0,001 |
| Communication savante | 0,004 | 0,006 |
| Science ouverte | 0,001 | 0,000 |
| Intégrité de la recherche | 0,000 | 0,000 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,000 | 0,000 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; les deux têtes enseignantes s’accordent sur ce qui est montré ici.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».