MétaCan
Menu
Back to cohort
Record W4377230410 · doi:10.2118/0423-0008-jpt

Comments: AI Language Tools Hit the Books . . . and Technical Content?

2023· article· en· W4377230410 on OpenAlexaboutno aff
Pam Boschee

Bibliographic record

VenueJournal of Petroleum Technology · 2023
Typearticle
Languageen
FieldMedicine
TopicArtificial Intelligence in Healthcare and Education
Canadian institutionsnot available
Fundersnot available
KeywordsChatbotWorld Wide WebNoticeComputer scienceUnicodeEngineeringArtificial intelligencePolitical science

Abstract

fetched live from OpenAlex

Are you among the millions of ChatGPT users/experimenters around the world? Or impatiently parked on the waitlist? OpenAI launched ChatGPT in late November 2022 with little fanfare. Within a week, the company reported 1 million users. Within the first month, it exceeded 57 million users. Since then, it has been updated several times and a UBS analyst estimated in February that the artificial intelligence (AI) language tool reached 100 million users. The website exceeded 13 million daily visitors as of January. On 1 February, OpenAI announced a ChatGPT Plus monthly subscription plan to give priority access to the chatbot. Clearly, AI language tools are fulfilling the needs and/or satisfying the curiosity of novices and more experienced AI users. Microsoft took notice and in January confirmed the extension of its partnership with OpenAI, an investment rumored to be $10 billion (not confirmed by Microsoft). Microsoft Azure will also continue as the exclusive cloud provider for the tool since OpenAI uses Azure to train its models. ChatGPT (generative pretrained transformer) is not the only AI tool available. Microsoft upgraded its Bing AI search engine, which is powered by an upgraded model of ChatGPT. ChatSonic, also built on top of ChatGPT, can access the internet. Jasper Chat, based on GPT 3.5 with OpenAI as its partner, was built for advertising and marketing businesses. Google Bard is an experimental conversational AI service powered by Google’s own next-generation language model. Character AI revolves around the concept of personas, trained with conversations in mind. Users choose from various personalities (e.g., Elon Musk, Tony Stark, Socrates, US President Biden, Kayne West, etc.) vs. interacting with a single AI chatbot. YouChat is built into a search engine and trained on a ChatGPT model. It holds conversations with full access to the internet. Caktus AI, not available as a free service, is aimed toward students and touted by the company as “the first-ever educational AI tool.” It helps with student content ranging from essays to writing paragraphs and extends to discussions, questions, and coding. There’s also a bot for programmers, GitHub Copilot X, which suggests and completes code and functions in real time. There are many more options and sure to be plenty more developed with specific user types in mind (e.g., technical specialties, business communications, conversational, special interests). All will have their own pros and cons. A retired systems engineer, formerly with General Dynamics Canada, Rob Miller, wrote a column in Canada’s National Observer about his experience using ChatGPT as a research tool for a story about carbon capture and storage—he gave a mixed review. As AI tools gain attention and provide assistance to a wide range of authors and researchers, SPE recently announced its policy for authors who use the tools to generate content for their papers. AI-generated content may be used within SPE publications, but under specific conditions. Open-source libraries, such as those used in ChatGPT and others, are not fail-safe solutions, as evidenced on 20 March when OpenAI took the chatbot offline due to a bug “which allowed some users to see titles from another active user’s chat history,” according to the company. “It’s also possible that the first message of a newly created conversation was visible in someone else’s chat history if both users were active around the same time.” OpenAI also discovered that “the same bug may have caused the unintentional visibility of payment-related information of 1.2% of the ChatGPT Plus subscribers who were active during a specific 9-hour window. … it was possible for some users to see another active user’s first and last name, email address, payment address, the last four digits (only) of a credit card number, and credit card expiration date.” I’d like to hear about your experience with ChatGPT or any of the other options. How did you use it? What type of content were you generating? If you were aiming for assistance with technical content related to oil/gas/energy, how useful was the chatbot? Share your comments and pros and cons at JPT Comments.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame machine prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.

metaresearch head score (Codex)0.003
metaresearch head score (Gemma)0.022
Version: metacan-v3-hybrid-931329e0061cValidation status: machine_predicted_unvalidated
Candidate categoriesMetaresearch
Consensus categoriesnone
DomainCandidate signal: Methods · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: Not applicable
GenreCandidate signal: Commentary · Consensus signal: Commentary
Teacher disagreement score0.997
Threshold uncertainty score0.501

Distilled classifier scores by category (both heads)

CategoryCodexGemma
Metaresearch0.0030.022
Meta-epidemiology (narrow)0.0010.000
Meta-epidemiology (broad)0.0010.001
Bibliometrics0.0010.001
Science and technology studies0.0040.001
Scholarly communication0.0030.005
Open science0.0010.002
Research integrity0.0080.005
Insufficient payload (model declined to judge)0.1500.067

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.138
GPT teacher head0.416
Teacher spread0.277 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.

Study designNot applicable
DomainMethods
GenreCommentary

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations10
Published2023
Admission routes1
Has abstractyes

Explore more

Same venueJournal of Petroleum TechnologySame topicArtificial Intelligence in Healthcare and EducationFrench-language works237,207