FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W4322208105> ?p ?o ?g. }

• W4322208105 abstract "Summary Background The global launch of ChatGPT on November 30, 2022 has sparked widespread public interest in large language models (LLMs), and interest in the medical community is growing. Indeed, recent preprints on medRxiv have examined ChatGPT and GPT-3 in the context of standardized exams, such as the United States Medical Licensing Examination. These studies demonstrate modest performance relative to national averages. In this work, we enhance OpenAI’s GPT-3 model through zero-shot learning, anticipating that it outperforms experienced neurosurgeons in written question-answer tasks for common clinical and surgical questions on vestibular schwannoma. We aimed to address LLM accountability by including in-text citations and references to the responses provided by GPT-3. Methods The analysis involved (i) creating a dataset through web scraping, (ii) developing a chat-based platform called neuroGPT-X, (iii) enlisting expert neurosurgeons across international centers to create and answer questions and evaluate responses, and (iv) analyzing the evaluation results on the management of vestibular schwannoma. The survey had a blinded and unblinded phase. In the blinded phase, a neurosurgeon with 30+ years of experience curated 15 questions regarding common clinical and surgical contexts of vestibular schwannoma. Then, four neurosurgeons, ChatGPT (January 30, 2023 model, aka naive GPT), and a context-enriched GPT model independently provided their responses. Three experienced neurosurgeons blindly evaluated the responses for accuracy, coherence, relevance, thoroughness, speed, and overall rating. Then, all seven neurosurgeons were unblinded to all responses and provided their thoughts on the potential of expert LLMs in the clinical setting. Findings Both the naive and content-enriched GPT models provided faster responses to the standardized question set (p<0.01) than expert neurosurgeon respondents. Moreover, responses from both models were consistently non-inferior in accuracy, coherence, relevance, thoroughness, and overall performance, and were often rated higher than expert responses. Importantly, context enrichment of GPT with relevant scientific literature did not significantly affect speed (p>0.999) or performance across the aforementioned domains (p>0.999). Of interest, all expert surgeons expressed concerns about the reliability of GPT in accurately addressing the nuances and controversies surrounding the management of vestibular schwannoma. Further, we developed neuroGPT-X, a chat-based platform designed to provide point-of-care clinical support and mitigate limitations of human memory. neuroGPT-X incorporates features such as in-text citations and references to enable accurate, relevant, and reliable information in real-time. Interpretation A context-enriched GPT model provided non-inferior responses compared to experienced neurosurgeons in generating written responses to a complex neurosurgical problem for which evidence-based consensus for management is lacking. We show that context enrichment of LLMs is well-suited to transform clinical practice by providing subspecialty-level answers to clinical questions in an accountable manner. Research in Context Evidence before this study We searched PubMed for “(vestibular schwannoma OR acoustic schwannoma) AND (GPT-3 OR Generative Pretrained Transformer OR large language model)” with no filters and identified no relevant articles. We then searched PubMed using the string “(subspecialty OR neurosurgery OR physician) AND (GPT-3 OR Generative Pretrained Transformer OR large language model) AND (fine-tuning OR context enrichment)” with no filters and identified three studies. One study noted that domain-specific knowledge enhanced pre-trained language models. Added value of this study To our knowledge, this is the first study to show the non-inferiority of a context-enriched LLM in a question-answer task on common clinical and surgical questions compared to experienced neurosurgeons worldwide, determined by their neurosurgical colleagues. Furthermore, we developed the first online platform incorporating an LLM, chat memory, in-text citations, and references regarding comprehensive vestibular schwannoma management. To assess the model’s performance, a neurosurgeon with 30+ years of experience managing patients with vestibular schwannoma curated 15 questions to the model, ChatGPT, and four international expert neurosurgeons. A separate, blinded group of three expert neurosurgeons assessed these answers for accuracy, coherence, relevance, thoroughness, speed, and overall rating. This study demonstrated the capability of context-enriched LLMs as point-of-care informational aids. Importantly, all expert surgeons raised questions regarding the nuances and role of human experience and intuition that GPT may not capture in generating opinions or recommendations. Implications of all the available evidence The present study, with its subspecialist-level performance and interpretable results, suggests that context-enriched LLMs show promise as a point-of-care medical resource. Evaluations from experienced neurosurgeons showed that a context-enriched GPT model was rated similarly to neurosurgeon responses across evaluation domains in this study. This work serves as a springboard for expanding this tool into more medical specialties, incorporating evidence-based clinical information, and developing expert-level dialogue surrounding LLMs in healthcare." @default.
• W4322208105 created "2023-02-27" @default.
• W4322208105 creator A5006848706 @default.
• W4322208105 creator A5012587406 @default.
• W4322208105 creator A5031529877 @default.
• W4322208105 creator A5037506643 @default.
• W4322208105 creator A5044195004 @default.
• W4322208105 creator A5044363422 @default.
• W4322208105 creator A5048395808 @default.
• W4322208105 creator A5048624729 @default.
• W4322208105 creator A5055460485 @default.
• W4322208105 creator A5063516601 @default.
• W4322208105 creator A5076132435 @default.
• W4322208105 creator A5079713113 @default.
• W4322208105 creator A5083573815 @default.
• W4322208105 creator A5091828448 @default.
• W4322208105 date "2023-02-26" @default.
• W4322208105 modified "2023-10-16" @default.
• W4322208105 title "neuroGPT-X: Towards an Accountable Expert Opinion Tool for Vestibular Schwannoma" @default.
• W4322208105 cites W1599764279 @default.
• W4322208105 cites W1878893887 @default.
• W4322208105 cites W1994307831 @default.
• W4322208105 cites W2077434826 @default.
• W4322208105 cites W2123441258 @default.
• W4322208105 cites W2127797729 @default.
• W4322208105 cites W2134598164 @default.
• W4322208105 cites W2136778797 @default.
• W4322208105 cites W2461460038 @default.
• W4322208105 cites W2479527281 @default.
• W4322208105 cites W2564966959 @default.
• W4322208105 cites W2766894906 @default.
• W4322208105 cites W2809361269 @default.
• W4322208105 cites W3004612364 @default.
• W4322208105 cites W3037023615 @default.
• W4322208105 cites W3176098057 @default.
• W4322208105 cites W3200759624 @default.
• W4322208105 cites W4205574973 @default.
• W4322208105 cites W4226278401 @default.
• W4322208105 cites W4281557260 @default.
• W4322208105 cites W4295951577 @default.
• W4322208105 cites W4297253404 @default.
• W4322208105 cites W4309468977 @default.
• W4322208105 cites W4309674289 @default.
• W4322208105 cites W4313384047 @default.
• W4322208105 cites W4319662928 @default.
• W4322208105 doi "https://doi.org/10.1101/2023.02.25.23286117" @default.
• W4322208105 hasPublicationYear "2023" @default.
• W4322208105 type Work @default.
• W4322208105 citedByCount "4" @default.
• W4322208105 countsByYear W43222081052023 @default.
• W4322208105 crossrefType "posted-content" @default.
• W4322208105 hasAuthorship W4322208105A5006848706 @default.
• W4322208105 hasAuthorship W4322208105A5012587406 @default.
• W4322208105 hasAuthorship W4322208105A5031529877 @default.
• W4322208105 hasAuthorship W4322208105A5037506643 @default.
• W4322208105 hasAuthorship W4322208105A5044195004 @default.
• W4322208105 hasAuthorship W4322208105A5044363422 @default.
• W4322208105 hasAuthorship W4322208105A5048395808 @default.
• W4322208105 hasAuthorship W4322208105A5048624729 @default.
• W4322208105 hasAuthorship W4322208105A5055460485 @default.
• W4322208105 hasAuthorship W4322208105A5063516601 @default.
• W4322208105 hasAuthorship W4322208105A5076132435 @default.
• W4322208105 hasAuthorship W4322208105A5079713113 @default.
• W4322208105 hasAuthorship W4322208105A5083573815 @default.
• W4322208105 hasAuthorship W4322208105A5091828448 @default.
• W4322208105 hasBestOaLocation W43222081051 @default.
• W4322208105 hasConcept C15744967 @default.
• W4322208105 hasConcept C166957645 @default.
• W4322208105 hasConcept C205649164 @default.
• W4322208105 hasConcept C2779343474 @default.
• W4322208105 hasConcept C512399662 @default.
• W4322208105 hasConcept C71924100 @default.
• W4322208105 hasConceptScore W4322208105C15744967 @default.
• W4322208105 hasConceptScore W4322208105C166957645 @default.
• W4322208105 hasConceptScore W4322208105C205649164 @default.
• W4322208105 hasConceptScore W4322208105C2779343474 @default.
• W4322208105 hasConceptScore W4322208105C512399662 @default.
• W4322208105 hasConceptScore W4322208105C71924100 @default.
• W4322208105 hasLocation W43222081051 @default.
• W4322208105 hasOpenAccess W4322208105 @default.
• W4322208105 hasPrimaryLocation W43222081051 @default.
• W4322208105 hasRelatedWork W1506200166 @default.
• W4322208105 hasRelatedWork W1995515455 @default.
• W4322208105 hasRelatedWork W2048182022 @default.
• W4322208105 hasRelatedWork W2080531066 @default.
• W4322208105 hasRelatedWork W2604872355 @default.
• W4322208105 hasRelatedWork W2748952813 @default.
• W4322208105 hasRelatedWork W2899084033 @default.
• W4322208105 hasRelatedWork W3031052312 @default.
• W4322208105 hasRelatedWork W3032375762 @default.
• W4322208105 hasRelatedWork W3108674512 @default.
• W4322208105 isParatext "false" @default.
• W4322208105 isRetracted "false" @default.
• W4322208105 workType "article" @default.