Matches in SemOpenAlex for { <https://semopenalex.org/work/W4387294150> ?p ?o ?g. }
Showing items 1 to 53 of
53
with 100 items per page.
- W4387294150 abstract "Large Language Models (LLMs) are advancing at a rapid pace, with significant improvements at natural language processing and coding tasks. Yet, their ability to work with formal languages representing data, specifically within the realm of knowledge graph engineering, remains under-investigated. To evaluate the proficiency of various LLMs, we created a set of five tasks that probe their ability to parse, understand, analyze, and create knowledge graphs serialized in Turtle syntax. These tasks, each embodying distinct degrees of complexity and being able to scale with the size of the problem, have been integrated into our automated evaluation system, the LLM-KG-Bench. The evaluation encompassed four commercially available LLMs - GPT-3.5, GPT-4, Claude 1.3, and Claude 2.0, as well as two freely accessible offline models, GPT4All Vicuna and GPT4All Falcon 13B. This analysis offers an in-depth understanding of the strengths and shortcomings of LLMs in relation to their application within RDF knowledge graph engineering workflows utilizing Turtle representation. While our findings show that the latest commercial models outperform their forerunners in terms of proficiency with the Turtle language, they also reveal an apparent weakness. These models fall short when it comes to adhering strictly to the output formatting constraints, a crucial requirement in this context." @default.
- W4387294150 created "2023-10-03" @default.
- W4387294150 creator A5024616757 @default.
- W4387294150 creator A5053761025 @default.
- W4387294150 creator A5072609503 @default.
- W4387294150 creator A5092474640 @default.
- W4387294150 creator A5092732638 @default.
- W4387294150 date "2023-09-29" @default.
- W4387294150 modified "2023-10-04" @default.
- W4387294150 title "Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?" @default.
- W4387294150 doi "https://doi.org/10.48550/arxiv.2309.17122" @default.
- W4387294150 hasPublicationYear "2023" @default.
- W4387294150 type Work @default.
- W4387294150 citedByCount "0" @default.
- W4387294150 crossrefType "posted-content" @default.
- W4387294150 hasAuthorship W4387294150A5024616757 @default.
- W4387294150 hasAuthorship W4387294150A5053761025 @default.
- W4387294150 hasAuthorship W4387294150A5072609503 @default.
- W4387294150 hasAuthorship W4387294150A5092474640 @default.
- W4387294150 hasAuthorship W4387294150A5092732638 @default.
- W4387294150 hasBestOaLocation W43872941501 @default.
- W4387294150 hasConcept C111919701 @default.
- W4387294150 hasConcept C154945302 @default.
- W4387294150 hasConcept C199360897 @default.
- W4387294150 hasConcept C204321447 @default.
- W4387294150 hasConcept C2522767166 @default.
- W4387294150 hasConcept C41008148 @default.
- W4387294150 hasConcept C511192102 @default.
- W4387294150 hasConcept C88006597 @default.
- W4387294150 hasConceptScore W4387294150C111919701 @default.
- W4387294150 hasConceptScore W4387294150C154945302 @default.
- W4387294150 hasConceptScore W4387294150C199360897 @default.
- W4387294150 hasConceptScore W4387294150C204321447 @default.
- W4387294150 hasConceptScore W4387294150C2522767166 @default.
- W4387294150 hasConceptScore W4387294150C41008148 @default.
- W4387294150 hasConceptScore W4387294150C511192102 @default.
- W4387294150 hasConceptScore W4387294150C88006597 @default.
- W4387294150 hasLocation W43872941501 @default.
- W4387294150 hasOpenAccess W4387294150 @default.
- W4387294150 hasPrimaryLocation W43872941501 @default.
- W4387294150 hasRelatedWork W1569841287 @default.
- W4387294150 hasRelatedWork W2293457016 @default.
- W4387294150 hasRelatedWork W2351428524 @default.
- W4387294150 hasRelatedWork W2369308426 @default.
- W4387294150 hasRelatedWork W2789919619 @default.
- W4387294150 hasRelatedWork W3169305685 @default.
- W4387294150 hasRelatedWork W4234402940 @default.
- W4387294150 hasRelatedWork W4318617392 @default.
- W4387294150 hasRelatedWork W1551406738 @default.
- W4387294150 hasRelatedWork W2610387714 @default.
- W4387294150 isParatext "false" @default.
- W4387294150 isRetracted "false" @default.
- W4387294150 workType "article" @default.