Matches in SemOpenAlex for { <https://semopenalex.org/work/W2056576456> ?p ?o ?g. }
Showing items 1 to 85 of
85
with 100 items per page.
- W2056576456 endingPage "305" @default.
- W2056576456 startingPage "281" @default.
- W2056576456 abstract "The Message Understanding Conferences (MUCs) represent one of the earliest and longest running efforts to evaluate language understanding technology. This article reviews the history of the MUCs and their evolution towards the use of common training and blind test sets, automated scoring, task decomposition into modular building blocks and tools for portability across languages and applications. Now that evaluation has become an accepted part of the developer's toolkit, it is important to understand the interplay between evaluation methods and the state of research. MUC was successful in generating excitement about text processing problems and in attracting talented researchers to the area. It also provided a functional decomposition of the information extraction problem into a series of simpler problems, thus allowing researchers to demonstrate successful systems and to spin off commercial products. However, the ultimate goal of accurate information extraction has been elusive; systems have become faster and cheaper to build, the evaluations have become harder, but overall accuracy in information extraction has improved only modestly. The MUC experience contrasts with experiences in other evaluations. For example, the spoken evaluation in the Air Travel Information System (ATIS) has shown dramatic improvement in error rate over time, but those evaluations were limited to a single domain and the metrics did not address interaction, even though real-time interactive systems were available. Looking across the history of MUC in the context of related evaluations, we can draw important lessons about the need for evaluation to evolve with the technology it evaluates, to balance costs against benefits and to weigh the divergent needs of the multiple stake-holders— developers, funders and users—in order to provide continuity while also providing the next set of challenges to the research community." @default.
- W2056576456 created "2016-06-24" @default.
- W2056576456 creator A5068115280 @default.
- W2056576456 date "1998-10-01" @default.
- W2056576456 modified "2023-10-02" @default.
- W2056576456 title "The Evolution of evaluation: Lessons from the Message Understanding Conferences" @default.
- W2056576456 doi "https://doi.org/10.1006/csla.1998.0102" @default.
- W2056576456 hasPublicationYear "1998" @default.
- W2056576456 type Work @default.
- W2056576456 sameAs 2056576456 @default.
- W2056576456 citedByCount "77" @default.
- W2056576456 countsByYear W20565764562013 @default.
- W2056576456 countsByYear W20565764562014 @default.
- W2056576456 countsByYear W20565764562015 @default.
- W2056576456 countsByYear W20565764562016 @default.
- W2056576456 countsByYear W20565764562017 @default.
- W2056576456 countsByYear W20565764562018 @default.
- W2056576456 countsByYear W20565764562019 @default.
- W2056576456 countsByYear W20565764562020 @default.
- W2056576456 countsByYear W20565764562021 @default.
- W2056576456 countsByYear W20565764562022 @default.
- W2056576456 countsByYear W20565764562023 @default.
- W2056576456 crossrefType "journal-article" @default.
- W2056576456 hasAuthorship W2056576456A5068115280 @default.
- W2056576456 hasConcept C101468663 @default.
- W2056576456 hasConcept C107457646 @default.
- W2056576456 hasConcept C124681953 @default.
- W2056576456 hasConcept C127413603 @default.
- W2056576456 hasConcept C134306372 @default.
- W2056576456 hasConcept C151730666 @default.
- W2056576456 hasConcept C154945302 @default.
- W2056576456 hasConcept C18903297 @default.
- W2056576456 hasConcept C195807954 @default.
- W2056576456 hasConcept C199360897 @default.
- W2056576456 hasConcept C201995342 @default.
- W2056576456 hasConcept C23123220 @default.
- W2056576456 hasConcept C2522767166 @default.
- W2056576456 hasConcept C2779343474 @default.
- W2056576456 hasConcept C2780451532 @default.
- W2056576456 hasConcept C33923547 @default.
- W2056576456 hasConcept C36503486 @default.
- W2056576456 hasConcept C41008148 @default.
- W2056576456 hasConcept C63000827 @default.
- W2056576456 hasConcept C86803240 @default.
- W2056576456 hasConceptScore W2056576456C101468663 @default.
- W2056576456 hasConceptScore W2056576456C107457646 @default.
- W2056576456 hasConceptScore W2056576456C124681953 @default.
- W2056576456 hasConceptScore W2056576456C127413603 @default.
- W2056576456 hasConceptScore W2056576456C134306372 @default.
- W2056576456 hasConceptScore W2056576456C151730666 @default.
- W2056576456 hasConceptScore W2056576456C154945302 @default.
- W2056576456 hasConceptScore W2056576456C18903297 @default.
- W2056576456 hasConceptScore W2056576456C195807954 @default.
- W2056576456 hasConceptScore W2056576456C199360897 @default.
- W2056576456 hasConceptScore W2056576456C201995342 @default.
- W2056576456 hasConceptScore W2056576456C23123220 @default.
- W2056576456 hasConceptScore W2056576456C2522767166 @default.
- W2056576456 hasConceptScore W2056576456C2779343474 @default.
- W2056576456 hasConceptScore W2056576456C2780451532 @default.
- W2056576456 hasConceptScore W2056576456C33923547 @default.
- W2056576456 hasConceptScore W2056576456C36503486 @default.
- W2056576456 hasConceptScore W2056576456C41008148 @default.
- W2056576456 hasConceptScore W2056576456C63000827 @default.
- W2056576456 hasConceptScore W2056576456C86803240 @default.
- W2056576456 hasIssue "4" @default.
- W2056576456 hasLocation W20565764561 @default.
- W2056576456 hasOpenAccess W2056576456 @default.
- W2056576456 hasPrimaryLocation W20565764561 @default.
- W2056576456 hasRelatedWork W1504101963 @default.
- W2056576456 hasRelatedWork W1788528807 @default.
- W2056576456 hasRelatedWork W2153799433 @default.
- W2056576456 hasRelatedWork W2352337653 @default.
- W2056576456 hasRelatedWork W2367301249 @default.
- W2056576456 hasRelatedWork W2379157006 @default.
- W2056576456 hasRelatedWork W2393978999 @default.
- W2056576456 hasRelatedWork W2725657302 @default.
- W2056576456 hasRelatedWork W2784803070 @default.
- W2056576456 hasRelatedWork W2889874405 @default.
- W2056576456 hasVolume "12" @default.
- W2056576456 isParatext "false" @default.
- W2056576456 isRetracted "false" @default.
- W2056576456 magId "2056576456" @default.
- W2056576456 workType "article" @default.