FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2561858436> ?p ?o ?g. }

• W2561858436 abstract "Talking Through Graphics: An Empirical Study of the Sequential Integration of Modalities Ichiro Umata (umata@mic.atr.co.jp) 1 ATR Media Integration & Communications Research Laboratories; Seika Soraku Kyoto, 619-0288 Japan Atsushi Shimojima (ashimoji@jaist.ac.jp) Japan Advanced Institute of Science and Technology; 1–1 Asahi Tatsunokuchi Nomi Ishikawa 923-1292 Japan Yasuhiro Katagiri (katagiri@mic.atr.co.jp) ATR Media Integration & Communications Research Laboratories; Seika Soraku Kyoto, 619-0288 Japan Abstract An empirical investigation was conducted on the character- istics of language use in graphics communication settings. Graphics communications, such as dialogues using maps, drawings, or pictures, provide people with two independent systems of representation, spoken language and graphics. Drawing on our dialogue data, we will show that the presence of a graphical representation significantly changes the way the spoken language is used, extending its expressive capacity in most cases. As two remarkable uses of language affected in this way, we will report the phenomena of mediated reference and dual description, illustrating them with actual examples from our data. Finally, a quantitative analysis of our data will show that these special uses of language are indeed as common as conventional uses of language in the presence of graphical representations. Introduction Conversational exchanges that involve external graphical rep- resentations are quite common in our daily lives. People often give and ask directions by referring to maps, or they might draw a floor plan in discussing where to place furniture in a living room. Now, linguistic expressions denote objects and relations in the world. This denotation relation is governed by conventions inside the language. An utterance of a lin- guistic expression carries unique information about the world through these conventions. This is what the standard view of the semantics of language tells us. However, when we look at speech in conversational exchanges involving graphics, re- gardless of the language used, we will immediately notice utterances that do not conform to this standard picture. This paper is a detailed examination of the impact of graph- ics on the use of language. Our data consist of actual two- party dialogues where participants draw or consult a map dur- ing verbal exchanges. We will focus on two remarkable uses of language, called “mediated reference” and “dual descrip- tion,” that we found through an examination of our data. Both phenomena are clearly specific to dialogues involving some graphical representation, or at least, some external represen- tation other than speech. Briefly, mediated reference is a case where a linguistic ex- pression reaches its “final” referent due to the fact that its “immediate” referent has a referential connection to this final one in the system of graphics. For example, our subjects often use the indexical “kore” (this) to refer to a building or some other landmark, although its immediate referent is clearly an icon on the map; the icon refers to the landmark in the system Also with Kobe University. of map, and this fact somehow enables the indexical expres- sion to do so too. We will discuss more examples of mediated reference later, and introduce three more varieties of the phe- nomenon. Dual description is a case where a declarative sentence is used to describe a fact that holds in the graphic as well as the corresponding fact in the situation represented by the graphic. Suppose, when asked about the number of stations between two particular stations, one counts the number of icons on a railroad map and says, “There are three stations in-between them.” Is this report concerned with the map itself, or with the mapped railroad? Is it reporting that the railroad map has three station icons between two particular station icons, or that the railroad system has three stations between the two stations? Whichever the answer may be, it seems clear that the speaker has managed to describe both facts with this sen- tence. Note that, on the semantics associated with the railroad map, the first fact means the second fact, and this semantic relation somehow underwrites the duplicative use of the sen- tence. Both uses of language are so natural and common in a di- alogue involving a graphical representation that people may not even be aware of the phenomena. In fact, their frequent occurrence in such settings suggests that they are not a de- viant but rather a perfectly legitimate use of language. Yet the empirical research on the integration of linguistic and graph- ical representations has focused on the issue of how speech is used to disambiguate a graphic (Neilson and Lee 1994) or how a graphic is used to disambiguate speech (Lee and Zeevat 1990). The linguistic-graphic integration has been also stud- ied from a logical point of view, but the focus has been on how a graphic expresses what cannot be easily expressed by a linguistic representation (Barwise and Etchemendy 1996, Shimojima 1999). For both views, the fundamental form of linguistic-graphic integration is a parallel one, where each mode of representation expresses information in its own way, but since one mode of representation expresses what the other form does not, they may work complementarily to each other. In contrast, the two phenomena that we are highlighting in this paper point to a rather different form of integration, where the presence of one mode of representation extends the expressive capacity of the other by affecting the way it is used. Our goal is to draw due attention to this sequential form of graphic-linguistic integration by demonstrating that the in- stances of that type of integration are commonly observed in actual human dialogues, as opposed to mere logical possibil- ities. In the next section, we will describe the methods through" @default.
• W2561858436 created "2017-01-06" @default.
• W2561858436 creator A5016055217 @default.
• W2561858436 creator A5028549350 @default.
• W2561858436 creator A5032231073 @default.
• W2561858436 date "2000-01-01" @default.
• W2561858436 modified "2023-09-23" @default.
• W2561858436 title "Talking Through Graphics: An Empirical Study of the Sequential Integration of Modalities" @default.
• W2561858436 cites W1496245333 @default.
• W2561858436 cites W160900276 @default.
• W2561858436 cites W2001722583 @default.
• W2561858436 cites W2053500205 @default.
• W2561858436 cites W2074229945 @default.
• W2561858436 cites W27045035 @default.
• W2561858436 cites W2912445413 @default.
• W2561858436 hasPublicationYear "2000" @default.
• W2561858436 type Work @default.
• W2561858436 sameAs 2561858436 @default.
• W2561858436 citedByCount "1" @default.
• W2561858436 crossrefType "journal-article" @default.
• W2561858436 hasAuthorship W2561858436A5016055217 @default.
• W2561858436 hasAuthorship W2561858436A5028549350 @default.
• W2561858436 hasAuthorship W2561858436A5032231073 @default.
• W2561858436 hasConcept C107457646 @default.
• W2561858436 hasConcept C111472728 @default.
• W2561858436 hasConcept C120936955 @default.
• W2561858436 hasConcept C121684516 @default.
• W2561858436 hasConcept C138885662 @default.
• W2561858436 hasConcept C144024400 @default.
• W2561858436 hasConcept C166957645 @default.
• W2561858436 hasConcept C17744445 @default.
• W2561858436 hasConcept C199539241 @default.
• W2561858436 hasConcept C205649164 @default.
• W2561858436 hasConcept C21442007 @default.
• W2561858436 hasConcept C2776359362 @default.
• W2561858436 hasConcept C2776505523 @default.
• W2561858436 hasConcept C2779903281 @default.
• W2561858436 hasConcept C36289849 @default.
• W2561858436 hasConcept C41008148 @default.
• W2561858436 hasConcept C41895202 @default.
• W2561858436 hasConcept C94625758 @default.
• W2561858436 hasConceptScore W2561858436C107457646 @default.
• W2561858436 hasConceptScore W2561858436C111472728 @default.
• W2561858436 hasConceptScore W2561858436C120936955 @default.
• W2561858436 hasConceptScore W2561858436C121684516 @default.
• W2561858436 hasConceptScore W2561858436C138885662 @default.
• W2561858436 hasConceptScore W2561858436C144024400 @default.
• W2561858436 hasConceptScore W2561858436C166957645 @default.
• W2561858436 hasConceptScore W2561858436C17744445 @default.
• W2561858436 hasConceptScore W2561858436C199539241 @default.
• W2561858436 hasConceptScore W2561858436C205649164 @default.
• W2561858436 hasConceptScore W2561858436C21442007 @default.
• W2561858436 hasConceptScore W2561858436C2776359362 @default.
• W2561858436 hasConceptScore W2561858436C2776505523 @default.
• W2561858436 hasConceptScore W2561858436C2779903281 @default.
• W2561858436 hasConceptScore W2561858436C36289849 @default.
• W2561858436 hasConceptScore W2561858436C41008148 @default.
• W2561858436 hasConceptScore W2561858436C41895202 @default.
• W2561858436 hasConceptScore W2561858436C94625758 @default.
• W2561858436 hasIssue "22" @default.
• W2561858436 hasOpenAccess W2561858436 @default.
• W2561858436 hasRelatedWork W1490528146 @default.
• W2561858436 hasRelatedWork W1499050190 @default.
• W2561858436 hasRelatedWork W1572515735 @default.
• W2561858436 hasRelatedWork W1589174919 @default.
• W2561858436 hasRelatedWork W165330127 @default.
• W2561858436 hasRelatedWork W1674334186 @default.
• W2561858436 hasRelatedWork W1977189151 @default.
• W2561858436 hasRelatedWork W2058199183 @default.
• W2561858436 hasRelatedWork W2217631055 @default.
• W2561858436 hasRelatedWork W2227733487 @default.
• W2561858436 hasRelatedWork W2401314204 @default.
• W2561858436 hasRelatedWork W2490416600 @default.
• W2561858436 hasRelatedWork W2588887699 @default.
• W2561858436 hasRelatedWork W2612770834 @default.
• W2561858436 hasRelatedWork W2742280187 @default.
• W2561858436 hasRelatedWork W2766553027 @default.
• W2561858436 hasRelatedWork W30026100 @default.
• W2561858436 hasRelatedWork W3021948304 @default.
• W2561858436 hasRelatedWork W3206007934 @default.
• W2561858436 hasRelatedWork W58377777 @default.
• W2561858436 hasVolume "22" @default.
• W2561858436 isParatext "false" @default.
• W2561858436 isRetracted "false" @default.
• W2561858436 magId "2561858436" @default.
• W2561858436 workType "article" @default.