Hypertext semiotics in the commercialized Internet
©2001
Doktorarbeit / Dissertation
235 Seiten
Zusammenfassung
Inhaltsangabe:Abstract:
Building on approaches that have succeeded in applying semiotic principles and methodology to computer science, such as computer semiotics, computational semiotics, and semiotic interface engineering, this dissertation establishes a systematic account for those researchers who are ready to look at hypertext from a semiotic point of view. Rather than a new hypertext model, this work presents the prolegomena of a theory of hypertext semiotics, interlacing the existing models with the findings of semiotic research, on all levels of the textual, aural, visual, tactile and olfactory channels. A short history of hypertext, from its prehistory to today's state of the art systems and the current developments in the commercialized World Wide Web creates the context for this approach which should be seen as a fortification of the connection between the media semiotic approach and computer semiotics. While computer semioticians claim that the computer is a semiotic machine and Artificial Intelligence scientists underline the importance of semiotics for the construction of the next hypertext generation, this paper makes use of a much broader methodological basis. These findings are placed in the context of the commercialization of the Internet. Besides identifying the main challenges for eCommerce from the viewpoint of hypertext semiotics, the author concentrates on information goods and the current limitations for a new economy, such as restrictive intellectual property and copyright laws. A semiotic analysis of iMarketing techniques and the Toywar complete the dissertation.
Zusammenfassung:
Diese Dissertation legt einen systematischen Ansatz für all jene Forscher dar, die bereit sind, Hypertext aus einer semiotischen Perspektive zu betrachten. Durch die Verknüpfung existierender Hypertext-Modelle mit den Resultaten aus der Semiotik auf allen Sinnesebenen der textuellen, auditiven, visuellen, taktilen und geruchlichen Wahrnehmung skizziert der Autor Prolegomena einer Hypertext-Semiotik-Theorie, anstatt ein völlig neues Hypertext-Modell zu präsentieren. Eine Einführung in die Geschichte der Hypertexte, von ihrer Vorgeschichte bis zum heutigen Entwicklungsstand und den gegenwärtigen Entwicklungen im kommerzialisierten World Wide Web bilden den Rahmen für diesen Ansatz, welcher als Fundierung des Brückenschlages zwischen Mediensemiotik und Computer-Semiotik angesehen werden darf. Während Computer-Semiotiker wissen, dass der Computer […]
Building on approaches that have succeeded in applying semiotic principles and methodology to computer science, such as computer semiotics, computational semiotics, and semiotic interface engineering, this dissertation establishes a systematic account for those researchers who are ready to look at hypertext from a semiotic point of view. Rather than a new hypertext model, this work presents the prolegomena of a theory of hypertext semiotics, interlacing the existing models with the findings of semiotic research, on all levels of the textual, aural, visual, tactile and olfactory channels. A short history of hypertext, from its prehistory to today's state of the art systems and the current developments in the commercialized World Wide Web creates the context for this approach which should be seen as a fortification of the connection between the media semiotic approach and computer semiotics. While computer semioticians claim that the computer is a semiotic machine and Artificial Intelligence scientists underline the importance of semiotics for the construction of the next hypertext generation, this paper makes use of a much broader methodological basis. These findings are placed in the context of the commercialization of the Internet. Besides identifying the main challenges for eCommerce from the viewpoint of hypertext semiotics, the author concentrates on information goods and the current limitations for a new economy, such as restrictive intellectual property and copyright laws. A semiotic analysis of iMarketing techniques and the Toywar complete the dissertation.
Zusammenfassung:
Diese Dissertation legt einen systematischen Ansatz für all jene Forscher dar, die bereit sind, Hypertext aus einer semiotischen Perspektive zu betrachten. Durch die Verknüpfung existierender Hypertext-Modelle mit den Resultaten aus der Semiotik auf allen Sinnesebenen der textuellen, auditiven, visuellen, taktilen und geruchlichen Wahrnehmung skizziert der Autor Prolegomena einer Hypertext-Semiotik-Theorie, anstatt ein völlig neues Hypertext-Modell zu präsentieren. Eine Einführung in die Geschichte der Hypertexte, von ihrer Vorgeschichte bis zum heutigen Entwicklungsstand und den gegenwärtigen Entwicklungen im kommerzialisierten World Wide Web bilden den Rahmen für diesen Ansatz, welcher als Fundierung des Brückenschlages zwischen Mediensemiotik und Computer-Semiotik angesehen werden darf. Während Computer-Semiotiker wissen, dass der Computer […]
Leseprobe
Inhaltsverzeichnis
ID 6687
Neumüller, Moritz: Hypertext semiotics in the commercialized Internet
Hamburg: Diplomica GmbH, 2003
Zugl.: Wien, Wirtschaftsuniversität, Dissertation / Doktorarbeit, 2001
Dieses Werk ist urheberrechtlich geschützt. Die dadurch begründeten Rechte,
insbesondere die der Übersetzung, des Nachdrucks, des Vortrags, der Entnahme von
Abbildungen und Tabellen, der Funksendung, der Mikroverfilmung oder der
Vervielfältigung auf anderen Wegen und der Speicherung in Datenverarbeitungsanlagen,
bleiben, auch bei nur auszugsweiser Verwertung, vorbehalten. Eine Vervielfältigung
dieses Werkes oder von Teilen dieses Werkes ist auch im Einzelfall nur in den Grenzen
der gesetzlichen Bestimmungen des Urheberrechtsgesetzes der Bundesrepublik
Deutschland in der jeweils geltenden Fassung zulässig. Sie ist grundsätzlich
vergütungspflichtig. Zuwiderhandlungen unterliegen den Strafbestimmungen des
Urheberrechtes.
Die Wiedergabe von Gebrauchsnamen, Handelsnamen, Warenbezeichnungen usw. in
diesem Werk berechtigt auch ohne besondere Kennzeichnung nicht zu der Annahme,
dass solche Namen im Sinne der Warenzeichen- und Markenschutz-Gesetzgebung als frei
zu betrachten wären und daher von jedermann benutzt werden dürften.
Die Informationen in diesem Werk wurden mit Sorgfalt erarbeitet. Dennoch können
Fehler nicht vollständig ausgeschlossen werden, und die Diplomarbeiten Agentur, die
Autoren oder Übersetzer übernehmen keine juristische Verantwortung oder irgendeine
Haftung für evtl. verbliebene fehlerhafte Angaben und deren Folgen.
Diplomica GmbH
http://www.diplom.de, Hamburg 2003
Printed in Germany
5
Abstract
Hypertext theory makes use of the same set of terms that have been explored in decades of
semiotic investigation, such as sign, text, communication, code, metaphor, paradigm, syn-
tax, etc. Building on approaches that have succeeded in applying semiotic principles and
methodology to computer science, such as computer semiotics, computational semiotics,
and semiotic interface engineering, this dissertation establishes a systematic account for
those researchers who are ready to look at hypertext from a semiotic point of view. Rather
than a new hypertext model, this work presents the prolegomena of a theory of hypertext
semiotics, interlacing the existing models with the findings of semiotic research, on all
levels of the textual, aural, visual, tactile and olfactory channels. A short history of hyper-
text, from its prehistory to today's state of the art systems and the current developments in
the commercialized World Wide Web creates the context for this approach which should be
seen as a fortification of the connection between the media semiotic approach and computer
semiotics. While computer semioticians claim that the computer is a semiotic machine and
Artificial Intelligence scientists underline the importance of semiotics for the construction
of the next hypertext generation, this paper makes use of a much broader methodological
basis. The range of subtopics include hypertext applications, paradigms, structure, naviga-
tion, Web design, and Web augmentation. The interdisciplinary spectrum of methodology
also enables detailed analyses, e.g. of the Web browsers' pointing device, the @ sign, and
emoticons. The "icon" (a small picture known from the GUI desktop and used in hyper-
text) is identified as a misnomer and replaced by a new generation of powerful "Graphical
Link Markers". These findings are placed in the context of the commercialization of the
Internet. Besides identifying the main challenges for eCommerce from the viewpoint of
hypertext semiotics, the author concentrates on information goods and the current limita-
tions for a new economy, such as restrictive intellectual property and copyright laws. These
anachronistic regulations are based on the problematic assumption that for information
too value is based on scarcity. A semiotic analysis of iMarketing techniques, such as
banner ads, keywords, and link injection and two digressions on the Browser War, and the
Toywar complete the dissertation.
6
Zusammenfassung
Die Hypertext Theorie verwendet die selbe Terminologie, welche seit Jahrzehnten in der
semiotischen Forschung untersucht wird, wie z.B. Zeichen, Text, Kommunikation, Code,
Metapher, Paradigma, Syntax, usw. Aufbauend auf jenen Ergebnissen, welche in der An-
wendung semiotischer Prinzipien und Methoden auf die Informatik erfolgreich waren, wie
etwa Computer Semiotics, Computational Semiotics und Semiotic Interface Engineering,
legt diese Dissertation einen systematischen Ansatz für all jene Forscher dar, die bereit sind,
Hypertext aus einer semiotischen Perspektive zu betrachten. Durch die Verknüpfung ex-
istierender Hypertext-Modelle mit den Resultaten aus der Semiotik auf allen Sinnesebenen
der textuellen, auditiven, visuellen, taktilen und geruchlichen Wahrnehmung skizziert der
Autor Prolegomena einer Hypertext-Semiotik-Theorie, anstatt ein völlig neues Hypertext-
Modell zu präsentieren. Eine Einführung in die Geschichte der Hypertexte, von ihrer
Vorgeschichte bis zum heutigen Entwicklungsstand und den gegenwärtigen Entwicklungen
im kommerzialisierten World Wide Web bilden den Rahmen für diesen Ansatz, welcher
als Fundierung des Brückenschlages zwischen Mediensemiotik und Computer-Semiotik
angesehen werden darf. Während Computer-Semiotiker wissen, dass der Computer eine
semiotische Maschine ist und Experten der künstlichen Intelligenz-Forschung die Rolle
der Semiotik in der Entwicklung der nächsten Hypertext-Generation betonen, bedient sich
diese Arbeit einer breiteren methodologischen Basis. Dementsprechend reichen die Teil-
gebiete von Hypertextanwendungen, -paradigmen, und -strukturen, über Navigation, Web
Design und Web Augmentation zu einem interdisziplinären Spektrum detaillierter Anal-
ysen, z.B. des Zeigeinstrumentes der Web Browser, des Klammeraffen-Zeichens und der
sogenannten Emoticons. Die Bezeichnung "Icon" wird als unpassender Name für jene
Bildchen, welche von der graphischen Benutzeroberfläche her bekannt sind und in Hyper-
texten eingesetzt werden, zurückgewiesen und diese Bildchen durch eine neue Generation
mächtiger Graphic Link Markers ersetzt. Diese Ergebnisse werden im Kontext der Kom-
merzialisierung des Internet betrachtet. Neben der Identifizierung der Hauptprobleme des
eCommerce aus der Perspektive der Hypertext Semiotik, widmet sich der Autor den In-
formationsgütern und den derzeitigen Hindernissen für die New Economy, wie etwa der
restriktiven Gesetzeslage in Sachen Copyright und Intellectual Property. Diese anachronis-
tischen Beschränkungen basieren auf der problematischen Annahme, dass auch der Infor-
mationswert durch die Knappheit bestimmt wird. Eine semiotische Analyse der iMarketing
Techniken, wie z.B. Banner Werbung, Keywords und Link Injektion, sowie Exkurse über
den Browser Krieg und den Toywar runden die Dissertation ab.
Preface
"Why is this a book?" is the first sentence of Jakob Nielsen's seminal work on
hypertext
and
hypermedia
, [
387
]. The same question should be asked to the author of a doctoral thesis
on hypertext semiotics. For Nielsen, there were still "so many disadvantages connected
with electronic publishing" that he decided to stay with paper a little longer. Many of these
disadvantages seem to have disappeared with the rise of the WWW. Yet, a lot of technical
and administrative factors still favor the dissemination of scientific works on paper and
online. The strategy to deliver a printout of a Web project's
HTML
pages to the library
staff (as described in [
65
]) does not solve the problem. Thus, I opted for a hybrid form of
electronic and paper publishing, the PDF format with embedded hyperlinks and a
HTML
version derived from it, cf. [
196
].
1
The cross-references to other sections, figures or footnotes are active links that save the
reader of the electronic text from leafing or scrolling around. The definiton links to the
glossary, the "backlinks" in the bibliographic section and the active hyperlinks to World
Wide Web sites are especially useful in the online version.
Accordingly, the table of contents alone cannot fully represent the structure of this docu-
ment. To get an overview over this work and for navigation purposes, the reader should
also study figure
1
, the graphical map of the dissertation.
This dissertation would not exist without the sustained efforts of many people. Allow
me first and foremost to stress the depth of my obligation to my adviser, Veith Risak.
In personal conversations and numerous e-mails, his insights and commentaries on this
dissertation helped me discover what it was about. I am also thankful for the advise
and contributions of several collegues, among them Wolfgang Panny and Andreas Geyer-
Schulz at the Vienna University of Economics and Business Administration (
WU Wien
),
Herbert Hrachovec and Klaus Hamberger at the University of Vienna (
Uni Wien
), Jeff
Bernard at the Institute for Socio-Semiotic Studies (ISSS) in Vienna, Karin Wenz, Guido
Ipsen and Joseph Wallmannsberger at the
University of Kassel
, Dagmar Schmauks at the
TU Berlin
, Linda Colet at the Museum of Modern Art (
MoMA
), Lauretta Jones at the
Thomas J. Watson Research Center
, Sigi Reich at the
University of Linz
, Frank Shipman
at
Texas A&M University
, and Brian Proffitt at
BrowserWatch
. In my efforts to write this
work, I was supported by Claudia Hundius (who also helped with the compilation of the
glossary), my parents, my brother and my closest friends.
1
This thesis is written with the text editor LYX on L
A
TEX, a document preparation system designed by
Leslie Lamport in 1985. It, in turn, was built up from a typesetting language called TEX, created by Donald
Knuth in 1984. The embedded hyperlinks use the package hyperref (developed by Sebastian Rahtz, Heiko
Oberdiek and others). The PDF version was created with ps2pdf, the
HTML
version was generated using the
L
A
TEX2HTML translator by Nikos Drakos. The online version of this dissertation can be found on my Home Page
www.unet.univie.ac.at/
a9108095
or ordered free of cost via
email
.
7
8
Systems
HT
HT-Appli-
cations
gation
HT-Navi-
WWW
Usability
eCommerce
Commer-
cialization
iMarketing
Computer
Semiotics
Symbolic
Forms
Media
Semiotics
Structure
Hypertext
Semiotics
Hypertext
Theory
Hypertext
HT-Para-
digms
Definitions
& Limitations
Sign
Code
Textuality
Semiotics
Figure 1: Graphical overview of this doctoral thesis.
Contents
1
Introduction
14
2
The Semiotic Approach
19
2.1
Introduction to Classic Semiotics . . . . . . . . . . . . . . . . . . . . . . .
19
2.2
From Symbolic Forms to (Post-)Structuralist Semiotics . . . . . . . . . . .
23
2.3
Definitions and Limitations of Semiotics . . . . . . . . . . . . . . . . . . .
28
2.4
Sign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30
2.5
Semiosis
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
2.6
The Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
38
2.7
Media Semiotics
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
40
2.8
Textuality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
53
2.9
Computer Semiotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
54
3
Hypertext Theory
59
3.1
History of Hypertext . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
62
3.1.1
Prehistory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
62
3.1.2
Memex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
64
3.1.3
Xanadu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
65
3.1.4
Augment/NLS . . . . . . . . . . . . . . . . . . . . . . . . . . . .
66
3.1.5
KMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
67
3.1.6
Intermedia
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
67
3.1.7
NoteCards
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
68
3.1.8
HyperCard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
68
3.1.9
HyperTies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
68
3.1.10 Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
68
3.1.11 Writing Environment . . . . . . . . . . . . . . . . . . . . . . . . .
69
3.1.12 gIBIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
69
3.1.13 The Aspen Movie Map . . . . . . . . . . . . . . . . . . . . . . . .
69
3.1.14 Storyspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
70
9
CONTENTS
10
3.1.15 Spatial Hypertext . . . . . . . . . . . . . . . . . . . . . . . . . . .
70
3.1.16 Open Hypermedia Systems . . . . . . . . . . . . . . . . . . . . . .
71
3.1.17 Brief History of the World Wide Web . . . . . . . . . . . . . . . .
72
3.1.18 Hyper-G and Hyperwave . . . . . . . . . . . . . . . . . . . . . . .
74
3.1.19 The XML Family and the Semantic Web
. . . . . . . . . . . . . .
74
3.2
Hypertext Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . .
76
3.2.1
Hypertext and Learning
. . . . . . . . . . . . . . . . . . . . . . .
77
3.2.2
Groupware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
80
3.3
Hypertext Paradigms . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
81
3.4
Hypertext Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
83
3.4.1
Graph Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
83
3.4.2
Beyond Graph Theory . . . . . . . . . . . . . . . . . . . . . . . .
89
3.4.3
Spatial Hypertext . . . . . . . . . . . . . . . . . . . . . . . . . . .
90
3.4.4
Time-based Hypermedia . . . . . . . . . . . . . . . . . . . . . . .
99
3.4.5
Personalizable Hypertext and Adaptive Systems . . . . . . . . . . .
100
3.5
Hypertext Semiotics
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
101
3.6
The Usability Approach to Hypertext . . . . . . . . . . . . . . . . . . . . .
112
3.7
Navigation in Hypertext Systems . . . . . . . . . . . . . . . . . . . . . . .
117
3.7.1
The Hypertext Session . . . . . . . . . . . . . . . . . . . . . . . .
121
3.7.2
Bookmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
122
3.7.3
Guided Tours . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
124
3.7.4
Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
126
3.7.5
Landmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
127
3.7.6
Backtrack Function . . . . . . . . . . . . . . . . . . . . . . . . . .
128
3.7.7
History lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
131
3.7.8
Agents, Narrative, Personal and Social Navigation . . . . . . . . .
133
3.7.9
Navigating by Query . . . . . . . . . . . . . . . . . . . . . . . . .
134
3.7.10 Navigation in non-text hypermedia
. . . . . . . . . . . . . . . . .
137
3.7.11 Semiotic Aspects of Hypertext Navigation
. . . . . . . . . . . . .
139
4
Reading the Signs of the World Wide Web
146
4.1
The WWW metaphor . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
146
4.2
Hypertext functionalities of the WWW
. . . . . . . . . . . . . . . . . . .
147
4.2.1
Web Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
148
4.2.2
WWW or What's Wrong with the Web? . . . . . . . . . . . . . . .
149
4.2.3
Web Augmentation . . . . . . . . . . . . . . . . . . . . . . . . . .
150
4.3
The Internet as a Global Agora . . . . . . . . . . . . . . . . . . . . . . . .
151
CONTENTS
11
4.3.1
The Global Swarm . . . . . . . . . . . . . . . . . . . . . . . . . .
152
4.3.2
Information Gluttony . . . . . . . . . . . . . . . . . . . . . . . . .
153
4.4
Cyberspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
154
4.5
Cybersigns
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
155
4.6
Iconicity of Graphic Link Markers . . . . . . . . . . . . . . . . . . . . . .
158
4.7
The Commercialization of the Internet . . . . . . . . . . . . . . . . . . . .
163
4.7.1
eCommerce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
164
4.7.2
Intellectual Property and Copyright . . . . . . . . . . . . . . . . .
165
4.7.3
Authenticity of Information . . . . . . . . . . . . . . . . . . . . .
168
4.7.4
Banners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
168
4.7.5
Keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
169
4.7.6
Digression I: The Browser War
. . . . . . . . . . . . . . . . . . .
172
4.7.7
Digression II: The Toywar . . . . . . . . . . . . . . . . . . . . . .
174
5
Summary
176
A Glossary
182
B Bibliography
202
List of Figures
1
Graphical overview of this doctoral thesis. . . . . . . . . . . . . . . . . . .
8
2.1
The Saussurean sign model. Source: [
469
, p. 78]. . . . . . . . . . . . . . .
20
2.2
The Peircean sign model. Source: [
353
, p. 136]. . . . . . . . . . . . . . . .
23
2.3
Syntagmatic chain. Source: [
469
, p. 137]. . . . . . . . . . . . . . . . . . .
27
2.4
Lacan's signifier/signified model. Source: [
300
, vol. 2, p. 24].
. . . . . . .
31
2.5
Peircean sign classes. Source: [
375
].
. . . . . . . . . . . . . . . . . . . .
32
2.6
Classification of signs by their source. Source: [
155
, p. 37]. . . . . . . . . .
34
2.7
Classification of signs by their sign-functionality. Source: [
155
, p. 44]. . . .
34
2.8
Language systems. Source: [
155
], p. 86. . . . . . . . . . . . . . . . . . . .
35
2.9
Graphein as the root of numbers. . . . . . . . . . . . . . . . . . . . . . . .
36
2.10 Misleading pairing of terminology. Source: [
154
, p. 190]. . . . . . . . . . .
37
2.11 Unlimited semiosis. Source: [
353
, p. 137]. . . . . . . . . . . . . . . . . . .
37
2.12 Communication process. Source: [
153
, p. 139]. . . . . . . . . . . . . . . .
41
2.13 Sonar chart of the water beneath a fishing boat. Source: Eagle Electronics.
43
2.14 Borromean rings. Source: based on [
160
, p. 86] and [
101
]. . . . . . . . . .
44
2.15 Counting technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
2.16 Langenscheidt's OhneWörterBuch. Source: [
1
, p. 28]. . . . . . . . . . . . .
51
2.17 Depictions of eagles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
51
2.18 Linus' "security blanket". Source: www.unitedmedia.com. . . . . . . . . .
56
3.1
Hypertext, multimedia and hypermedia. Source: [
216
, p. 53]. . . . . . . . .
60
3.2
Berners-Lee's Proposal. Source: [
51
]. . . . . . . . . . . . . . . . . . . . .
73
3.3
Hypertext structure with six nodes and eight links
(Source: [
387
, p. 1]) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
82
3.4
Two hypertext graphs base on fig.
3.3
. . . . . . . . . . . . . . . . . . . . .
84
3.5
Hypertext (fig.
3.3
) as matrix and table. . . . . . . . . . . . . . . . . . . .
84
3.6
Three depictions of the same graph. Source: [
222
]. . . . . . . . . . . . . .
84
3.7
Detail of a screen-shot from [
361
]. . . . . . . . . . . . . . . . . . . . . . .
87
3.8
Multiple links in the Lotus Organizer 4.1. . . . . . . . . . . . . . . . . . .
89
12
LIST OF FIGURES
13
3.9
Manhattan metric, Euclidian&topological space. Source: based on [
142
].
.
92
3.10 Viewing details in the cityscape-visualization of a tree. Source [
275
].
. . .
94
3.11 Depth by Overlapping. Source: [
17
, Fig. 186].
. . . . . . . . . . . . . . .
98
3.12 AHM components and timing relations. Source: [
216
, p. 58] . . . . . . . .
100
3.13 Parameters associated with system acceptability. Source: [
387
, p. 145].
. .
112
3.14 Yahoo!3D map. Source: [
145
] . . . . . . . . . . . . . . . . . . . . . . . .
120
3.15 3D cityscape view of the Web generated by Map.Net. Source: [
145
] . . . .
121
3.16 Web link attributes in the HyperScout prototype, Source: [
536
]. . . . . . . .
123
3.17 Metaphor misuse: The help function of www.bankaustria.com. . . . . . . .
125
3.18 Loosing track with the backtracking command. . . . . . . . . . . . . . . .
129
3.19 Online Catalogue of the Vienna University: Book list. . . . . . . . . . . . .
130
3.20 Vienna University: Multiple-windows book-ordering. . . . . . . . . . . . .
131
3.21 Search request for "eagle" on Google Image Search.
. . . . . . . . . . . .
137
4.1
Transition from pictographic to abstract representation types. Source: [
375
].
160
4.2
Development of object representations according to Dieberger [
142
, p. 61].
161
4.3
Development of object representations. Source: [
235
].
. . . . . . . . . . .
162
4.4
Graphic Link Markers. Source: based on [
235
] and [
385
].
. . . . . . . . .
163
4.5
Midstream Marketing Program. Source: based on [
276
]. . . . . . . . . . .
170
4.6
Katz Fodor Tree for /visa/. Source: [
382
, p. 239]
. . . . . . . . . . . . . .
171
4.7
Browser War. Source: [
425
]. . . . . . . . . . . . . . . . . . . . . . . . . .
172
4.8
etoy soldier and bomber. Source: www.toywar.com.
. . . . . . . . . . . .
175
Chapter 1
Introduction
While many people think of
hypertext
in terms of the World Wide Web, hypertext was
conceptualized in the mid-1940s by Vannevar Bush (cf. [
80
]) and practical research by
Douglas Engelbart and Ted Nelson has been ongoing since the early 1960s, cf. [
162
,
379
].
In 1965, Nelson coined the word "hypertext" and defined it as "a body of written or pictorial
material interconnected in a complex way that it could not be conveniently represented on
paper. It may contain summaries or maps of its contents and their interrelations; it may
contain annotations, additions and footnotes from scholars who have examined it" [
379
].
Since the late 1980s, Hypertext Theory has established a new battle-field for literary stud-
ies,
semiotics
, linguistics, philosophy and media theory. Prima facie, there seems to be a
vast discrepancy between this theoretical approach and the phenomenon of the commer-
cialized
WWW
at the turn of the century: "Discussion of hypertext from five or ten years
ago now seems strangely idealistic. Although the possibilities identified by semioticians
are still present, the implementation of hypertext within the Web closely resembles tradi-
tional print media. Indeed, the structure often imposed upon hyperlinks actively negates
the radical, writerly, intertextual qualities previously envisaged," [
346
]. Yet, that does not
mean that academic research must adapt itself fully to the current standards of corporate
Web design and electronic publishing, cf. [
287
]. On the contrary, it can critically analyze
these developments and provide a theoretical and scientific basis for a public discussion
that involves the industry and governments, the hypertext research community, the W3C
and the Internet Engineering Task , cf. [
266
,
506
].
Today, navigation tools and even the content of typical Web pages consist to a large extent
of graphical information, whether we call them
images
,
icons
, buttons, or even animations.
However, most hypertext theorists view the terms
hypertext
and
hypermedia
as synony-
mous and use them interchangeably with a preference to sticking to hypertext "since there
does not seem to be any reason to reserve a special term for text-only systems," [
387
, p. 5];
cf. [
62
] . Concerning the difference between multimedia and hypermedia, it has been said
that the difference between multimedia and hypermedia is similar to that between watching
a travel film and being a tourist yourself, cf. [
387
, p. 10]. In the question whether hyperme-
dia is a subset of hypertext or,
hypertext
a subset of
hypermedia
, I follow Schulmeister: "A
special subset of multimedia, then, is hypertext at the same time, and the decisive criterion
is interactivity. If we collate both arguments, we arrive at the following definition: hyper-
media is a subset of hypertext, and at the same time hypermedia is a subset of multimedia.
It is probably better to view multimedia and hypertext as two independent entities with an
intersection that might be called hypermedia" [
476
].
The importance of all our senses for understanding cannot be overestimated, yet there are
no major hypertext systems that incorporate olfactory, gustatory, or haptic elements (neither
14
CHAPTER 1. INTRODUCTION
15
as
nodes
, nor as
links
). Video and audio clips can be found in many systems and are
available widely on the
WWW
, but involve many questions, e.g. how to link within, out of,
and into a sound
1
[
320
], [
20
], [
215
]. Nielsen points out that "even though many hypertext
systems are in fact hypermedia systems and include many multimedia effects, the fact that a
system is multimedia-based does not make it hypertext [. . . ] Only when users interactively
take control of a set of dynamic links among units of information does a system get to
be hypertext," [
387
, p. 10]. Therefore, the main focus will be on the distinction between
textual and
graphic
elements of hypertext systems, whether they are simply part of a
node
or whether they have linking functionality.
The semiotic approach has proved to be a suitable and most elaborate tool when working
with both words and
images
. It has been adopted by art history and media theory and is
nowadays a standard tool when analyzing the domain of
images
, cf. [
66
,
535
,
141
,
160
].
"Computer Semiotics" is a term which has been gaining currency in recent years. Estab-
lished by Peter Bøgh Andersen (cf. [
9
,
8
,
10
,
13
]) it may be an emergent field of inquiry, but
as of yet there is little academic consensus as to its scope. By elaborating the concept of
Hypertext Semiotics, I intend to test the stability of the Computer Semiotics construct and
its applicability of its methods on hypertext structures which has often been implied but not
yet fully explored, cf. [
8
,
106
,
382
,
383
,
478
,
384
,
128
]. The validity of a semiotic approach
to computer science has been emphatically underlined by Nadin:
"Computation is about meaning, not electrons. Regardless of the type of
computation, what interests computer users is not the electrons moving along
sophisticated circuits, but the various bearers of meaningful information signs
subjected to their programmed processing. Whether electron, light, quantum,
or DNA-based, the computer is a medium for sign processes! Numbers turned
into images, simulations, database operations, etc. are examples of how the
signs of the object of our practical interest are processed according to our
goals" [
375
].
Peirce's classic distinction between
iconic
,
indexical
and
symbolic
signs has been cited
in connection with hypertext theory by Colón [
107
]. It has also been pointed out that
some of the "
icons
" employed in
GUIs
, within the toolbar of Web
browsers
and on Web
sites are in fact
symbols
, cf. [
346
]. However, many of these bridges between
semiotics
and hypertext theory are not quite theoretically founded. Besides a thorough consideration
of classical semiotic approaches, I will draw on a broader theoretical framework of the
symbol
, including Cassirer, Langer and Lévi-Strauss on the one hand and Freud, Lacan
and Derrida on the other.
The analysis of signs in hypertext systems can also profit from the latest advances in image
theory, namely Elkins [
160
], who weds Wittgenstein's Bildtheorie with Goodman's crite-
ria of
notation
. The visualization of hypertext architecture depends largely on graph the-
ory, itself a
notation
system. Some readers might be surprised by the exemplary regresses
on cultural and artistic phenomena to illustrate my points of view. Yet, considering that
the average Web page designer (knowingly or not) seems to recur on the same concepts,
memories and experiences of a common visual culture, this strategy will present itself as
appropriate for the purpose.
The intention to make Web pages more appealing to users of different age and education
from around the globe have pushed forward a wave of non-text media: Graphic and pho-
tographic elements of hypermedia design promise to ring in a Renaissance of the
image
while the semiotic limitations of picture languages have been long identified, cf. [
163
,
471
,
473
]. Keeping these limitations in mind, I will still try to make use of Otto Neurath's
1
Hopefully, the insights gained from building non-visual
hypermedia
systems for blind users can soon be used
for navigation in auditory hyperspace, cf. [
367
], [
255
].
CHAPTER 1. INTRODUCTION
16
International Picture Language
to elaborate a basic scheme for a new generation of "
icons
",
which I call Graphic Link Markers (
GLM
). I will also reflect on the future involvement of
other human senses into
hypermedia
, even if they are still restricted by bandwidth
2
(such
as moving
images
and sound) or other technological limitations (such as haptic, olfactory
and gustatory inputs).
Hypertext theory has always been strongly linked to usability and human factors, epito-
mized by Jakob Nielsen who is commonly referred to as a Web design guru today. Impor-
tant as Nielsen's usability studies are to understand the success or failure of corporate Web
sites, online services and DotCom enterprises, they often lack a broader analytical basis.
The discussion of the commercialization of the Internet has produced a large body of theo-
retical and practical work. The elaboration of my hypertext semiotic approach is placed in
the framework of this sociological and economic research.
It is commonly agreed upon that eCommerce, the expansion of the Information Technology
branch (sometimes called Wintelism, cf. [
69
]) and the increasing capital market orientation
on New Markets have changed our economy, cf. [
111
,
331
,
437
,
481
,
29
,
30
,
32
,
214
]. The
New Economy phenomenon, if understood on a macroeconomic level, has promised higher
non-inflationary economic growth due to increases in productivity caused by the digital rev-
olution. Evidence put forward by mainstream protagonists of the New Economy suggests
the end of the economic cycle and permanent stability of a finance-led regime of accumu-
lation on the basis of the digital production
paradigm
. However, Scherrer [
470
] reasons
that some basic causal relationships of such a regime, especially the connections between
investment and profits; profits and wealth; and wealth and consumption seem too fragile for
suggesting that a stable new regime of accumulation has emerged. Furthermore, the New
Economy thesis suffers from serious problems in measuring productivity in the service in-
dustries and seems to be based on massive borrowing by both companies and households,
whose debts now stand at a record high, cf. [
165
]. As the expansion of the late 1990s
comes to an end, the vast inflows of financial capital from abroad are turning around and
thus become a threat for the US economy. This paper will not analyze the recent market
crash of the technology and information sector. Nevertheless, future research might well
prove that my semiotic reflections on commercialized hypertexts tackle some of the prob-
lems that have led to the "DotCom-crisis" on a deeper level than current usability statistics
and financial market models.
The Internet is a phenomenon that has inspired the literateness of many scholars, but the
methods of each academic discipline facilitate certain ways of knowing and inhibit oth-
ers. For instance, those which involve quantitative
paradigms
(such as Economics) involve
the selection only of those aspects of experience which can be measured. Linguistic and
Literary studies, on the other hand, concentrate on the purely academic and artistic use
of the medium, and tend to ignore the rapid growth of its commercial use (advertising,
public relations, eCommerce, etc.). While Computer Science is primarily concentrating of
technical innovations and implementation, traditional methods of Business Administration
encounter difficulties if applied to an environment that (at least in the future) deals pri-
marily with non-tangible goods, agents and points of sale. Such selectivity has, of course,
dramatically empowered "the scientific method", but it can do so only in limited domains.
As Aldous Huxley wryly noted, "our universities possess no chair of synthesis" [
242
, p.
276]. As a matter of fact, the way of knowing favored in the broad arena of academia
is specialization. The intention of this paper is to contribute to a broader discussion of
theoretical and practical issues related to
hypertext
and the World Wide Web. It is also a
wide-ranging exploration of the commercialization of the Internet and focuses on a variety
of ways in which the development is framed. It highlights major steps in the history of the
2
Nielsen's Law of Internet bandwidth states that "a high-end user's connection speed grows by 50% per year
[but] you don't get to use this added bandwidth to make your Web pages larger until 2003". For him, average
bandwidth increases slowly for three reasons: 1. Telecom companies are conservative, 2. Users are reluctant to
spend much money on bandwidth, and 3. The user base is getting broader, cf. [
390
].
CHAPTER 1. INTRODUCTION
17
medium and its users, including the dynamic changes which have enabled the Internet to
become a major marketplace of the 21st century. It involves a particular focus on its aca-
demic and corporate structures, although many references are also made to private users.
I aim to give an account which will serve, by its generality and interdisciplinarity, to lift
the scholars' heads from what sometimes seems to me too narrow a focus. The need for
this kind of enterprise becomes apparent in the context of the recent "DotCom-crisis". The
main challenges for online businesses that want to establish themselves on the market are,
for consulter Helmut F. Meier, language and cultural problems: "Sprach- und Kulturbarri-
eren [sind] noch schwerer zu überwinden [als nationalstaatliche Grenzen]" [
3
]. To analyze
these barriers, it seems favorable to employ methods of Ethnology, Information and Me-
dia Theory, Cultural Studies,
Semiotics
, Philosophy and Art History, hand in hand with
an Economic approach. Of course, this multidisciplinary advance runs the risk of ignor-
ing academic-bureaucratic structures but the point is not so much to dismantle disciplinary
boundaries as to be able to move across them, cf. [
152
, p. 27].
"Hypermedia is on the one hand multidisciplinary [. . . ] on the other hand
it is pervasive, with applications in many areas, instruction, information, enter-
tainment, commerce, engineering. [. . . ] Despite the recent upsurge of interest
in the commercial applications of the WWW and e-commerce generally, many
of the fundamental issues surrounding successful commercial exploitation of
hypertext remain unsolved" [
114
, p. 40].
That line of thought makes it less surprising why this dissertation was handed in at a
business school, not at another faculty or school that deals with Communication Science,
Aesthetics
, or Philosophy. Obviously, a primarily theoretical approach to communication
networks and new media seems congruent with the current flow of research at these fac-
ulties. Furthermore, theoretical disciplines tend to face the commercialization of the In-
ternet on a rather superficial and polemical level. Furthermore, the Vienna University of
Economics and Business Administration (
WU Wien
) houses a variety of disciplines which
have long been dealing with communication problems (such as Marketing, especially In-
ternational Marketing and Advertising), and a
Department
of Information Business that is
actively involved in hypertext research issues, cf. [
427
].
Working inter-, or transdisciplinary is more than the sum of the involved disciplines; it is the
exploration of a common ground for researchers from different fields by contrasting estab-
lished points of view. In the following sections, I will try to elaborate a theory of Hypertext
Semiotics that adheres less strictly to literary theories and linguistics than comparable ap-
proaches (e.g. [
265
,
304
,
448
,
369
,
247
,
56
,
539
]). It will include a
diachronic
account of
the historic and technological efforts to implement advanced
hypertext functionalities
. The
insights and principles will then be applied to analyze the current state of the World Wide
Web
3
.
While the tidiness of academic texts often misleadingly suggests the enduring nature of the
positions which they represent
4
, I think to have left a lot of open ends and loose threads for
critique and future research. The methodology and presentation of this dissertation have
been chosen in accordance with the demands of the field of study to which it contributes,
cf. [
312
]. While the paper version follows most technical conventions of a dissertation, the
online version of this text is enhanced with
hypertext functionalities
5
. Naturally, I hope that
3
In
semiotics
, this strategy is called a synchronic analysis.
4
"Seamlessness and sequential structures reinforce an impression of the ground having been covered, of all
the questions having been answered, of nothing important having been left out. Though it is a lie, closure suggests
mastery of the material through its control of form" [
95
, Syntagmatic Analysis].
5
Besides the uncountable bibliography
links
, this dissertation includes more than 2500 handcrafted
links
to
the
glossary
, to external Web sites and between sections.
CHAPTER 1. INTRODUCTION
18
the present work will be linked into rich relationships with other
resources
to disseminate
my findings.
In Austrian academic terms, this dissertation should be regarded as part of the studies "Han-
delswissenschaften" (Commerce) with a strong inclination towards an interdisciplinary ap-
proach of the subject, depending strongly on what is usually summarized in German as
"
Geisteswissenschaften
" (humanities). Accordingly, the
glossary
supplies key terminology
and definitions of
semiotics
, as well as economic and technical terms (as certain definitions
mean completely different things in the various disciplines). In British, or north American
academic terms, the paper might be considered as a melding of Business Administration,
Economics, Computer Science, Communication Studies (blending psychology, sociology,
semiotics
and linguistics), Composition Research (a slightly more focused hybrid of dis-
ciplines) and Media Theory. This paper was written in the English language for broader
accessibility. Nevertheless, the author has employed the scientific principles and tradition
of his native country, Austria.
Chapter 2
The Semiotic Approach
According to Andersen, a major contributor to the field of computer semiotics (see section
2.9
), "semiotics may be helpful in enhancing the interpretation of computer based signs
and creating understandable interaction" [
11
]. Before presenting my prolegomena for a
theory of hypertext semiotics, it will be necessary to describe the methods and practices,
definitions and limitations of the semiotic approach. What Andersen says about semiotic
interface engineering is basically true for the whole field of computer semiotics, including
hypertext semiotics:
"Semiotics is an abstraction of individual disciplines such as linguistics, art
theory, drama theory and film theory. Therefore it can serve as a common
language for transferring insights from one domain to another in a systemati-
cal way. This is useful in designing computer interfaces, since computers are
inherently multimedia where codes from these diverse fields meet and amal-
gamate in practice" [
11
].
For such a transfer to take place, there is a strong need to build a common base of departure
which is furnished with a shared methodology and terminology. In this chapter, I will
introduce the the basic semiotic principles and comment on their applicability on hypertext
theory.
2.1
Introduction to Classic Semiotics
After speaking of
language
as a system of signs that express ideas ("un système de signes
exprimant des idées"), the Swiss linguist Ferdinand de Saussure points out that
language
is only the most important of these systems and that a science that studies the life of signs
within society is well conceivable: "On peut donc concevoir une science qui édudie la vie
des signes au sein de la vie sociale" [
468
, p. 33]. This new science would be a part of social
psychology and consequently of general psychology:
"I shall call it
semiology
(from the Greek semeîon 'sign'). Semiology
would show what constitutes signs, what laws govern them. Since the science
does not yet exist, no one can say what it would be; but it has a right to exis-
tence, a place staked out in advance. Linguistics is only a part of the general
science of
semiology
; the laws discovered by
semiology
will be applicable to
19
CHAPTER 2. THE SEMIOTIC APPROACH
20
linguistics, and the latter will circumscribe a well-defined area within the mass
of anthropological facts."
1
Ferdinand de Saussure sharply distinguished the
diachronic
from the
synchronic
study of
language
.
2
In order to make his
synchronic
studies persuasive, Saussure was forced to draw
another sharp distinction. He argued for dividing
language
into three levels,
langage
,
by which he meant the human capacity to evolve structured communication
systems,
langue
,
what we think of as a language, such as English or French, and
parole
,
any individual speaker's particular use of the language.
Langue
is, according to Saussure, a self-contained whole and thus appropriate object of
synchronic
study; it refers to the system of rules and conventions which is independent of,
and pre-exists, individual users, cf. [
102
, p. 86], [
95
, Introduction]. Accordingly, Saussure
was chiefly interested in
langue
as an a-historical phenomenon.
Parole
, however, refers to
the use of this system in particular instances.
In contemporary
semiotics
, the distinction
langue
parole
has been generalized to a differ-
entiation between the semiotic system and its usage in specific (con-)texts: "The distinc-
tion is one between between code and message, structure and event or system and usage
(in specific texts or contexts)" [
95
, Introduction]. The system includes rules of use which
constrain but do not determine usage (this is analogous to Chomsky's distinction between
competence and performance, cf. [
97
]). To the traditional, Saussurean semiologist, what
matters most are the underlying structures and rules of a semiotic system as a whole rather
than specific performances or practices which are merely instances of its use, cf. [
102
, p.
86].
signifier
signified
Figure 2.1: The Saussurean sign model. Source: [
469
, p. 78].
While the Saussurean definition of
semiology
may have inspired much semiotic research
of this century, it falls short for many contemporary semioticians. For Saussure, the
sign
is
a composition of the
signifier
, /dog/
3
and the
signified
(our concept of a dog), as shown in
figure
2.1
. The real animal (in semiotic terminology, the
reference
) does not interest him as
1
[
468
]. English translation according to [
246
, p. 34-35] and [
95
].
2
A
diachronic
investigation traces the development or evolution of language, whereas a synchronic inquiry
examines language as a system, a network of relationships co-existing in the present. Saussure was reacting to the
Neogrammarians of his own day, linguists who contended that the only valid approach to the study of language was
a historical or
diachronic
approach. It is arguable that one extreme approach called forth the opposite extreme, for
an exclusive concern with the history of language was displaced by a systematic denial of this history's relevance
for an understanding of language, cf. [
469
,
102
,
95
,
154
].
3
It has become an academic convention among scholars of
semiotics
to use the forward-slash character to
emphasize a referral to the
signifier
. Thus, /dog/ refers to the word formed by the letters d, o, and g, as opposed to
the
meaning
of the word, "dog" and the actual animal (which is not taken into account by the Saussurean system),
cf. [
155
, p. 30f.].
CHAPTER 2. THE SEMIOTIC APPROACH
21
a linguist, cf. [
155
, p. 31].
4
In the context of natural language, Saussure stressed that there
is no necessary, or inherent connection between the
signifier
and the
signified
between
the sound or shape of a word and the concept to which it refers. The relationship is purely
conventional dependent on social and cultural conventions. This is not to suggest that
the form of a word is random, of course: While the words fish and man are
unmotivated
,
composita like fishermen illustrate the relative
arbitrariness
of
language
: the intra-linguistic
determination of
grammar
, cf. [
401
, p. 340].
Saussure emphasized the differences between signs. He argued that concepts are purely
differential and defined not by their positive content but negatively by their relations with
the other terms of the system: Their most precise characteristic is in being what the others
are not, cf. [
469
, p. 128]. The Saussurean model, with its emphasis on internal structures
within a sign system, can be seen as supporting the notion that
language
does not "reflect"
reality but rather constructs it, cf. [
545
].
Peirce's
semiotic triangle
has become the counter-model to Saussure's
dichotomy signified
/
signifier
. The American philosopher Charles Sanders Peirce (1839-1914), indeed, was the
other key figure in the classic development of
semiotics
:
"I am, as far as I know, a pioneer, or rather a backwoodsman, in the work of
clearing and opening up what I call
semeiotic
, that is, the doctrine of the es-
sential nature and fundamental varieties of possible
semiosis
. . . " [
416
, 5.488].
This
semiosis
is "an action, an influence, which is, or involves, a cooperation of three
subjects, such as a sign [
representamen
, MN], its
object
, and its
interpretant
, this thri-
relative influence not being in any way resolvable into action between pairs" [
416
, 5.484].
While Saussure's system needs an active
sender
of signals to make the semiotic process
work, Peirce's
semiosis
-trias can be applied to phenomena that have no
sender
, such as
natural symptoms of an illness that can be detected and interpreted by a medic.
Peirce was clearly fascinated by tripartite structures and made a phenomenological distinc-
tion in his three universal categories of firstness,
secondness
, and
thirdness
5
: "Everything
is something in itself; this Peirce calls firstness. We might call this initselfness. Everything
either actually or potentially reacts against, or opposes itself, to other things; this he calls
secondness
(over-againstness). Everything is, in some measure, intelligible, if only because
it can be related by me to something else" [
102
, p. 194]. Formally and abstractly defined,
thirdness
is betweenness or mediation.
6
In contrast to Saussure's self-contained two-termed model of
sign
(sign as an
arbitrary
correlation between
signifier
and
signified
), Peirce offered a
triadic
relation between the
representamen
, the
interpretant
and the
object
. Nöth has substituted these terms for more
intuitive terminology: the
sign vehicle
, the
sense
and the
reference
object.
The representamen (sign vehicle) is the broadest form in which the sign takes place.
In Peircean terminology, "a sign, or representamen
7
, is something which stands to
somebody for something in some respect or capacity" [
416
, 2.228].
4
Thus, many subcategories of today's semiotic studies (such as zoosemiotics, computer semiotics, and com-
putational semiotics) would have to remain outside the semiologic building that Saussure sketches, cf. [
469
,
396
].
5
"Such unfamiliar terms are relatively modest examples of Peircean coinages, and the complexity of his ter-
minology and style has been a factor in limiting the influence of a distinctively Peircean semiotics" [
95
, Signs].
6
One of Peirce's own favorite examples of
thirdness
or mediation is an act of giving. For him, giving exhibits
an irreducibly triadic structure or form that is, any attempt to break it down into a simpler affair loses its
meaning
. In any act of giving, there is a giver, a recipient, and a gift. One half of this act is divestiture (the giver
diverts herself of something she owns); the other half is appropriation (the recipient appropriates or comes to own
something new). But, in giving, these two dyads (giver and gift-as-divested; recipient and gift-as-acquired) are
integrally united. If the giver simply gets rid of her property and, a little while later, the recipient comes along and
finds it, we have two accidentally related dyads but no act of giving, cf. [
102
, p. 195]
7
He proposed this term because he believed that the English word "sign" and most, if not all, of its equivalents
in English and other languages were too closely tied to a mentalist understanding of the
sign
.
CHAPTER 2. THE SEMIOTIC APPROACH
22
The interpretant of a sign is the
sense
made of the
sign
. Apart from mental interpre-
tants (for example a concept in the Saussurean sense), the Peircean
pansemiotic
view
of the world also considers not-mental interpretants (for example, the plant turning
towards the sun), cf. [
102
, p. 171]. The
interpretant
should not be confused with
the
interpreter
: The
interpretant
is that in which a
sign
as such results, whereas the
interpreter
is a personal agent
8
who takes part in and presumably exerts control over
a process of interpretation.
9
The (reference) object is that to which the
sign
points. The difference between the rather
conceptual sense of the
sign
and the concrete
referent
can be shown in an easy ex-
ample: The
reference
semiotician includes the historic persons Charles S. Peirce,
Ferdinand de Saussure, Charles Morris, Roland Barthes, and the like; these are some
of the beings to whom this
sign
refers. In contrast, the
meaning
of semiotician is any-
one who investigates, especially in a self-conscious way, the nature and properties of
signs.
10
What might seem like the same triad already offered by Platon and Aristoteles as semainon,
semainomenon and pragma, is more complex. The
triadic
model of the sign and the concept
of the
semiosis
was expanded by Umberto Eco to designate the process by which a culture
produces signs and/or attributes
meaning
to signs. Eco and others criticize the simpli-
fied variants of Peirce's triad which keep reappearing under the name "
semiotic triangle
",
cf. [
368
,
406
]. While there seems to be a broad consensus on the triangular shape of the
model, the names of the three poles are not only terminological differences: They represent
different ways of seeing the process as a whole. Nadin calls this lack of consistent termi-
nology the post-Morris
11
syndrome, "an intellectual disaster from which semiotics does not
seem to recover. The consequences are obvious: the outcome of applied semiotics rarely
justifies expectation", cf. [
375
].
Floyd Merrell thinks that the triangle shape, too, is a misinterpretation of Peirce's sign triad.
According to him, the triangle has been chosen because it is pleasing to the eye:
"Its shape is quite familiar to any elementary schooler who has studied a few
geometrical figures, and it coincides nicely with our penchant for Euclideaniz-
ing the world. But it is not genuinly triadic. It consists of a set of three binary
relations. . . " [
353
, p. 135]
Like Marty and Marty [
341
, p. 100] before, he proposes a model (figure
2.2
) that ties each
sign component to the other two, and, in addition, to the relation between them, cf. [
401
, p.
140].
It is important to note that Saussure's term, "
semiology
" is sometimes used to refer to the
Saussurean tradition, whilst "
semeiotic
" (or "semeiotics") sometimes refers to the Peircean
8
In computer semiotics, the
interpreter
can be a machine or a process, e.g. a compiler or, of course, an
"interpreter" in the technical sense: "There are many interpreters at work in the design and use of computer
systems. In fact, systems development mainly consists in writing and reading, so semiotics may not only shed
light over the interface, but also inform technical concepts such as program verification, program specification,
compilation, etc." [
9
, p. 11].
9
The
interpretant
is not any result generated by a
sign
. Something functioning as a sign might produce effects
unrelated to itself as a sign: "For example, a fire indicating the presence of survivors of an airplane crash might
set a forest ablaze. The forest fire would be an incidental result and thus not an
interpretant
of the sign calling for
help (or indicating the whereabouts of the survivors)" [
102
, p. 121].
10
Imagine Saussure's neighbor in Geneva at the turn of the last century: Even if she did not know the
meaning
of the word semiotician, she would still know Monsieur Saussure. It is also possible for a
sign
to have a
meaning
but no reference: for example, the "unicorn" has no real animal to whom it points, cf. [
155
, p. 29].
11
Besides presenting a simplified
semiotic triangle
model, Morris provided useful extensions to the Peircean
theory. His division of
semiotics
into the following three branches has been widely accepted:
semantics
(the
meaning
of signs; the relationship of signs to what they stand for); syntactics or syntax (the structural relations
between signs); and pragmatics (the ways in which signs are used and interpreted), cf. [
368
, p. 6-7]; [
396
, p. 50].
CHAPTER 2. THE SEMIOTIC APPROACH
23
R
I
I
O
R: Representamen (sign vehicle)
I : Interpretant (sense)
O: Object (reference object, referent)
Figure 2.2: The Peircean sign model. Source: [
353
, p. 136].
tradition. Colapietro gives a most concise differentiation of the two classic semiotic ap-
proaches:
"For Peirce, anything properly designated as a sign has an
object
; more over,
this object is conceived in such a way that it can constrain or guide the process
of
semiosis
or sign generation. In other words, whereas Saussure's view of
language
as a self-contained system of formal differences suggests something
free-floating, Peirce's conception of
semiosis
suggests something firmly rooted
in an objective world. In Saussure's
semiology
, the link between language and
reality is severed or, at best, extremely attenuated; in Peirce's semiotic, the
connection between signs and objects is commonsensically assumed" [
102
, p.
151].
While the European tradition of
semiology
evolved from the science of linguistics, the
American tradition of
semeiotic
is seen as part of a much more comprehensive philosophi-
cal system. The work of Louis Hjelmslev, Roland Barthes, Claude Lévi-Strauss, Julia Kris-
teva, Christian Metz and Jean Baudrillard follows in the semiological tradition of Saussure
whilst that of Charles W. Morris, Ivor A. Richards, Charles K. Ogden and Thomas Se-
beok is in the
semeiotic
tradition of Peirce. Nöth complements these two traditions with
a rich study of the history of concepts (Begriffsgeschichte) of the terms that derive from
Greek
µ
~io
and
~
µ
, which have been used in philosophy and medicine since the late
antiquity.
Many other leading semioticians (e.g. Umberto Eco) have also helped bridging these two
traditions since the late 1960s. Today, the term
semiotics
is primarily used as an umbrella
term to embrace the whole field, [
401
, p. 3], [
95
, Introduction]. The same is true for this
dissertation.
2.2
From Symbolic Forms to (Post-)Structuralist Semiotics
Even if Ernst Cassirer has long been recognized as a semiotician ante litteram and a thinker
worthy of the semiotic Olymp
12
, his influence on today's semiotic viewpoints has been
neglected, cf. [
342
]. There are only faint historical connections between Peirce and Cas-
sirer
13
, and "there is also no hint that Cassirer would have appreciated Peirce's philosophy
12
As pointed out to me by Jeff Bernard, Institute for Socio-Semiotic Studies (ISSS), in a personal interview on
July 19th, 2001.
13
"One of the few relations that can be drawn between Cassirer and [American Pragmatism] is constituted by
the Cassirer-disciple Edgar Wind [. . . ] another link between Cassirer and Peircean philosophy can be seen in
the work of Susanne K. Langer (1895-1985) who is often considered as deeply influenced by Cassirer" [
342
, p.
332-333].
CHAPTER 2. THE SEMIOTIC APPROACH
24
even if he had known about it" [
342
, p. 333]. Yet, there is a consistent line of thought on
Symbolic Forms from Cassirer to Langer and Lévi-Strauss that intersects with the semiotic
path at several points.
Cassirer's definition of man as an animal symbolicum that lives in a symbolic universe is
inherent in any discussion about human thinking, understanding, culture, and communcia-
tion, including telecommunication and
hypertext
(see section
3.5
): "No longer in a merely
physical universe, man lives in a symbolic universe. Language, myth, art and religion are
parts of this universe. They are varied threads which weave the symbolic net, the tangled
web of human experience" [
90
, p. 25]. Peter Marx insists that Cassirer cannot be clearly
categorized into either the European or the American semiotic tradition, as his symbolic
forms (which must not be confused with the Peircean definition of the
symbol
) relate to a
broader cultural concept. Marx [
342
] also points out that Cassirer deals with two different
questions:
"On the one hand, the philosophy of symbolic forms is concerned with the
questions of knowledge, it is concerned with the matter that is called Erkennt-
nistheorie [. . . ] On the other hand, Cassirer tries to develop a new science of
culture and this project is very close to the project of European structuralism"
[
342
, p. 337].
Susanne Langer's work [
306
,
305
] is based on the findings of Ernst Cassirer [
89
], and
inherent in the work of Claude Lévi-Strauss [
315
,
316
].
14
Langer remarks that, in modern science, the object of study is no more the center of atten-
tion because it has been displaced by measuring and control technology, displays, visual
representation
material:
Indices
have taken the place of the cause, and observation has
become almost entirely indirect: "The sense-data on which the propositions of modern sci-
ence rest are, for the most part, little photographic spots and blurs, or inky curved lines on
paper. The problem of observation is all but eclipsed by the problem of
meaning
.
15
And the
triumph of empiricism in science is jeopardized by the surprising truth that our sense-data
are primarily symbols" [
306
, p. 20-21]. Langer makes clear that mathematical construc-
tions are
symbols
as well, as mathematics does not need to refer to an external world at all:
its
signifieds
are indisputably concepts and mathematics is a system of relations. However,
her investigations of the
symbol
do not draw upon that discipline's technical conventions,
cf. [
306
, p. 18ff.]. These observations, of course, are based on Cassirer's statement that
all truly strict and exact thought is sustained by the symbolics and semiotics on which it is
based.
16
The other great influence for Langer comes from Alfred North Whitehead's exposition of
the two pure modes of perception [
543
], one of which is essentially perception in space,
and the other in time. Whitehead designates them presentational immediacy and causal
efficacy.
17
Presentational immediacy is perception of what is immediately present without
14
The inspiration for this line of thought are owed to Klaus Hamberger's lecture on social structures and sym-
bolic forms which included a thorough discussion of its epistemological sources (University of Vienna, summer
term of 2001).
15
In her longing to implement the concept of transformation (i.e. the theoretical basis for any virtual reality)
into epistemology, Langer uses the term analogy in her theory of isomorphism. The concept of transformation
between structures was also used by Lévi-Strauss, cf. [
316
]. Chandler's notion that "signifying systems impose
digital order on what we often experience as a dynamic and seamless flux" [
95
, Signs] still draws on the Kantian
evolution of cognition from "Mannigfaltigkeit", via "Synthesis", to "Einheit", cf. [
267
, B 102-104].
16
"Every law of nature assumes for our thinking the form of a universal formula and a formula can be ex-
pressed only by a combination of universal and specific signs. Without the universal signs provided by arithmetic
and algebra, no special relation in physics, no special law of nature would be expressible. It is, as it were, the
fundamental principle of cognition that the universal can be perceived only in the particular, while the particular
can be thought only in reference to the universal" [
89
, p. 86].
17
It is important to note that these two modes rarely occur in their "pure" form. Neither pure mode in itself can
be called consciousness as we know it, but these two pure modes together constitute symbolic reference.
CHAPTER 2. THE SEMIOTIC APPROACH
25
reference
to anything in the past or anything that may lie in the future (neither memory
nor awareness of potentiality). Causal efficacy is the pure mode of inheritance of feeling
from past data and the precondition for awareness of future potentialities. In accordance
to Whitehead's
dichotomy
, Langer distinguishes between rational discursive language and
presentational language. To begin with the discursive function of symbolism, we see a
linear construction which is limited to sequencing. Its nature is very exacting and precise,
as it is used in science.
"Language in the strict sense is essentially discursive; it has permanent
units of meaning which are combinable into larger units; it has fixed equiva-
lences that make definition and translation possible [. . . ] The meanings given
through language are successively understood, and gathered into a whole by
the process called
discourse
. . . " [
306
, p. 89].
Discursiveness in this context is similar to sequentiality: Words cannot be piled one upon
the other, neither can they be arranged arbitrarily into a sentence (they have to follow a pre-
defined
grammar
); it takes time to form (and listen to) each word of a sentence and only
once you have heard the last word of a sentence you can be sure of its
meaning
. Langer
thought that, even if they are nested, we have to string our ideas in order to communicate
them to others in a language; like clothes that are draped around a body, but hanging out
to dry on a clothes-line. You place one piece of language at a time into a straight line; at
the end of the process the parts add up to a whole argument or proposition, cf. [
307
, p.
88]. The argument of hypertext is that ideas do not have to be arranged on an infinitely
long clothes-line. In fact,
hypertext
represents variable structure that permits an interlinked
presentation of ideas, as outlined in section
3.5
. The act of navigation means a linearization
of those
nodes
that the hypertext user chooses to read along a personal thread that is laid
upon the network.
For Langer, denotation and connotation are central to discursive symbolism, cf. [
306
], see
section
2.2
. While denotation makes reference to a specific object or person, connotation
brings up a concept of a name.
18
While denotation is intended, connotation is inferred.
Therefore, Langer believes that denotation is the essence of language, because its use is
deliberate, and directly relates the concept to the real thing. The
syntax
and
grammar
of
discursive language tie together
symbols
into complex
thoughts
, allowing intricate ideas to
be communicated.
In contrast with the rather strict discursive form, presentational symbolism is simultaneous
in its nature and open to broad interpretations. Using art as an example, Langer writes,
"visual forms lines, colors, proportions, etc. are just as capable of articulation, i.e., of
complex combinations of words" [
306
, p. 93]. This is because the relationship that exists
in a picture, between an object and
image
, is similar to the relation of a word to its object.
They evoke a presence rather than defining a present fact.
With a basic understanding of the two functions of
symbols
, Langer presents her case as
to their role in the origin of
language
. Although
language
is clearly one of the keys to
human survival, it is not well explained as derived from such motivations. The psycho-
genetic theory of language contends that human language developed from the simple signs
that animals use to communicate; in time, these mechanisms became more complex and
developed into
symbols
. Langer rejects this theory and argues that human rationality is
much too varied and distinct from that of animals to have developed from basic needs for
survival. She believes that humans have had, since their earliest days, a peculiar need not
18
One is reminded here of Wittgenstein's concept of names: "One name stands for one thing, and another for
another thing, and they are connected together. And so the whole, like a living picture, presents the atomic fact
[Sachverhalt, MN]" [
548
, 4.0311]. For an overlook of the use and reference of names in Wittgenstein's Tractatus,
cf. [
249
].
CHAPTER 2. THE SEMIOTIC APPROACH
26
found in other animals: to express their inner life. The mere expression of ideas seems to
be a uniquely human exercise, as animals do not see any need for crying, laughing, super-
stition, rituals and scientific ingeniousness. Presentational language did not develop from
a need for something in addition to self preservation; but rather, Langer believes that the
need for expression gave birth to discursive language.
19
Though
language
may be the most important natural outcome of this symbolic process, the
other transformations of experience in the human mind have quite different overt endings
and end in acts that are neither practical nor communicative, though they may be both
effective and communal, e.g. rites and art, cf. [
307
, p. 49-51].
Signs (as
symbols
not as symptoms) help us to refer to objects in absentia. Rather than
aliquid pro aliquo,
symbols
, for Langer, serve as vehicles for the conceptions of objects,
cf. [
307
, p. 39 and 69]. Symbolization precedes the actual act of thinking (rather than being
the act itself, as formulated by Ritchie [
444
, p. 238]). Thus, symbolic transformation of
sensual information is an inherent necessity, just like eating, looking and moving. It is a
fundamental, continuous process of the human mind, a fountain of ideas.
20
This is the point
where Langer's theory coincides with structuralism, especially Lacan and Lévi-Strauss,
who have employed Saussure's structural analysis with psychoanalysis and ethnology.
Structuralism drew heavily on linguistic concepts, partly because of the influence of Saus-
sure and because linguistics was a more established discipline than the study of other sign
systems, cf. [
131
]. Lévi-Strauss noted that "language is the semiotic system par excellence;
it cannot but signify, and exists only through signification" [
317
, p. 48]. Accordingly, the
structuralists adopted
language
as their model in exploring a much wider range of social
phenomena: Lévi-Strauss for myth, kinship rules and totemism; Lacan for the unconscious;
Foucault for disciplining systems of power; Barthes and Greimas for the "
grammar
" of
narrative. Structuralism is an analytical method which has been employed by many semi-
oticians and which is based on Saussure's linguistic model of binary oppositions.
21
According to Saussure,
meaning
arises from two kinds of differences between
signifiers
:
syntagmatic (concerning positioning) and paradigmatic (concerning substitution). Whilst
syntagmatic relations are possibilities of combination (the dimension of "and"), paradig-
matic relations are functional contrasts they involve differentiation (the dimension of
"or").
On the syntagmatic axis, a word is assigned
meaning
by the words surrounding it.
22
A
19
Freud has shown that human behavior is not a mere strategy to provide food, but also a language, as every
movement is, at the same time, a gesture, cf. [
300
, vol. 1, p. 115ff.], [
307
, p. 59]. Consequently, Jacques Lacan
drew his famous conclusion that the unconscious itself is structured like a
language
(see section
3.5
).
20
"Language, in its literal capacity, is a stiff and conventional medium, un-adapted to the expression of gen-
uinely new ideas, which usually have to break in upon the mind through some great and bewildering metaphor"
[
306
, p. 45f.].
Metaphors
are the next step in understanding the development of symbolic language.
Metaphors
expand the literal meanings of symbols. In this stretching of meaning, is the abstraction of presentational sym-
bolism. Symbolic transformation is the process in which man transforms experience into new ideas.
Metaphors
are the means by which new abstractions of symbolism are born;
metaphors
are the key to symbolic transforma-
tion; they dictate the laws of
language
. Both the presentational and discursive functions of symbols are expanded
through metaphorical means, cf. [
371
].
21
"People have believed in the fundamental character of binary oppositions since at least classical times. [. . . ]
As for methodologies, Saussure's theories constituted a starting point for the development of various structuralist
methodologies for analyzing texts and social practices. These have been very widely employed in the analysis of
a host of cultural phenomena. [. . . ] Semiotics is probably best-known as an approach to textual analysis, and in
this form it is characterized by a concern with structural analysis." [
95
, Introduction]. As a metatheory, a theory
about theories, "structuralism insists upon the necessity to conceive any object of inquiry as a structure" [
102
, p.
187]; cf. [
289
, p. 197f.].
22
The syntagm "waxing hot" was used for a visual punch by the American artist Bruce Nauman: In the se-
ries The Artist at Work, he shows snapshot-like situations of the artist's activities, where the boundary between
everyday doings and the practice of art is undefined. In the photograph Waxing Hot, his hands are seen at work,
polishing 3 cast letters, H, O and T, which are standing on the floor. The pun is realized by means of visualization.
The oscillation of the word hot between the role of an adverb in the syntagm (waxing how? hot!) and an object
(waxing what? H, O, and T.) formulates a tension on the syntagmatic axis.
CHAPTER 2. THE SEMIOTIC APPROACH
27
syntagm
is an orderly combination of interacting
signifiers
which forms a meaningful
whole within a
text
sometimes, following Saussure, called a syntagmatic chain (see figure
2.3
). Such combinations are made within a framework of syntactic rules and conventions
signifier
signified
signifier
signified
signifier
signified
Figure 2.3: Syntagmatic chain. Source: [
469
, p. 137].
(both explicit and inexplicit). In written language, a sentence, for instance, is a
syntagm
of words; so too are paragraphs and chapters. Saussure himself noted that visual
signifiers
(he instanced nautical flags) can exploit more than one dimension simultaneously, while
auditory
signifiers
are presented one after another on a time line, cf. [
469
, p. 82]. Chandler
expands this concept of multiple syntagmatic dimensions:
"Syntagms are often defined as 'sequential' (and thus temporal as in
speech and music), but they can represent spatial relationships. [. . . ] Spatial
syntagmatic relations are found in drawing, painting and photography. Many
semiotic systems such as drama, cinema, television and the world wide web
include both spatial and temporal syntagms" [
95
, Paradigms and Syntagms].
I will return to spatial syntagmatic relations in section
3.5
on hypertext semiotics.
Paradigmatic analysis involves comparing and contrasting each of the
signifiers
in a
text
with absent
signifiers
which in similar circumstances might have been chosen, and con-
sidering the significance of the choices made. Saussure had actually called the words on
this axis "associative" relations, cf. [
469
, p. 147ff.], but Roman Jakobson's term is now
used.
23
Paradigmatic analysis can be applied at any semiotic level, from the choice of a
particular word,
image
or sound to the level of the choice of style, genre or medium. The
use of one
signifier
rather than another from the same
paradigm
is based on factors such
as technical constraints,
code
, convention, connotation, style, rhetorical purpose and the
limitations of the individual's own repertoire. Paradigmatic analysis has been used to trace
binary oppositions in visual
images
.
24
The distinction between the syntagmatic and the paradigmatic axes is a key one in struc-
turalist semiotic analysis. The value of a
sign
is determined by both its paradigmatic and
its syntagmatic relations (see figure
2.8
). Syntagms and
paradigms
provide a structural
context within which signs make sense; they are the structural forms through which signs
are organized into
codes
. Roland Barthes [
35
] outlined the paradigmatic and syntagmatic
23
"Saussure's notion of 'associative' relations was broader and less formal than what is normally meant by
'paradigmatic' relations. He referred to 'mental association' and included perceived similarities in form (e.g.
homophones) or meaning (e.g. synonyms)" [
95
, Paradigms and Syntagms]. Such similarities were diverse and
ranged from strong to slight, and might refer to only part of a word (such as a shared prefix or suffix). He noted
that there was no end (or commonly agreed order) to such associations. In the syntagm "waxing hot", introduced
in footnote
22
, the associative axis for the word "waxing" might include:
waxing
rewaxing
polishing
ordering
waxwing
dewaxing
applying wax
teaching
vaccine
waxer
augmentation
wearing
Maxim
waxed
crescendo
having
mixing
radical
analogy
suffix
sound
Saussure noted that there was no end (or commonly agreed order) to such associations, cf. [
469
, p. 150ff.]
24
Jean-Marie Floch compares and contrasts the company
logos
of IBM and Apple, revealing their differences
to be based on a series of associated binary oppositions, cf. [
172
].
CHAPTER 2. THE SEMIOTIC APPROACH
28
elements of the "garment system".
25
Jonathan Culler analyzed Restaurant menus in similar
terms.
26
Chandler notes that formal
semiotics
is difficult to disentangle from structuralism [
94
,
95
].
However, he also quotes Deborah Cameron who suggests that structuralism is merely "a
method you can use" in
semiotics
, cf. [
84
, p. 25]. John Hartley describes structuralism
as "an analytical or theoretical enterprise, dedicated to the systematic elaboration of the
rules and constraints that work [. . . ] to make the generation of meanings possible" [
408
, p.
302]. Teresa de Lauretis describes the movement away from structuralist semiotics which
began in the 1970s.
27
Contemporary social semiotics has moved beyond the structuralist
concern with the internal relations of parts within a self-contained system and is also some-
times allied with a Marxist approach which tends to stress the role of ideology, cf. [
153
, p.
168-179], [
210
]. The other emphasis in "poststructuralist semiotic theory" is a
semiotics
focused on the subjective aspects of signification and strongly influenced by Lacanian psy-
choanalysis, where
meaning
is construed as a subject-effect (the subject being an effect of
the
signifier
), cf. [
126
, p. 166-67].
2.3
Definitions and Limitations of Semiotics
Semiotics
is not yet widely institutionalized as an academic discipline. It is still under
discussion whether
semiotics
is a science, a discipline, or a field of study. Some commen-
tators adopt Morris's definition of
semiotics
(in the spirit of Saussure) as "the science of
signs" [
368
, p. 1-2] For others, the term science is misleading.
28
Hodge/Kress think that "semiotics offers the promise of a systematic, comprehensive and
coherent study of communications phenomena as a whole, not just instances of it" [
227
, p.
1]. For John Fiske and John Hartley, the central concerns of semiotics are "the relationship
between a sign and its meaning; and the way signs are combined into codes" [
170
, p. 37].
One of the broadest definitions of
semiotics
is that of Umberto Eco, who states that "semi-
otics is concerned with everything that can be taken as a sign" [
154
, p. 7]. Accordingly,
semiotics
is often criticized as "imperialistic", since some semioticians appear to regard it
as concerned with, and applicable to, anything and everything, trespassing on almost ev-
ery academic discipline, cf. [
95
]. David Sless sees
semiotics
as consolidating, rather than
invading.
29
25
The paradigmatic elements are the items which cannot be worn at the same time on the same part of the body
(such as hats, trousers, shoes). The syntagmatic dimension is the juxtaposition of different elements at the same
time in a complete ensemble from hat to shoes, cf. [
35
].
26
"In the food system [. . . ] one defines on the syntagmatic axis the combinations of courses which can make
up meals of various sorts; and each course or slot can be filled by one of a number of dishes which are in
paradigmatic contrast with one another (one wouldn't combine roast beef and lamb chops in a single meal; they
would be alternatives on any menu)" [
113
, p. 104].
27
"In the last decade or so, semiotics has undergone a shift of its theoretical gears: a shift away from the clas-
sification of sign systems - their basic units, their levels of structural organization - and towards the exploration
of the modes of production of signs and meanings, the ways in which systems and codes are used, transformed
or transgressed in social practice. While formerly the emphasis was on studying sign systems (
language
, litera-
ture, cinema, architecture, music, etc.), conceived of as mechanisms that generate messages, what is now being
examined is the work performed through them. It is this work or activity which constitutes and/or transforms the
codes, at the same time as it constitutes and transforms the individuals using the codes, performing the work; the
individuals who are, therefore, the subjects of
semiosis
" [
126
, p. 166-67].
28
"Semiotics is not, never has been, and seems unlikely ever to be, an academic discipline in its own right. It
is now widely regarded primarily as one mode of analysis amongst others rather than as a 'science' of cultural
forms" [
95
, Criticisms]. Yet, "it is at least a focus of enquiry, with a central concern for meaning-making practices
which conventional academic disciplines treat as peripheral" [
95
, Strenghts].
29
"We consult linguists to find out about language, art historians or critics to find out about paintings, and
anthropologists to find out how people in different societies
signal
to each other through gesture, dress or decora-
tion. But if we want to know what all these different things have in common then we need to find someone with a
semiotic point of view, a vantage point from which to survey our world" [
493
, p. 1].
CHAPTER 2. THE SEMIOTIC APPROACH
29
Semiotics
involves the study not only of what we refer to as "signs" in everyday
speech
,
but of anything which "stands for" something else aliquid stat pro aliquo, such as words,
images
, sounds, gestures and objects. For the linguist Saussure, "sémiologie" was a science
which studies the role of signs as part of social life and linguistics was one of its branches;
cf. [
468
], [
469
, p. 19]. For the philosopher Charles Peirce, "
semeiotic
" was the formal
doctrine of signs which was closely related to Logic. For him, "a sign [. . . ] is something
which stands to somebody for something in some respect or capacity" [
416
, 2.227-8]. His
pansemiotic
view of the world led Peirce so far as to declare that "every thought is a sign";
indeed he claimed that: "The entire universe [. . . ] is perfused with signs, if it is not com-
posed exclusively of signs" [
416
, 5.448], cf. [
102
, p. 154]. Contemporary semioticians
study signs not in isolation but as part of semiotic "sign systems" (such as a medium or
genre) and in relation to a
code
.
Semioticians do not always make explicit the limitations of their techniques, and
semiotics
is sometimes uncritically presented as a general-purpose tool: "Saussurean semiotics is
based on a linguistic model but not everyone agrees that it is productive to treat photog-
raphy and film, for instance, as 'languages"' [
95
]. While it cannot be the purpose of this
paper to contribute to this discussion, it shall be noted that according to my fields of ex-
pertise I will make use of other approaches apart from
semiotics
to tackle the problems
of
hypermedia
.
Today, Seboek's Encyclopedia [
477
] and Nöth's Handbook of Semiotics [
396
] have be-
come the standard works for the history, and methodology of this field of study.
Danesi's work [
119
] might achieve an equal status as an interdisciplinary approach to
semiotics
and the related fields of media studies and communication theory. Tradition-
ally, communication theory has been seen as one of
Semiotics
' major rivals in the struggle
for truth finding: "Communication theorists generally focus more on the study of message-
making as a process, whereas semioticians center their attention more on what a message
means and on how it creates meaning" [
118
], quoted in [
105
]. Marcel Danesi implies that
both communication science and
semiotics
are systematic studies of signs. These defini-
tions and distinctions about
communication
science and
semiotics
seem to have captured
the interest of Carlos Colón. In his article Communication Science vs. Semiotics, Colón
goes back to the term information as their common destiny.
30
In defining
communication
"as the transfer of information from a source to a receiver" and keeping in mind Danesi's
statement that semiotics studies signification first and communication second, he arrives
at a point where he sees "more a stitch than a line between communication science and
semiotics" [
105
].
Insofar as
semiotics
tends to focus on
synchronic
rather than
diachronic
analysis (as it
does in Saussurean
semiotics
), it underplays the dynamic nature of media conventions (for
instance, television conventions change fairly rapidly compared to conventions for written
English). It can also underplay dynamic changes in the cultural myths which signification
both alludes to and helps to shape, cf. [
95
, Criticisms].
Besides a few "full-time semioticians", those involved in
semiotics
include linguists, philoso-
phers, psychologists, sociologists, anthropologists, literary,
aesthetic
and media theorists,
psychoanalysts, art historians and educationalists.
Semiotics
has changed over time, since
semioticians have sought to remedy weaknesses in early semiotic approaches: Yet, it is only
fair to note that much of the criticism of semiotics has taken the form of self-criticism by
30
"Information is the core element of communication science and probably of semiotics as well. I consider
information to be the raw material for message construction and the creation of meaning. Signs are a collection
of bits and pieces of information. Information is what we decipher from signs. Notice that decoding has to be
performed because some sort of coding is always a part of the 'creation' of a
sign
. Even iconic signs which are
'a direct representation of a referent' as defined by Danesi, have to be encoded in order to make them deliverable
through any given medium" [
105
].
CHAPTER 2. THE SEMIOTIC APPROACH
30
those within the field. Morris and Kristeva have both criticized the auto-reflexivity of semi-
otics: Morris describes
semiotics
as a discipline that wants to be science and meta-science
the same time. For Kristeva,
semiotics
tries to be a meta-language (a science of
text
) while
also being an object-language (a signification practice), cf. [
401
, p. XII]. Nöth [
401
] argues
that this question depends mainly on the underlying scientific theory: And which concept
of science could be more applicable than Peirce's? Thus, the circulus vitiosus, for him,
has long become a circulus virtuosus of consequent semiotic self-reflection that establishes
semiotics
as a scientific activity.
31
The theoretical literature of
semiotics
reflects a constant
attempt by many semioticians to grapple with the implications of new theories for their
framing of the semiotic enterprise" [
95
, Criticisms]. This elaboration also showed its result
in a transformation of terminology, which means that, even with the most basic semiotic
terms, there are multiple definitions, cf. [
154
,
477
,
396
,
401
,
95
]. In this dissertation, I have
tried to define most controversial terminology in the
glossary
of this dissertation.
Recent semiotic work in the commercial and economic context, especially papers submitted
to the Vienna University of Economics and Business Administration (
WU Wien
) concen-
trate on the fields of Marketing, Advertising, and Business English [
110
,
123
,
182
,
314
,
364
,
508
,
523
].
2.4
Sign
While logical reasoning à la "cogito, ergo sum" seems to demand complex brain-work, the
association of a
signifier
with its
meaning
(resp. of a
sign vehicle
with its
sense
) looks like
a rather automatic process
32
, cf. [
154
,
396
,
401
]. Many people will not even be aware of
the fact that there is difference between the
signifier
/dog/ and its
signified
. Saussure takes
this as reason enough to speak of the
sign
as their sum. From the experience that a child
will automatically say "dog" whenever it sees a drawing of one, we can see that the process
works in both directions, as depicted in figure
2.1
. In the semiotic terminology, a
sign
is
to be distinguished from what we often call a "sign" in colloquial language: the
signal
and
the
sign vehicle
.
33
As mentioned before, Saussure sees the
signifier
and
signified
as
relata
in the sign-relation.
Unlike Peirce's
triadic
model of the sign (see fig.
2.2
), Saussure's model excludes reference
to an
object
in the world (the actual animal known as "dog"). Saussure's model includes
only a mental concept and a "sound-image"
34
. His conception of
meaning
was purely
structural the
meaning
of signs was seen as lying in their (syntagmatic and paradigmatic)
relation to each other.
31
"Denn die konsequente Anwendung der Semiotik auf sich selbst läßt die Theorie von den Zeichenprozessen
zu einer stets selbstkritischen wissenschaftlichen Tätigkeit werden, welche somit ihre dynamische Weiterentwick-
lung zu sichern in der Lage ist" [
401
, p. XII].
32
"In signs, one sees an advantage for discovery that is greatest when they express the exact nature of a thing
briefly and, as it were, picture it; indeed, the labor of thought is wonderfully diminished" Gottfried Wilhelm von
Leibnitz, cited in [
407
].
33
The failure to due so can lead to difficulties when working in the border-field of
semiotics
.
Andreas
Dieberger's dissertation "Navigation in Textual Virtual Environments using a City Metaphor" is one of many
valuable approaches to the same problems that I intend to tackle with my "Hypertext Semiotics". Dieberger,
Nielsen, Bieber and other authors will be heavily quoted in my reflections on hypertext navigation (I could not
agree more to Lunefeld's charming assertion that "we love to correct those who treat closest to our own path, for
who else would listen?" [
325
, p. xviii]). However, it should be noted that those approaches which ignore the key
results of decades of semiotic investigations (whilst using the terms
sign
,
text
,
communication
,
code
,
metaphors
,
etc.) are truly not state of the art user-centered research. Dieberger, for example, defines the
sign
as a general
form of an information provider: "Signs can be read. They can be attached to facades or walls or can be located
on their own in the environment. Some signs are well visible all day, others are unreadable in the dark. Signs can
change their appearance repeatedly these signs provide rapidly changing information, like departures in a train
station" [
142
, p. 57].
34
For the linguist Saussure, the "image acoustique" is a psychological imprint of the sound of the word.
CHAPTER 2. THE SEMIOTIC APPROACH
31
Lacan was one of the main engineers in the conversion of the Saussurean sign model,
appropriating Saussure through Jakobson to Freud: He related
metaphor
to Verdichtung
(condensation) and
metonymy
to Verschiebung (displacement).
35
Saussure had stressed
that the
signifier
and the
signified
were as inseparable as the two sides of a piece of paper
(see figure
2.1
), they were intimately linked in the mind by an associative link, and wholly
interdependent, neither pre-existing the other, cf. [
469
, p. 77]. Lacan substituted Saus-
sure's model of the
sign
in the form of the quasi-algebraic formula
S
s
and a "variation" of
the diagram that describes the relation between the
signifier
and the
signified
(figure
2.4
).
In the formula, the
signifier
(represented by a capital "S") is now placed over the
signified
(a lower case and italicized "s"), separated by a horizontal "bar". Lacan's diagram shows
the doors of a public restroom, the crossroads which forcefully separate male from female
necessities. What he means is that we think in categories that we have not chosen our-
selves: While entering the symbolic order of language enables us to
express
ourselves, we
have to subordinate under its rules of
grammar
and social habits, cf. [
300
]. The parlêtre,
or "speaking-being" cannot brabble whatever he feels like without having to fear social
consequences, e.g. when using rude or non-sense words. Barry notes that "symbols in
the form of written language may be inaccessible to those outside the culture, yet each
symbol within that culture of necessity carries a history of representation, association and
relation" [
34
, p. 119].
DAMES
HOMMES
Figure 2.4: Lacan's signifier/signified model. Source: [
300
, vol. 2, p. 24].
Throughout his life, Peirce tried to construct a comprehensive and systematic classification
of signs. Peirce reduced his ten trichotomies of signs, which would have sufficed to furnish
us with 3
10
¡
59049 sign classes, to sixty-six not completely independent classes of signs,
cf. [
416
, vol. 2, p. 330]. These ten trichotomies are not to be confused with his ten not
completely independent classes of signs described hereafter. The latter originate from only
three trichotomies.
This threefold consideration "is at the center of what is perhaps his most successful attempt
at such a classification" [
102
, p. 132]. It is based on the very nature of a
sign
, as defined
by Peirce: anything (thus, something in itself) standing for some other (called its
object
)
and giving rise to an
interpretant
. Accordingly, signs might be considered in themselves,
or in relationship to their
object
, or finally in relationship to their
interpretants
. These three
35
Simply put, his theory goes from the assumption of a fundamentally split subject and thus comes up with a
model of subjectivity that grounds itself on a constitutive lack rather than wholeness. According to Lacan, the
human being is entangled in three registers, which he calls the symbolic, the imaginary, and the real. Whereas the
imaginary constitutes the (perceptual) realm of the ego, the register that accounts for a (however illusive) notion
of wholeness and autonomy, the symbolic is the field of mediation that works according to a differential logic.
Whereas the imaginary constantly tries to "heal" the lack-of-being of the subject, the symbolic accepts castration.
The human subject is thus doubly split: on the imaginary level between the ego and its mirror
image
, while on
the symbolic level it is language and the inscription into a specific socio-cultural reality and its rules that bars the
subject from any unity. Thus, this forever lost unity belongs to the third register: the real, which is simply that
which eludes any representation, imaginary or symbolic. Because of this lack, the subject, which, according to
Lacan, is an effect of the
signifier
, aims at recreating that lost unity. The 'strategy' of desire emerges as a result
of the subject's separation from the real and the "means" by which the subject tries to catch up with this real, lost
unity again, cf. [
300
,
225
,
239
].
CHAPTER 2. THE SEMIOTIC APPROACH
32
considerations yield three trichotomies: a
sign
considered in itself might be a quality and
thus a
qualisign
, an individual thing or event, thus a
sinsign
, or a law, hence a
legisign
;
the relation of a
sign
to its
object
can be an
iconic
or
indexical
or symbolical; the relation
of the
sign
to its
interpretant
is a
rheme
or
dicent
or argument, cf. [
102
, p. 132]. This
R
I
I
O
quali-
sin-
legisign
rhema
dicent
argument
icon
index
symbol
representation
communication
signification
Figure 2.5: Peircean sign classes. Source: [
375
].
concept, of course, can be seen as going beyond Saussure's emphasis on the paradigmatic
and syntagmatic value of a
sign
in its relation to other signs. To demonstrate the logics of
the Peircean classification, Colapietro takes the example of a knock on the door:
"If there is a knock on the door announcing the arrival of guests, this rap
is a
sinsign
. More accurately, it is a
dicent
,
indexical sinsign
. It is a
dicent
(or
dicisign
) since it in effect performs the function of an asserted proposition
('The guests have arrived'). It is
indexical
since there is an actual, physical
connection between the
sign vehicle
and its
object
(the knocking sound and
the guests announcing their arrival by means of knocking). Finally, it is a
sinsign
because the knocks as they are occurring here and now the sounds in
their individuality serve as the
sign vehicle
" [
102
, p. 132].
According to the Peircean
pansemiotic
view of the world, signs may be considered in them-
selves; that is, in terms of what the
sign vehicle
(in Peirce's terminology:
representamen
) is
in itself, for different things play the role of signs. When a quality plays this role, we have a
qualisign
; when something general or law-like performs this function, we have a
legisign
;
and when an individual or actual existent assumes the role of
sign
, we have a
sinsign
.
36
The
trichotomy
of
qualisign
,
sinsign
, and
legisign
is part of an intricate classification of
signs devised by Peirce, for he also considers the
sign
in its relation to its
object
and in its
relation to its
interpretant
.
The second
trichotomy
has been cited more than once in isolation from the two other tri-
chotomies. In relation to its dynamic
object
, a
sign
may be either an
icon
, an
index
, or a
symbol
(see figure
2.5
). An iconic relation is a mode in which the
signifier
physically or
perceptually resembles or imitates the
signified
, recognizably looking or sounding like it
possessing some of its qualities (e.g. a portrait, a scale-model, onomatopoeia, the sound
of a gun in a violent computer game, imitative gestures).
Index
is a usage established by
Charles S. Peirce and widely adopted by contemporary semioticians to denote a specific
type of
sign
or sign function in which a
sign vehicle
represents its
object
(of
reference
)
by virtue of a causal or physical connection, cf. [
102
, p. 118]. This linkage can be ob-
served or inferred (e.g. a weather vane to indicate the direction of the wind, smoke for fire,
36
For example, a photograph of a woman may stand for some broad category such as women (
legisign
) or may
more specifically represent only the particular woman (sinsign) who is depicted.
CHAPTER 2. THE SEMIOTIC APPROACH
33
footprints for a path in use, fingerprints in biometrics, knock on door announcing a visit,
medical symptoms for certain illnesses, greyed-out
link marker
for an inactive link, etc.).
In a
symbolic
relation, the
signifier
does not resemble the
signified
but is
arbitrary
or purely
conventional, like most words of our natural languages, road signs, exclamation marks that
accompany error meassages, an arrow on a Web page or the
browser
tool bar that points to
the left for "back", etc. Naturally, different cultures show very different conventions, e.g.
the arrow to the left in Arabic or Hebrew means "forward"; shaking the head does not mean
"no" in all cultures, etc.
It is important to note that these broadly cited modes are not mutually exclusive: a
sign
can be an
icon
, a
symbol
and an
index
, or any combination. "A map is [. . . ] indexical (it
indicates where places are) and iconic (it represents places in topographical relation to each
other) and
symbolic
(its notational system must be learned)" [
118
, p. 77]. Furthermore, a
signifier
resembling that which it depicts is not necessarily purely
iconic
. As will be elab-
orated in section
2.7
, photographs have iconic, as well as indexical and symbolic relations.
hypermedia
show all three types aswell, as will be shown in section
3.5
. Whether a
sign
is symbolic, iconic or indexical depends primarily on the context in which the
sign
is used
and the code it is embedded into, so the "typical" examples chosen to illustrate the various
modes can be misleading.
In relation to its
interpretant
, the
sign
may be either a
rheme
, a
dicisign
, or an argument
(see figure
2.5
). Peirce's third
trichotomy
of signs is the most confusing one and can only
be understood as a reflection to the traditional logic categories. That is,
rheme
,
dicent
, and
argument more or less correspond to terms (concepts), propositions (statements), and argu-
ments, respectively. A
rheme
(Greek
~
µ
= word) represents a possible, not a concrete
object
; it can be every
sign
which is neither false nor true, such as nearly any word except
"yes" or "no", cf. [
416
, 2.309, 2.250].
37
A
dicent
is an informative
sign
which is slightly
more defined in its relation to the
interpretant
: It is not an assertion, but a
sign
capable of
being asserted, cf. [
416
, 8.337]. The difference to the
rheme
is that a
dicent
can either be
true or false: For example, if a man on the street shouts "apple", it would be considered a
rheme
, while the street vendor's promotion of "fresh apples!" is a
dicent
38
. An argument is
a complex
sign
whose elements (
rhemata
and
dicents
) are governed by general rules, e.g. a
sonnet, a syllogism, cf. [
401
, p. 66f.].
Peirce doesn't stop here: He goes on to explore the possibilities of combining the specific
types of sign or, perhaps better, sign functions identified in these three trichotomies. The ten
resulting categories (not all 27, i.e. 3
3
combinations make sense) are listed with examples
in [
401
, p. 67] and [
353
, p. 2].
Let us now turn back to the most fundamental taxonomies of the
sign
. Especially authors
who are exploring the frontiers of the semiotic field, tend to maintain a classification that
points to fields of studies such as zoosemiotics (animal
communication
) and endosemiotics
(communication of the organs within the body); see figure
2.6
.
Whereas Saussure had only considered non-verbal signs that explicitely function as such
(military signals, etiquette, sign-language etc.), contemporary
semiotics
includes not only
gestures but also objects. The function of an umbrella is to shelter against the rain. Thus,
the umbrella has become a
symbol
for shelter, it becomes the exemplary of a model, or
rite.
39
The apparent secondary function of architecture as a communication system can sometimes
even effect its primary function, or "functionality", cf. [
155
, 43ff.]. The skirt of a balle-
37
Peirce used rheme virtually synonymously with the way contemporary logicians used the term "predicate":
The predicate "x is red", for example, is a
sign
which cannot be spoken of as being true or false (until a quantifier
is added to tell which or how many xs we are talking about), cf. [
416
, 4.438ff.].
38
I chose this example, of course, to demonstrate the relationship to Wittgenstein's concept of names in relation
to his propositions and his theory of the Sprachspiel, cf. [
548
,
549
].
39
This semiotic of the consumer good has inspired many authors, e.g. [
154
,
40
].
CHAPTER 2. THE SEMIOTIC APPROACH
34
Source of Signs
anorganic
organic
artificial
natural
extra-terrestric
terrestric
homo sapiens
animals
parts of
organism
organism
parts of
organism
organism
Figure 2.6: Classification of signs by their source. Source: [
155
, p. 37].
rina has very little, or even negative primary functions (it reveals more than it covers) and
very dominant secondary functions. Thus, Eco proposes to replace Sebeok's classification
(figure
2.6
) by his own (figure
2.7
).
artificial
natural
explicitely
produced for
functionality
- signs of
primary functions
- signs of
secondary funct.
explicitely
produced to
designate
natural
processes
unwillingly
produced by
human being
- medical symptoms
- psychological s.
- signs of dispositions
- Race, Regional or
class adjectives
- others
SIGNS
Figure 2.7: Classification of signs by their sign-functionality. Source: [
155
, p. 44].
Signs could also be classified according to the grade of consciousness and the intention
of the
sender
, cf. [
155
, pp. 45-50]: Besides the communicative signs, that are sent out
explicitely as tools to "get something across", every individual also sends along unwanted
signals, or expressive signs. While the first kind of signs are consciously coded, signs of the
second kind are intuitive. If a dancing partner has sweaty hands, or uses a certain perfume,
that might reveal more than what the person wanted to say. Paralinguistics is a discipline
that concentrates on the sound of the voice rather than the spoken words. Buyssens [
82
]
uses the example of a fake descendant of the Platagenet family, who discloses his false
play with his rude manners and inaccurate pronunciation while telling his stories. Whether
a
sign
is sent/received intendedly or not, whether the receiver thinks it was intentional and
finally whether the the
sender
wanted the receiver to think one or the other, creates a whole
new "Semiotics of Dissimulation" [
155
, p. 49].
Another important classification of signs is that according to their physical channel of trans-
mission (especially in the context of media semiotics). While really natural signs
40
can be
carried by virtually every thinkable medium, or channel, the reception of signs is, com-
40
Natural signs, according to Eco, are physical processes, such as thunder, starry nights, etc., cf. [
155
, pp.
45-50].
CHAPTER 2. THE SEMIOTIC APPROACH
35
monly spoken, restricted to the five channels of human sensibility.
The question of
arbitrariness
vs.
motivation
of a
sign
has produced a large production
of scholarly discourse: Although the
signifier
is treated by its users as "standing for" the
signified
, Saussure emphasized the
arbitrariness
of the
sign
: there is no intrinsic, direct or
"transparent" relationship between the
signifier
and the
signified
. John Fiske comments
that Saussure believed that the
arbitrary
nature of verbal language is the main reason for
its complexity, subtlety and ability to perform a wide range of functions, cf. [
168
]. Each
language
involves different distinctions between one
signifier
and another (e.g. "tree" and
"free") and between one
signified
and another (e.g. "tree" and "bush"). The
arbitrariness
of signs underlines the scope for their interpretation (and the importance of context). Signs
have multiple rather than single meanings. Within a single language, one
signifier
may
refer to many
signifieds
(e.g. puns draw on homonyms) and one
signified
may be referred
to by many
signifiers
(synonyms). Eco shows that the
signified
does not only depend on
the
signifier
, but also on the position within a system, for example the system of a language
(Figure
2.8
). In a
triadic
sign model,
arbitrariness
is given if the cognitive experience of the
Figure 2.8: Language systems. Source: [
155
], p. 86.
French
German
Danish
Italian
English
arbre
Baum
albero
Holz
legno
bosco
foresta
foret
bois
Wald
trae
skov
tree
wood
woods
forest
sign-user with the
referent
has no influence on the
sign vehicle
, cf. [
401
, p. 340]. Grades
of
arbitrariness
, for Peirce are given by the
iconic
,
indexical
and
symbolic
types of
object
-
relation, cf. [
401
, p. 341]. This system is especially helpful (and often cited) when defining
the
arbitrariness
of visual signs (see section
2.7
). Some semioticians, among them Fiske
and Eco, maintain that convention is necessary to the understanding of any
sign
, however
iconic or
indexical
it is. Guy Cook asks whether the iconic
sign
on the door of a public
lavatory for men actually looks more like a man than like a woman (the two signs on the
bottom left of figure
2.16
). For a sign to be truly iconic, it would have to be transparent to
someone who had never seen it before - and it seems unlikely that this is as much the case
as is sometimes supposed. We see the resemblance when we already know the
meaning
,
cf. [
110
]. This is especially true with onomatopoeia which supposedly imitate the sound of
their
referent
. Onomatopoeic words like "to miaow", (Dutch: miauwen; French: miauler;
German: miauen; Italian: miagolare; Spanish: maullar) were seen by Saussure as being no
threat the
arbitrary
relationship of the
signifier
to the
signified
. The same is true for excla-
mations (English: ouch!, Spanish: ¡ay!, French: aïe!, German:au!, etc.), cf. [
469
, p. 78ff.].
Although Saussure's approach was a
synchronic
one, he was aware that the relationship
between the
signified
and the
signifier
in
language
was subject to change over time. John
Hartley notes that over time, what were once motivated signs can "become arbitrary and
radically change their signified" [
220
, p. 31]. Many signs which in their original use could
be seen to be motivated to bear some discernible relationship to their
referent
come to be
used more metaphorically and may subsequently lose even their metaphorical association
for their users.
41
Some of the letters in the Greek and Latin alphabets, of course, derive
from
iconic
signs in Egyptian hieroglyphics (cf. [
160
, p. 132ff.]. The first three Arabic
41
The word "miniature" derives from Latin word "minium", the red lead color that was used for medieval
illumination of manuscripts. Therefore, the Latin word miniâre, soon became a synonym for producing small
painting executed with great detail and the
signified
of /miniature/ shifted from "painted with lead color" to
"being on a small or greatly reduced scale" [
7
].
CHAPTER 2. THE SEMIOTIC APPROACH
36
numbers can be shown to derive from the same counting technique as the Latin numbers
by a little trick: While the number one is obviously a scratch of the prehistoric accountant,
the following two numbers have to be rotated to reveal
graphein
as their common root, see
figure
2.9
.
I
II
III
1
2
3
Figure 2.9: Graphein as the root of numbers.
Another important taxonomic distinction is that between analogical and digital signs: "Ana-
logical signs (such as visual images, gestures, textures, tastes and smells) involve graded
relationships on a continuum. They can signify infinite subtleties which seem 'beyond
words"' [
95
, Signs]. As Guy Cook puts it: "one cannot specify the number of different
smiles and [. . . ] laughs available in one person's repertoire" [
110
, p. 67].
Digital signs, on the other hand, involve discrete units such as words and numerals and
depend on the categorization of what is signified. The appearance of the "digital watch"
in 1971 and the subsequent "digital revolution" in audio- and video-recording have led
us to associate the digital mode with electronic technologies, cf. [
95
, Signs]. Yet, digital
codes (e.g. the Morse code) have existed before the invention of binary coding, cf. [
401
, p.
224]. In fact, "digital codes have existed since the earliest forms of language and writing
is a 'digital technology"' [
95
, Signs]; see section
2.4
. Fiske notes that "turning nature
into culture and thus making it understandable and communicable involves codifying it
digitally", [
169
, p. 313]. Digital differences are either/or while analog distinctions are
more-or-less.
An example for an analog representation system is the speedometer on an automobile.
The speeds of the vehicle are represented in a dense class of states of affairs (positions
of the pointer) that can hold in the meter. The light on the dashboard that registers oil
pressure affords a digital representation system because there are only two states of affairs
(on and off) that indicate information (high and low) about the oil pressure. Both classes
of indicating states of affairs and indicated states of affairs are discrete. Of course, a single
representation system can be both analog and digital with respect to different subsets of
information within its coverage, cf. [
482
, 105].
One may be tempted to use this distinction between analog and digital to draw a line be-
tween graphical systems and linguistic systems. Thus, it might be proposed that a linguistic
system is digital with respect to the entire set of information it covers, while a graphical
system is analog with respect to the entire set of information it covers [
482
]. However,
Goodman has effectively cut this line of proposal.
42
Linguistic systems might be all digi-
tal, but the property of being digital is shared by some graphical systems, cf. [
482
, p. 106].
Umberto Eco has criticized the apparent equation of the terms "
arbitrary
", "conventional"
and "digital" by some commentators, cf. [
154
, p. 178-180]. He notes the way in which
the following widespread pairings misleadingly suggest that the terms vertically aligned
in figure
2.10
are synonymous. He observes, for instance, that a photograph may be both
42
"Diagrams, whether they occur as the output of recording instruments or as adjuncts to expository texts
or as operational guides, are often thought because of their somewhat pictorial look and their contrast with
their mathematical or verbal accompaniments to be purely analog in type. Some such as scale drawings for
machinery, are indeed analog; but some others, such as diagrams of carbohydrates, are digital; and still others,
such as ordinary road maps, are mixed" [
194
, p. 68]. Since diagrams of carbohydrates are digital with respect to
CHAPTER 2. THE SEMIOTIC APPROACH
37
digital
vs.
analog
arbitrary
vs.
motivated
conventional
vs.
natural
Figure 2.10: Misleading pairing of terminology. Source: [
154
, p. 190].
"motivated" and "digital".
Based on Peirce's views especially the differentiation of sin-, quali-, and legisigns Eco
offers another distinction between
sign vehicles
, relating to the linguistic concept of
tokens
and
types
. In relation to words in a text, a count of the
tokens
would be a count of the
total number of words used, whilst a count of the types would be a count of the different
words used (regardless of repetition). Eco lists the following kinds of
sign vehicles
: signs in
which there may be any number of tokens (replicas) of the same type (e.g. exactly the same
model of car in the same color
43
); "signs whose tokens, even though produced according to
a type, possess a certain quality of material uniqueness" (e.g. a word spoken or handwritten
by different people); and "signs whose
token
is their type, or signs in which type and
token
are identical" (e.g. an original oil-painting), cf. [
154
, p. 178-180]; [
95
,
396
,
44
]. In the
context of
hypertext
,
nodes
and
link markers
are typically signs of the first kind, or "copies
without originals" [
95
, Signs].
2.5
Semiosis
Semiosis
, a term borrowed from Charles Sanders Peirce, is expanded by Eco to designate
the process by which a culture produces signs and attributes
meaning
to signs. Umberto
Eco coins the phrase "unlimited
semiosis
" to refer to the way in which a series of suc-
cessive Peircean
interpretants
lead to a (potentially) ad infinitum process, as any initial
interpretation can be re-interpreted; cf. [
416
, 1.339, 2.303], [
154
,
396
]. Merrell [
353
, p.
137] illustrates "the ongoingness of this semio[t]ic process" as figure
2.11
, an extension of
his tripod model of the
sign
(figure
2.2
).
R
I
I
O
R
I
I
O
R
I
I
O
1
1
1
2
2
2
3
3
3
Figure 2.11: Unlimited semiosis. Source: [
353
, p. 137].
Chandler equates unlimited semiosis with the process of browsing a dictionary: "That a
signified
can itself play the role of a
signifier
is familiar to anyone who uses a dictionary
the entire sets of information they cover, Goodman's observation also precludes the suggestion that a graphical
system is analog with respect to at least a part of the information set it covers, cf. [
160
, p. 13ff.].
43
See footnote
36
.
CHAPTER 2. THE SEMIOTIC APPROACH
38
and finds themselves going beyond the original definition to look up yet another word
which it employs" [
95
, Signs].
44
Bolter [
68
] and McGuire [
346
] expand the equation to
hypertext navigation, as described in section
3.7.11
.
The notion of the importance of sense-making
45
(which requires an
interpreter
though
Peirce doesn't feature that term in his triad) has had "a particular appeal for communication
and media theorists who stress the importance of the active process of interpretation, and
thus reject the equation of 'content' and
meaning
. Many of these theorists allude to semiotic
triangles in which the
interpreter
(or 'user') of the
sign
features explicitly (in place of
'
sense
' or '
interpretant
')" [
95
, Signs]. Nadin insists that "each time someone interprets a
sign
, that person becomes part of the sign and thus of the process of its interpretation" [
375
].
David Sless reminds us that statements about users, signs or
referents
can never be made in
isolation from each other: "A statement about one always contains implications about the
other two" [
493
, p. 6].
2.6
The Code
In information theory and computer science,
codes
play a major role in programming, data
transmission, cryptography
46
, etc. In semiotic texts, two meanings of
code
are encountered
most frequently: In one sense, code means a set of rules prescribing how to act or how
to do, and in another, a key (or set of instructions) for translating a
message
. Morse code
is a key for correlating particular patterns of clicks and silences to letters of the alphabet.
Codes as sets of rules are normative: They provide us the norms to judge whether we
are acting appropriately. Judgments of mispronunciation are only possible in reference to
the codification of sounds found in the alphabet. Of course, not all violations of a
code
signal ineptitude or incompetence; some result from deliberate or conscious decision: "For
example, when a person desiring to shock people shows up on a formal occasion dressed
in a bathing suit, thereby breaking the fashion code. This example suggests an important
distinction: A
code
needs to be explicitly formulated. In fact, most codes might be sets of
more or less implicit (or unstated) rules: They are acquired through imitative behavior and
are followed, in a sense, unconsciously" [
102
, p. 64].
Eco points out that the semiotic field includes the traditional field of
Aesthetics
because
every
code
(visual codes, cultural codes, natural languages, musical codes etc.) permits
an
aesthetic
usage of its signs, cf. [
153
, p. 25]. Thus, even the non-semiotic aspects of
aesthetics, such as the psychology of artistic creation, the analysis of the relation of art
and society and the physio-psychological definition of
aesthetic
pleasure, could be directed
from a semiotical point of view. Although many of these issues shall not be approached
directly in this paper, they will permanently reappear on the horizon of the field of study.
In the semiotic context, however,
codes
are interpretive frameworks which are used by both
producers and
interpreters
of
texts
and help to simplify phenomena in order to make it eas-
44
This seems even more true for browsing an encyclopedia. The encyclopedia paradigm of
hypertext
will be
described in section
3.3
.
45
"Although for Eco meaning production or
semiosis
is a social activity, he allows that subjective factors are
involved in each individual act of
semiosis
. The notion then might be pertinent to the two main emphases of
current, or post-structuralist, semiotic theory" [
126
, p. 167].
46
Whilst the fascination of hidden
messages
(e.g. of Kabbalistic and alchemistic kinds) seems to be as old
as language itself, the deciphering of the Egyptian hieroglyphs may be seen as the starting point of modern
cryptography. Royal names were historically, along with the Rosetta stone, the key to the understanding of the
Egyptian hieroglyphs: The Abbot Barthélémy had already suggested in the eighteen century that the cartouches
enclosed royal names. Thus, after the Rosetta stone had been found, Akerblad and Young were able to read some
Greek and Roman royal names. Jean Champollion, using his knowledge of the Coptic language, proved that the
phonetic system wasn't only used for foreign names, thus getting the clue that allowed him to translate quite
accurately many texts during the ten years that followed his discovery, cf. [
450
]. The importance of cryptography
to ensure secure data exchange and authenticity on the Internet will be treated in section
4.7.3
).
CHAPTER 2. THE SEMIOTIC APPROACH
39
ier to communicate experiences, cf. [
193
, p. 35]. Yet, "codes are not simply 'conventions'
of communication but rather procedural systems of related conventions which operate in
certain domains. Codes organize signs into meaningful systems which correlate
signifiers
and
signifieds
" [
95
, Code]. Society itself depends on the existence of such signifying sys-
tems, or, as Stuart Hall puts it, "there is no intelligible discourse without the operation
of a code" [
210
, p. 131]. In accordance to Kristeva's
intertextuality
thesis [
291
],
codes
transcend single texts, linking them together in an interpretative framework.
It has become the common way to start a semiotic account of the
code
by citing Ernst
Gombrich's commentary on the golden plaque aboard of Pioneer 10, an interstellar probe
sent into deep space by the NASA in 1972, cf. [
95
,
458
]. Gombrich explains why even
in that unlikely case that "intelligent scientifically educated beings" (as expected by the
NASA) had sense organs that responded to the same band of electromagnetic waves as
our eyes the aliens could not possibly get the message: "Reading an image, like the
reception of any other message, is dependent on prior knowledge of possibilities; we can
only recognize what we know" [
193
, p. 151]. Accordingly, primal tribes experience initial
difficulties in decoding photographs and film to the same degree that Westeners fail to
understand Chinese writing or a Tibetian mandala, cf. [
18
, p. 11, 80].
Roman Jakobson emphasized that the production and interpretation of
texts
depends upon
the existence of
codes
or conventions for
communication
, cf. [
253
, p. 570-79]. Since the
meaning
of a
sign
depends on the
code
within which it is situated,
codes
provide a frame-
work within which signs make sense: "Indeed, we cannot grant something the status of
a
sign
if it does not function within a code" [
95
, Code]. According to the Gestalt psy-
chologists [
281
,
280
] there are certain universal features in human visual perception which
in semiotic terms can be seen as constituting a perceptual
code
.
47
Perceptual constancy
ensures that "the variability of the everyday world becomes translated by reference to less
variable codes. The environment becomes a text to be read like any other text" [
386
, p. 26].
Whilst these basic principles have to be observed in the design of any information system,
it is impossible to base an intuitive hypertext interface on them alone: To the first-time
computer user, the basic
codes
of Human-Computer Interaction (
HCI
) have to be learned
first. Only thereafter will they soon be internalized, or "naturalized" [
210
, p. 132], see
section
3.6
.
Unlike researchers that ignore the semiotic approach, semioticians have tried to capture the
difference between
code
and
language
in verbal and non-verbal
communication
: With re-
gard to photography (though one might say the same for film and television), Victor Burgin
insists that: "There is no 'language' of photography, no single signifying system (as op-
posed to technical apparatus) upon which all photographs depend (in the sense in which all
texts in English depend upon the English language); there is, rather, a heterogeneous com-
plex of codes upon which photography may draw" [
79
, p. 143]. Jakobson observed that
"the image of language as a uniform and monolithic system is oversimplified. Language is
a system of systems, an overall code which includes various subcodes" [
254
, p. 30]. How-
ever, "the term language is often used by semioticians and others in a very general sense
to mean any system of signs. It is also frequently used in a narrower sense to designate a
system of verbal signs, talking verbal here to include both spoken (or auditory) and written
signs. Third, language is used in a still narrower sense by some linguistics [. . . ] and others
to mean a system of auditory signs" [
102
, p. 128].
Semioticians have seeked to identify
codes
and the tacit rules and constraints which un-
derlie the production and interpretation of
meaning
within each
code
. Different theorists
have found it convenient to divide codes into groups, yet they favor different taxonomies.
47
"In addition to introducing the terms 'figure' and 'ground', the Gestalt psychologists outlined what seemed
to be several fundamental and universal principles (sometimes even called 'laws') of perceptual organization.
The main ones are as follows (some of the terms vary a little): proximity, similarity, good continuation, closure,
smallness, surroundedness, symmetry and prägnanz" [
95
, Code], cf. [
57
,
336
,
384
]. See figure
3.11
.
CHAPTER 2. THE SEMIOTIC APPROACH
40
The most widely mentioned in the context of media,
communication
and cultural studies
are social, textual, and interpretative codes (and subcodes), cf. [
95
, Code]; [
401
, p. 216ff.].
As the various kinds of
codes
overlap, the semiotic analysis of any
text
or practice involves
considering several codes and the relationships between them. The "tightness" of semiotic
codes themselves varies from the rule-bound closure of logical codes (such as codes in
computer science) to the interpretative looseness of poetic codes.
48
The deliberate intention to communicate tends to be dominant in
digital
codes, whilst
communication
in
analog
codes (through gesture, posture, facial
expression
, intonation and
so on) takes place on a largely uncontrollable and unconscious level. Actors, politicians and
managers are trained to control these analog codes to a certain extent. To the receiver, these
codes unavoidably "give us away", revealing such things as our moods, attitudes, intentions
and truthfulness (or otherwise), cf. [
95
, Signs].
2.7
Media Semiotics
Nöth sketches the relation of two neighboring research disciplines
Semiotics
and Media
Studies
49
as appearing to be "predestined to fruitful transdisciplinary cooperation" [
399
, p.
1].
Semiotics
began to become a major approach to media theory in the late 1960s, partly
as a result of the work of Roland Barthes. The translation into English of his popular
essays in a collection entitled Mythologies (1957), followed in the 1970s and 1980s by
many of his other writings, greatly increased scholarly awareness of this approach, cf.
[
401
, p. 107-111]. Semioticians have been interested in the media both as an area of
applied semiotic research and as an area of testing, questioning, or even revisiting its own
theoretical premises. Consequently, there is a plurality of semiotic approaches to the media,
cf. [
399
, p. 1f.].
The term "medium" is used in a variety of ways by different theorists, and may include
such broad categories as
speech
and writing or print and broadcasting or relate to spe-
cific technical forms within the mass media or the media of interpersonal
communication
,
cf. [
95
, Introduction]. Chandler's own definition of a medium "is similar to the definition
of a semiotic system as a symbolic system which serves to support the construction of real-
ity" [
94
, p. 3]. Other theorists classify media according to the physical channels involved
(visual, auditory, tactile, olfactory, gustatory
50
). Human experience is inherently multisen-
sory, yet every
representation
of experience is subject to the constraints and affordances of
the medium involved. Peirce's consideration of the medium instead of signs has often been
revisited, reminding us that the study of the
sign
just like the study of the media is the study
of the process of mediation between ourselves and the world outside, cf. [
399
, p. 3]. Of
course, every medium is constrained by the channels which it utilizes and an awareness of
this phenomenon of transformation by media has often led media theorists to argue deter-
ministically that our technical means and systems always and inevitably become "ends in
48
Nöth shows some exemplary transcodings, e.g. between the Latin alphabet, Morse code, Braille alphabet (for
the blind) and the naval alphabet, cf. [
401
, p. 223]. Janko demonstrates en- and decoding the
ASCII
, the Huffman
code, error correcting codes, etc., cf. [
256
, p. 16ff.].
49
According to Daniel Chandler, media studies, or "the study of the mass media" is an offshoot of communi-
cation studies, cf. [
94
, p. 2]. Media Theory, "a term which has been gaining currency in recent years" is even less
of an established discipline than semiotics or media studies: "Many of the concerns of media theory are shared by
scholars in a variety of disciplines, including the anthropologists, linguists and rhetoricians. [. . . ] And a particular
kind of media theory is also a concern of those involved in semiotics [. . . ] The best-known 'theorist' of media
in the broadest sense, Marshall McLuhan (who enjoyed widespread popular attention in the 1960s and '70s), can
hardly be regarded as having developed a coherent theoretical framework for the study of media" [
94
, p. 3].
50
Buyssens [
82
] calls these categories "semies".
CHAPTER 2. THE SEMIOTIC APPROACH
41
themselves" [
95
, Introduction].
51
.
The same is true, of course, for hypermedia
nodes
. While aural and visual signs seem
to be privileged in current hypertext systems, other channels are helpful for the reception
of expressive signs. In media theory, the use of a certain channel, or "medium" can be
as important as the
message
itself. In his account on "the computer, hypertext and the
history of writing", Jay David Bolter argues that "signs are always anchored in a medium.
Signs may be more or less dependent upon the characteristics of one medium they may
transfer more or less well to other media but there is no such thing as a sign without
a medium" [
68
, 195f.]. Chandler points to the fact that it would be more precise to say
that the
sign vehicle
cannot be without a medium, cf. [
95
, Signs]; [
551
, p. 17]. Hodge
and Tripp note that, "fundamental to all semiotic analysis is the fact that any system of
signs (semiotic
code
) is carried by a material medium which has its own principles of
structure" [
228
, p. 17]. The medium is not "neutral"; each medium has its own constraints
and, as Umberto Eco notes, each is already "charged with cultural signification" [
154
, p.
267]. John Fiske insists that "each medium is capable of transmitting codes along a channel
or channels" [
408
, p. 176] and that "the physical characteristics of the channel limit the
medium and codes that it can carry". For Schmauks, media differ from each other in their
ability to illustrate abstract objects and concepts
52
. Such differences lead Emile Benveniste
to argue that the "first principle" of semiotic systems is that they are not synonymous: "We
are not able to say 'the same thing' in systems based on different units" [
47
, p. 235]. This
assumption seems even more striking in the consciousness that the general scheme of the
communication
process is always the same (see figure
2.12
).
Sender
Signal
Channel
Message as
Source of
Information
Receiver
Message as
Meaning
Signal
Subcodes
Code
Subcodes
Code
semantic
noise
physical
noise
context
Figure 2.12: Communication process. Source: [
153
, p. 139].
As an approach to
communication
which focuses on
meaning
and interpretation,
semiotics
challenges the reductive transmission model [
480
] which equates "
meaning
" with "
message
"
(or content), cf. [
102
,
442
,
399
]. Meaning is not 'transmitted' to us we actively create it
according to a complex interplay of
codes
or conventions of which we are normally un-
aware. Becoming aware of such
codes
is both inherently fascinating and intellectually
empowering. We learn from
semiotics
that we live in a world of signs and we have no
way of understanding anything except through signs and the
codes
into which they are
organized. Signs do not just "convey" meanings, but constitute an environment in which
meanings are constructed. Generalizing the findings of those urban semioticians who have
highlighted the limitations of the Lynchian and cognitive perspective (cf. [
198
]), one could
say that
meaning
gets "conceived", not "perceived" (see section
3.7.11
).
Semiotics
helps
51
This is a common interpretation of Marshall McLuhan's famous aphorism, "the medium is the message"
[
350
,
351
], and has even led some to present media as wholly autonomous entities with "purposes" (as opposed to
functions) of their own. "However, one need not adopt such extreme stances in acknowledging the transformations
involved in processes of mediation. When we use a medium for any purpose, its use becomes part of that purpose.
Traveling is an unavoidable part of getting somewhere; it may even become a primary goal" [
95
, Introduction].
These reflections are important in the consideration of hypertext navigation, see section
3.7
.
52
"Medien unterscheiden sich unter anderem darin, inwieweit sie abstrakte Objekte darstellen können. Die
Sprache ist außerordentlich flexibel [. . . ]. Bilder hingegen können abstrakte Objekte nur auf "semiotischen
Umwegen" darstellen" [
473
].
CHAPTER 2. THE SEMIOTIC APPROACH
42
us to realize that
meaning
is not passively absorbed but arises only in the active process of
interpretation, cf. [
63
,
399
].
Lacanian psychoanalysis has been successfully applied to media theory, e.g. the relation
of the spectator to the cinema in general, cf. [
225
]. Christian Metz, in his seminal study of
cinema as The Imaginary Signifier, has tackled the problem of the position of the spectator
with respect to a film from within a Lacanian framework. His analysis starts off from the
notion of perception "The cinema's
signifier
is perceptual (visual and auditory)" [
355
, p.
42] and goes on to distinguish the cinema from other arts inscribed into the perceptual
register (such as painting, sculpture etc.) by stating that the cinema is "more perceptional"
[
355
, p. 43] by involving more perceptional axes.
53
Semiotics
will have to play a major role in the successful introduction of gustatory, olfac-
tory and tactile data into (hyper-)media systems. Merrell makes clear that the specifica of
those senses make them extremely interesting for the human understanding, learning and
interpreting of data: for example, "odors, and even tastes [. . . ] hardly know more than long-
term memory" and "touch is sort of at the crossroads regarding sensory modes" [
353
, p.
151, 157]. As the technological means for their integration are not yet available, most
researchers concentrate on audio-visual inputs:
"Images, language and writing are what's meant when I talk of media in the
narrow sense. They have been at the centre of many philosophical discussions
in the twentieth century which were mostly concerned with identifying one or
more of these media as being a binding base structure for human understanding
of reality altogether, or at the very least as the foundation of the world-
picture characteristic of Western culture" [
465
].
54
The transformation of the WWW from
hypertext
to
hypermedia
and the evolution of inter-
faces from
symbolic
to
iconic
(in Brown's sense [
77
]), echoes the transition from a society
of
text
to a society of the (moving)
image
. At this point the Web is primarily a static visual
medium, but higher access speeds and the convergence with other media, such as TV and
radio will bring more time-based hypermedia soon, cf. [
388
,
393
].
As pointed out by Risak, optical presentation of home pages can communicate a lot of
messages
to the user, cf. [
442
,
443
]. Screen designs that resemble glossy magazines and
brochures rely of the eye-catching quality of the
image
, and also on an immediate (and
easy) understanding of its meaning. In other words, that what Langer calls presentational
immediacy
55
. The investigation of popular cultures is the realm of cultural studies and
semiotics
alike. Some ask if the intrusion of the
image
catapults us back into an age when
most people were educated by narration and murals on church or cave walls and only few
were able to read and write. Others face this development by tearing down the wavering
distinctions between "art" and "nonart," "expressive" and "inexpressive" that have been
53
Bignell argues that "the whole of our social world is pervaded by messages which contain visual as well as
linguistic signs, or which are exclusively visual. Gestures, dress codes, traffic signs advertising images, newspa-
pers, television programmes and so on are all kinds of media which use visual signs. The same principles underlie
the semiotic study of visual signs and linguistic signs. In each case, there is a material signifier, which expresses
the
sign
and a mental concept, a
signified
, which immediately accompanies it" [
63
, p. 14].
54
"The spectrum reaches from analytic philosophy's 'linguistic turn' and the diverse misunderstandings trig-
gered by Derrida's early concept of a philosophical 'grammatology' in the realm of postmodern thinking, through
to contemporary proclamations of a 'pictorial turn"' [
465
].
55
In Philosophy in a New Key, Langer writes, "Visual forms are not discursive. They do not present their
constituents successively, but simultaneously, so the relations determining a visual structure are grasped by one
act of vision" [
306
], see section
2.2
. Mircea Eliade puts it this way in Images and Symbols: "Images by their very
structure are multivalent. If the mind makes use of images to grasp the ultimate reality of things, it is just because
reality manifests itself in contradictory ways and therefore cannot be expressed in concepts. [. . . ] It is therefore
the image as such, as a whole bundle of meanings, that is true, and not any one of its meanings, nor one alone of
its many frames of reference" [
159
, p. 15].
CHAPTER 2. THE SEMIOTIC APPROACH
43
obstructing the way to a wider panorama on visual
codes
, cf. [
160
, p. 31f.]. Of course, this
question falls far beyond the borders of this dissertation. Yet, a short disquisition on the
state of art in image theory may clarify the potential of
graphics
and photos in
hypermedia
.
Elkins breaks with the tradition of art history that concentrates on fine art
images
leaving
non-art, or "informational images"
56
to other disciplines, such as archaeology, cognitive
psychology, mathematics, philosophy of science. As "art history is centrally positioned
in the emerging field of image studies because it possesses the most exact and developed
language for the interpretation of images", existing art-historical methods "can embrace
images of any kind, from graphs to ideographic writing" [
160
, p. 6]. He takes a sonar chart
used for fishing (similar to figure
2.13
) as an example of "a composite of very different
Figure 2.13: Sonar chart of the water beneath a fishing boat. Source: Eagle Electronics.
routes of reference: It is an x-y
graph
, a naturalistic scene, and a collection of
symbols
for the motion of fish. It needs to be read, seen, and deciphered, and a viewer must switch
between modes of interpretation in order to comprehend it" [
160
, p. 36]. Recurring on Nel-
son Goodman's terminology of "routes of reference" ( [
195
, p. 55-70]), Elkins finds mixed
routes of reference not only in sonar charts for fishing, but also "on ancient artifacts, and
on contemporary images such as computer screens (which are partly naturalistic pictures
and partly notations)" [
160
, p. 36].
According to Elkins, nonart
images
could be called notations, "if it were not that the word
has been coopted by Nelson Goodman to describe especially systematic images such as
printed music and Labanotation
57
" [
160
, p. 54].
In the context of
hypertext
, Goodman's notational criteria
58
concern the presentation layer
56
"In general, art history tests its boundaries by working with popular, medieval, and non-Western images. But
the domain of images is substantially larger. In particular there is another group of images that seems to have
neither religious nor artistic purpose, and that is images principally intended in the dry language of communica-
tion theory to convey information. There is no good name for such images, which include graphs, charts, maps,
geometric configurations, notations, plans, official documents, some money, bonds, patents, seals and stamps,
astronomical and astrological charts, technical and engineering drawings, scientific images of all sorts, schemata,
and pictographic or ideographic elements in writing: in other words, the sum total of visual images, both Western
and non-Western, that are not obviously either artworks, popular images, or religious artifacts. In general, art
history has not studied such images. . . " [
160
, p. 4]. Thus, "it makes sense to use "informational images as con-
venient labels rather than as definitions, because they say less about pictures than about the disciplines that study
them" [
160
, p. 5].
57
Labanotation is a standardized system for analyzing and recording any human motion. Mainly it is used at
theaters to archive ballets. The original inventor is the (Austrian-) Hungarian Rudolf von Laban (1879-1958).
58
Goodman lists five criteria of notation, of which two are syntactic and three semantic. It is crucial that a
mathematic symbol is syntactically disjoint, that is, that a mark cannot be assigned to two characters. The nota-
tional system itself must also be syntactically finitely differentiated or articulate, i.e. different symbols must not
be mistaken for one another. Furthermore, a notation must be semantically unambiguous, semantically disjoint,
and semantically finitely differentiated, or dense, cf. [
194
]. Veith Risak has drawn my attention on the parallels
between the syntactic rules of notation and the scope rules in computer programming.
CHAPTER 2. THE SEMIOTIC APPROACH
44
(the "run-time layer") and the within-component layer of the
Dexter Reference Model
(see
section
3.4
), as
link markers
must be clearly defined for the user and lead to the right
link target
. In order to display
node
contents, these
notation
criteria are relevant for de-
sign questions, such as typography and computer fonts, questionnaires, etc. For Elkins,
Goodman's system becomes interesting at the point of breakdown.
59
Figure 2.14: Borromean rings. Source: based on [
160
, p. 86] and [
101
].
After several attempts to formalize the relation between writing, picture and
notation
, he
recurs on Lacan's idea of interlocking the registers Symbolic, Imaginary and Real with
Borromean rings:
"For Lacan, Borromean rings are a suggestive image of the state of the psy-
che, formed of encircled emptiness rather than bounded psychic 'registers'.
The same could be said of the triad of writing, picture and notation. Inter-
locked rings do justice to the commonsense belief that all images are somehow
related: A page of text might fall on the circumference of the 'writing ring' at
a point diametrically opposed to the unnamed center, but even there will be
intimately connected with the other rings" [
160
, p. 86].
Elkins distinguishes seven kinds of
images
on the way from writing via picture to
notation
60
,
cf. [
160
, p. 95ff.]. Of course, he is well aware that there are overlappings and hybrid
forms in the categories (writing,
notation
, picture) just as well as in the
image taxonomy
(allography, semasiography, pseudowriting, subgraphemics, hypographemics, emblemata,
schemata), cf. [
160
].
Wittgenstein's Bildtheorie
61
(picture theory) is, according to Elkins "the strongest and most
consistent form of the desire to have pictures make determinate sense" [
160
, p. 56]. Elkins
claims that Wittgenstein wants to demand of pictures what Goodman demands of notations.
While any further commentary reaches beyond the limits of this dissertation, I do agree
to Elkin's claim that "there might be good reason to reconsider the 'picture theory' and
Wittgenstein's Tractatus logico-philosophicus in light of contemporary visual theory" [
160
,
p. 58].
Apart from his picture theory, Wittgenstein, especially in the Philosophische Untersuchun-
gen [
549
] was interested in natural language and its use. Wittgenstein, Bertrand Russell,
59
Kaplan/Moulthrop take a similar position towards the
breakdown
of the graph model for hypertext, cf. [
268
].
60
This way could be called a logocentric sequence, the ostensive progression from a pure picture to the alphabet,
cf. [
160
, p. 86]
61
The idea for a theory of the Bild (he could have tried Darstellung or Vorstellung, with their echoes of Hegel
and Schopenhauer) probably came from Gottlob Frege who writes "It would be desirable to have a special term
for signs having only sense if we name them, say pictures [Bilder], the words of an actor on the stage would
be pictures; indeed the actor himself would be a picture" [
178
], cited in [
160
, p. 64]. For different viewpoints on
Wittgenstein's Bildtheorie, cf. [
236
,
250
,
87
].
Details
- Seiten
- Erscheinungsform
- Originalausgabe
- Jahr
- 2001
- ISBN (eBook)
- 9783832466879
- ISBN (Paperback)
- 9783838666877
- DOI
- 10.3239/9783832466879
- Dateigröße
- 3.5 MB
- Sprache
- Englisch
- Institution / Hochschule
- Wirtschaftsuniversität Wien – Marketing, Informationsverarbeitung und Informationswirtschaft
- Erscheinungsdatum
- 2003 (April)
- Note
- 1,0
- Schlagworte
- semiotik zeichentheorie navigation e-commerce world wide
