“SIGHT SOUND and SENSE” in “Sight, Sound, and Sense”
Semiotics: A Discipline or an Interdisciplinary Method?
In 1974 the editorial board o f VS set out to compile a comprehensive bibliography of semiotic research and asked scholars from Israel to the USSR to Denmark to Australia to pull together their resources and prepare a critical list of works produced in their respective countries. Although the stylistic criteria were stipulated by VS, certain questions posed by the collaborators were left unanswered, such as : "Which books or papers are considered to be those dealing with semiotics—the ones which explicitly use the term semiotics within their title? Or those that we consider useful to our own semiotic perspective? For example, is Locke's Essay Concerning Human Understanding a book on semiotics, since its last chapter proposes that semiotics be one of the three main branches of science? Is Husserl's Logische Untersuchungen a semiotic work even though the term semiotics is never used, while another of this scholar's works, entitled Semiotik, has barely influenced semiotic thought? On the one hand, there seem to be books that are explicitly designated as pertaining to the 'semiotics of the visual arts' but that only reiterate the traditional cliches of academic art criticism, while, on the other hand, there are other books, such as Gombrich's Art and Illusion and Goodman's Languages of Art , that are masterpieces of visual semiotics but do not even mention the word semiotics. Please give us some guidelines." In response to this dilemma, we answered: "No, gentlemen, that's your business! Why do you think we are preparing this special issue if not to find out how semiotics is defined?"
Frequently, while traveling, I meet people who ask: "Semiotics? [Those in the U.S. sometimes say "symbiotics"; less frequently, on the Bloomington campus, they say "sebeotics."] What's that? A disease?" Although I suspect that my friend Tom Sebeok and I feel that semiotics is our Apollonian disease, this is not the point. Semiotics does not concern any irregularity in human functions. On the contrary, it deals with the most characteristic human function, that is, the sign-function, or, in more elementary terms, signs. According to C. S. Peirce, a sign is "something which stands to somebody for something in some respect or capacity." In other words, to have a process of communication between two human beings (or, perhaps, between two living beings in general) requires the material presence of a certain object, be it an artifact or a natural event, which refers back to something that is not itself. This process of referring back may change from phenomenon to phenomenon, and the type of rule (or convention or attitude) that allows one to correlate a given presence to a supposed object or concept may assume different forms. In any case, there is an object for semiotic curiosity every time there is, as Roman Jakobson has stated, a “relation de renvoi” or—in more classical terms—when aliquid stat pro aliquo.
This relationship of referring back must be taken into account by the linguist, who studies the connection between signans and signatum; by the meteorologist, who infers from certain clues forthcoming atmospheric conditions; by the medical doctor, who establishes the relation between a given spot on an X-ray and an anatomical peculiarity; by the political scientist, who is concerned with the links between a given public behavior and an ideological attitude; and so forth. Therefore, when speaking of the semiotic field today, one is compelled to list an impressive, wide-ranging array of approaches, all of which are concerned in some way with the process of signification (aliquid stat pro aliquo) at different levels of complexity and discernibility.
When we take a closer look, there seems to exist, from studies of the genetic code to those of information theory, an underlying foundation to the pseudo-communication' between an unintentional sender and an unintentional receiver. Zoosemiotics (as studied particularly by Sebeok), in its turn, analyzes all the processes involved in the exchange of information between nonhuman beings. On other fronts, research is being conducted on olfactory signs (Hall ), tactile communication (Frank), gestural codes (Efron), and culinary conventions (LéviStrauss ). Motor signs used by handicapped people have been found to be ruled by complex codes that are independent of linguistic ones (Stokoe, Bellugi). Paralinguistics has listed and systematically organized every kind of suprasegmental feature and toneme (Trager, La Barre, Fónagy); and medical semiotics, established since the time of Hippocrates, is now considered a highly developed branch of general semiotics. Following the linguistic reinterpretation of Freud that was undertaken by Lacan, psychoanalysis now claims to be a semiotics of the unconscious. Kinesics, from its first pioneers De Jorio, Kleinpaul, and Mallery through the work of Efron to the recent researches of Birdwhistell, Ekman, and others, has made it clear that to gesture is to speak according to certain precise codes. In addition, proxemics (Hall) has revealed that spatial distances (that can be qualified) acquire different significations within the framework of different cultural systems. There is even a growing school concerned with the semiotics of music (even though musicology has been, from the time of Pythagoras and Boethius, nothing more than a theory of musical codes), as well as of grammatology (from the work of Gelb to Derrida), which deals with the autonomous roles of those writing conventions that cannot be considered uniquely dependent on the verbal codes that they translate into another medium. The study of iconic signs is reaching more and more complicated levels of sophistication (cf. the work of Panofsky and the rediscovery of the iconological tradition as one of several possible approaches to the study of signification in painting; Gombrich, Goodman, Metz, Krampen, Eco, and others have dealt with the dialectics between motivation and convention in images). Further, those visual signs that are more clearly and simply coded, such as traffic signs, have been investigated extensively and reduced to strict logical rules (Prieto).
It therefore goes without saying that from formalized languages to natural ones, logic and linguistics are two basic aspects of the semiotic endeavor. I am not arguing here whether these disciplines are two branches of a general semiotics (as maintained by Saussure) or whether semiotics itself should be posited under their headings. It should be enough to realize that each throws light on the problems of the others. When linguistics does try to go beyond its boundaries and to elaborate either discourse analysis or text grammars, it is unable to resolve all the questions raised without comparing the verbal issues with the complex of semiotic circumstances that makes a text understandable (for instance, think of the concept of behavioreme in Pike or of the recent analyses of Petőfi ). The comprehension of a text—which constitutes not only a semantic problem but also a pragmatic one—is strictly dependent upon a series of cultural codes that constitute a semiotic typology of cultures (e.g., Lotman and the Tartu School). As far as social systems are concerned, let us recall the semiotic approach used in cultural anthropology (Lévi-Strauss) and the semiotic notion of coding applied to social interaction as presented by Bernstein.
In the last decade a semiotics of architecture and of various objects has been elaborated (Eco, Prieto, Garroni, De Fusco) ; the various research projects on film are well known (Metz) ; and studies on verbal texts as well as other visual phenomena converge toward a more articulated study of mass media in general. And, I hardly need to cite the semiotic research done in literature (Jakobson, Barthes, Todorov, Segre) and on the more general problems of esthetics and the philosophy of art, as well as rhetoric.
I have so far outlined a complex field of interrelated research that advocates a semiotic perspective. Is it enough to state that therefore a discipline (or a science) called semiotics exists, or are we simply witnessing the passing of a rather snobbish fad? What are, in fact, the criteria for a discipline? First, one needs to have a precise subject; and second, a set of unified methodological tools. We could also list among the requirements—since a discipline is a science—the capability of producing hypotheses, the possibility of making predictions, and—as in the hard sciences—the possibility of modifying the actual state of the objective world. With regard to this latter set of requirements, let us take it for granted that many semiotic inquiries have produced, along with their hypotheses, the conditions for their falsification, as well as for their successful prediction. What seems to be more urgent is the need to establish whether there is a unique subject and a unique set of categories, and I will conclude my essay by also asking whether semiotics is even able to permit the practical transformation of its own object, that is, whether semiotics is a form of social practice.
1. First of all, let us consider the unity of the object. If one looks at the problems approached by the disciplines I have listed above, one realizes that in every case the core of each problem revolves around the process of referring back (aliquid stat pro aliquo ). That is, that there is a semiotic phenomenon every time something present can be used in order to lie—to refer one back to something that does not correspond to an actual situation. In other words, there is a semiotic phenomenon every time it is possible to speak of a possible world.
One could say that genetic processes cannot lie. That is true, and in fact I am not convinced that the genetic code should really be considered a code. The representation of the genetic process by geneticists, however, is a semiotic event. Chromosomes and DNA cannot lie; geneticists can. I am not sure that a machine can lie to another machine, but a man can lie to a computer (software being smarter than hardware); and perhaps the same applies to animals. The possibilities given above are not being excluded by any means, but the doubts raised also need to be examined. We are sure that one can lie using words, that it is possible to ring a bell in a church even though transubstantiation has not even taken place, to gesture like a Russian although one is a CIA agent, to dress like a bishop while being an atheist, to emit odors by chemical means, to mistake the social status of a person because of the megalomaniac size of his desk, to build a restaurant as though it were a pagoda (a very American performance indeed! ), to show a photograph of John and say that it is a picture of Paul, to touch up a photograph of Paul, to edit a film by presenting the effects as the causes, and so forth. Let us look at another analogy: if two molecules of hydrogen combine with one molecule of oxygen, the result is water; there is no alternative because, in this case, nothing stands for something else: something stands with or against something. But the chemical formula of water stands for water, and there is the possibility of representing the same compound by another conventionalized formula. Making chemical reactions is not a matter for semiotics; the representations for these reactions are.
In this sense, many disciplines are concerned with semiotic phenomena even when no one realizes it. Let us now consider two fields that are usually examined from the viewpoint of material interaction rather than symbolic interaction: the production of material tools and the exchange of these tools as commodities.
When an Australopithecine used a stone to split the skull of a baboon, there was as yet no culture—even if the Australopithecine had in fact transformed an element of nature into a tool. We would say that culture is born when (i) a thinking being establishes the new function of the stone (irrespective of whether he works on it, transforming it into a flint stone); (ii) he calls it “a stone that serves for something” (irrespective of whether he calls it that to others or to himself); (iii) he recognizes it as “the stone that responds to the function F and that has the name Y” (regardless of whether he uses it as such a second time, it is sufficient that he recognizes it). These three conditions result in a semiotic process of the kind shown in Fig. 1.
S1 represents the first stone used for the first time as a tool, and Sշ is a second stone, different in size, color, and weight from the first one. Now suppose that our Australopithecine, after having used the first stone by chance and having discovered its possible function, comes upon a second stone (S 2)) some days later and recognizes it as a token, an individual occurrence of a more general model (St), which is the abstract type to which S! also refers. Encountering S 2 and being able to subsume it (along with S!) under the type St, our Australopithecine regards it as the sign vehicle of a possible function F.
S1 and Sշ, as tokens of the type S t,, are significant forms referring hack to and standing for F.
The possibility of giving a name to this type of stone (and to every other stone having the same properties) adds a new semiotic dimension to Fig. 1. A name denotes the stone-type as its content, but at the same time it connotes the function of which the object-stone (or stone-type) is the expression. In principle, this represents no more than a system of signification and does not imply an actual process of communication (except that it is not possible to conceive the reason for the institution of such signifying relationships if not for communicative purposes). These conditions, however, do not even imply that two human beings actually exist: it is equally possible for the same situation to occur in the case of a solitary, shipwrecked Robinson Crusoe. It is necessary, though, for whoever uses the stone first to consider the possibility of 'transmitting' to himself the information he has acquired; and in order to do so he would elaborate a mnemonic device that was able to establish a permanent relationship between a given type of object and a given type of function. A single use of the stone does not constitute culture. To establish how the stone can be used again and to transmit this information to himself again does imply culture. The solitary man then becomes both transmitter and receiver of a communication (which is based on a very elementary code ).
The moment that communication occurs between two men, one might well imagine that what can be observed is the verbal or pictographical sign with which the sender communicates to the addressee, by means of a name, the object-stone and its possible function (for example: ‘headsplitter’ or ‘weapon’). In fact, the sender could communicate the function of the object even without necessarily using its name, by merely showing it. Thus, when the possible use of the stone has been conceptualized, the stone itself becomes the concrete sign of its virtual use. That is, once society exists, every function is automatically transformed (as Barthes has said) into a sign of that function. This is possible once culture exists, but, at the same time, culture can exist only when such transformations are possible.
We shall now move on to other phenomena as economic exchange. Here we need to eliminate the ambiguity whereby every ‘exchange’is a communication (just as some people think that every communication is a ‘transfer’ ). It is true that every communication implies an exchange of signals (just as the exchange of signals implies the transfer of energy) ; but there are exchanges such as those of goods in which not only signals but also consumable entities are transferred. It is possible to consider the exchange of commodities as a semiotic phenomenon not because the exchange of goods itself implies a physical exchange, but because, in the transaction, the use value of the goods is transformed into their exchange value —and therefore a process of signification, or symbolization, takes place, this later being further refined by the appearance of money, which stands for something else.
The economic relationships ruling the exchange of commodities (as described in the first book of Das Kapital by Karl Marx) may be represented in the same way as was the sign-function performed by the tool-stone (Fig. 2).
C1 and С2 represent two commodities devoid of any use value. In Das Kapital, Marx not only shows how all commodities in a general exchange system can become signs standing for other commodities, but also suggests that this relation of mutual significance is made possible because the system of commodities is structured by means of oppositions (similar to those that linguistics has elaborated in order to describe, for example, the structure of phonological systems ). Within this system, Commodity #1 becomes the commodity through which the exchange value of Commodity #2 is expressed (Commodity #2 being the item of which the exchange value is expressed by Commodity #1). This relationship is made possible by the existence, in culture, of an exchange parameter that we shall designate Ev (exchange value). If in a use value system all the items referred back to a function F (corresponding to the use value), in an exchange value system Ev refers back to the quantity of human labor necessary for the production of both С1 and С շ (this parameter being labelled HL). All these items can be correlated, in a more sophisticated cultural system, with a universal equivalent, i.e., money (which corresponds in some respects to the cultural name representing both commodities and their abstract and ‘type’equivalents, H L and Ev). The only difference between a coin (as a sign-vehicle) and a word is that the word can be reproduced outside of an economic context while a coin is irreproducible (it therefore shares some of the properties of its commodity-object). This simply means that there are different kinds of signs that must be differentiated according to the economic value of their expression-matter. The Marxist analysis shows, in addition, that the semiotic diagram ruling a capitalistic economy distinguishes both H L and Ev (which are in fact mutually equivalent) from a third element, the salary received by the worker who performs HL. The gap between HL, Ev, and Salary constitutes the plus value. But this fact, although highly significant from the point of view of an economic inquiry, does not contradict our semiotic model; on the contrary, it shows how semiotics can clarify certain aspects of cultural life and how, from a certain point of view, a scientific approach to economics consists in discovering the one-sidedness of some obviously apparent semiotic codes—i.e., their ideological quality.
Therefore the relation of referring back may be the subject of a science called semiotics, even though, in any case, it is the object of other disciplines. Every discipline at a certain stage of its metatheoretical development should be concerned with semiotic phenomena, and if one does not want to consider semiotics as a discipline per se, one should at least consider it as a methodological approach serving many disciplines. But in order to do this, semiotics must have a unified set of methodological tools; this takes us back to our second criterion.
2. How does one go about establishing a unified set of categories for a ‘pseudo’ science that has grown incognito for 2,000 years under the headings of the other sciences? One could start by saying that there exist at least three systematic approaches to semiotics, each distinguished by a growing degree of abstraction—i.e., the ones propagated by Peirce, Morris, and Hjelmslev, respectively. Each approach takes into account, by a unified point of view and the use of a coherent ensemble of categories, all possible systems of signs. I can now see your disappointment and frustration: lured by the promise of the revelation whether semiotics is indeed a science or an interdisciplinary method, we find that it is in fact three sciences.
It may be more challenging, perhaps, to say that, given the present state of its growth, semiotics is moving toward a merger of all three perspectives in order to establish a more flexible set of tools. It is, moreover, undergoing a series of transplantations from various scientific domains, of which linguistics is only one. It should be mentioned here that the scholar who has recognized this merger to the greatest extent and who therefore has been the most influential in establishing a unified set of methodological tools is Roman Jakobson.
Let me list among the paraphernalia of this methodological koine the following: the overwhelming use of the synonymous terms signans/ signatum, signifiant/signifié, expression/ content, sign-vehicle/significatum, and so forth, to describe the semiotic relationship in any sign system; the linguistic criterion of pertinence as applied to other code systems from gestural to folkloristic; the psychological notion of frustrated expectation, the mathematical one of information, and the poetic one of deviation from the norm, applied together to the analysis of messages; the list of the functions of language from Bühler to Jakobson as applied to any form of communication; the extension of the notion of binarism to certain syntactic systems and even to structural semantics; the concept of distinctive feature working outside the domain of phonology, from visual signals to genetic units; the opposition between selection and combination to explain phenomena of various languages, from movies to music; the unified use of the Peircean notion of interpretant, which, even in linguistics, does not work if not viewed as an intersemiotic substitution of a sign by another sign and so on; and, finally, the pair code/message originally derived from the mathematical theory of communication, borrowed by linguists, and subsequently used in other semiotic analyses. The list could go on and include other categories that come from other disciplines and are widely used today as ‘pansemiotic’ categories.
Many of the categories are very specific, but it is still possible to foresee a more wide-ranging and universal application of them. The notion of presupposition (linked with referential indices ), for instance, has proved to be useful outside of its original domain, the logic of natural languages. It is impossible not to consider presuppositions when analyzing pictorial texts, for example. In the same way many researchers have borrowed from rhetoric the classical figures of speech and applied them to visual phenomena, such as are found in advertising. On the other hand, by analyzing visual communication, it was found that mathematical tools such as transformation (from geometry) and isomorphism between graphs (from topology) as well as the more elementary notions of spatial and temporal directionality or vectoriality, could be applied to explain the phenomenon of iconicity as well as certain verbal phenomena. One cannot, for instance, represent the relations of command and embedding in deep semantic structure of a phrase without referring to these notions.
At present, there are many semiotic methods, each of which is focussing on the same object and yet is organizing the same set of categories differently.
3. As for the third requirement, a science (be it a hard science or a soft one) should be able to change the state of things of which it speaks. Let me say that semiotics is not a science in the same sense as anatomy is. It seems to me a scientific carrefour more similar to medicine, which does not limit its task to a given state of affairs but attempts to improve it—it is concerned with the way in which the human body functions and tries to influence that in a specific way.
Obviously I am not saying that semiotics is like a medicine of signs, or a therapy of the processes of communication. Such a utopia has already been advocated by general semantics, and I do not trust it. My metaphor (in which aliquid stood pro aliquo, as in every figure of speech) suggests that semiotics, more than a science, is an interdisciplinary approach—even though I could have also surreptitiously put forth the idea that it may be a sort of unified metatheoretical point of view governing a new encyclopedia of unified science. In any case, this interdisciplinary tendency (this is the second aspect of my metaphor) cannot avoid, at least at certain levels, an operational destiny.
The semiotician should thus always question both his subject and his categories in order to decide whether he is dealing with the abstract theory of pure competence of an ideal sign-producer (a competence that can be posited in an axiomatic and highly formalized way) or whether he is concerned with a social phenomenon that is subject to changes and restructuring, resembling a network of intertwined incomplete and transitory competencies rather than a crystal-like, unchanging model. The object of semiotics may somewhat resemble either (1) the surface of the sea, where independent of the continuous movement of water molecules and the interplay of the underlying currents, there is an all-encompassing form called the Sea, or (2) a carefully ordered landscape where human intervention continuously changes the form of settlements, dwellings, plantations, canals, etc. If one accepts the latter, one must also accept another condition of the semiotic approach, which is not like exploring the sea, where a ship’s wake disappears as soon as it has passed, but is more like exploring a forest, where paths or footprints do modify the explored landscape, so that the description that the explorer gives must also take into account the ecological variations that he has produced.
The semiotic approach is ruled by a sort of indeterminacy principle: insofar as signifying and communicating are social functions that determine both social organization and social evolution, to ‘speak’ about ‘speaking’, to signify signification, or to communicate about communication cannot but influence the universe of speaking, signifying, and communicating.
We use cookies to analyze our traffic. Please decide if you are willing to accept cookies from our website. You can change this setting anytime in Privacy Settings.