Logical Basis of Semiotics

Henry Hіż

When we want to say something, often we say it in words. We convey information, an opinion, an opposition, a question, a command; we tell a story, give an impression, reprimand, or confess in words, in sentences, in linguistic utterances. How the meaning is carried in words is the business of semantics, a part of linguistics.

Often meanings are not conveyed in words. By our dress we tell something about ourselves or about the occasion. The hostess—in some circles— shows respect to the guests by wearing a formal gown. An Appeals Court in New York barred a Roman Catholic priest from wearing clerical garb while serving as a lawyer in a criminal trial. The court held that the clerical collar was “a continuing visible communication to the jury” and this would prevent a fair trial.^1. A Bachrach photograph shows a distinguished businessman, wearing his immaculate shirt and necktie, with the standard expression on his face radiating success, comfort, confidence, and reliability. These businessmen have a different look about them than the kings in the eighteenth-century (or, for that matter, most centuries) pictures in the style of “king-realism.” The standard expression on the face of a monarch is perhaps pompous but does not show any curiosity. This was to show that a discussion with him was not possible. (In that respect, David’s picture of Napoleon is different; it portrays an intense and alert man, standing near his working-desk, with a stain of ink on his white trousers. Napoleon, a self-made monarch, liked this picture and did not allow the stain on the trousers to be painted over.)

Semiotics deals with meaning conveyed by any medium, not only by speech. Semiotics is therefore a generalization of semantics.^2. I will use the term semiotic property to refer to those properties which create meaning. Semantic properties are those semiotic properties that belong to speech. Speech has not only semantic properties. A sentence may be long; it may contain seventy words, twenty of which are monosyllabic. These are not semantic properties though they may in some unusual circumstances contribute to semantic properties, i.e., to the meaning of the sentence. Many objects, paintings, buildings, music compositions, customs, have meaning. But not all properties of these objects have semiotic roles. A painting has length and width; it weighs a few pounds and is worth some sum of money. Those properties ol the painting are not semiotically significant. A painting may have aesthetic values, but neither are these semiotic properties. One of the main difficulties in establishing semiotics as a field of systematic studies is to distinguish those properties of objects which play a role in meaning from extrasemiotic properties. In particular, semiotics may abstract from the aesthetic values of objects. Linguistics developed rapidly when it abstracted from considering which texts are beautiful, or in good style. In the long run, the fact that linguistics does not study sentences as artistically valuable has proven useful even for stylistics, which today uses some results and techniques of general linguistics. Progress in a science requires delimitation of topics. Social history of art records what different classes liked in different periods—and does not say whether they were right or whether they had good taste or poor taste. Iconography deals with the subject matter of works of art and not with their social role or with their artistic values.^3. Iconography is a special case of semiotics, as semantics is. Semiotics can learn from semantics and from iconography, although most of the particular methods of semantics or of iconography are too closely adjusted to the objects they study, namely languages and arts, to be directly applicable to semiotics, which is to be the study of meaning wherever it occurs. Another science which is a particular case of semiotics is model theory, which is that part of logic which deals with the relations between formal languages and their possible interpretations. Again, it is clear that semiotics cannot follow in details what model theory is doing, because model theory in its essential steps uses the structure of a formalized language and puts a limitation on models which it will consider. Only certain very general investigations of model theory may be helpful to semiotics and will be used below.

To delimit properly the semantic role of linguistic utterances, and similarly the semiotic role of our other communicative behavior, one may note that our utterances not only communicate their content but show something more about us. Our way of speaking betrays us as coming from a dialectal region, a social class, or a profession, or as pretending to know something about the subject matter, as coherent or disorganized, stylish or stupid, arrogant or polite. Our listeners are forming their opinion about us on the basis of what we say and how we speak, even if we do not speak about ourselves but about birdwatching, Giacometti, or category theory. The listeners conclude from the fact of our utterance not only what it says but also that we, the speaker, say it. But this is not the same thing. A sentence does not say that it is said. Nothing about the sentence or about the speaker follows from the sentence. From a sentence follows only what follows from its content.

If I say that Epicure performed euthanasia on himself, you are directly informed that Epicure performed euthanasia on himself. You may conclude from this that he took his life voluntarily. Indirectly, from my saying it, and not from the sentence I have uttered, you know something about me. First of all, you know that I know that Epicure performed euthanasia on himself. Furthermore, with varying degree of substantiation, you gather that I have read a book about ancient philosophers, that I am interested in ancient philosophy, or in Epicurean philosophy, or in the problem of euthanasia. You may wonder why I chose to call it euthanasia performed on himself rather than suicide. You may suppose that I have read about the event in Diogenes Laertios where it is described in detail. But this is not what was said. The difference is between what you were told and what you noticed. You were told something about the mode of Epicure’s death. And from that you rightly conclude that he decided to end his life. From what you noticed you infer my acquaintance with Diogenes Laertios. From the Euclidean axioms follow many statements about figures, lines, sets of points, but nothing about Greek culture. From the fact that Euclid wrote the axioms of geometry follows a lot about Greek culture of his time. We notice the presence of an advanced abstract intuition and use of axiomatic systems.

The concept of noticing is not semantic or even semiotic, though sometimes the term sign is used in connection with it. We notice that wood gets soft and take it as a sign of termites; we notice that a friend is coughing and conclude that he has bronchitis. But we were not told by anybody about softening of wood, termites, coughing, or bronchitis. Our ways of noticing things and generally of acquiring knowledge do not belong to semiotics. The only acquisition of knowledge with which semiotics deals is that of being told. A piece of wood does not tell us that there are termites in it. Nature does not tell us anything, though writers often use this metaphor. We are told this or that by elements of various systems of communication.

I am not trying here to define systems of communications; I am only drawing a distinction between noticing and being told. The latter is always in a “language,” in a “system,” in English or Italian, in conventions governing dress, in a tonal system of Western music (before Webern and Schoenberg). It is plausible that all our perception is done in some “categories of mind” and even that these categories are related to a language in which reports of our perception are stated, but those are different questions than the simple one which I am not addressing. We see ink as black if our vision is proper. But we understand the sentence The ink is black if our command of English is proper. We are told that ink is black if we hear the sentence Ink is black and understand English or if we hear the sentence Inchiostro è nero and we understand Italian. Similarly, we are told something by a painting when besides perceiving the colors we group the color patches into proper sets, if we are trained in interpreting them, if we are familiar with the style in which the painting was made. Otherwise we may either see it asemiotically, as just an interplay of color patches, or we may misunderstand it. An asemiotic interpretation occurs, for example, in listening to music if one perceives the rhythm in three with the first stressed and distinguishes it from the rhythm in three with the third stressed but does not know that the first is music to dance a waltz by and the other is a mazurka. A lot of the so-called aesthetic experiences may be asemiotic: experiencing the rhythm of the distribution of pilasters in the façade of a Renaissance palace, for instance.^4. Even when one has already given a semiotic interpretation to an object one may in addition give an asemiotic interpretation and by this assign to the object a more penetrating structure. For instance, the façade of the Notre Dame de Paris can be viewed, and is often viewed, as composed of two equal squares overlapping each other by half. Whether this structure has a semiotic counterpart—whether it expresses something—is a matter of dispute among art historians.^5. We observe the proportions of the façade, but we are not told anything by them.

There are borderline cases between being told and noticing. A recording tells us that the telephone number that we have dialed is disconnected. Here the original speaker did not address us directly. When the recording was made, the person was addressing anybody who might be calling a disconnected number. A busy signal tells us that the other party is using the line; the long ringing signal tells us that the line is not busy. The speaker in all those cases may be taken to be the telephone system which in our culture is a partner in communication. We tell it, by dialing, that we want such and such a party and it follows our request or tells us that it cannot satisfy it and why. But when in our receiver there is no signal whatsoever we are not told anything by the telephone system. We did not reach it. And we notice that we did not reach it and gather that our telephone is out of order. This example also shows that the intention of communication can be fairly remote in an act of communication.

An explanation of the meaning of a sentence or of the meaning of a picture, a building, or a sonata is a difficult task. From all previous discussions (and there have been many, from Plato on) it is clearly naive to think that a part of a picture represents something and that the meaning of the picture is the sum of the meanings of the parts. Moreover, it does not seem very useful to consider that a picture has symbols which stand for, or represent, or denote some entities. And that holds not only for abstract, nonobjective art, but also for realistic pictures. A representational, denotational semiotics says that a part of David's picture Marat Assassinated^6. represents the body of Marat, another part a knife, another a towel, another a drop of blood. One will not go very far with a representational semiotics. No part of a picture, no symbol, stands for death, or for the body of Marat being in the bathtub. As a matter of fact, the bathtub itself is not shown at all; it is all covered by towels, sheets, and blankets. We also may infer that there is a source of light somewhere behind Marat’s head. But the lamp, candle, or window does not appear in the picture. We infer it from the distribution of shadows and from the technique of painting the wall darker closer to the supposed source of light and progressively lighter toward the other side. It is as if the scene is larger than its part shown in the frame. There are many facts conveyed by the picture. More precisely, there are many English sentences which we infer from the picture. Those sentences are jointly the meaning of the picture. Thus I will not say that the picture represents Marat, the towels, the knife, etc., but that from the picture one infers that Marat’s body lies in a bathtub, that his blood is coagulating, that there is a lamp, candle, or window to the left of Marat’s head, etc.

The meaning of a sentence in a natural language is also, roughly speaking, the set of its consequences. Less roughly, to the meaning of a sentence belongs any sentence that is a consequence of the first sentence taken not alone but with other sentences which we know, or assumed before. For we do not draw consequences from an isolated sentence but from it taken with a lot of other, mostly banal, sentences which often remain tacit. Still more precisely, the meaning of a sentence is the difference between what one can infer with the new sentence and what one can infer without it.

Let A be a set of assumptions,^7. Cn(X) the set of consequences of X, X U Y the set of X augmented by the set Y, X — Y the set X with all the members of Y deleted, {α} the set having α as its only element; αX is read α is a member of X.^8.

In this way, the meaning of a sentence may change, if the set A of accepted sentences changes. For some studies the set A of accepted sentences is taken to be constant (for instance, the common knowledge of a linguistic group). In others, it may change as the conversation progresses.^9. In the extreme case, A may be empty and then we have the meaning of the sentence α in isolation. Of course, this is only one of many senses of meaning as applied to utterances. It is the information meaning and not, for instance, a psychological meaning, which may be useful if one studies the surprise value or the commercial value of an utterance. The information meaning is useful in linguistic semantics, and one may try to utilize a similar idea for a more general semiotics where one takes as α not a sentence but a behavior, a costume, a picture, a musical composition, a building, etc. To study this extention of logical semantics to general semiotics, let us look more closely at the concept of consequence by means of which meaning is defined. Logicians accept the following principles for it.^10.

If α is a consequence of A, then α is a sentence and A is a set of sentences. Sentences are, of course, in a language and the concept of consequence is relative to that language.

A set of sentences is included in the set of its consequences; in other words, each sentence from among sentences A is a consequence of sentences A. In particular, each sentence is a consequence of the set consisting of that sentence alone.

This is the closure principle. The consequences of consequences of a set are just the consequences of it.

This is called the compactness principle. If a sentence is a consequence of a set of sentences, then it is also a consequence of a finite part of the set. Whatever follows from infinitely many sentences follows already from a finite selection of those sentences.

There are usually other principles of consequence given which show how this concept combines with specific constructions present in the considered language. For the language of logic, one states that a contradictory set has every sentence as a consequence of it.

In principles 2-4, the concept of consequence is used very generally. There may be several more specific concepts of consequence. Often, one uses a specification of a consequence which is provability . α is provable from A if there is a finite sequence of applications of some rules of inference which lead from A to α. Another specification of the concept of consequence is the semantic consequence which is the preservation of truth; α is a semantic consequence of, A if a is true whenever all sentences of .A are true; that means, no matter how sentences of A are understood, no matter how they are interpreted, if they are true in that interpretation, then the sentence α is true in that interpretation as well.^12. The various specific concepts of consequence are not at all equivalent but they all have the properties stated in principles 2-4.

To come now to semiotics, we want to make consequences not from sentences but from a picture and a set of sentences. From what we know or assume (however provisionally, fictitiously, or for fun) and from a picture we infer consequences. The consequences are English sentences. Let us therefore include in our study some objects in addition to sentences. Those objects may be called meaningful objects (abbreviated as Mo). Sentences are among meaningful objects and therefore semantics of a natural or of a formal language is a particular case of semiotics. Just as sentences are in some language or another, so meaningful objects are in one or another convention. It does not make much sense to speak of a sentence in abstraction of a particular language and it should not make any more sense to speak about the meaning or the content of a picture outside a convention. What has meaning in Christian iconography, say a halo, may not have meaning in Moslem or in Buddhist iconography. Or it may have a different meaning. An eagle on an American coin is not a symbol of St. John’s Gospel but of the U.S. government. What counts as a picture of a flower is relative to a style, a period, a culture.^13. In all statements here when S appears it is to be taken as S in the language L and when Mo occurs it is to be read as Mo in the Convention C.^14. One may generalize the concept of consequence to the concept of semiotic consequence (symbolically Iocn) and one may state some principles for it.

If α is a semiotic consequence of the set A, then a is a sentence in a language and every member of A is either a sentence in that language or a meaningful object in a convention. A particular case of conventions in which meaningful objects are formed are languages with sentences. Thus semiotic consequences are sentences inferable from a meaningful object (a ritual, a drawing) and from sentences assumed.^15. Semiotic consequences contain the semantic consequences of those assumed sentences:

The closure principle is similar to 4:

The problem of compactness with respect to the semiotic consequence is a subtle one. Applying the idea of 5, if α is a semiotic consequence of a meaningful object and of a set of assumptions, then a finite number of those assumptions should suffice to infer it:

The truth of principle 10 may be questioned. And so is the truth of 5. There are higher order formal languages for which 5 is false. Therefore, before one assents to an application of 10 one has to examine the convention in which the studied objects are coined and the grammatical features of the sentences that are assumed. The matter is complicated and future research on semiotics should bring results concerning the applicability and limitations of 10 which will contribute to our understanding of the differences between the formal structures of various conventions. Incidentally, another kind of compactness may be of interest. Namely, instead of, or in addition to, the compactness of the set of assumptions, we may speak about the compactness of the picture itself. There are meaningful objects in a convention which have as their part (in the case of a picture, a physical part) another meaningful object in that convention. If you divide David’s painting in half vertically, the left half is still a painting in the classical convention together with the usual assumptions and gives as a semantic consequence A man was killed with a knife. A part of a picture may be discontiguous; it may consist of a fragment here and a fragment there. Triptychs and comic strips are discontiguous meaningful objects. Normally, a picture can be taken to be a two-dimensional continuum of points. Can a finite selection of points of the picture be considered a picture? And is it true that whatever is conveyed semiotically by a continuum of points can also be conveyed by a finite selection of them? An affirmative answer may be called the hypothesis of pointillism:

The pointillist hypothesis is a statement which may be viewed as a claim in psychology or physiology of perception, or in epistemology. But neither a point (in the geometric sense) nor a finite set of points is perceivable in isolation. I do not claim that a geometric point is not perceivable at all; certainly we perceive the intersection of two lines, that is, we see the point of intersection. But it requires the context of the two lines. The pointillist hypothesis is false if a point is understood as a circle with radius zero. However, if a point is understood as a minimal meaningful object, a meaningful object of which no proper part is a meaningful object, then the hypothesis may well be true. Then it is not a physiological hypothesis. Speaking about minimal meaningful objects is like speaking about words or morphemes. They are repeatable entities. It does not matter whether you take a word or a picture or a musical composition to be a physical event with definite time and space coordinates or whether you think about them more abstractly. Among some objects there is a relation of repetition and this is what makes the objects meaningful. Nothing unrepeatable is meaningful. But further analysis of repetion may lead to the conclusion that in a natural language only sentences and texts are repeatable primarily, and words, morphemes—or perhaps phonemes—are so only derivatively. Repetition of a sentence is a particular case of the relation of consequence. The hypothesis of compactness says that a consequence of a picture is a consequence of a finite selection of meaningful objects which are parts of the picture.^16. Before one can really say whether the hypothesis is true, much more study must be done about the foundations of semiotics. Here, an important idea is that not all properties of a picture are semiotic properties. It may very well be that, as Goodman wants, a picture is unrepeatable—with all its qualities.^17. But semiotically it is repeatable; there may be another picture with the same consequences.^18. Aesthetic properties and technical mastery of a painting or of a building do not go into consequences which are English sentences.^19.

The semiotic meaning of a meaningful object depends on the set of assumed sentences or other meaningful objects and consists of the increase of the field of consequences of the set augmented by that meaningful object.

The semiotic meaning of an object depends both on what kind of meaningful object α is, that is, to what convention it belongs, and on the content of the set A of assumed sentences and possibly of other meaningful objects. In what language we have to read an inscription or hear an utterance is not stated in the inscription or in the utterance. We understand the utterance as being in a particular language. Similarly, we guess what convention or style a painting is in and we “read” it accordingly. The reading of a text with understanding and the understanding of a painting consist in drawing the consequences from it plus an appropriate set of assumptions. In an ordinary discourse the assumptions are taken from common knowledge. In art they are found in the real or fantastic realm. A picture of Bacchus awakes in the spectator a number of stories from Greek mythology and they are used in our drawing of inferences from the picture. The semiotics with which people look at a picture varies depending on what they know and what other pictures they have seen and remember. Our ability to shift the system of assumptions, the story in which we place a sentence or a picture, is an important feature of our thinking. For a sentence in a conversation, what was said before is accepted as true and relevant, and is used for making consequences and therefore affecting the meaning of the sentence. And the environment of a picture may affect its meaning. If in conversation a sentence is used twice, then the meaning of its second occurrence is quite different from its meaning the first time. Because the meaning, according to 1, is only what the sentence said that is new, no part of the meaning of the first occurrence of the sentence is in the meaning of the second occurrence. Maybe it will be considered meaningless without, at least, some variation in intonation. If tolerable at all, the meaning of the second contains, perhaps, statements about the meaning of the first, that it is important, that it really is so, or that it was said. Now, suppose you see two identical pictures (say, two copies of the same photograph) hanging side by side. Though we can force ourselves to contemplate one of them separately, in the arrangement they are affecting or annihilating each other. We will make only comments about their similarity. This technique may be used to add to the meaning of the picture. Andy Warhol put in a picture 20 pictures of Marilyn Monroe, and we conclude that her appearance was for mass consumption.^20.

A sentence, or a text, or a picture often makes an allusion to another meaningful object; a religious painting to the Gospel, Brahms’ violin concerto in D to a čardash. We can define allusion using the terminology just developed.

Two objects in different texts allude to each other when there is a consequence of the two objects which is not a consequence of each of them separately. For instance the sentence To buy or not to buy has as an allusion the consequence: Do not worry about being, think about buying. In this and similar cases we form a text of which the first sentence is the old one and the second sentence is the new allusion. So that α locn ({β} U {ɣ} A)in 13 could be replaced by α Iocn({βɣ}U A). But this specification is inappropriate when β and ɣ are in different conventions, different media like painting and the Gospel, and we cannot form a text by juxtaposing them. Note that according to 13, when β alludes toɣ, then ɣ alludes to β as well. There may be an advantage in keeping the relation of alluding symmetric. A Renaissance Entombment alludes to the Gospel and the Gospel to the Renaissance Entombment. The meaning of the text of the Gospel may stay the same but the set of its allusions changes with time. We associate with the text of the Gospel Renaissance paintings.

The definition of allusion (13) presents some interesting difficulties. If β and ɣ are axioms in an independent axiomatization, then often there are consequences of the two axioms which are not consequences of each of them separately. We say in such a case that the axiomatic system does not accept a split. But then from 13 it seems to follow that the two axioms are allusions to each other—which is counterintuitive. To rule out such cases, it may suffice to require that β and ɣ be in different texts. A proof, or a system, is a typical text. To the fact that a proof is a text testifies the ease of inserting into a proof such phrases as hence, and, on the other hand. Two axioms of an unsplittable axiom system are fragments of the same text. But if β and ɣ are to be of different texts, the definition fits well. However, this calls for a definition of a text. Or for stating some principles governing texts as such. The task of characterizing the concept of text is particularly difficult, as we do not know enough facts about texts. The grammar of texts, or discourse analysis, is not a very advanced science.

To return for a moment to David’s picture; on the table by the bathtub is carved “À Marat, David.” However, we do not conclude that just after Marat was killed and before his body was removed from the bathtub David carved an inscription on the table. It is not a unusual signature; we take it as an essential part of the picture, we draw conclusions from that part. Still, it is on a different order, as if in a different language or manner of speaking. The realism of a painting does not require that there be only one convention used in it. Here there are two. The carving is not for real in one convention and the scene of Marat’s death is not for real in the other. Note also that the shapes of the letters make the carving look like a Roman inscription. This is an allusion to the fact that during the French Revolution Roman civic virtues were revered. And the inscription is a personal tribute of David to Marat.^21. The history of painting has taught us to operate with more than one system in a picture and to see their interplay. In Nativity by Caravaggio there is Mary and the Baby and Joseph and an ox—and, in addition, St. Francis and St. Lawrence. There is also a flying angel. We are not told that St. Francis was present at the Nativity as we are told the ox was. We know that we must conclude from this picture that St. Francis was there in another—perhaps spiritual—sense. A similar mixture of conventions occurs in all the many pictures in which, beside a scene of Nativity or of Crucifixion, a benefactor of a church is kneeling. We know he was not there. We learn the combined convention of the two orders though each order by itself may be in the same convention, as it is in Caravaggio. This kind of combination of conventions may be distinguished from a joined representation of two orders of supposed reality. For instance, as in very many paintings, in Pietà by Perugino the body of Christ does not have much weight. It is lying on two Marys’ laps but their dresses have no folds. What is conveyed is that the celestial order is different from the terrestrial one. There are numerous Crucifixions where the body of Christ does not really hang on the cross but floats in the air. Here we do not have two conventions; we have two worlds spoken about by one convention. A consequence of Perugino’s Pietà is that there are two kinds of reality and that Christ’s body is not totally subject to physical laws.

The logical or semantical concept of consequence requires some principles which will combine it with some important words in the language—in particular, with the logical connectives. Statement 6 binds the concept of consequence with negation. How about negation in general semiotics? Among the consequences of a picture or a building, there are negative sentences. A consequence of Perugino’s Pietà is, The body of Christ is not a purely physical object. It is not easy to study directly the connection between consequence and various parts of a painting or various ways the painting is put together from its component parts. An imposition of linguistic structures on a painting or on a building, or even on a poem, is a dangerous enterprise. But from a painting may follow a negative sentence, or a conjunction, and therefore indirectly there is a link between a painting and logical connectives. Similarly, a Renaissance palace tells us that the landlord is not afraid of military attack. A Roman palace makes a negative allusion to a castle. There are on it no corbels supporting a machicolation. But many Florentine palaces actually do have relictish corbels and minute machicolations, making a positive allusion to military architecture. If a meaningful object is making a negative allusion to another meaningful object, they may be considered to be in some sort of negation relation. But one has to be careful with negation—both in English grammar and in general semiotics. There are paintings which together with a usual set of assumptions are contradictory: the set of consequences is the set of all sentences. An interrogative can be a consequence of a picture as well. In Caravaggio’s Calling of St. Matthew, Christ and St. Peter approach a pool of gamblers. The gamblers are not surprised to see the Biblical figures. One of them asks:Do you call my gambling partner? Negation, conjunction, interrogation are therefore among consequences of pictures.

There is an old controversy whether art or art criticism is more difficult. About poetry Montaigne wrote that il est plus aisé de la faire que de la conoistre. Later, in the eighteenth century Philippe Destouches, a theatre writer, was of the opposite opinion: La critique est aisée, l’art est difficile.^22. To me it seems that the theory of art, especially its semiotics, is in a rather primitive stage, but I cannot say that about art. Art exists in all cultures, theory of art in some cultures only. Reflection about art and customs is like reflection about language; people speak in all cultures but write grammars only in a few. Therefore, I am siding with Montaigne.