Notes on the Cybernetics of Language and Video

by Stephen Jones.

Sydney, August, 1979

FOREWORD

The material within consists more or less of notes about certain aspects of the interrelationships of language, knowledge and social being.

The cybernetic nature of the world and the interdependence of all systems in the world has only recently entered into the discourse of the social sciences of the West. There is a major lack, in the language we use, of means for describing and dealing with this interactiveness and the processes of change inherent in all eco-systems. Without the words to talk about change and system interactiveness we are effectively prevented from knowing of these aspects of the world in which we have our being. We do not see ourselves in inter-relationship, we do not recognise the contingency of the systems containing us, until we see the inter-relatedness of our social structures as systems of relations.

We exist within a social framework which has a myth structure interactive with the history of society. We are socialised into this framework as we become members of society. Our parents, our learning language and our education along with that ever so prominent purveyor of information, roles and attitudes the media, in particular television, provide this overall socialisation.

I would like to thank Ruth Waller for critical input and Nell Smith for the typesetting. Also I would like to invite any feedback you may have. Any correspondence should be addressed to me ...

A short list of references follows:

Barthes, Roland: Mythologies

Bateson, Gregory: Steps to an Ecology of Mind. Paladin, London 1972.

Battcock, Gregory (ed): New Artist's Video. Dutton, New York, 1978.

Burgin, Victor: Victor Burgin, Stedelijk van Abbemuseum. Eindhoven, 1977.

Hall, Sue and Hopkins, John: "The Meta Software of Video", Studio International Video Art issue May/June 1976, London, pp.260ff.

Hopkins, John, et al: Video in Community Development, Centre for Advanced Television Studies, London, 1972.

Levine, Les: Using the Camera as a Club. Museum of Mott Art Inc, New York, 1976.

Jones, Stephen: interview in Ozone, ed. Terry Reid, Sydney, 1978.

Reynish, Richard: "Notes on the Politics of Information", JCATS, 3 no. 1, London 1975.

Ryan, Paul: Cybernetics of the Sacred, Doubleday, Anchor, New York, 1974.

Wiener, Norbert: Cybernetics, M I T, 1948.

COMMUNICATIONS and the SOCIAL CONTEXT

The mind and its content form the context within which all incoming information is interpreted.

What we see is understood in terms of what we already know, we have each a unique history – mediated socially via language – against which we compare all incoming data – the redundancy of the system might be as high as 90% so that we develop apparently constant appearances over a longish timebase. A building tends to stay put, as does the language, weathering only slowly over time.

We perceive something new because we see a difference – it might be a new case of something we already have a name for, or it might be something that is as yet nameless (noting that all experience is in some way perceived).

A name implies a history – at its emergence a new (technological) medium lacks history, its potential content as yet unrealised. In practical terms the appearance of a new medium is governed largely by an activity of pushing old content from previous media through a new channel: to the degradation of both the old content and the new medium. Witness the cinema on T.V.

So video as a new medium appears, its properties are not explored, it becomes broadcast television: one way and serving the specific needs of the broadcaster only. We use it in a manner we already know: centralised distribution of information, all the more carefully packaged through the expense of the production process.

We neglect those aspects of the medium which would be the most humanly valuable. These properties of video which I am suggesting we should be exploring, are that it is

• a two way, "realtime", instantaneous and simultaneous process through time;
• a mirror on ourselves, both realtime and at one or several removes, on later replay;
• a communication with ourselves and with others, with the feedback function of the system setting up a learning situation.

For example: when used in the realtime mode, camera directly linked to monitor we see and modify our behaviour according to the feedback we receive from the display.

The camera looks at you from over there (part of you but separate), it relays an image on to the video monitor screen, you can then look, or not, each is still a response to this visual information loop providing feedback to you about your appearance, behaviour and relationship to the context provided by whatever space this system occupies.

This response activity is a process displaying communicative form in that there is a feedback of information, from another viewpoint, about oneself. We are in an information loop, not in a one way situation, but in a two way, action and response, situation; in this case with one's image on the screen, an extended self, a part of you but outside of you.

(Of course, why do we think of it as outside of us at all, or anything else "out there" for that matter. It is in one's perception that these things have their existence, and that is our perception within our organism/mind. The objective world is an appearance only shaped by our organism/mind, social being and language.)

This information/feedback loop is a process, the relationship between the elements of which changes constantly, in the present. We behave and watch our behaviour, we feedback off ourselves and by this trick of separation we enter into communication with ourselves. This form is analogous to the more general communicative form of a conversation between two people acting and responding in concert.

There is no possible dialogue with T.V. but video as a technology is perfectly capable of being responsive – i.e. of being used in a dialectical manner.

For example, on another scale, we have the teleconferencing link-up where (usually) business people and others are able to link up with their offices and others', in a fully two way video/audio link-up and carry out a full conference with cameras, cables and monitors interfacing them.

People within a social milieu become part of a system of interrelating entities. We can describe physical boundaries: that is the walls and floor of the room for example. We can describe the social limits of a particular system/group in an agreed 'conventional' rnanner. (And ultimately all boundaries are only conventional even the physical.) This system exists over time and undergoes change as energy and information pass through it. People's interactions and communications provide some of the energy and information within the system (we are looking at the informational aspects of the system, i.e. communications). As the system of communications develops we may observe the processes operating within the sub-systems of one-to-one and one-to-n relationships developing between and among the participants in the system. (Much art has already discussed this kind of problem but not especially within the video medium.)

These interactions show up the network of relationships within the system. The properties of networks provide the rneans for the feedback of information and the control of the communicative processes' development. You can ignore the feedback potential, as in broadcast T.V., if you choose, but if you're at the bottom of the hierarchy (as is the audience) nobody cares anyway. If you're at the top of the hierarchy: the T.V. station; then you've already got the control by avoiding the possibility of feedback completely. The only feedback the T.V. station looks for is the financial feedback from the advertisers. Feedback and control go hand in hand.

In small groups, in regions, in communities the networks of organisations and individuals are the sources and the media of information flow through the various interlocking systems. These systems display processes of interaction over time, and by memory and documentation develop a history. Upon the history of this system is all the new information and meaning predicated. Interlocking systems each with their own histories provide the structure of society. The avoidance of change despite all this is a process of institutionalisation and in the semantic world myth provides those apparently natural conditions of relations between people which are the content of the institutions (of, for example, the family).

All information flowing through social systems is in some sense communications. Of course rnost communications are mediated by various technologies. The individual face-to-face conversation and small groups are among the very few communication structures which are not mediated via some technical system.

Hall and Hopkins (in Studio International, May-June 1976, p. 262) describe the communications process thus: people

“exist in a dynamic relation with each other and their contexts. The interrelation of their contexts (including the material conditions of life) metaprogramme and expectations causes them to engage in activities which are intended to satisfy needs as perceived by the people themselves. The activities result in achievements which in turn modify the metaprogrammes, act on the contexts and generate new expectations. This can be better understood as an ongoing, dynamic and cyclical process rather than a linear chain of events" (the metaprogramme is a "set of instructions, descriptions and means of control of sets of programmes" ibid.)

Because people's needs are not precisely the same from individual to individual each person will feed into any operating communications networks from different points of view, with different priorities, etc., and it is from these different weightings on the system that much of the energy driving the system derives. Also it is this intersubjectivity which brings complexity and renders these systems indescribable within conventional linear notions and language. A descriptive system, known as General Systems Theory, based on complex causal chains is being developed.

All these patterns of relationships; feedback, etc. appear at all levels of the world we know, in the biochemistry of organisms and the ecology of the biosphere, in psychological and social activity and social structures. These patterns also appear in the technological sphere (with language as the prime social technology) in communications and technical control systems. Video models these patterns of feedback at many levels of their appearance, in particular at the social and technical levels.

Semantics has to do with the function of meaning in language. Information and meaning is a function of the context in which it is found, so that the same image in two different contexts will have two entirely separate meanings. If you're working with images and you're working in such a way that as the images are appearing in various contexts at various times, in random or structured order, then the way the image works, the meaningfulness of that image at various times, is going to be changing. (Where the signification of the image becomes static despite changing contexts we have the appearance of mythic information in the sense of Barthes.)

LANGUAGE and OBJECTIVITY

I want to show here that the concept of objectivity is misleading, and therefore, so is the concept of subjectivity. The argument is based on Wittgenstein's 'private language argument' coupled with Heisenberg's 'principle of uncertainty'. Wittgenstein argues that all our knowledge of the world, all that we think about, talk about and so on, is gained out of experience, interaction and action, i.e. is gained out of linguistic processes within the environment. One can have no existence as a (social) human being outside of a social framework and the social framework is constantly mediated by shared language and that in the long run that there can be no private language that would have any communicative function. My understanding of myself, i.e. my identity, is disclosed by my interactions. I do not exist as an island – 'no one is an island'.

Heisenberg, in attempting to explain certain curious results in physics, recognised that the tools being used to make observations in the subatomic field were in fact having a major effect on events in that field and were thus giving misleading results. The presence of the observer in the field effects the events being observed. This, of course is obviously true in social behaviour, though it is only in recent years that the social sciences have begun to accept it. The uncertainty principle means that one cannot enter an environment without effecting it in some way.

Effect is interaction, or at least will lead to interaction or the avoidance of interaction which in itself is a kind of interaction. To reach some sort of state of 'objectivity' within an environment is impossible. Language proposes the illusion of objectivity merely as an abstraction. The failure to recognise this illusory nature of language will inevitably lead to bad results. The corollary is that as 'subjectivity' and 'objectivity' are linguistically opposed, the disappearance of objectivity implies the similar disappearance of the concept of subjectivity.

This argument suggests that one cannot gain information from a system without entering into it and acting upon it, in some way altering the conditions of the system, if only by talking about it.

There is no subjectivity or objectivity. The presence of the observer within the environment being observed, or even outside the environment being observed, alters the processes going on in that environment. (Heisenberg's principle of uncertainty).

The subjectivity/objectivity illusion exposes one of the major problems of language; that language allows for the appearance of concepts that in no way reflect the reality of things. Another problem related to this one is that language, as we use it, does not allow for an adequate description of things in process but tends to objectify and make static. So that we consider persons, things and events, normally in process, as static unchanging objects to be maintained in this static condition. Thus the fear of change so inherent in much of society. We employ no language with which to accommodate change. A further consequence of this can be seen, for example, in the objectification of women in advertising. Of course, these comments refer largely to the western language, English in particular. Our language doesn't really effectively deal with processes. In attempting to describe what's going on language tends to make things static, it holds and pins things down and defines them.

The processes of a social framework are basically linguistic, and display certain characteristics such that the communicative function goes something like this: A makes an action and talks to B, B listens and responds in some way to A, who can continue to develop the conversation by further response. It is not A acting and then B acting in isolation from what A has done, unless you purposefully go about that, which is the Absurd thing. It is A acting, B responding and A responding to B's response and so on in a developing loop of interaction. So that you get feedback operating, B feeding back to A who feeds back to B, constantly modulating each other as long as the conversation lasts. A to B and back again. I am defined by your response to me (i.e. by your cumulative response). 

If B chooses not to respond, this applies an inhibitive feedback on to the conversation loop, which will then, in the normal course of events, probably die. I.e. the inhibited response (or no response at all) is just another response from which one derives information and further modulates one's communicative activity. So all response, whether it is empty or very full, influences how you carry on. You can stop talking with each other, you can go away; or you can, if it's a positive or full response, then continue further into the conversation. Positive and negative feedback modulate the flow of the interaction loop.

The communicative function, which is also language, is governed by interactive processes. I am defined by the way that I relate to you and by your response to me. I become the synergy (the synthesis) of all responses in all situations constantly updating.

This results from the way that we become human; get socialised. The way we know about the world, the whole business of cognition and perception is underpinned and continuously modified by what we already know of the world. Because we cannot know about something until we've named it. We don't know what it is until we talk about it. All knowledge of the world is socially mediated via language.

The social set from which we perceive, i.e. our framework, also governs the way in which what is perceived is interpreted. We live within an ongoing condition of intersubjectivity with our physical and social environs. No two people can ever occupy the same space at the same time, consequently we all have different histories, different consciousness of ourselves and others. This history and consciousness is our framework from within which we relate to all phenomena, information, other people, social structures and institutions. As we are contained within various social structures we become part of others' consciousness. What each of us does and says influences the way the other sees us. The way you see me is a function of your framework plus the interaction we have had or have avoided. Similarly the way I see you is a function of my framework plus our interactions. I exist within you as a perception and a name and the set of linguistic artefacts (some active, some mythic), contained within your general framework for functioning within society. Likewise are you perceived and defined within my framework. This operation generalises out into all social relations.

All these things occur over time and are therefore undergoing constant renewal and change. The redundancy or repeating elements of this communicative system provide some sense of continuity.

We exist within an inter-subjective process that is the meta-framework of society, a dynamic and complex web or network of interaction and relationship.

MYTH

The myths which accrue around a social structure provide much of the framework, or the context, within which the apparent givens (the institutions, etc.) of the society are couched.

Roland Barthes, in Mythologies, proposes that there are in the environment; the social, cultural, built/natural environment, images which have become the cultural archetypes of our world, our society. These images are the mythic images, the conspicuous signs of what is normal in social behaviour.

Myth informs the basic frameworks within which we interpret the goings on around us. Myth forms the frameworks from which we perceive and moulds the way we work and the roles we take. We are governed and constrained by these images because we are in various ways forced into the roles presented by these image structures. We are offered no other information in the presentation of the myth so that we then have no critical grasp of the wider context in which that myth operates. Advertising is the major purveyor of mythic imagery with our only proposed role that of the consumer.

The political myth that we must preserve our social institutions at all costs is well served by a language structure which hides the inherent changeableness of things, leaving people to believe that these institutions have always been there, as they are the "natural" forms. Fred Flintstone and Star Wars assure us that society has been like it is now from the far distant past into the far distant galactic future. Myth sets up the historical so that it is perceived as natural.

FEEDBACK

Social feedback...

Bio-feedback has direct biological consequence, whereas social feedback has social consequence rather than direct biological consequence. Recognizing that the integration between biological and social action is very strong, and they cannot really be separated, it is only a trick of language that allows us to make a fragmentation of this sort.

There are many classes of feedback in society and it is to do with the cybernetic process and the action of inhibition and control in social processes ... or in the action of feedback as inhibitive or positive control of a process ... or in a self-corrective process itself. Inhibitive in the sense that if you're an actor working/rehearsing in front of a camera so that you can see what's happening, you then can correct your actions as an actor, according to the things you see in the video, so that you will inhibit the mistakes and enhance the quality of the thing in some way.

Video feedback...

One can take this approach to video, the feedback function, to advantage within the context of performance or installation where the data is processed through a feedback video environment, feeding back to the audience information about their own actions ... where they can control the actions and feedback off the monitor and camera relationship and one's relationship to these things starting to do things within it creatively.

So we begin to see feedback as a system process, a class of action within processes, having a lot of levels of operation.

Though it has a direct electronic one (endemic to video), it has the macro scale social function (video access, video as a means of social facilitation) and intermediate functions as in installations, etc. Obviously feedback is a generalised function, not at all restricted to video, and has a great many areas in which it manifests, e.g. the conversation, all the other processes of social interaction. Feedback is the action that takes place in all processes of interaction within a state, or within an existence, or within an ecology of some sort; a social ecology, a techno-ecology, a biological ecology or any self-corrective system.

The thing about video here is that we find a means for modelling many of the processes of consciousness. Feedback is the response of the context to an output from the consciousness to which the consciousness responds becoming a process in action, live, containing and maintaining itself, open-ended, subject to change, and non-conservative. PROCESS. So via its nature as an analogue of consciousness – no matter how partially that nature is revealed – video becomes a tool that allows one to operate on the processes of cognition and one's environment and the social relations within that environment.

PROCESS and KNOWING

One aspect of the processes that we must deal with is the nature of change itself. To the Taoist there is nothing constant in the universe excepting change itself. We use language which tends to negate change, or at least slow it down. However we are looking for a language of process.

Our language is not at all strictly verbal. I couldn't talk to you if I didn't move about and gesture as I talk. These are the peripheral activities of proxemics and body language. Communication is broadened by these deeper socially structured aspects and we get our first hint of the intuitive here.

We are trying to get to a language which enables us to understand open structures in time, the structuring of knowledge through time, history, the function of history, and all those sorts of things.

The Chinese Taoist view (as expressed in The Classic of Change, The Classic on the Tao, Chuang Tzu, etc.) seems to be something like this:

The important thing one has to do is to not grasp for the information. To refrain from directing: to allow the processes that are involved to occur and go on as they will and to accommodate to those processes and to therefore become carried along in them and realise the work as it should be rather than to try and push the environment into a situation in which it is not willing to go perhaps ... it's the line of least resistance ... and that's a cybernetic process. Yet one must remain entirely detached and entirely critical.

Of course current socio-political conditions make this approach somewhat idealistic. But this kind of approach comes close to exposing some of the aspects of the intuitive mode of knowing.

I consider intuition to be a mode of knowing wherein phenomena are considered within the context of their framework and the complexity of relationships entailed in the structures underpinning the presence of the phenomena.

This is generally done without the mediation of verbal language, so that the problems of categorization and fragmentation are avoided. But intuitive activity takes place usually below the threshold of everyday consciousness and the data gained is consequently very difficult to access. So we need to develop other tools for exposing things in process.

Video may be a tool of this kind. Perhaps we can use video; with its feedback, simultaneity, real-time control/response (responsiveness), in such a manner as to expose processes as changing, interactive, interdependent.

Perhaps video with its footing in social structures as well as other more abstract structures will be interactive enough across structural levels to be a general tool for examining processes at many levels within many kinds of situations.

What I am doing in the studio and in my video work in general is basically experimenting, I'm playing... because I don't feel that I know the answers to these things that I am proposing. There are processes upon processes by which one develops information, messages, and one can then make actions which generate feedback in one form or another. (And that is the function of exhibiting the work one does.) The function of what I am discussing is that it provides a framework from which I can operate upon everything that I am doing. 

VIDEO

Video (form) is formed and constrained by the nature of the technology itself. The low resolution, the 4 x 3 format, the ephemeral nature of the tape, the requirement for a machine for replay, etc., all contribute to a particular set of factors which make video (art) unique.

The particular qualities of the medium always will determine how the content is affected when worked through that medium. In video, we have, very much for the first time in the use of technology in the arts, a recognition of the conditioning factors of the medium, and the employment of those factors in and of themselves to code, process and transmit information. (By medium, here, I refer to the actual means of production of software, i.e. the hardware, the equipment). We also see a situation in which the technology itself; the hardware, provides a source of content in itself alone.

The video synthesiser appears very early in the history of video art and though perhaps, to some, inadequate as a source of content ("one should use material that has relevance to some set of social conditions") as a processor of images and coupled with other sources, it becomes an expansion of the parameters of visual imagery. Video allows us to make a variety of hypotheses about communications within the visual/auditory modes and to test these out. The simultaneity and instantaneity of response and feedback leave us free to modify as we go, clarifying issues during the process, i.e. we have real time control over the process.

I find that in my own work, that what develops out of the formalistic aspects of the medium; the significatory function operates off a symbolic, 'mythological' datum which is then, via the aid of video processing, worked upon in such a way as to expose syntactical relationships which may be unique to video, and are certainly outside of the usual framework of language; i.e. non-verbal, multi-layered, etc.

Thus there develops, through the inextricable relationship between form and content, new semantic relationships depending on the video context for their transmission. The communication might be idealised as the manner in which an alien being might communicate in a non-specific language situation.

We want to encode certain information about something, but the structures imposed by available verbal languages are inappropriate to the information and consequently degrade it in the basic encoding, so we look for other ways of encoding information which will solve the problems of the particular kinds of information degradation (noise) being imposed in the verbal-linguistic encoding. Of course we add all kinds of other noise factors inherent in whatever coding system we use whether it be software dependent (a conventional language in some sense) or derives its syntactic/semantic activity from purely formal manoeuvres.

Much of this so-called 'noise' is a function of patterns of structure and myth inherent in the languages used, verbal and social. So, I am dealing with a semiotic the source for which is in the hardware itself and the new arrangements and juxtapostions of image/information which this hardware makes possible. 

INTERACTIVE PERFORMANCE

The performer initiates an action ... the detector will detect this action and translate it into an electronic signal varying in frequency and amplitude with the performers actions. This signalcan then be

• (a) fedback live to the performer if in the audio range,
• (b) used as a control signal in an audio or video synthesiser to alter the sound and images in a manner proportional to the movements of the performer.

The performer can then respond to, and learn to exercise a degree of control over, the sound and images (mixes) produced. I.e. the performer, by the functioning of the feedback loop generated in this system, can learn to play the system as though it were a musical instrument. But the performer need not be encumbered with direct contact, the remote detectors allowing free movement in space.

A corollary of this model is demonstrated in the approach I have dubbed "interactive sculpture", in which some arrangement of video equipment forms a system in which the observer is enveloped in such a way as every action sets up some kind of response within the video system.

The observers' movement within the space is recorded with cameras and other detectors (radar, proximity) and the information (delayed, treated, mixed with other input) is fedback to the observer in some, often surprising manner. The observer is then free to play with the system and explore his/her interaction with it. This element of surprise, the difference that makes a difference, is elucidated by the realtime operation of the feedback, the nowness of the system.

This kind of structure can show the process that is communication – realtime action-response-interaction feedback loop, which, when fed by your presence, goes live (takes off). You enter a conversation with the video system and its image of you.

BIOFEEDBACK – DANCE/VIDEO/MUSIC

Model: a dancer is wired up with an array of biological-process detectors/amplifiers; 

• GSR – galvanic skin response
• EEG – electro-encephalogram – beta, alpha, theta wave monitoring
• EMG – electro-myogram – muscle movement
• EKG – electro-cardiogram – heart rate monitoring
• Respiration monitoring

The dancer initiates by moving – the signals emanating are fed into various audio and video switching and treatment channels. Coupling must be via opto-couplers or a transmitter-receiver. Video and audio from the treated bio-signals are then fed back to the dancer who then can learn how to work with this instrument and 'play' it. The bio-signals as monitored and amplified are generally applicable to synthesisers as control voltages and clocks.

Performance:

Video: real-time multi-camera vision mixing systems with several cameras on the dancer, on video feedback biofeedback signal displays, using voltage-controlled mixers and colourisers... (e.g. Fairlight Colouriser 108, EMS Spectre).

Audio: real-time, multi-channel bio-signals treated with and controlling an audio synthesiser.

Feedback: essential for the proper integrated operation of the instrument... at all levels of the system... to tune the human components of the system (the dancer, video operators, sound mixer, etc.) to the process... Can develop programmed audio treatment patches and video approaches.

This kind of activity is obviously applicable to both studio realisation and to live performance.

BIO-MUSIC

Using a bio-signal pre-amplifier, e.g. an alpha wave monitor, listen to and learn the rhythms of your system. The alpha-wave monitor is especially good because it implies a contemplative state, the body system is quiescent. The signal is then used as a base structure for musical improvisation. Using a keyboard instrument one can set up rules for relating to the bio-signal as a framework within which to improvise. 

Rule possibilities:

1. play only while in alpha; as you drift out of alpha devote yourself to regaining that state.
2. relate the rhythms of your playing to the rhythm of the bio-signal being used.
3. generate your own rules.

Look through all tape and image bank material, being selective for image, events.

By processes of juxtaposition and cut up, images thrown almost randomly together.

Start to look for patterns in the relationships of images via form, content, semantic juxtaposition, etc..

Develop (if there) integrating systems of thought and action processes and their dynamic interrelationship.

In a sense the task is to explicate these interior processes and to demonstrate inner/outer correspondence leading to an understanding of the identity of nature – of the nature of identity...

This is an activity of mental eco-logic.
It is the logic of the stone in the circle
the logic of the Tao
the logic of ecology and of the mind.

Dreams are plays upon, playing with
the logic and the form and the content
for logic and form and content are inseparable, identical
each is a way of perceiving
a part of perception
and the ability to perceive.

Myth, Dream and Reality
Myth the structure
Dream the growth
Reality the manifestation through language and consensual validation.

Through the structure grows the
dream into reality, and
reality into dream.

Our perception is the medium through which we realize these processes and bring them into meaning.

Logos, the breath of life
enters the fire of knowledge
into the wellsprings of consciousness.
Fire and Water are the dynamic opposites
in whose conjunction the breath, Air, manifests
its own conjuncted opposite, Earth, manifest reality.

This mythic primordial dynamic
contains the seed of a concept of process,
dynamic through the multitudinal diversity
of manifestation
of relation and interrelation.

Relationships between things in deep and intricate structure.
Image, language and the world.