Monorepo for Aesthetic.Computer
aesthetic.computer
Move Menkman guest papers from index to platter readings
OCR'd three Rosa Menkman PDFs to text and added them to the
platter readings under Media & Critical Theory. Removed guest
papers section from the papers index since they belong on the
platter, not the publication list.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
+6253
-58
+2
-33
papers/cli.mjs
+2
-33
papers/cli.mjs
···
449
449
},
450
450
];
451
451
452
-
// Guest papers — external works hosted on the platter as related reading
453
-
const guestPdfs = [
454
-
{
455
-
file: "menkman-glitch-momentum-2011.pdf",
456
-
title: "The Glitch Moment(um)",
457
-
detail: "Glitch as critical practice — signal, noise, and the politics of failure · Institute of Network Cultures · 70pp",
458
-
abstract:
459
-
"Menkman's book treats glitch as a critical practice rather than an accident. It remains a useful companion for thinking about failure, image politics, and media noise.",
460
-
author: "Rosa Menkman",
461
-
year: "2011",
462
-
metaKey: "menkman-glitch",
463
-
},
464
-
{
465
-
file: "menkman-vernacular-of-file-formats-2010.pdf",
466
-
title: "A Vernacular of File Formats",
467
-
detail: "Compression artifact taxonomy — databending one self-portrait through every codec · 20pp",
468
-
abstract:
469
-
"A tight taxonomy of compression artifacts and databending methods. It works as both a visual glossary and a reminder that file formats have aesthetics.",
470
-
author: "Rosa Menkman",
471
-
year: "2010",
472
-
metaKey: "menkman-vernacular",
473
-
},
474
-
{
475
-
file: "menkman-beyond-resolution-2020.pdf",
476
-
title: "Beyond Resolution",
477
-
detail: "Resolution as ideology — optics, standards, and the invisible norms of the image pipeline · 2020",
478
-
abstract:
479
-
"Beyond Resolution reframes image resolution as an ideological standard, not just a technical setting. It is a strong companion for thinking about how pipelines shape what counts as clarity.",
480
-
author: "Rosa Menkman",
481
-
year: "2020",
482
-
metaKey: "menkman-resolution",
483
-
},
484
-
];
452
+
// Guest papers — moved to platter readings (OCR'd text files)
453
+
const guestPdfs = [];
485
454
const extras = [];
486
455
for (const ex of extraPdfs) {
487
456
const fp = join(SITE_DIR, ex.file);
+2748
system/public/assets/papers/readings/text/Menkman-Beyond-Resolution-2020.txt
+2748
system/public/assets/papers/readings/text/Menkman-Beyond-Resolution-2020.txt
···
1
+
Rosa Menkman
2
+
3
+
LXIII
4
+
5
+
BR
6
+
7
+
eyond
8
+
esolution
9
+
10
+
LXVII
11
+
12
+
Refuse to let the syntaxes of (a) history direct our futures
13
+
14
+
I
15
+
16
+
Colophon
17
+
18
+
Beyond Resolution is compiled of texts that have been extended and reworked by
19
+
Rosa Menkman, 2115 – 2020.
20
+
Published by the i.R.D.
21
+
A pdf of this publication can be freely downloaded at:
22
+
https://beyondresolution.info/beyond-resolution
23
+
For more information please contact me:
24
+
rmenkman@gmail.com
25
+
ISBN: 978-90-828273-0-9
26
+
COPY < IT > RIGHT !
27
+
A warm thank you for the support I have received:
28
+
Transfer Gallery: Kelani Nichole
29
+
Akademie Schloss Solitude: Jean-Baptiste Joly, Clara Herrmann,
30
+
Mareen Wrobel and Bruce Sterling, Taietzel Ticalos and Jakob Weiss
31
+
Ward Janssen and Mieke Gerritzen
32
+
Landers: Per Platou, Amsterdam: Annette Wolfsberger.
33
+
Tokyo: CG-Arts (Japan Media Arts Festival), the ICC and ARTnSHELTER
34
+
Oregon Story Board, Upfor Gallery, EyeBeam: Thomas Wester and Theo Downes-Le Guin
35
+
DiMoDA: Will Robertson and Alfredo Salazar Caro
36
+
#Additivism: Daniel Rourke and Morehshin Allahyari
37
+
Parallax, Brett Wallace, Lune Magazine: Nathan Jones
38
+
Nora ní Mhurchú and Bogomir Doringer
39
+
A4 and Matus Kobolka
40
+
Transmediale: Kristoffer Gansing
41
+
Kunsthochschule Kassel: Joel Baumann and my students
42
+
CERN and Fabra i Coats
43
+
SJSU and L’Unique Caen
44
+
Mario de Vega
45
+
Lotte, Ben, Caspar Menkman Crum
46
+
Beyond Resolution is supported by Stimuleringsfonds Creatieve Industrie
47
+
48
+
II
49
+
50
+
BEYOND RESOLUTION
51
+
52
+
Introduction
53
+
54
+
0000
55
+
0001
56
+
0010
57
+
0011
58
+
0100
59
+
60
+
Whiteout
61
+
Behind White Shadows
62
+
Night of the Unexpected
63
+
Refuse to Let the Syntaxes of (a) History Direct Our Futures
64
+
institutions of Resolution Disputes [i.R.D.]
65
+
66
+
Conclusion
67
+
68
+
I
69
+
70
+
“the pandemic of mental anguish that afflicts our time cannot be properly understood, or
71
+
healed, if viewed as a private problem suffered by damaged individuals.”
72
+
- Mark Fisher.
73
+
74
+
I
75
+
76
+
I
77
+
78
+
ntroduction
79
+
80
+
I opened the institutions of Resolution Disputes [i.R.D.] in March 2015, hosted by Transfer Gallery in New
81
+
York. In September 2017, its follow-up Behind White Shadows opened, also at Transfer. Finally, in 2020
82
+
Shadow Knowledge found its way to SJSU Galleries in San José. At the heart of all three solo shows is my
83
+
research into resolutions and together these exhibitions form a triptych framework for this publication: Beyond
84
+
Resolution.
85
+
This little book also represents a journey spanning over 5 years, starting on a not so fine Saturday morning in
86
+
early January 2015, when I signed the contract for a research fellowship in Amsterdam, to write a book about
87
+
Resolution Studies. For this opportunity I immediately moved back from London. But unfortunately, and out of
88
+
the blue, three days before my contract was due to start, my job was put on indefinite hold. During this time in
89
+
limbo I developed the i.R.D, as a critique on institutions. But after the i.R.D. were finally opened and nothing in
90
+
my private life was resolved, I fell into a financial and ultimately a mental black hole ! . Finally in 2016, when
91
+
Goldsmiths decided to kill off the department of Cultural Studies, the department where I was undertaking my
92
+
PhD, I gave up and resigned of all.
93
+
After some months, I finally reorganized some of my finances and relocated temporarily to the Mojave
94
+
desert, where I took some time for myself, considering my next steps. There, from the porch of a little cabin
95
+
looking out over a dust road, I could feel the infrasound produced by bombs dropped on Little Baghdad, a
96
+
Twentynine Palms military training ground, just miles away on the slope of a hill. I became fascinated with this
97
+
obscure military location where things happened beyond my understanding, yet in my direct field of perception.
98
+
The situation made me revisit Trevor Paglen’s book: I Could Tell You But Then You Would Have to Be
99
+
Destroyed by Me. 1 Paglen’s main field of research is mass surveillance and data collection. His work often deals
100
+
with photography as a mode of non-resolved vision and a system for the production of invisible images. I felt as
101
+
if I was in the proximity of such a military dark-ops space, where different forms of opaqueness were present
102
+
day and night, and it got to me.
103
+
During my stay in the Mojave Desert, I undertook my first pilgrimage to the 1951 USAF resolution test
104
+
target located west of Cuddeback Lake, where documentary film maker and writer Hito Steyerl’s video essay
105
+
How Not to be Seen. A Fucking Didactic Educational .MOV File (2013) is set. In this iconic video essay,
106
+
Steyerl presents the viewers with an educational manual that critically considers how resolution is embedded in
107
+
both digital and analogue surveillance technologies. She argues that whatever is not captured by resolution is
108
+
1
109
+
110
+
Paglen, Trevor. I Could Tell You But Then You Would Have to be Destroyed by Me: Emblems from the Pentagon's Black
111
+
World. NY: Melville Publishing House, 2007.
112
+
II
113
+
114
+
invisible and thus carries political importance. During this visit I felt finally excited again to re-start my
115
+
research into resolutions, but from this time onward, independently.2
116
+
Soon after returning to Europe, I was incredibly lucky to receive a 5 month residency at Schloss
117
+
Solitude (2017), which offered me a Schloss (a palace) to come ‘home’ to, and the time to develop a colloquium
118
+
around Resolution Studies as an artistic practice. My research was further fine-tuned during a Vertretungs
119
+
professorship in Kassel (2018 - 2020) where I shared, reworked and extended Resolution Studies with my
120
+
students –– an opportunity for which I am very grateful.
121
+
Resolution Studies attempts to uncover how resolutions inform both machine vision and human perception. I
122
+
believe it is incredibly important to unpack the ways in which resolutions organize our contemporary
123
+
(technological) processes. Considering that resolutions do not only impose how or what gets run or seen, but
124
+
also what images, settings, ways of rendering and points of view are forgotten, obfuscated, or simply dismissed
125
+
or unsupported. In short: resolutions are not just a determination of how something or someone is run, read and
126
+
seen, but also of who or what options are compromised and unresolved.
127
+
Although not written as such, this booklet on Resolution Studies, Beyond Resolution, can be seen as a sequel to
128
+
the Glitch Moment/um (Menkman, 2011). In a way, Resolution Studies encompasses glitch and glitch art,
129
+
offering a more zoomed out perspective. The works I present in this publication are produced exemplary of the
130
+
artifacts that appear when experimenting with the affordances of certain resolutions. Rather than a breaking a
131
+
flow of operation (as the genre of glitch art is known to do), these works also question the ‘normal’, standard
132
+
modes of operation and offer alternatives. This is why if positioned within the realm of glitch art, then definitely
133
+
these works are a form of ‘tactical glitch’.
134
+
What finally sits in front of you, Beyond Resolution, is an independently developed and published work - a
135
+
collection of different types of texts ranging from short stories to basic optics and a manifesto like text,
136
+
accompanied by a collection of artworks that I developed during the time of writing, presented in an achronological order. The organisation of this publication could be considered modular; the chapters can be read
137
+
independently. However I did choose to order them the way they are for a reason: to present a consistently
138
+
additive flow.
139
+
140
+
2
141
+
142
+
In the 1950s and 1960s, the United States Air Force installed various versions of these test charts to calibrate aerial
143
+
photography and video. For more information on resolution test charts see research by The Center for Land Use
144
+
Interpretation, “Terrestrial Test Patterns Used for Aerial Imaging.”
145
+
Available here: http://www.clui.org/newsletter/winter-2013/photo-calibration-targets (20.02.20)
146
+
III
147
+
148
+
Beyond Resolution starts at Whiteout, a short story about loss induced by over-saturation. Whiteout is an
149
+
account of a visceral experience: it describes the journey of climbing a mountain in a snowstorm; of being
150
+
engulfed by input, yet not being able to render anything.
151
+
Whiteout is followed by Behind White Shadows, a chapter written after the recurring loss of authorship
152
+
over my own face. During my research into this type of loss, I found it striking to learn that certain faces have
153
+
become so ubiquitous, they are no longer considered to be a representation of someone; instead they have
154
+
turned into “just pixels”. In the age of hyper visibility, the face is still an often overlooked battleground.
155
+
Contiguous to this, within the field of image processing, the image of the Caucasian female face has
156
+
systematically been used (or stolen) for the production of color test cards, resulting into biased industry norms
157
+
for color calibration and ultimately racist technologies — a fact that became a turning point into my own
158
+
understanding and usage of my face within resolution studies.
159
+
The Night of the Unexpected, the following third chapter of this compendium, offers an older account of
160
+
an evening in Russia, which initiated my first thinking through the material politics of screens and digital
161
+
material in general and led to a need for a more substantial definition of the term ‘resolution’ — a redefinition
162
+
that I developed in the consecutive chapter: Refuse to Let the Syntaxes of (a) History Direct Our Futures.
163
+
Finally, the fifth chapter of Beyond Resolution, institutions of Resolution Disputes [i.R.D.], offers several
164
+
statements on resolution studies in a manifesto-like style.
165
+
In conclusion, I would like to state, that after releasing Beyond Resolution, the i.R.D. (institutions of Resolution
166
+
Disputes) will remain an active research platform, which, as of now, has followed a pentagon of institutions,
167
+
covering the effects of scaling and the habitual, material, genealogical, and institutional use (and abuse) of the
168
+
settings, protocols, and affordances shaping our resolutions, free, open and updated at
169
+
www.beyondresolution.info
170
+
171
+
IV
172
+
173
+
LXVII
174
+
175
+
XII
176
+
177
+
I
178
+
179
+
W
180
+
181
+
hiteout
182
+
183
+
p
184
+
185
+
oint No Point Light
186
+
11 February | temp: -5c to -11c | Snow: at least 10-30 cm, often more.
187
+
It is grey. Snow in the form of fine dust is coming down. The trees bow slightly under its weight and
188
+
occasionally shed a cache, which then falls down onto the forest floor, landing with a dampened “phlouf.” This
189
+
is not the ideal weather to hike up a mountain, but we are doing just that.
190
+
Soon most of the trees give way, the path is now completely covered in a thick and fresh pack of snow
191
+
–– footsteps and the occasional boulder mark its surface. We use GPS for navigation and also carry a small
192
+
hand-held device called Limenia, that picks up particular bands of the electromagnetic frequency spectrum,
193
+
which it translates into humanly audible sounds. 3 Limenia serves as an interface: it distills the intangible and
194
+
typically imperceptible infrastructures that are beamed around us –– and even straight through us –– all the
195
+
time. 4
196
+
We are on a mission to document the top of this mountain, The Brocken, 5 which like any other mountain
197
+
these days is marked by humans. All over the world we have elongated nature's highest terrains with antennas,
198
+
without respect for the natural stratifications they are built upon, or consideration of the panoramas interrupted.
199
+
These antennas usually function as repeaters: they receive, retransmit and propagate invisible electromagnetic
200
+
waves of information through the ether, as well as the natural environment. Notably, the antenna on top of this
201
+
particular mountain is a remnant of the Cold War. As such, it is not only anchored in our current
202
+
telecommunication networks, but in a previous –– and quite possibly ongoing –– position as an ear that listens
203
+
in on confidential networks of telecommunication.
204
+
While we move along the trail, the atmosphere becomes denser and the wind picks up. As spatial
205
+
orientation withers and our perception of the distance is curtailed to 50 meters, a visual gradient fills my
206
+
viewing plane, oscillating between grey whites and white greys. This place is known for the Brocken spectre,
207
+
which sometimes, when conditions are just right, appears to its visitors: an optical phenomenon in which a
208
+
3
209
+
210
+
A previous version of this text was first written for AX15, a project about invisible infrastructures initiated by Mario de
211
+
Vega, see: http://axis.mariodevega.info/ (11.12.19).
212
+
4 Schmidt, Paul. “Perceiving the Invisible: Speculation as Interface at Sonic Acts Academy 2018,” (Re)mistify!
213
+
214
+
see: https://remystifyblog.wordpress.com/ (11.12.19).
215
+
5
216
+
217
+
Located in a region of Saxony Anhalt, the Brocken, or Blocksberg, is the highest peak of Northern Germany, though its
218
+
elevation is below alpine dimensions: 1,141 meters.
219
+
II
220
+
221
+
Limenia is a small antenna which translates certain frequencies of the electromagnetic spectrum to the realms of audible
222
+
sound. This little handheld device was conceived and build by Mario de Vega and Victor Mazón Gardoqui,
223
+
see: https://web.archive.org/web/20190811005423/http://r-aw.cc/-/limen/research/ (20.02.20)
224
+
225
+
person’s magnified shadow is cast upon clouds, surrounded by a haloed rainbow.6 Like Limenia, this apparition
226
+
relies on the convergence of the intangible, but unlike radio signals, the Brocken Spectre is an embodiment of
227
+
natural imaging phenomena.
228
+
The weather is not right for the phenomenon and the wind in my face brings me back to our climb. A hut
229
+
offers shelter for two walkers who seem to be taking a rest. We pass them without much of a greeting as it is
230
+
hard to speak. A fork in the path leads us to the right. Introduced to a barrier of fog, a roaring wind in our ears
231
+
and frictious bed of snow under our feet, we find ourselves reduced to an ever slower pace. Limenia has now
232
+
become the only proof of life beyond this grey enclosure. It is comforting to hear its sounds in my ears as it
233
+
pings invisible infrastructures: concretizing and assigning dimensions to a space my eyes cannot see.
234
+
235
+
I
236
+
237
+
mpossible Images
238
+
239
+
Last month I had the opportunity to speak to several particle physicists and ask them about their favorite
240
+
“impossible image.” One scientist expressed a wish to have antennas mounted in his eye sockets so he could
241
+
see through walls, straight into the electromagnetic spectrum. This 20/20 electromagnetic vision would reveal
242
+
everything from electricity to cosmic rays, and transform our grey enclosure into a universe of colors ––
243
+
including colors far beyond our visible color wheel. It would clearly capture the environment introduced by the
244
+
antenna: the beacon on the top of this mountain that relentlessly emits signals into the invisible-to-me realms of
245
+
the radio frequency spectrum.
246
+
6
247
+
248
+
Atmospheric Optics, “Brocken Spectre”, see: https://www.atoptics.co.uk/droplets/globrock.htm (11.12.19).
249
+
III
250
+
251
+
As I scale the mountain, I wonder: could there be creatures out there, in the fog, that can
252
+
perceive the noise of the antenna, blasting through the ether? How much of the landscape
253
+
–– hidden to me through these layers of snow and fog –– exists to organisms or networks
254
+
that have passive electronic properties, besides my Limenia? Do my human qualities
255
+
simply render me incapable of understanding the violence of these intangible signals? In
256
+
short, to really read this place I would need a way to capture and transcode a vertically
257
+
thickened geography: a strategy of counter-mapping that adds layers, stacks and other
258
+
vertical elements to my imaging of this space.
259
+
Eyal Weizman, founder and director of Forensic Architecture, once described this
260
+
as a “politics of verticality.” He suggests a re-visioning of existing cartographic techniques
261
+
and an “Escher-like representation of space, a territorial hologram in which political acts
262
+
of manipulation and multiplication of the territory transform a two-dimensional surface
263
+
into a three-dimensional volume.”7 Weizman also advocates for the superimposition of
264
+
discontinuous modes of mapping, to construct a “sceneography” rather than a map –– a
265
+
form of landscape that contains a mesh of significations, which can nonetheless be read
266
+
together, instead of simply viewed as a formal unity.
267
+
I wish I could create a territorial hologram of this trail, by mounting different
268
+
types of antennas or other sensing technologies onto my eye sockets. I wish I could conjure
269
+
a thick mesh of the sceneographies I am traversing. I wish I could visually compute the
270
+
world around me. For some time, thoughts and references like these float and sink in the
271
+
rhythms of the beating wind.
272
+
After some hours, we pass a signpost that signals another five kilometers awaits us. As
273
+
we cross a barrier of snow the fog suddenly vanishes and in its place encroaches an
274
+
enveloping white noise. Like a spacecraft going into warp drive, the wind shoots the snownoise in our faces and eyes. I think about the different forms and shapes these tiny crystals
275
+
must have and how they owe their forms to their previous environments; all this only to
276
+
finally end up as a drop on my cheek –– another impossible image, that makes me feel
277
+
small.
278
+
The path, contoured by the light, is now twisting, crossing, intersecting,
279
+
connecting, bending and elongating. Yet, as my eyes stare into the greyish white, it seems
280
+
as though all may be dark: white without light / white in the dark, I had never previously
281
+
imagined the darkness of white.
282
+
7
283
+
284
+
Weizman, Eyal. "Maps of Israeli Settlements." The Politics of Verticality: Open Democracy,
285
+
2002. see: http://www.opendemocracy.net/ecology-politicsverticality/article_631.jsp (11.12.19).
286
+
IV
287
+
288
+
M
289
+
290
+
oving on a Line
291
+
292
+
Some years ago, through a fluke of destiny, I found myself inhabiting a little cabin in the Mojave Desert.
293
+
Having rented an SUV in the hopes of gaining some horsepower I found myself disappointed with what the
294
+
machine offered, but at the same time nonetheless excited to traverse this vast desert expanse. I drove for
295
+
days–– with gallons of water and spare batteries for my mobile device –– without a route, aimlessly following
296
+
the heat aberrations and dust devils into the unforeseen.
297
+
On one such drive I took a route marked with a thick line on Google Maps yet found myself on a dusty
298
+
stretch of road that became narrower and narrower as I progressed –– suddenly I was left without space to turn
299
+
around. I was trapped on a path that slowly changed from dust to stones and then to rocks: big rocks, sometimes
300
+
larger than my suspension should bear. I spent hours on that path, scared to puncture a tire. The more I
301
+
progressed, the more I feared breaking down in a location more desolate than those passed in the preceding
302
+
hours.
303
+
Singing to myself while probably driving slower than I would move by foot, I finally reached the end of
304
+
that rocky road. I still remember the extreme relief of entering a clean stretch of asphalt, looking over a barren
305
+
valley marked by the sediments of a dried lake. The valley was carved in two by the straight line of interstate
306
+
15, which stretched into the horizon. Peering into the expanse before me, I saw something surprising: a pillar of
307
+
light peeking over the rim of a mountain, a strange daylight lantern, shining with an intensity that puzzled me.
308
+
Forty-five minutes later, I had crossed the distance that separated me from the daylight lantern. As I
309
+
pulled up on the private road of a centralized solar farm, I swung my door open and hurried toward the edge of
310
+
the field to experience its energy, half expecting to be told to move on by some security guard. Yet, it remained
311
+
quiet. Endlessly quiet. There, in front of the whitest light I had ever seen, I was alone –– there was nothing else:
312
+
no wind, no cars, not even a hum to signal the concentration of energy right in front of me. It was just me and
313
+
the whitest light in the middle of the High Mojave Desert. I had never seen such white. I closed my eyes and
314
+
suddenly I lost my references to the light. It might as well have been vantablack.
315
+
V
316
+
317
+
Menkman, Rosa. Whiteout (2020) Two Channel Video, 15 minutes
318
+
319
+
M
320
+
321
+
aking Space and Place
322
+
323
+
I pull myself back onto the mountain where it is cold and definitely not that type of white. The weather
324
+
escalated from a gentle snowfall into a storm. Vague reflections of trees and objects lost in swirling grey tones
325
+
pixelate my sense of the horizon: a line that otherwise could have been a guiding beacon. Wassily Kandinsky
326
+
once wrote: “A particular capacity of [the] line [is] its capacity to create surface.” 8 The trail presents a space
327
+
devoid of any lines and what remains is a place of distortion: lacking surface, resolution, progress, depth or end.
328
+
In the Dictionary of the English Language (1755), Samuel Johnson observed seventeen definitions of the line.
329
+
He differentiated between its function as a longitudinal extension, a string, a contour, a letter and a family line.
330
+
The line is not a singular thing...
331
+
332
+
8
333
+
334
+
Kandinsky, Wassily. Point and Line to Plane. US: Dover Publications, 1982: p. 576.
335
+
VI
336
+
337
+
Maybe I could seek lines – I could try to find a line to lead me back: a border, a boundary, a frontier, an
338
+
outline, a separator, a perspective, a line of flight, a guideline, a progress bar, a ruler, a red thread, a storyline, a
339
+
chronology, a timeline, a vector, a ray of light, a beam, a sequence, an axis, a connector… I have none of these.
340
+
Lines can offer myriad shapes and directions: horizontal, vertical, diagonal, rastrum, grid or wave; a maze, a
341
+
mesh, a labyrinth. The snow usurps not only one but all possible lines. In front of me could be anything, but all
342
+
remains resolutely unresolved.
343
+
Like any map, every map of this mountain is made out of lines. A thick black line describes the Railway,
344
+
which swirls around the mountain’s summit like the cochlea of the inner ear; while the hikers’ trail approaches
345
+
its top like the Eustachian tube. 9 More advanced maps feature contour lines indexing the heights of the
346
+
mountain's surfaces; historical maps document the lines drawn during the Cold War, when it was partitioned by
347
+
a vertical border wall and its summit was an exclusion zone (Sperrzone), dedicated to espionage by the East.
348
+
During this time, the Brocken was no more than an imagined space to civilians. Today, the region is open and
349
+
its lines and borders construct a visual record: one of experience and history inscribed in layers of linear
350
+
(re)imaginings that signify mapping rather than simply a map.
351
+
Weizman states that maps exist to make it easier to navigate the world. They inform with insights and
352
+
overview, yet, generally, they bear no witness to their creator(s). Rather, the map appears as a pre-composed
353
+
artifact: an object derived from a space rather than a reading that is imposed on a space, which transforms it
354
+
into a place –– much like photographic inscriptions of experience that transform space to place. But, like
355
+
photographs, maps are highly political objects. They define space and impose particular readings. “Twodimensional maps, fundamental to the understanding of political borders, have been drawn again and again. But
356
+
these maps are two-dimensional. Attempting to represent reality on two-dimensional surfaces, they not only
357
+
mirror but also shape the thing they represent. As much as describing the world, they create it.”10 In the
358
+
domains of both politics and law the concept of space is understood only in terms of the map (or sometimes, the
359
+
plan) and territorial claims marked on two dimensional maps assume that claims are applicable simultaneously
360
+
above them as well as below –– underneath –– the space that is mapped. This type of one-dimensional scaling
361
+
is violent and undermines the meshes that transform space into a specific place. The spaces represented on
362
+
maps are those without inhabitants: no one is present, nothing moves or makes a sound. In the map of the
363
+
Brocken, we are merely blue dots scaling a two-dimensional Eustachian tube; presented with minimal
364
+
9 The Eustachian tube is part of the middle ear. Also known as the auditory tube, it connects the nasopharynx to the
365
+
366
+
middle ear.
367
+
10
368
+
369
+
Weizman, Eyal. "Maps of Israeli Settlements." The Politics of Verticality: Open Democracy, 2002.
370
+
see: http://www.opendemocracy.net/ecology-politicsverticality/article_631.jsp (11.12.19).
371
+
VII
372
+
373
+
references for the complexities folded into the landscape, or the histories and folklore we cross.
374
+
375
+
O
376
+
377
+
ver-Saturated Opaqueness
378
+
379
+
At CERN (Conseil Européen pour la Recherche Nucléaire [European Organization for Nuclear Research]), I
380
+
spoke to a scientist who told me that humans can only remember one direction of time: the direction that
381
+
accounts for the past. He speculated about the possibility of four-dimensional matter, which could help us to
382
+
remember the future. Having access to such matter, we could carve slices of time –– including the future ––
383
+
which would then help us navigate the present. A more inclusive description of the Brocken might mean the
384
+
formation of a multi-dimensional volume, layered with folkloric, historical and political strata. The Brocken as
385
+
a mesh that exceeds the conventional outlines drawn on the “normal” map. To conjure such a form would imply
386
+
VIII
387
+
388
+
collecting all the lines of history, ecology, thought and beyond.
389
+
But on the mountain, there are just endless snowflakes, in which I can see stories and adventures from my
390
+
future and my past… But it feels like I can carve nothing. The twirls around me are not uniformly defined. The
391
+
thoughts in my head are spinning as if they are dancing to the movements of the flakes. The white noise is no
392
+
longer coming down from the sky but instead from all directions –– up, down, left and right –– without a focus,
393
+
a line or a path. They have become the length and depth of my hike. All I see is grey.
394
+
In this intense flattening of perspective, we have become the distance. We are forms, lines, energies; one
395
+
upon another, the matter of an intangible –– though far from immaterial –– situation. Walking here together, the
396
+
trail has taken us deeper inside ourselves.11 The mountain has entered us. Like Walter Benjamin once wrote, “I
397
+
have become my snow globe.” 12 I wonder, is the sun still there, is it fighting to reach me, above my head?
398
+
A terrible feeling of dread and loneliness comes over me. I remember this grey feeling. It is being
399
+
together with people, yet still being alone. Completely and utterly alone, lost in ubiquity, like there is an
400
+
invisible wall that won’t let me join the state of “togetherness.” Sometimes a space does not allow you to make
401
+
place, to plot and situate yourself in connection to space. Sometimes a space is made of invisible rules and
402
+
viscosities that keep you from inhabiting it. Although, these material properties of space may just be qualities I
403
+
am not trained in. I have to learn to understand space through properties beyond my senses, my literacy and my
404
+
habits. Blurred is this blizzard: a blizzard of snow and a blizzard of thoughts. Timothy Ingold wrote that “drawn
405
+
threads invariably leave trailing ends that will, in their turn, be drawn into other knots with other threads.” 13
406
+
Yet, things seem to be simply unravelling. I do not see the knots or threads. I feel less and less human. I am just
407
+
a cold body stuck inside a sack of clothes that is trying to carry itself up a mountain.
408
+
We have walked for a long time. I am exhausted. The twirls of white noise make space for a flurry of
409
+
references and images that take up the space beyond my retina. Shadows and shapes appear in the corner of my
410
+
eyes, apparitions that disappear when I try to see them. The ghosts are dancing in a fog of flakes as my hair is
411
+
dancing with them, reaching out like tentacles in front of my eyes. I take Limenia out again: the little
412
+
instrument gives me static and a hint of something else. For a moment it sounds like music. Maybe I mishear a
413
+
deep EVP (electronic voice phenomenon), or the ghosts of the weather have now started to speak through the
414
+
device.
415
+
Then, suddenly there is a little clearing in the sky and I can see a shimmer of light. It is like the sun
416
+
wants to tell me it is still there. I don´t know how long or how far we still have to go, but the Sun has given me
417
+
back a sense of perspective: a vertical perspective. I have been pulled out of my grey state of being, on a
418
+
mission to make it to the top of this track. On the side of the road I see spectacular installations: collaborations
419
+
11
420
+
421
+
Leslie, Esther. Walter Benjamin. UK: Reaktion Books, Critical lives series, 2007: p. 78.
422
+
423
+
12 Benjamin, Walter. Gesammelte Schriften, vol. VII, pt.1. Frankfurt: Suhrkamp, 1991.
424
+
13
425
+
426
+
Ingold, Timothy. Lines: A Brief History. London: Routledge, 2007.
427
+
IX
428
+
429
+
between the trees, the wind, snow and ice. The final stretch of the track is suddenly there, it is not too long
430
+
before we reach the end of the trail and arrive at the summit of the mountain.
431
+
Cold, stiff and tired, there are only a few trees to keep us protected from the wind. The weather makes it
432
+
impossible to see much, and our video and photographic equipment fails to document what remains of “the
433
+
spectacle” as our lithium batteries are depleted. The full length of the Brocken’s antenna is not visible, I just
434
+
recognize its base, its mast shrouded in grey. There, in the snow, it is sending and repeating inaudible messages,
435
+
encrypted on different frequencies, waiting for technology to pick them up for repetition or translation to
436
+
another medium or node. We clearly hear its overpowering strength through the Limenia and the different
437
+
rhythms at which data is sent through the ether, as well as through us, offering me just a small insight into the
438
+
immensity of unresolved spectral resolutions (that I am blind to/do not perceive) with my human body.
439
+
440
+
It is not enough to regard the surface as a taken-for-granted backdrop for the lines that are inscribed
441
+
upon it. The history of writing belongs within the history of notation, and the history of notation within
442
+
the history of the line, so there can be no history of the line that is not also about the changing relations
443
+
between lines and surfaces.
444
+
- Timothy Ingold14
445
+
446
+
P
447
+
448
+
lotting Lines: Windows and Messengers
449
+
450
+
Whiteout is a story about climbing a mountain during a snowstorm –– an experience that led me into a space
451
+
without dimension: white in the dark and references without lines. The story of a time when I manoeuvred
452
+
through slices of consciousness; traversed a virtual axis to nowhere and, finally, was pulled back into space,
453
+
only for such space to collapse on me.
454
+
The Brocken Spectre never appeared to us, but for a moment, the Brocken morphed into a landscape
455
+
with multiple horizons, in which the orientation discerning top from bottom did not exist except outside the
456
+
mind of the wanderer. And even though events seemed to happen in the same space, this state created different
457
+
places within that space, layered upon one another.
458
+
As I scaled that mountain in the snowstorm, the mountain sublimated itself into a space without lines.
459
+
For a moment there were no threads, traces or cracks to fall into; lines to follow, grasp or get a hold of. The
460
+
mountain showed me I need lines to understand space. In a perfect whiteout, I missed visual markers and, as a
461
+
result, all lines of reference. I became my own surface. And all I had to inscribe with meaning was my own
462
+
internalization of space: a quantum axis I got lost in. I then learned that a node by itself is nothing: it is just
463
+
14
464
+
465
+
Ingold, Timothy, ‘Transformations of the line: Traces, Threads and Surfaces’, in: Textile (no.8.1, 2010): p. 12.
466
+
X
467
+
468
+
infinitely small. A node needs lines and connections, it needs links. And although these are ubiquitous –– and
469
+
therefore often ignored –– they actually provide an important rationale for grounding: they organize images,
470
+
vision and context.
471
+
The line is a beautiful and radical object in its capacity to represent both operation and representation.
472
+
Some lines even defy classification. When dealing with materials that are non-static, opaque or invisible, we
473
+
can shape their so-perceived immateriality or conjure them into materiality through approaching them as an
474
+
axis. Lines can be the basis of all materiality because they invoke a surface. They form our matter. If a visitor
475
+
temporarily commits to a line, they re-territorialize the environment. It is this kind of thinking about space and
476
+
place that allows for seeing through time; through different spectra and frequencies to open different or new
477
+
perspectives of visual, experiential and spatial narratives.
478
+
Maybe, to a mountain, all that matters is verticality, quantified as height above sea-level. That is why
479
+
mountains are generally defined by their summits and not by greatness of rock mass or the degree of muscular
480
+
exertion the hiker has to endure when scaling. 15 To me the Brocken is also a place of many imaginary lines or
481
+
descriptions, that can be highlighted and then collapsed onto the mountain. Every time this occurs, the
482
+
mountain is freed again and again because the act of repositioning these membranes of perspectives create a
483
+
forever unfolding, non-vertical abyss of lines. Our immaterial objects are shaped by the vectors we choose, the
484
+
lines we draw around them, the norms and expectations, traditions and rules; and, finally, the lines of meaning
485
+
we inscribe them with. It not only matters what surface a line is drawn on, but also what lines we draw upon a
486
+
surface. Finally, the scale at which we draw lines, or the spectrum through which we radiate is of note. If we
487
+
can open up these levels of navigation, we can open up new “windows” (ways of seeing) to other
488
+
“messengers” (beyond for instance the humanly visible part of the electromagnetic spectrum) through which we
489
+
radiate. And when doing that, we may even be able to jump scales.
490
+
491
+
15
492
+
493
+
Ingold, Timothy. The Life of Lines. London: Routeledge, 2015: p. 32.
494
+
XI
495
+
496
+
I
497
+
498
+
II
499
+
500
+
B
501
+
502
+
ehind White Shadows
503
+
504
+
Blankness is as much a state of mind as it is a material condition, and as such it can be deceptive,
505
+
hallucinatory, unintentional, poetic or spiritual. It can be legible or illegible, present or absent, perfect
506
+
or imperfect. But it is never empty or devoid of meaning.
507
+
– Michael Gibbs16
508
+
509
+
W
510
+
511
+
hite Shadows
512
+
In her 2012 lecture “White Shadows: what is missing from images” at the Gdansk Academy of Fine Arts in
513
+
Poland, Hito Steyerl speaks about how new technologies force us to reformulate important questions about
514
+
making visible, capturing and documenting information. In the first part of her talk, Steyerl focuses on the use
515
+
of 3D scanning in forensic crime-scene investigations. Steyerl explains how the 3D scanner, or LiDAR
516
+
technology (Light Detection And Ranging), sends laser beams that reflect the surfaces of the scanned objects. In
517
+
this process, each point in space is measured and finally compiled into a 3D facsimile point cloud. In turn,
518
+
Steyerl observes that this kind of capturing does not just provide a new image of reality but transforms our very
519
+
relation to the world. Yet, Steyerl takes issue with the general belief that these emergent technologies should be
520
+
accepted as the ultimate documentary and forensic devices: as tools that produces 100 percent reliable, true,
521
+
evidence. 17
522
+
Just like any other technology, Steyerl argues, VR has its affordances and with these affordances come
523
+
blind spots. For instance, only a few scanning rigs are advanced enough to capture a moving object: they
524
+
generally become a blur or are not picked up at all. A 2.5D scanning rig (a rig with just one 3D scanner) can
525
+
only provide the surface data for one side of a scanned object. As a result, the final scan of an object, or space,
526
+
includes blind spots: the backs of the objects and the shadows cast by objects in front of an object which,
527
+
depending on the display technology, may show up as empty white shells.
528
+
To really process an environment properly, its scans would have to be taken from every angle. The barrier
529
+
16 Gibbs, Michael. “All or Nothing and Other Pages,” ed. by Gerrit Jan de Rook & Andrew Wilson. Uniformbooks, 2016:
530
+
531
+
p. 10.
532
+
17
533
+
534
+
Steyerl, Hito. “White Shadows: what is missing from images,” Gdansk Academy of Fine Arts, Youtube, 2012.
535
+
see: https://www.youtube.com/watch?v=PoZa707a91s
536
+
III
537
+
538
+
“What becomes visible on the back of the image is the space that is not captured. The space that is missing, missing
539
+
data, the space where an object covers the view. A shadow. […] Documentary truth and evidence also including the
540
+
missing of information. The missing people themselves.”
541
+
- Steyerl, Hito. “White Shadows: what is missing from images,” Gdansk Academy of Fine Arts, Youtube, 2012.
542
+
543
+
presented for such a task in the case of most 3D scanning equipment is that certain elements of the scanned
544
+
environment will always exist in the shadow of the next object; resulting in white patches, blank spaces, or
545
+
hollowed-out shells remaining in the dataset as unseen and unregistered content.
546
+
An important question, then, not just for 3D, but for any technology is: who decides the point of view and
547
+
who stands behind the perspective from which a LiDAR –– or any scanning or imaging technology –– is
548
+
operated? In order to formulate a possible entry point for tackling this problematic, in what follows you will
549
+
find a history of resolutions, specifically the history of the color test card.
550
+
551
+
IV
552
+
553
+
T
554
+
555
+
he White Shadows of Image Processing:
556
+
557
+
Shirley, Lena, Jennifer and the Angel of History18
558
+
A fundamental part of the history of image processing and setting standardization –– within both analogue and
559
+
digital compression as well as codec technologies –– is the test card, chart, or image. This standard test image is
560
+
an image (file) used across different institutions to evaluate: image processing, compression algorithms,
561
+
rendering and display quality. One type, the test pattern or resolution target, is typically used to test the
562
+
rendering of a technology or to measure the resolution of an imaging system. Such a pattern often consists of
563
+
reference line patterns with clear, well-defined thicknesses and spacings. By identifying the largest set of
564
+
indistinguishable lines, one determines the resolving power of a given system; by using identical standard test
565
+
images, different labs are able to compare results –– both visually, qualitatively and quantitatively.19
566
+
A second type of standard test image, the color test card, was created to facilitate skin color balancing or
567
+
adjustment and can be applied to test color rendering on different displays. While technologies such as
568
+
photography, television, film and software all have their own color test images, these types of test images all
569
+
typically involve a referencing norm that depicts a Caucasian woman wearing a colorful, high-contrast dress.
570
+
Even though there were many different Shirleys (in analogue photography) or China Girls (in color film
571
+
chemistry) that modeled for these test cards, they were never selected to facilitate variation. Although the
572
+
identities of the many Shirleys who modeled for these norms have stayed unknown, they have nonetheless
573
+
formed a collective public identity, which is often used as a standard on color test cards.20 As such, the cards
574
+
cultivated a gendered and race biased reference, which even today continues to influence our image processing
575
+
technologies. In his 1997 book White, British film studies professor Richard Dyer observed the following: In
576
+
the history of photography and film, getting the right image meant finding the image that conformed to
577
+
prevalent ideals of humanity. This included ideals of whiteness, of what color –– what range of hue –– white
578
+
people wanted white people to be.21
579
+
18 This chapter is based on a talk I gave during Elevate Festival 2016 (Graz, Austria), for which I was kindly invited by
580
+
581
+
Nora O’Murcu. I would also like to extend my gratitude to Bogomir Doringer for inviting me to present my story and
582
+
research as part of his Faceless panel during Resonate 2017 (Belgrade, Serbia) and publish in his book of the same title.
583
+
And last, but not least, to Ward Jansen who has been a driving force behind the joint acquisition of a Vernacular of File
584
+
Formats by the Stedelijk Museum Amsterdam and Moti in 2016, which initiated my thinking on topics covered in this
585
+
paper. This text also accompanied my solo show of the same name, at Transfer Gallery, New York in 2017.
586
+
19
587
+
588
+
Thorlabs, ‘Resolution Test Targets’. Website. (11.12.19).
589
+
see: https://www.thorlabs.com/NewGroupPage9_PF.cfm?ObjectGroup_ID=4338
590
+
20 Gross, Benjamin. ‘Living Test Patterns: The Models Who Calibrated Color TV’, in: The Atlantic (06/28/2015).
591
+
592
+
see: https://www.theatlantic.com/technology/archive/2015/06/miss-color-tv/396266/
593
+
21
594
+
595
+
Dyer, Richard. White: Essays on Race and Culture. UK: Routledge, 1997.
596
+
V
597
+
598
+
VeriColor II testcard by Kodak.
599
+
VeriColor III, a professional portrait film developed in the early 80s by Richard Wien and his team at Kodak Park,
600
+
became known for its more flexible accommodation of a range of skin colours. (Lorna Roth, Looking at Shirley: 2009)
601
+
602
+
The de-facto ideal standard that has been in play since the early part of the twentieth century for most
603
+
analogue photo labs has thus been positively biased toward white skin tones, which naturally have a high level
604
+
of reflectivity. As a result it was not only difficult to capture darker and blacker skin tones, but it also proved
605
+
impossible to capture two highly contrasting skin tones within the same shot: when trying to capture a black
606
+
person sitting next to a white person, the African American facial image would often lose details and pose
607
+
lighting challenges. In consequence, film material generally came to depict ashen-looking facial skin colors that
608
+
contrasted harshly with the whites of eyes and teeth. The Caucasian test card is thus not about variation, but
609
+
about setting a racist standard, which has been dogmatically implemented for more than 40 years.
610
+
611
+
A
612
+
613
+
nalogue Photographies’ Shirley Cards
614
+
615
+
The failure of photographic film stock to capture dark skin tones isn’t a technical issue, but a choice. Reacting
616
+
to these findings, scholar Lorna Roth writes in her 2009 article “Looking at Shirley, the Ultimate Norm” that
617
+
film emulsion could have been designed with more sensitivity to the continuum of yellow, brown and reddish
618
+
skin tones. The willful disregard by film companies, such as Kodacolor (1928) and Kodachrome (1935), is
619
+
reasoned in terms of the markets centering of white consumers.
620
+
It was only when “chocolate” production companies and manufacturers of wooden furniture complained
621
+
about the impossibilities they faced when trying to reproduce different shades of brown, that Kodak’s chemists
622
+
started changing the sensitivities of their film emulsions (the coating on the film base which reacts with
623
+
chemicals and light to produce an image). The company gradually extended the capacities of their film stock for
624
+
VI
625
+
626
+
greater dynamic range/ratio between the maximum and minimum measurable
627
+
light intensities (white and black, respectively). 22 While progress was made
628
+
over the 1970s and 1980s, it was only in 1997 that Kodak’s dynamic range
629
+
made a real leap forward, with the introduction of its popular consumer film
630
+
Gold Max. Roth notes how Kodak executive Richard Wien described this
631
+
development in terms of ability of the companies film to capture the details of a
632
+
dark horse in low light.23 Still, in the real world, true white and black do not
633
+
exist –– only varying degrees of light source intensity and subject reflectivity.
634
+
Moreover, the concept of dynamic range is complex and depends on whether
635
+
one is calculating a capturing device (such as a camera or scanner), a display
636
+
device (such as a print or computer display), or the subject itself.
637
+
This is why around the same time that these changes in sensitivity of film emulsion took place, the color
638
+
test card was also revisited, albeit only slightly. First, in the mid-90s, Japanese photography companies
639
+
redesigned their Shirley cards using their own stock images from their own color preference tests. Since, the
640
+
local reference card featured Japanese women with light yellow skin.24 Finally, in 1995, Kodak designed a
641
+
multiracial norm for their reference card. 25 From the single Caucasian woman surrounded by the necessary
642
+
color balancing information codes, Kodak’s Shirley has now evolved into an image of three women with
643
+
different skin colors (Caucasian, Asian, African American), all dressed in brightly colored, contrasted clothing.
644
+
645
+
22 Roth, Lorna. ’Looking at Shirley, the Ultimate Norm: Colour Balance, Image Technologies, and Cognitive Equity,' in:
646
+
647
+
Canadian Journal of Communication (no. 34.1. 2009): p. 111.
648
+
23
649
+
650
+
Ibid.
651
+
652
+
24 Ibid.
653
+
25
654
+
655
+
Ibid.
656
+
VII
657
+
658
+
Have your highlights lost their sparkle?
659
+
And the midtones lost their scale?
660
+
Are your shadows going smokey?
661
+
And the colors turning stale?
662
+
Have you lost a little business to labs whose pictures shine?
663
+
Because to do it right –– takes a lot of time.
664
+
Well, here’s a brand new system. It’s simple as can be!
665
+
Its name is LAD – an acronym for Laboratory Aim Density.
666
+
– John P. Pytlak26
667
+
668
+
M
669
+
670
+
otion Picture Color Correction: China Girls vs. Maureen the LAD girl
671
+
672
+
In a similar vain to analogue photography, from the 1920s to the early 1990s, the analogue motion picture
673
+
industry had its own color test equivalent, “color-timing.” The term timing hails from the days before
674
+
automated printers, when the photo chemical process used a timer to determine how long a particular film strip
675
+
had to sit in the developer. During the decades of color-timing, hundreds of female faces or China Girls (which
676
+
some have described as a reference to the porcelain mannequins used in early screen tests) appeared in the film
677
+
leaders, typically only for one to four frames, never intended to be seen by anyone other than the projectionist.
678
+
The color-timing practice was not completely reliable as it involved a different China Girl and slightly
679
+
different lighting arrangement each time. Around the 1980s it was gradually superseded by the Laboratory Aim
680
+
Density (LAD) system, developed by John Pytlak. Along with color-timing, the anonymous China Girls, whose
681
+
occupations ranged from studio workers to models, became artifacts of an obsolete film history. Only one LAD
682
+
Girl was to become the model for the color reference card: Maureen Darby.27 Pytlak accounts that it was
683
+
primarily intended as “representative” footage and not as a standard. By filming two 400-foot rolls of 5247
684
+
film, all film supplied since the introduction of LAD is made from the same original negative, either as a
685
+
duplicate negative or now as a digital intermediate. 28
686
+
Two decades later –– after spending a year and a half restoring lost color strip images –– Julie Buck and
687
+
26
688
+
689
+
Pytlak, John. ‘Leader Ladies Project’, Chicago Film Society.
690
+
see: http://www.northwestchicagofilmsociety.org/projects/leaderladies/ (20.02.20)
691
+
27 Pytlak, John. "China Girl” in: Film Tech Forums (22.02.2006)
692
+
693
+
see: http://www.film-tech.com/ubb/f8/t004510.html (20.02.20)
694
+
28
695
+
696
+
Monaghan, Peter. ‘China Girls, Leading Ladies, Actual Women,’ Moving Image Archive, 2014.
697
+
see: http://www.movingimagearchivenews.org/china-girls-leading-ladies-actual-women/ (20.02.20)
698
+
VIII
699
+
700
+
archivist Karin Segal finally found a way to bring the China Girls, or women of color-correction, to the
701
+
spotlight. Rescuing the China Girls from the margins of cinema, they intended to “recast them as movie stars in
702
+
their own right.”29 In their 2005 “Girls on Film” exhibition statement, Buck and Segal write: “Even though
703
+
these women were idealized, they were only seen by a handful of men. Their images exist on the fringes of
704
+
film. They were abused and damaged. We wanted to give them their due.” 30 Unable to find any cases of ChinaGirls-turned-film-actresses, Buck and Segal used their collection of images to create the short, Girls on Film
705
+
(2008); 31 wherein these absent figures were recast as stars.
706
+
707
+
You know what a black-and-white test pattern is?
708
+
she told The New York Times in 1953.
709
+
Well, I’m it for color. I’m the final check.
710
+
– Marie McNamara32
711
+
712
+
O
713
+
714
+
ne Standard Does not Fit All (or: Physics is not Just Physics)
715
+
716
+
With the onset of color television there came no significant shifts: producers hired Caucasian ladies as their test
717
+
models, reinforcing longstanding gender and race biases. The only difference being that in television, the
718
+
objectified test model, was known by her real name. The red haired model Marie McNamara, for instance,
719
+
became known in the 1950s when she modeled to calibrate the NBC television cameras, while Carole Hersee is
720
+
known as the face of the famous Test Card F (and later J, W, and X), which aired on BBC Television from 1967
721
+
722
+
29 ‘Artists Reveal and Reinterpret Captivating Imagery,’ in: Artdaily, 2005
723
+
724
+
see: http://artdaily.com/news/14311/Artists-Reveal-and-Reinterpret-Captivating-Imagery. (20.02.20)
725
+
30
726
+
727
+
Gewertz, Ken. ‘A Bevy of Unknown Beauties,’ in: Harvard News Office, 2005.
728
+
see: http://news.harvard.edu/gazette/story/2005/07/a-bevy-of-unknown-beauties/ (20.02.20)
729
+
31 Buck, Julie and Karin Segal, “Girls on Film.” video, 2008.
730
+
731
+
see: https://www.youtube.com/watch?v=-nVBDX3P5TY (20.02.20)
732
+
A collection of China and Lily films can be found in this article: Tom Warner, “China Girls on Film, 2017.
733
+
see: http://accelerateddecrepitude.blogspot.com/2017/03/china-girls-on-film.html (20.02.20)
734
+
32
735
+
736
+
McNamera, Marie, in: New York Times, 1953.
737
+
Gross, Benjamin. ‘Living Test Patterns: The Models Who Calibrated Color TV’, in: The Atlantic (06/28/2015).
738
+
see: https://www.theatlantic.com/technology/archive/2015/06/miss-color-tv/396266/
739
+
IX
740
+
741
+
to 1998.33
742
+
Cameramen continued to use Caucasian color girls –– either as live models
743
+
or as photographs –– to test their color settings. If an actor with a different skin
744
+
color entered the scene, the calibration process was supplemented with special
745
+
lighting or makeup techniques to ensure that the non-white participants looked
746
+
good on screen –– a task that is not always easy and deferred the development
747
+
and implementation of adequate, non-biased technologies. Such conditions led
748
+
Lorna Roth to conclude, in her seminal article, that the habitual racism
749
+
embedded within color reference cards did more than just influence major
750
+
standard settings: the tone of hue, chroma, contrast, quantization, and lightness
751
+
(luminance) values. To her, it is also responsible for the highly deficient renderings of non-Caucasian skin
752
+
tones, which have resulted in an ongoing need for compensatory practices.
753
+
This one size fits all is problematic, as a technician once explained to Roth: physics is physics approach
754
+
has become the standard, in reality, the various complexions reflect light differently. What this reveals is a
755
+
composite interplay between the different settings involved when capturing the subject. Despite the obvious
756
+
need to factor in these different requirements for different hues and complexions, television technically only
757
+
implemented support for one – the Caucasian complexion. 34
758
+
Moreover, the history of color bias did not end when old analogue standards were superseded by digital
759
+
ones because digital image (compression) technologies too, inherited these standards. As a result, even
760
+
contemporary standards are often rooted within these racist, habitual practices. New digital technologies still
761
+
feature embedded racial biases. For instance, in 2009 and 2010 respectively, HP webcams and the Microsoft’s
762
+
X-Box Kinect controller had difficulties tracking the faces of African American users. Consumer reports later
763
+
attributed both problems to low-level lighting, again moving the conversation away from important questions
764
+
about skin tone to the determination of a proper lighting level –– discussions continue to echo naive physics
765
+
justifications.
766
+
767
+
L
768
+
769
+
ena JPEG
770
+
771
+
In his retrospective article “How I Came Up with the Discrete Cosine Transform (DCT),” Nasir Ahmed
772
+
describes his conception of the use of a cosine transform in the field of image compression. Ahmed recollects
773
+
33 Gross, Benjamin. ‘Living Test Patterns: The Models Who Calibrated Color TV,’ in: The Atlantic (06/28/2015).
774
+
775
+
see: https://www.theatlantic.com/technology/archive/2015/06/miss-color-tv/396266/
776
+
34
777
+
778
+
Roth, Lorna. ”Looking at Shirley, the ultimate norm: Colour balance, image technologies, and cognitive equity" in
779
+
Canadian Journal of Communication (no. 34.1, 2009): p. 118.
780
+
X
781
+
782
+
how he proposed the National Science Foundation (NSF) study for the application of the cosine transform and
783
+
how much to his disappointment, the NSF did not support the proposal, because the whole idea seemed too
784
+
simple.35 Ahmed decided to keep working on the problem, ultimately publishing his results in the January 1974
785
+
issue of IEEE Computer Transactions. Today, more than 40 years after Ahmed’s proposal, DCT is widely used
786
+
in digital image compression. For instance the algorithm has become a core component of JPEG image
787
+
compression technology, which is developed by the JPEG Experts Group. 36
788
+
I remember dedicating the whole summer of 1973 to work on this problem. The results that we got
789
+
appeared too good to be true, and I therefore decided to consult Harry Andrews later that year at a
790
+
conference in New Orleans. […] When I sent the results back to Harry Andrews, he suggested that I
791
+
publish them. As such, I sent them to the IEEE Computer Transactions, and the paper was then published
792
+
in the January 1974 issue. […] Little did we realize at that time that the resulting ›DCT‹ would be widely
793
+
used in the future!37
794
+
Shortly after Ahmed’s initial proposal, the implementation of DCT for digital image compression also became a
795
+
subject of experiments at the University of Southern California’s (USC) Signal and Image Processing Institute.
796
+
In a 2001 newsletter, Jamie Hutchinson offers an insightful reflection on how the testing of DCT again focused
797
+
on the implementation of a Caucasian female color test card. Hutchinson quotes Alexander Sawchuk, who
798
+
reminisces on his efforts to implement the test card during his time as assistant professor of electrical
799
+
engineering. Sawchuk explains that he and his colleagues were tired of the normal test images or “dull stuff.”
800
+
They wanted something glossy to ensure good output in dynamic range and they wanted a human face. Just
801
+
then, somebody happened to walk in with a recent issue of Playboy. 38 Sawchuk moves on to describe how they
802
+
ripped out the centerfold of the Playboy and scanned its top third with their Muirhead scanner, which they had
803
+
customized with analog-to-digital converters to create a three-channel, 512 x 512px, test image. After the tricky
804
+
processing stage was finished, Sawchuk realized that they had lost a line while scanning. Moreover, the timing
805
+
of the analogue-to-digital converters was off, making the final test image slightly more elongated than the
806
+
original. Faced with time pressure, the engineers settled for the distorted version and simply replicated the top
807
+
35 Ahmed, Nasir: How I Came Up with the Discrete Cosine Transform. Electrical and Computer Engineering Department,
808
+
809
+
University of New Mexico, Albuquerque, New Mexico in: Digital Signal Processing 1.1 (1991): p. 4-5.
810
+
36
811
+
812
+
For an overview of the JPEG standard see: https://jpeg.org/jpeg/index.html (20.02.20)
813
+
814
+
37 Ahmed, Nasir: How I Came Up with the Discrete Cosine Transform. Electrical and Computer Engineering Department,
815
+
816
+
University of New Mexico, Albuquerque, New Mexico in: Digital Signal Processing 1.1 (1991): p. 4-5.
817
+
38
818
+
819
+
Hutchinson, Jamie. ‘Culture, Communication, and an Information Age Madonna,’ in IEEE Professional
820
+
Communication Society Newsletter (v.5 no,3, 2001): p. 1–7.
821
+
XI
822
+
823
+
XII
824
+
825
+
line to arrive at 512. Those three sets of 512 lines –– one set for each color, created imperfectly –– would
826
+
become the de facto industry standard.
827
+
The Miss November 1972 centerfold, that the USC employees used for testing the implementation of
828
+
DCT, featured the Caucasian model Lena Sjööblom (or today Lena Forsén). Lena quickly became the single
829
+
most used picture in image-processing research and notably one of the first pictures uploaded to ARPANET (the
830
+
precursor of today’s internet). In A Note on Lena (1996), David Munson, University of Illinois professor and
831
+
editor-in-chief at IEEE Transactions on Image Processing, explains why he believes the Lena image became an
832
+
industry standard: “First of all the image contains a nice mixture of detail, flat regions, shading, and texture that
833
+
do a good job of testing various image processing algorithms. It is a good test image! Second, the Lena image is
834
+
a picture of an attractive woman. It is not surprising that the (mostly male) image processing research
835
+
community gravitated toward an image that they found attractive.” 39 Munson goes on to describe why the Lena
836
+
image has become such an issue: “Some members of our community are unhappy with the source of the Lena
837
+
image. I am sympathetic to their argument, which states that we should not use material from any publication
838
+
that is seen (by some) as being degrading to women.” 40
839
+
While the use of the Lena image remained a topic of discussion –– not to mention the rights of use never
840
+
being confirmed or even checked with Playboy –– by 1991, USCs Signal and Image Processing Institute (SIPI)
841
+
actually started distributing the image of Lena for a fee, to researchers all over the world. While Lena was
842
+
regularly found on the pages of image processing journals, books and conference papers, Playboy finally
843
+
became aware of the transgressions after the Journal of Optical Engineering featured Lena on its July cover. In
844
+
August 1991, Optical Engineering received a letter from Playboy Enterprises Inc. asking them, as fellow
845
+
publishers, to cease any unintentional, unauthorized use of the image and to contact Playboy for permission for
846
+
any future use of their copyrighted material. The International Society for Optical Engineering (SPIE)
847
+
responded, arguing that, “The image is widely used in the worldwide optics and electronics community. It is
848
+
digitized and its common use permits comparison of different image processing techniques and algorithms
849
+
coming out of different research laboratories.” 41 They also pointed out that SPIE is a nonprofit scientific society
850
+
and that the material published by SPIE is intended for educational and research purposes.
851
+
SPIE eventually reached an agreement with Playboy, disclosing a warning in their January 1992 editorial
852
+
39
853
+
854
+
The Lena image was actually not the first Playboy image used as testcard for image calibration. The first Playboy
855
+
magazine centerfold known to be utilized to illustrate image processing algorithms dates back to 1961, when Lawrence G.
856
+
Roberts used two cropped 6-bit grayscale scanned images from Playboy's July 1960 issue (featuring Playmate Teddi
857
+
Smith), for his MIT master's thesis on image dithering.
858
+
40 Munson, David C., ‘A Note on Lena,’ in: IEEE Transactions on Image Processing (no. v.5 no.1, 1996): p. 3.
859
+
41
860
+
861
+
Thompson, Brian J. ‘Editorial,’ in: OPTICAL ENGINEERING (V. 31 no.1): p. 5.
862
+
see: http://webcache.googleusercontent.com/search?q=cache:h_JsSzpi8coJ:www.lenna.org/optical.html
863
+
+&cd=2&hl=en&ct=clnk&gl=jp&lr=lang_de%7Clang_en
864
+
XIII
865
+
866
+
that it is each author's responsibility to make sure materials in their articles are either free of copyright or that
867
+
permission from the copyright holder has been obtained. Seeing an opportunity, Eileen Kent, vice president of
868
+
new media for Playboy, publicly commented on the issue: “We decided we should exploit this, because it is a
869
+
phenomenon.” 42 SPIE was granted authorization for all further use of the image. According to publications
870
+
director at SPIE Eric Pepper, it was almost as if Lena had entered the public domain by that time –– almost, but
871
+
not quite. 43
872
+
In May 1997, almost 25 years after being Miss November, Lena Söderberg attended the fiftieth
873
+
anniversary of the Imaging Science and Technology (IS&T) Conference in Boston. Jeff Seideman, the president
874
+
of the Boston IS&T, arranged for Lena to appear and after the event Seideman started working with a Playboy
875
+
archivist on rescanning Lena's image in the hopes of recovering missing information, such as: the type of photo
876
+
emulsion used to make the print featured in the magazine and the technical specifications of the scanner. As a
877
+
result, Seideman hoped that the image of Lena would remain a standard reference image for compression
878
+
technologies throughout the twenty-first century. Even today, the standard Lena test image is downloadable
879
+
from several laboratory sites.44
880
+
Notably, the controversy around the Lena image did not end in the 1990s. In 2001, David Munson, editor
881
+
of the IEEE’s image processing journal, wrote:
882
+
It was clear that some people wanted me to ban Lena from the journal […] People didn’t object to the
883
+
image itself, but to the fact that it came from Playboy, which they feel exploits women. Rather than ban
884
+
Lena, Munson wrote an editorial in which he encouraged authors to use other images. 45
885
+
In 2016, Scott Acton, editor of IEEE Transactions, proposed the journal’s editorial board instate a prohibition
886
+
on the use of Lena in any published research: “We could be fine-tuning our algorithms, our approaches to this
887
+
one image. […] They will do great on that one image, but will they do well on anything else? […].” In 2016,
888
+
demonstrating that something works on Lena isn’t really demonstrating that the technology works. Acton
889
+
42 Brown, Janelle. ‘Playmate Meets Geeks Who Made Her a Net Star’ in: Wired News, (20.05.97).
890
+
891
+
see: https://www.cs.cmu.edu/~chuck/lennapg/wired_backups/4000.html.
892
+
see: http://www.lenna.org/wired_backups/4000.html (20.02.20)
893
+
43
894
+
895
+
Hutchinson, Jamie. ‘Culture, Communication, and an Information Age Madonna,’ in: IEEE Professional
896
+
Communication Society Newsletter (v.5 no.3, 2001): p. 1–7.
897
+
44 Asuni, Nicola and Andrea Giachetti, ‘TESTIMAGES: A Large Data Archive for Display and Algorithm Testing,’ in:
898
+
899
+
Journal of Graphics Tools (no.17 v.4, 2013): p. 113–125.
900
+
45
901
+
902
+
Hutchinson, Jamie. ‘Culture, Communication, and an Information Age Madonna,’ in: IEEE Professional
903
+
Communication Society Newsletter (v.5 no.3, 2001): p. 1–7.
904
+
XIV
905
+
906
+
believed that the Lena image doesn’t send the right message to female researchers about their inclusion in the
907
+
field. But Acton’s strongest objections were technical in nature: Lena contains about 250,000 pixels, some 32
908
+
times smaller than a picture snapped with an iPhone 6. And then there’s a quality problem: The most commonly
909
+
used version of the image is a scan of a printed page. The printing process doesn’t produce a continuous image,
910
+
but rather a series of dots that trick your eye into seeing continuous tones and colors. Those dots, Acton says,
911
+
mean that the scanned Lena image isn’t comparable to photos produced by modern digital cameras. Short of an
912
+
all-out ban in the journal, he says, making authors aware of the image’s technical and ethical issues might be a
913
+
way to usher Lena gracefully into retirement.46
914
+
While it is clear that the use of the Lena image opened a discussion about embedded bias and the
915
+
consideration of gender in test card usage, many questions remain unanswered. How much are the performance,
916
+
texture and materiality of digital photography actually influenced by the use of the image of Caucasian Lena?
917
+
What would it have meant for the standardization of digital image compression if the image chosen for the test
918
+
card would have been the first African American Playboy centerfold Jennifer Jackson (March, 1965); or if the
919
+
512 x 512 pixel image had instead featured the image of Grace Murray Hopper, one of the first African
920
+
American pioneers in computer programming and the person responsible for inventing some of the first
921
+
compiler-related tools –– moreover, the woman who, coincidentally, coined the widely used computer slang
922
+
“bug.” How much do the compression standards we use on a day to day basis reflect the complexities of the
923
+
good 512 x 512 pixel Lena image and how well do these standard settings function when capturing another kind
924
+
of color complexity?
925
+
926
+
46
927
+
928
+
Iozzio, Corinne. ‘The Playboy Centerfold that Helped Create the JPEG,’ in: The Atlantic, 2016.
929
+
see: https://www.theatlantic.com/technology/archive/2016/02/lena-image-processing-playboy/461970/ (20.02.20)
930
+
XV
931
+
932
+
Dullaart, Constant. Jennifer in Paradise, 2013.
933
+
934
+
Dear Jennifer,
935
+
Sometime in 1987, you were sitting on a beach in Bora Bora, looking at To’opua island, enjoying a
936
+
holiday with a very serious boyfriend. […] This photograph of a beautiful moment in your personal
937
+
history has also become a part of my history, and that of many other people; it has even shaped our
938
+
outlooks on the world at large. John’s image of you became the first image to be publicly altered by the
939
+
most influential image manipulation program ever. […] In essence, it was the very first photoshop
940
+
meme –– but now the image is nowhere to be found online.
941
+
Did John ask you if he could use the image? Did you enjoy seeing yourself on the screen as much as he
942
+
did? Did you think you would be the muse that would inspire so much contemporary image making?
943
+
XVI
944
+
945
+
Did you ever print out the image? Would you be willing to share it with me, and so, the other people
946
+
for whom it took on such an unexpected significance? Shouldn’t the Smithsonian have the negative of
947
+
that image, not to mention digital backups of its endless variations?
948
+
All these questions have made me decide to redistribute the image ›Jennifer in Paradise‹ as well as I
949
+
can, somewhat as an artist, somewhat as a digital archeologist, restoring what few traces of it I could
950
+
find. It was sad to realize this blurry screen grab was the closest I could get to the image, but beautiful
951
+
at the same time. How often do you find an important image that is not online in several different sizes
952
+
already?47
953
+
954
+
J
955
+
956
+
ennifer in Paradise (Constant Dullaart, 2013)
957
+
958
+
A woman is sitting with her back toward us, topless, on a beach. Silver sand, blue water, a green island in the
959
+
distance. We can’t see her face but we know her name: Jennifer. This photo, taken in 1987 by one of the two
960
+
creators of Photoshop, John Knoll, became the standard test image for the development and implementation of
961
+
Photoshop and its suite of creative effects. Twirling, deleting and copying Jennifer were just some of the myriad
962
+
processes tested on the image. At that time, the early days of digital computing, there was not a large array of
963
+
digital images available, which is why this 24-bit scan of a holiday photo of John’s soon-to-be Jennifer Knoll
964
+
became a standard test image for all Photoshop's developments. It is also one of the reasons why the image did
965
+
not disappear when Photoshop moved out of its development phase. When Photoshop was finally ready for
966
+
public use, John and his brother Thomas used the image again and again in online demos. It was a good image
967
+
to do demos with, John Knoll recalls, as it was pleasing to look at and there were a whole bunch of things you
968
+
could do with that image technically. 48
969
+
As Dutch artist Constant Dullaart explains in his Chaos Computer Club presentation The Possibility of an
970
+
Army, John Knoll confirmed an age-old motif: a man objectifying a female body.49 Besides being critical,
971
+
Dullaart also underlined the special cultural-historical value of the artifact, which formed a key inspiration for
972
+
his 2013 Future Gallery solo show Jennifer in Paradise. In this show, Dullaart focused on the excavation and
973
+
47
974
+
975
+
Dullaart, Constant and John Knoll. ‘Jennifer in Paradise – the correspondence,’ in: Carroll / Fletcher Onscreen: 1
976
+
March 2016.
977
+
see: http://carrollfletcheronscreen.com/2016/03/01/jennifer-in-paradise-the-correspondence/.
978
+
48 Comstock, Gordon. ‘Jennifer in Paradise: the story of the first Photoshopped image,’ in: The Guardian, 2014.
979
+
980
+
see: https://www.theguardian.com/artanddesign/photography-blog/2014/jun/13/photoshop-first-image-jennifer-inparadise-photography-artefact-knoll-dullaart.
981
+
49
982
+
983
+
Constant Dullaart, The Possibility of an Army, presentation at 32c3, 2015.
984
+
see: https://media.ccc.de/v/32c3-7517-the_possibility_of_an_army.
985
+
XVII
986
+
987
+
exhibition of a reconstruction of the Jennifer image. In an open letter accompanying the show, Dullaart
988
+
describes the image of Jennifer as an important artifact in the history of software development and as an
989
+
anecdote in Adobe history. He asks Jennifer to share the original image file with the world. A sentiment that
990
+
was later echoed by Gordon Comstock in a 2014 piece for The Guardian, in which he describes the image as
991
+
central to the modern visual vernacular as Eadweard Muybridge’s shots of galloping horses or the first use of
992
+
perspective. In a way, just like the Lena image, Jennifer has become a phenomenon.
993
+
While Dullaart never obtained any rights or permissions for the use of the Jennifer image, he did digitally
994
+
reconstruct the original image, creating an image series consisting of Photshopped versions, materialized as
995
+
wallpaper, and a series of prints featuring enthusiastically filtered Jennifers (twirled, blurred, etc.). Dullaart also
996
+
spread the digitally reconstructed version of the original image with an added payload: steganographically
997
+
adding messages to the reconstructed JPEG image file. In doing so, he intended to treat the JPEG image not just
998
+
as an image, but as a unique container format for content. This, he hoped, would open a debate on the value of
999
+
the digital file (format). The reconstructed Jennifer JPEG is not just a format that carries the reconstructed
1000
+
image information. Via the steganographic technique it has become a unique container and placeholder for
1001
+
discussing the materiality of digital photography. In terms of monetization of the material, Dullaart only sells
1002
+
the password to the encrypted payload added to the reconstructed version of the original JPEG –– the access to
1003
+
his secrete message. Finally, in an effort to translate the work for the context of the gallery, Dullaart organized a
1004
+
performance in which he briefly showed his secret message written in phosphorescent paint on top of the
1005
+
wallpaper by shining a blacklight on its surface, followed by a destruction of the blacklight as a metaphor for
1006
+
encryption (and inaccessibility).
1007
+
Dullaart never received a direct response from Jennifer or John Knoll to his request to enter the original
1008
+
image into the public domain or to gift it to a (media) archeological institution such as the Smithsonian.
1009
+
Remarkably, for his article in the Guardian, Comstack did manage to get a short response from both:
1010
+
John Knoll seems unconvinced: “I don't even understand what he’s doing,” he says, bristling at the idea
1011
+
of the image being reconstructed without permission (ironically using Photoshop). Jennifer is more
1012
+
sanguine: “The beauty of the internet is that people can take things, and do what they want with them, to
1013
+
project what they want or feel,” she says. 50
1014
+
And maybe even more remarkable is the fact that the article features just one image: the original Jennifer
1015
+
in Paradise photo taken by John Knoll, embedded on the newspaper’s website (and thus finally entering the
1016
+
digital domain). Albeit indirectly, Dullaart had now fulfilled one of the main goals of his solo show.
1017
+
50
1018
+
1019
+
Comstock, Gordon. ‘Jennifer in Paradise: the story of the first Photoshopped image,’ in: The Guardian, 2014.
1020
+
see: https://www.theguardian.com/artanddesign/photography-blog/2014/jun/13/photoshop-first-image-jennifer-inparadise-photography-artefact-knoll-dullaart.
1021
+
XVIII
1022
+
1023
+
P
1024
+
1025
+
ique Nique pour les Inconnues
1026
+
1027
+
Desktop video installation 2019-2020.
1028
+
1029
+
Pique Nique pour les Inconnues considers the ways in which the history of technology has been defined by
1030
+
standardization, in particular through the use of color test cards for image processing. The work presents les
1031
+
Inconnues - unknown women whose images are linked to the history of image processing. In this work, test
1032
+
cards, bots, virtual assistants, stock photos and others find a voice, but fail to recover their personhood.
1033
+
Engineers used these female objects to evaluate the quality of image processing, rendering and
1034
+
composition of architecture and to make these latent spaces more amicable. While these women seem to be able
1035
+
to prolong their existence for as long as the (digital) realms will copy and reuse them, most of them have lost
1036
+
their name and identity. In this work, the viewer is haunted by the familiarity of these digital ghosts, while at
1037
+
the same time, privy to an uncanny experience when the historically mute images speak for the first time.
1038
+
XIX
1039
+
1040
+
P
1041
+
1042
+
ique Nique pour les Inconnues (2019-2020)
1043
+
1044
+
Telegram sticker set
1045
+
A fundamental part of the history of image-processing, webdesign, and the standardization of settings within
1046
+
both analogue and digital media are test cards, placeholder images, bots, digital actroids or virtual assitants.
1047
+
Engineers use these images to evaluate the quality of image processing, the rendering and composition of the
1048
+
architecture of a (digital) space, but also simply to make latent spaces more amicable.
1049
+
However, just like the canned laughter tracks of televisions shows, that are made by real people who were
1050
+
immortalized when they lend their laughs in the 30s or 40s, these women too seem to have prolonged their
1051
+
exitence for as long as the (digital) realms will copy and reuse them. They have become a "shell without a
1052
+
ghost".
1053
+
XIII
1054
+
1055
+
1:00 l’Inconnue de la Seine (after 1900)
1056
+
The Unknown Woman of the Seine is a death mask of an unidentified young woman that became a popular
1057
+
fixture on the walls of artists’ homes after 1900. She featured in various artists works ranging from books to
1058
+
theatre and film. But while many artists felt inspired by her borrowed visage, little but her moniker is known
1059
+
about her and she remains forever an asset with missing values.
1060
+
2:00 Audrey Munson Bust (1913-1915)
1061
+
The bust “The Spirit of Life”, by Daniel Chester French (1913-1915), inspired by ‘America’s First Supermodel’
1062
+
Audrey Munson. Munson was the inspiration for more than 12 other statures in New York City, and many,
1063
+
many others elsewhere. Chances are that when you cross a statue, it might be modelled after her.
1064
+
3:00 Color-timing control strips
1065
+
Officially known as color-timing control strips, these anonymous female film studio workers were
1066
+
affectionately dubbed "china girls" by the industry, but are also known as leader ladies or lilys. The images in
1067
+
this show were meant only for use by the processing lab to match color tones in the associated film. They were
1068
+
often film lab workers themselves.
1069
+
4:00 Miss NBC (1953)
1070
+
The onset of color television brought no big surprise; in this medium too, producers hired Caucasian ladies as
1071
+
their test models, reinforcing longstanding biases in gender and race—the only difference being that in
1072
+
television, the objectified test model was known by her real name. The red-haired model Marie McNamara, for
1073
+
instance, became known in the 1950s when she modelled to calibrate the NBC television cameras, while CBS
1074
+
used a girl named Patty Painter.
1075
+
5:00 Two Bit Teddi Smith (1961)
1076
+
Lawrence G. Roberts used two different, cropped 6-bit grayscale scanned images from Playboy's July 1960
1077
+
issue, featuring Playmate Teddi Smith, in his MIT master's thesis on image dithering.
1078
+
6:00 Carole Hersee (1967)
1079
+
Hersee is known as the face of the famous Test Card F (and latter J, W, and X), which aired on BBC Television
1080
+
from 1967 to 1998.
1081
+
More: https://beyondresolution.info/Les-Inconnues
1082
+
1083
+
XIV
1084
+
1085
+
Menkman, Rosa. A Vernacular of File Formats, 2010.
1086
+
see: https://beyondresolution.info/A-Vernacular-of-File-Formats (20.02.20).
1087
+
1088
+
I
1089
+
1090
+
n Front of the Angel of History
1091
+
1092
+
Covered in a heavy layer of white make-up, she shot her face on a DV tape. She wished to mask her flaws, to
1093
+
be perfect. But only a short time into the shoot, the illusion shattered and she found herself forced to visit the
1094
+
emergency room. An allergic reaction to the makeup hurt her eyes violently and left her sight affected for days.
1095
+
1096
+
B
1097
+
1098
+
ehind the Angel of History
1099
+
Seven years after shooting the source footage for A Vernacular of File Formats: An Edit Guide for Compression
1100
+
Design (2010), it has become strange to think and refer to the image that I shot that day as a self-portrait: that
1101
+
image being my image. When I shot it, it was a symbol of my own imperfect being. I tried to be perfect like
1102
+
porcelain, at least from the outside, but my body broke out and reminded me that there is no such thing as
1103
+
perfection. Not even in make believe video.
1104
+
As I aged, it wasn’t just the cliche of time –– which healed the wounds of bloodshot eyes, and slowly
1105
+
but naturally grayed my hair –– that changed my relation to this particular shot. As time passed, the relationship
1106
+
between me and that image fundamentally changed because of other more complex and unexpected reasons.
1107
+
1108
+
A
1109
+
1110
+
Vernacular of File Formats
1111
+
1112
+
A file format is an encoding system that organizes data according to a particular syntax or compression
1113
+
algorithm. The choice of a particular image compression algorithm is based on its foreseen mode and place of
1114
+
usage; therefore prompting questions such as: how much accuracy is necessary for a particular task? What hard
1115
+
or software will process the image? What data is important and what can be discarded?
1116
+
An image file format answers to certain affordances. Affordances or –– as described by James Gibson in
1117
+
1977 –– preferred object action possibilities, are created by considering settings such as speed, size and quantity
1118
+
XX
1119
+
1120
+
as relative to each other. 51 The bigger the file, the more time it will take to read and write it from memory and
1121
+
the slower the camera will respond. As Adrian Mackenzie wrote in 2008:
1122
+
Software such as codecs possess several analytical problems. Firstly, they are monstrously complicated.
1123
+
Methodologically speaking, coming to grips with them as technical processes may entail long
1124
+
excursions into labyrinths of mathematical formalism and machine architecture, and then finding ways
1125
+
of backing out of them bringing the most relevant features. […] Second, at a phenomenological level,
1126
+
they deeply influence the very texture, flow, and materiality of sounds and images.”52
1127
+
Reverse engineering as a standardization process is thus complex, if not generally impossible. Although
1128
+
standards are often set in a way that avoids or hides all traces of testing and standardization regimes, traces can
1129
+
(re)surface in the form of flaws, inherited dogmas, or (obsolete) artifacts. Every compression algorithm comes
1130
+
with its own set of rules and compromises, which, even though often invisible, influence our media on a
1131
+
fundamental, meaningful, and often compromising level. In A Vernacular of File Formats, I explore and
1132
+
uncover these otherwise hidden protocols and standards. Through a series of corrupted self-portraits, I illustrate
1133
+
the language of compression algorithms. A Vernacular of File Formats consists of one source image, the
1134
+
original portrait, and an arrangement of recompressed and disturbed iterations. By compressing the source
1135
+
image using different compression languages and subsequently implementing the same (or similar) error into
1136
+
each file, the normally invisible compression language presents itself on the surface of the image. Besides every
1137
+
iteration of the image I also try to give an explanation of how I disrupted the image and the basic affordances of
1138
+
the compression responsible for its aesthetic outcome. In doing so, A Vernacular of File Formats was not only a
1139
+
start for my ongoing research into the politics of file formats and their inherent resolutions, but is also a
1140
+
51 Gibson, James J. The ecological approach to visual perception: classic edition. Psychology Press, 2014.
1141
+
52
1142
+
1143
+
Mackenzie, Adrian: Codecs, in: Fuller, Matthew, ed. Software studies: A lexicon. Mit Press, 2008: p. 48-54.
1144
+
XXI
1145
+
1146
+
thesaurus, or handbook, for glitch aesthetics –– the aesthetics of digitally disturbed imagery.
1147
+
After releasing A Vernacular of File Formats, these images initially circulated quite naturally, following
1148
+
the random flow of the Internet. Some of them were republished with consent or attribution, others were badly
1149
+
copied without attribution. Once in a while, I found my face as a profile picture on someone else’s social media
1150
+
account. It became clear that particular iterations of the self-portrait had quite a bit more traction than others;
1151
+
these got frequent requests and pulls and were featured on the covers of books, magazines and online music
1152
+
releases. One of the images became a mascot for a festival in Valencia, included in a poster campaign
1153
+
throughout the city.
1154
+
It was only some years after the release of A Vernacular of File Formats that the displacement of the
1155
+
portrait made me rethink my relation to the image. The first time this happened was when I read a description
1156
+
of the work in a piece by Kevin Benisvy, at the time a student at the University of Massachusetts. Benisvy
1157
+
writes that the protagonist is presented in the act of brushing her hair, with an almost “come hither” expression,
1158
+
as if caught by surprise, having an intimate moment in a Playboy erotic fiction. 53 I never considered the image
1159
+
erotic, to me, the image contains painful memories –– being, after all, a documentation of me losing my vision
1160
+
for a certain amount of time. Reading this gave me insight into the various readings the image could evoke.
1161
+
Soon after, a sequence of nonconsensual, non-attributed instances of exploitation appeared: the face
1162
+
became an embellishment for cheap internet trinkets such as mugs and sweaters; was featured on the cover of a
1163
+
vinyl record by Phon.o released by the Berlin label BPitch Control; an application button for two proprietary
1164
+
glitch software apps for iPhone and Android; as an outline for the face of Yung Joey (a rapper who
1165
+
photoshopped his face onto mine), and was also used in a sponsorship campaign for a Hollywood movie about
1166
+
a woman being stalked; to name just a few surprising appearances. 54 The image, exploited by artists and
1167
+
creators alike, started to lose its connection to the source –– to me –– becoming the portrait of no one in
1168
+
particular, a specter, similar to a Shirley test image, though in this case, a Shirley for de-calibration.
1169
+
During the winter of 2016 –– six years after the creation of A Vernacular of File Formats –– the
1170
+
Vernacular was invited to be part of a large-scale, joint acquisition of Stedelijk Museum Amsterdam and
1171
+
MOTI.55 After thorough consideration, the institutions agreed that the best format for the purchase would be the
1172
+
full archive of digital files, which consists of over 16GB of data (661 files), including the original and glitched,
1173
+
53
1174
+
1175
+
Benisvy, Kevin. ‘The Queer Identity and Glitch: Deconstructing Transparency,’ 2012.
1176
+
http://www.kevinbenisvy.com/sites/default/files/2012%20-%20The%20Queer%20Identity%20and%20Glitch.pdf
1177
+
(site discontinued)
1178
+
54 I have been in touch with both Phon.o and the creators of the sponsorship campaign after their respective releases, and
1179
+
1180
+
now have my consent for using the image.
1181
+
55
1182
+
1183
+
Press Release: AANKOOP 17 TOPWERKEN DOOR STEDELIJK EN MOTI Nieuws — 15 dec 2016. Stedelijk
1184
+
Museum Amsterdam. see: https://www.stedelijk.nl/nl/nieuws/aankopen-stedelijk-en-MOTI (20.02.20).
1185
+
XXII
1186
+
1187
+
or broken image files, the PDF, videos, documentation and a collection of (unsolicited) appropriations. While
1188
+
the whole collection now remains in the archive of the Stedelijk, A Vernacular of File Formats will also remain
1189
+
freely available online, inviting artists and designers to use the files as source footage for their own work and
1190
+
research, following the spirit of COPY < IT > RIGHT !:
1191
+
First, it’s okay to copy! Believe in the process of copying as much as you can; with all your heart is a
1192
+
good place to start – get into it as straight and honestly as possible. Copying is as good (I think better
1193
+
from this vector-view) as any other way of getting‚ “there.”56
1194
+
1195
+
S
1196
+
1197
+
pecters Controlling Our Imaging Technologies
1198
+
1199
+
By now it should be clear that while taking an image of the face and saving it to memory seems like a simple,
1200
+
straightforward act, in fact a large set of protocols and standards intervene in the processes of saving the image
1201
+
to memory –– including, but not limited to: scaling, reordering, decomposing, and reconstituting image data.
1202
+
All of which is directed toward certain affordances, which cater to techno-conventional, political and
1203
+
historically biased settings often geared primarily to efficiency. Some of these biases can be traced back to the
1204
+
history of the color test card; a history which can offer an insightful perspective on how image compression
1205
+
standards have come to exist.
1206
+
The first color test cards were developed almost a century ago. They would feature a Caucasian,
1207
+
anonymous, brightly-dressed girl, smiling in a friendly way at the camera. Throughout the many legacies and
1208
+
histories of image processing –– including, but not limited to analogue photography, film, television, the JPEG
1209
+
algorithm and even Photoshop effects –– this trope grew into a habitual racial bias, violently lodged under the
1210
+
fold of efficient image processing. The habitual use of Caucasian test cards –– such as the Lena photo –– led to
1211
+
the development of certain affordances in the compression algorithm scaling; sometimes even cutting away
1212
+
image data. It is important to be aware that a bias does not just influence the final rendering of the image; the
1213
+
bias also exists in what a technology does not show: what it obscures or obfuscates and what image data simply
1214
+
deletes. The Shirleys, but more importantly the technicians that implemented the use of these Shirleys or color
1215
+
reference cards, cast white shadows: patches of unregistered information during image processing. And while
1216
+
artists such as Hito Steyerl or Constant Dullaart make an effort to spread awareness around the biases and
1217
+
habits that are embedded in the histories of resolutions, even today, a history of these specters influences our
1218
+
images –– albeit often invisibly from the perspective of the unaware observer.
1219
+
Six years after releasing A Vernacular of File Formats, and after close study of some of the histories of
1220
+
56
1221
+
1222
+
Morton, Phil. ‘Distribution Religion,’ 1973, republished in: Re:Copying-IT-RIGHT-AGAIN. Relive: Media Art
1223
+
Histories, ed. Jon Cates. MIT Press, 2013: p. 337.
1224
+
see: http://criticalartware.net/DistributionReligion/DistributionReligion.pdf
1225
+
XXIII
1226
+
1227
+
Menkman, Rosa: A Color Test Card for Image De-calibration, BLINX 1.
1228
+
From: A Vernacular of File Formats, 2011. published in Vogue 2018.
1229
+
1230
+
standardization and resolution setting, I realize that by using my own face as a Shirley card for de-calibration, I
1231
+
unintentionally aligned myself with the historical trope of the Caucasian test card. Just as Jeff Seidemann once
1232
+
said about Lena: when you use a picture often, it becomes merely pixels 57–– my face had become just pixels, or
1233
+
even, I had simply lost my face: it no longer belonged to me, but had become an anonymous image, ready for
1234
+
co-optation.
1235
+
Today I believe that one way to make the habitual whiteness of color test cards more apparent, is by
1236
+
insisting that these standard images that often are trapped in the histories of our technologies, become part of
1237
+
the public domain. These images need to lose their elusive power. The stories of standardization belong in high
1238
+
school textbooks, and the possible violence of standardization should be studied in any curriculum. By
1239
+
illuminating the histories of standardization, we will also see its white shadows.
1240
+
57
1241
+
1242
+
Iozzo, Corinne. ‘The Playboy Centerfold That Helped Create the JPEG. The story of a 1970s computer-science lab, a
1243
+
spare magazine, and one model’s unlikely technological legacy,’ in: The Atlantic, 2016.
1244
+
XXIV
1245
+
1246
+
3
1247
+
1248
+
65 Perfect Decalibration (2020)
1249
+
1250
+
Archival prints (5)
1251
+
In 365 Perfect, Menkman turns to mobile imaging softwares. 365 Perfect is “the best FREE virtual makeup app,
1252
+
period. It’s like having a glam squad in your pocket” - or so states the software. In this humorous, yet
1253
+
discomforting work, Menkman layers the standard features of beautifying software on her own image,
1254
+
enlarging her eyes, deleting blemishes and enhancing features, until the original face is nearly unrecognizable.
1255
+
Over and over, Menkman alters her likeness, seemingly to make herself appear more conventionally beautiful
1256
+
and more “perfect”. Saving her image at every new iteration, she arrives at a re-compressed pixelated JPEG,
1257
+
and a grotesque, if not almost inhuman self-portrait.
1258
+
365 Perfect is not only a commentary on the relationship between mobile technology and the beauty industry,
1259
+
but a careful exploration of resolution loss, and the visual artifacts that are created as a result.
1260
+
1261
+
IX
1262
+
1263
+
U
1264
+
1265
+
nresolved (2020)
1266
+
1267
+
64 meters, 32x32x2 pixel file mounted on hardware
1268
+
Inspired by the 2011 work Beyond Yes and No, by new
1269
+
media artist Beflix, Unresolved explores an alternative
1270
+
method of visualising data.
1271
+
Reproducing two 32x32 pixel bitmap images,
1272
+
Menkman painted a 64 meter long tape, and then
1273
+
mounted the double sided work on hardware (a
1274
+
wooden frame).
1275
+
Unresolved presents an image on one side and a jpeg
1276
+
compression on the other. The work illustrates that
1277
+
bitmap images are merely string of data that can be
1278
+
copied, pasted and otherwise altered. Through this
1279
+
work, Menkman explores the ways in which a bitmap
1280
+
file, when opened in different formats, can create
1281
+
alternate modes of reading.
1282
+
1283
+
X
1284
+
1285
+
LXVII
1286
+
1287
+
Not requested on rider:
1288
+
Setup during the night of the unexpected.
1289
+
1. Sound from Knalpot into circuit bent video mixer
1290
+
A. Analogue to HDMI converter
1291
+
2. Video mixer to subtitler
1292
+
B. Russian video police
1293
+
3. Canopus 110 (RCA to DV for computer)
1294
+
C. A black screen, ‘cleaned’ for better reflection
1295
+
4. Midi controller
1296
+
5. VGA to RCA converter
1297
+
[in: computer VGA. 2x out: to projector and to video mixer, to generate a feedback loop]
1298
+
6. Feedback loop into circuit bent video mixer, mixed with sound from Knalpot
1299
+
I
1300
+
1301
+
T
1302
+
1303
+
he Night of the Unexpected
1304
+
1305
+
In February 2013, I was invited to play a concert with the Dutch band Knalpot during The Night of the
1306
+
Unexpected, which was to take place October, of the same year, in Moscow. The invitation came from the
1307
+
Russian government as part of a celebration of 100 years of trade with the Netherlands. Coincidentally, the
1308
+
event was timed just after the implementation of a federal law passed on June 29, 2013, banning all
1309
+
“propaganda” about non-traditional sexual relationships. Given the circumstances, and with international
1310
+
stipulations about boycotting the country, the question of artistic participation was starkly raised. Needing the
1311
+
money and not feeling comfortable cancelling an event that me and Knalpot had been working on for quite
1312
+
some months, I did not cancel –– although in hindsight maybe I should have.
1313
+
A mediator working for the Dutch embassy in Moscow helped us through the process of obtaining our
1314
+
visa. Besides the normal requests, he demanded a map of my setup, a rider (the list of technological needs – I
1315
+
requested an RCA (analog) connection and a list of gear. He also spoke about permitted AV behavior,
1316
+
referencing the new ban on propaganda. I had to explain my intentions, describing the generation of synced live
1317
+
video, by using the sound of Knalpot.
1318
+
The Night of the Unexpected arrived. The venue was large and we set up in the middle on an island built
1319
+
of scaffolding. A big projector graced the prow of our island, pointing at a professionally suspended black
1320
+
screen. A technician handed me HDMI (digital, not the requested analogue RCA), and an analog-to-digital
1321
+
converter. Surprised by a seemingly rapid accumulation of problems, I turned to the event’s producer:
1322
+
< Where will I project?
1323
+
> This is the screen (pointing at the black screen)
1324
+
< Ah I see…
1325
+
I need to project on a white screen, will there be a white screen?
1326
+
> In your rider it did not specify a white screen.
1327
+
< A black screen will not reflect the light; it will absorb light, which means the projection will not show.
1328
+
I think we really need a white screen.
1329
+
> I ordered the best technology in Moscow. It is the most expensive.
1330
+
(silence)
1331
+
< Can I show you what I mean? We can test it by projecting onto it.
1332
+
… Could I please have the RCA I requested in my rider?
1333
+
> (producer points at HDMI cable and analog-to-digital converter)
1334
+
We have this for you. It’s better, it’s digital.
1335
+
< Oh…
1336
+
II
1337
+
1338
+
I requested analogue out. It is better for me.
1339
+
> HDMI is better resolution.
1340
+
< I need to send my analogue output unconverted, from my synthesizer to the projector, to keep it
1341
+
untransformed and synced with the music the band is playing.
1342
+
> Analogue is not possible. We have HDMI.
1343
+
With none of my initial issues resolved I entered the rehearsal only to encounter an even bigger problem: in the
1344
+
corner of the island, next to a video server, a Russian video engineer (member of what I would dub the
1345
+
“Russian video police”) was screening my live video for offensive content, taking the liberty to overlay or even
1346
+
cut my stream at any time. The digital video server of the Russian video police digitized my analog and synced
1347
+
the video stream, not only corrupting its intrinsic analogue qualities by replacing analogue scanning (line)
1348
+
artifacts with digital macroblocks –– introducing an aspect ratio conflict (the video got stretched) –– but also its
1349
+
timing, by adding a two-second delay. By the end of this process, my performance became a barely visible, unsynced and wrangled disaster.
1350
+
1351
+
R
1352
+
1353
+
esolving the Image Off-Screen
1354
+
1355
+
I then realized that even though a screen often illuminates a situation, what happens beyond the screen is
1356
+
obscured by that same screen. The screen acts as a veil, covering one part of the technology as a Japanese Shoji
1357
+
screen divides a room, concealing (most of) the technological processes involved in resolving the image. 58
1358
+
Further, it demonstrated shortcomings in my use and understanding of the term resolution. During the
1359
+
performance the differences between the resolved image on the black screen visible to the audience, and the
1360
+
image resolved on my check monitor differed not just in terms of brightness or aspect ratio, but also in terms of
1361
+
aesthetics, timing, and most importantly, in terms of power.
1362
+
Thus, the term resolution needs to be expanded to encompass more than just the dimensions of the screen
1363
+
or display –– a final resolution is, after all, not just a matter of width and height. A critical reflection of a
1364
+
screens’ resolution considers the technological procedures and trade-offs a programmer or artist has to deal with
1365
+
58
1366
+
1367
+
Shoji screens are made of a traditional paper called washi, consisting of fibers from the kozo tree. Washi paper has a
1368
+
specific thinness that allows just the right amount of light to go through: “By changing the fiber direction and thickness,
1369
+
washi can control two opposing optical factors such as reflection rate and transparency. Shoji’s paper surface scatters
1370
+
sunlight evenly, making it soft to the eye and allowing light to distribute evenly. […] Even at night, Shoji screens help
1371
+
light a room as their white surface reflects indoor light and brightens the room. Shoji paper is thus quite remarkable: it has
1372
+
no glare problem, maintains privacy, and allows the light to enter in a pleasant way.”
1373
+
Sukiya Living Magazine, “Shoji Screens,” 2004.
1374
+
see: http://www.rothteien.com/superbait/shojiscreens.htm
1375
+
III
1376
+
1377
+
beyond (or behind) the screen, i.e., the power relations and standardizations involved in creating or resolving a
1378
+
final image on the screen. In his thesis Movie/ Cinema: Rearrangements of the Apparatus in Contemporary
1379
+
Movie Circulation, Gabriel Menotti Miglio Pinto Gonring writes:
1380
+
Screens are normally treated as mere flat surfaces, composed of only height and width. One talks about
1381
+
their area, aspect ratio and resolution, as if these characteristics were all that mattered to the screen.
1382
+
Nevertheless, to be able to hold an image, the screen must have some density and in order to be dense,
1383
+
the screen has to be thick. Given this implies solidity and opacity, the thickness accounts for the most
1384
+
material aspect of the screen. It means that, if it is in the wrong position, the screen can actually hide the
1385
+
image from the gaze. 59
1386
+
The question is, what is this depth of the screen and what does it obfuscate and show –– what, in other words,
1387
+
are the qualities and compromises of the process of resolving the image on and off-screen?
1388
+
1389
+
59
1390
+
1391
+
Gonring, Gabriel Menotti Miglio Pinto, Movie / Cinema: Rearrangements of the Apparatus in Contemporary Movie
1392
+
Circulation. Doctoral thesis, Goldsmiths, University of London, 2011: p. 227.
1393
+
see: http://research.gold.ac.uk/6604/ (20.02.20)
1394
+
IV
1395
+
1396
+
R
1397
+
1398
+
efuse to Let the Syntaxes of (a) History Direct Our Futures
1399
+
1400
+
M
1401
+
1402
+
oving Beyond Resolution
1403
+
I wish I could open Google image search, query “rainbow,” and simply listen to any image of a rainbow Google
1404
+
has to offer me. I wish I could add textures to my fonts and that I could embed within such text videos that
1405
+
would play at particular moments. I wish I could render and present my videos as circles, pentagons and other
1406
+
more organic manifolds. If I could do these things I believe my use of the computer would be different: I would
1407
+
create modular relationships between my text files and my videos would have uneven corners, multiple time
1408
+
1409
+
Screenshot of QTzrk (2011) by Jon Satrom
1410
+
I
1411
+
1412
+
In 2011, Chicago glitch artist Jon Satrom released the video installation QTzrk. Technically, QTzrk consists
1413
+
primarily of two video elements. The first element, a 16:9 video, captured from a desktop perspective features
1414
+
a movie.mov file. The movie.mov is shown on top of the desktop, an environment deconstructed by QTzrk.
1415
+
The second type of video elements are Qtlets: smaller, looped videos, which are not quadrilateral. QTlets are
1416
+
constructed and opened via a now obsolete option in Quicktime 7 software. Using this option, Satrom
1417
+
employed a mask to change the shape of the otherwise four-cornered videos, transforming them into “video
1418
+
shards.” QTlet elements are featured in Qtzrk, but are also released as stand-alone downloadables, available
1419
+
on Satrom’s website. Unfortunately, they no longer play properly on recent versions of Mac OS X as the
1420
+
masking option is now obsolete, which makes playing these files a challenge.
1421
+
Story-wise, QTzrk begins when the movie.mov file is clicked. It opens inside a Quicktime 7 player, on
1422
+
top of what will later become visible as a clean desktop without a menu bar. Movie.mov shows a slow motion
1423
+
nature video of a great white shark jumping out of the ocean. Suddenly, a pointer clicks the pause button on
1424
+
the interface, and the Great White Shark turns into a fluid video, leaking out of the Quicktime 7 movie.mov
1425
+
interface. The shark folds up in a kludgy pile of video, resting on the bottom of the desktop, still playing, but
1426
+
now in a messily folded way. The Quicktime 7 movie.mov window changes into what looks like a terminal
1427
+
and is then commanded to save itself as a QTlet named “shark_pile.” The shark_pile is picked up by a mouse
1428
+
pointer, which kind of performs like an invisible hand, hovering the pile over the desktop, finally dropping it
1429
+
back into the Quicktime window, which now shows line after line of mojibake data. This action –– dropping
1430
+
the shark_pile inside the Quicktime window –– seems to be the trigger for the desktop environment to
1431
+
collapse.
1432
+
The Quicktime player breaks apart, no longer adhering to its quadrilateral shape, it now assumes the
1433
+
form of a second, downloadable, QTlet. On the desktop, 35 screenshots of the shark frame appear. A final
1434
+
new QTlet is introduced, this one consists of groups of screenshots, which when opened show glitched png
1435
+
files. These clusters themselves transform into new video sequences (a third downloadable QTlet), adding
1436
+
more layers to the collage. By now the original movie.mov seems to slowly disappear in the desktop
1437
+
background –– itself featuring a data-moshed shark video (data-moshing is the colloquial term for the
1438
+
purposeful deconstruction of an .mpeg, or any other video compression using intraframe/keyframe standards).
1439
+
After a minute of noisy droning of the QTlets on top of a data-moshed shark, the desktop suddenly starts to
1440
+
zoom out, revealing one Quicktime 7 window inside the next. Finally, the cursor clicks the close button in the
1441
+
Quicktime 7 interface, ending the presentation and revealing a clean white desktop with just the one
1442
+
movie. mov file icon in the middle. Just when the pointer is about to reopen the movie.mov file, and start the
1443
+
loop all over again, QTzrk ends.
1444
+
TITLE: QTzrk — DIMENSIONS: expandable/variable — MATERIALS: QuickTime 7
1445
+
YEAR: 2011 — PRICE: FREE
1446
+
II
1447
+
1448
+
lines and changing soundtracks. In short, I think my computational experience could be much more like an
1449
+
integrated collage if my operating system would allow me to make it so.
1450
+
The installation, consisting of four different video loops, introduced me to both the genre of desktop
1451
+
film and to non-quadrilateral video technology. As such, it left me both shocked and inspired. So much so that,
1452
+
in 2012, inspired by Satrom’s work, I set out to build Compress Process, an application that would make it
1453
+
possible to navigate video inside a 3D environment. I too wanted to stretch the limits of video, especially
1454
+
beyond its quadrilateral frame. However, upon release of Compress Process, Wired magazine reviewed my
1455
+
video experiment, calling it a “flopped video game.” 60 Ironically, the Wired reporter could not imagine video
1456
+
existing outside the confines of its traditional two-dimensional, flat, and standardized interface. From his point
1457
+
of view, this other resolution, 3D, signified a specific categorization of the work as a game rather than a video
1458
+
application and as such, it was reviewed (and burned) as ‘a flop’.
1459
+
This type of imposition on the interface is extensively described by NYU new media professor
1460
+
Alexander Galloway in his book The Interface Effect. Galloway writes that “an interface is not a thing, an
1461
+
interface is always an effect. It is always a process or a translation.”61 The interface thus always becomes part of
1462
+
the process of reading, translating and understanding of our mediated experiences. When considered in this
1463
+
light, one can make sense of the reviewer's reaction to Compress Process. Such a situation prompts the question
1464
+
of whether it is at all possible to escape the normative or habitual interpretation of the interface?
1465
+
As Wendy Hui Kyong Chun writes, “New media exist at the bleeding edge of obsolescence. […] We are
1466
+
forever trying to catch up, updating to remain (close to) the same.” 62 Today, the speed of the upgrade has
1467
+
become incommensurable: new upgrades arrive too fast and even seem to exploit this speed as a way to obscure
1468
+
their new options, interfaces, and (im)possibilities. Because of the speed of the upgrade, remaining the same, or
1469
+
using technology in a continuous manner, has become a mere aspiration. Chun’s argument seems to echo what
1470
+
Deleuze had already described in his 'Postscript on the Societies of Control’:
1471
+
Capitalism is no longer involved in production […] Thus it is essentially dispersive, and the
1472
+
factory has given way to the corporation. The family, the school, the army, the factory are no
1473
+
longer the distinct analogical spaces that converge toward an owner –– state or private power
1474
+
–– but coded figures –– deformable and transformable –– of a single corporation that now has
1475
+
only stockholders. […] The conquests of the market are made by grabbing control and no
1476
+
1477
+
60
1478
+
1479
+
Klatt, Oliver. ‘Compress Process’ in: Wired Germany edition, January 2013: p. 106–107.
1480
+
1481
+
61 Galloway, Alexander R. The Interface Effect. Cambridge: Polity Press, 2012: p. 33.
1482
+
62
1483
+
1484
+
Chun, Wendy Hui Kyong. Updating to Remain the Same: Habitual New Media. Cambridge: MIT Press, 2016: p. 1.
1485
+
III
1486
+
1487
+
longer by disciplinary training, by fixing the exchange rate much more than by lowering
1488
+
costs, by transformation of the product more than by specialization of production.63
1489
+
In other words, the continuous imposition of the upgrade demands a form of control over the user, leaving them
1490
+
with a sense of freedom, while they effectively become more and more restricted in their practices. Today, the
1491
+
field of image processing forces almost all formats to follow quadrilateral, ecology dependent, standard
1492
+
(re)solutions, which result in trade-offs (compromises) between settings that manage speed and functionality
1493
+
(bandwidth, control, power, efficiency, fidelity), while at the same time considering claims in the realms of
1494
+
value vs. storage, processing, and transmission. At a time when image-processing technologies function as
1495
+
black boxes, we desperately need to research, reflect, and re-evaluate our technologies of obfuscation. Yet,
1496
+
institutions –– schools and publications alike – appear to consider the same old settings over and over, without
1497
+
critically analyzing or deconstructing the programs, problems, and possibilities that come with newer media. As
1498
+
a result, there exists no support for the study of (the setting of) alternative resolutions. Users just learn to copy
1499
+
the use of the interface, to paste data and replicate information, but they no longer question, or learn to
1500
+
question, their standard formats.
1501
+
Mike Judge made some alarming forecasts in his 2006 science fiction comedy film Idiocracy, which,
1502
+
albeit indirectly, echos science fiction writer Philip K. Dick’s dystopian short story ‘Pay for the Printer’ (1956).
1503
+
In an era in which printers print printers, everything slowly resolves into useless mush and kipple. In a way, we
1504
+
have to realize that if we do not approach our resolutions analytically, the next generation of our species will
1505
+
likely be discombobulated by digital generation loss –– a loss of fidelity in resolutions between subsequent
1506
+
copies and trans-copies of a source. As a result, daily life will turn into obeying the rules of institutionalized
1507
+
programs, while the user will end up only producing monotonous output.
1508
+
In order to find new, alternative resolutions and to stay open to refreshing, stimulating content, I need to
1509
+
ask myself: do I, as a user, consumer, and producer of data and information, depend only on my conditioning
1510
+
and the resolutions that are imposed on me, or is it possible for me to create new resolutions? Can I escape the
1511
+
interface, or does every decontextualized materiality immediately get re-contextualized inside another, already
1512
+
existing, paradigm or interface? How can these kinds of connections block or obscure intelligible reading, or
1513
+
actually offer me a new context to resolve information? Together these questions set up a pressing research
1514
+
agenda but also a possible future beyond monotonous data. In order to try to find an answer to any of these
1515
+
questions, I will need to start at the beginning with a genealogy of the term resolution.
1516
+
1517
+
63
1518
+
1519
+
Deleuze, Gilles, ‘Postscript on the Societies of Control,’ in: October, (no.59, Winter 1992): p. 3–7.
1520
+
IV
1521
+
1522
+
O
1523
+
1524
+
ptical Resolution
1525
+
1526
+
To really get an insight into the complexity of resolution, I would like to use this section to describe some
1527
+
fundamental principles at the core of the basic formula for angular resolution, which hail from physics.
1528
+
Although they simply concern what the minimum angle necessary is to discern two points from each other, a
1529
+
much greater complexity is faced when such a formula is transposed into the realms of technology.
1530
+
In 1877, the English physicist John William Strutt succeeded his father to become the third Baron
1531
+
Rayleigh. While Rayleigh’s most notable accomplishment was the discovery of the inert (not chemically
1532
+
reactive) gas argon in 1895, for which he earned a Nobel Prize in 1904, Rayleigh also worked in the field of
1533
+
optics. Here he wrote a criterion that is still used today for quantifying angular resolution: the minimum angle
1534
+
at which a point of view still resolves two points, or the minimum angle at which two points become visible
1535
+
independently from each other. In an 1879 paragraph titled ‘Resolving, or Separating, Power of Optical
1536
+
Instruments,’ Lord Rayleigh writes: “According to the principles of common optics, there is no limit to
1537
+
resolving-power [of an instrument].” 64 But in a paper written between 1881 and 1887, Rayleigh asks: ‘How is it
1538
+
[…] that the power of the microscope is subject to an absolute limit […]? The answer requires us to go behind
1539
+
the approximate doctrine of rays, on which common optics is built, and to take into consideration the finite
1540
+
character of the wave-length of light.” 65
1541
+
When it comes to straightforward optical systems that consider light rays only from a limited spectrum,
1542
+
Rayleigh was right: in order to quantify the resolution of these optical systems, the contrast –– i.e. the amount
1543
+
of difference between the maximum and minimum intensity of light visible within the space between two
1544
+
objects –– is indispensable. Just like a white line on a white sheet of paper needs contrast to be visible (to be
1545
+
resolved), it will not be possible to distinguish between two objects when there is no contrast between these two
1546
+
objects. Contrast between details defines the degree of visibility, and thus resolution: no contrast will result in
1547
+
no resolution.
1548
+
But the contrast between two points, and thus the minimum resolution, is contingent on the wavelength
1549
+
of the light and any possible diffraction patterns between those two points in the image. The ring-shaped
1550
+
diffraction pattern of a point light (a light source), known as an Airy Pattern (named after George Biddell Airy),
1551
+
is the result of diffraction and is characterized by the wavelength of light illuminating a circular aperture. When
1552
+
two light points are moved into close proximity –– so close that the first Airy disk’s zero crossing falls inside
1553
+
the second Airy disk’s zero crossing –– the oscillation within the Airy Patterns will cancel most contrast of light
1554
+
1555
+
64 Rayleigh, Lord, ‘XXXI. Investigations in Optics, with Special Reference to the Spectroscope,’ in: The London,
1556
+
1557
+
Edinburgh, and Dublin Philosophical Magazine and Journal of Science, (no. 8, 1879): p. 261–74.
1558
+
65
1559
+
1560
+
John William Strutt, Baron Rayleigh, ‘On the electromagnetic theory of light,’ in: Scientific Papers, Vol II, 1881–1887.
1561
+
Cambridge University Press, 1900: p. 410.
1562
+
V
1563
+
1564
+
(a) Two monochromatic light sources pass through a small circular aperture and produces a diffraction pattern. (b) Two point light
1565
+
sources that are close to one another produce overlapping images because of diffraction. (c) Two light sources move so close together,
1566
+
that they cannot be resolved or distinguished.
1567
+
1568
+
between them. As a result, the two points will optically be blurred together, no matter the lens’ resolving power.
1569
+
Thus: even the biggest imaginable telescope has limited resolving power.
1570
+
Rayleigh described this effect in his Rayleigh criterion, which states that two points can be resolved
1571
+
when the center of the diffraction pattern of one point falls just outside the first minimum diffraction pattern of
1572
+
the other. When considered through circular aperture, Rayleigh states that it is possible to calculate the
1573
+
minimum angular resolution (the minimum distance between two points or light sources) as:
1574
+
θ = 1.22 λ / D
1575
+
1576
+
In this formula, θ stands for angular resolution (which is measured in radians), λ stands for the wavelength of
1577
+
the light used in the system (blue light has a shorter wavelength, which will result in a better resolution), and D
1578
+
stands for the diameter of the lens’ aperture (the hole with a diameter through which the light travels). Aperture
1579
+
is a measure of a lens’ ability to gather light and resolve fine specimen detail at a fixed object distance.
1580
+
VI
1581
+
1582
+
A smaller resolution thus means there is a smaller resolution angle (and thus less space) necessary
1583
+
between the resolved dots. However, real optical systems are complex and suffer from aberrations, flaws in the
1584
+
optical system and practical difficulties such as specimen quality. Besides this, in reality, most often, two dots
1585
+
radiate or reflect light at different levels of intensity. This means that in practice the resolution of an optical
1586
+
system is always higher (worse) than its calculable minimum.
1587
+
All technologies have a limited optical resolution, which depends on, for instance, aperture, wavelength,
1588
+
contrast and angular resolution. When the optical technology is more complex, the actors that are involved in
1589
+
determining the minimal resolution of the technology become more diverse and the setting of resolution
1590
+
changes into a more elaborate process. In microscopy, just like in any other optical technology, angular and
1591
+
lateral resolution refer to the minimum amount of distance needed (measured in rads or in meters) between two
1592
+
objects, such as dots, that still make it possible to tell them apart. However, a rewritten mathematical formula
1593
+
defines the theoretical resolving power in microscopy as:
1594
+
dmin = 1.22 x λ / NA
1595
+
1596
+
In this formula, dmin stands for the minimal distance two dots need from each other to be resolved, i.e. their
1597
+
minimal resolution. λ stands again for the wavelength of light. In the formula for microscopy, however, the
1598
+
diameter of the lens’ aperture (D) is swapped with NA, or numerical aperture, which consists of a mathematical
1599
+
calculation of the light-gathering capabilities of a lens. In microscopy, this is the sum of the aperture of an
1600
+
objective and the diaphragm of the condenser, which have set values per microscope. Resolution in microscopy
1601
+
is thus determined by certain physical parameters that not only include the wavelength of light, but also the
1602
+
light-gathering power of the objective and the lenses.
1603
+
The definition of resolution in this second formula is expanded to also include the attributed settings of
1604
+
strength, accuracy, or power of the material agents that are involved in resolving the image: the objective,
1605
+
condenser and lenses. At first sight, this might seem like a minimal expansion and lead to the dismissal of a
1606
+
simple rephrasing or rewriting of the earlier formula for angular resolution. However, the expansion of the
1607
+
formula with just one specific material agent, the diaphragm, and the attribution of certain values of this
1608
+
material agent (which are often determined in increments rather than a fluid spectrum of values) is actually an
1609
+
important step that illustrates how technology gains complexity. Every time a new agent is added to the
1610
+
equation, the agent introduces complexity by adding their own rules or possible settings; involving or
1611
+
influencing the behavior of all other material agents. Moreover, the affordances of these technologies, or the
1612
+
clues inherent to how the technology is built to tell a user how it can or should be used, also play a role that
1613
+
intensifies the complexity of the resolution of the final output. As James J. Gibson writes, “affordances are
1614
+
1615
+
VII
1616
+
1617
+
properties of things taken with reference to an observer but not properties of the experiences of the observer.”66
1618
+
In 1975, Estes and Simonett describe resolution in their paper 'Fundamentals of Image Interpretation' as
1619
+
the ability of an imaging system […] to record fine detail in a distinguishable manner. 67 While
1620
+
resolution is often understood in terms of spatial resolution –– the fineness of the details visible in an
1621
+
image –– but there are many different types of resolution: temporal resolution (the speed of the
1622
+
sequence in which images are recorded) or radiometric resolution –– the recording of different levels of
1623
+
brightness. Users can decide on the trade-offs between different types of resolution, for instance - a high
1624
+
temporal resolution might result in a lower range in brightness, while high contrast favours recording of
1625
+
fine spatial detail. “In very broad terms, resolution refers to the ability of a […] sensing system to record
1626
+
and display fine spatial, spectral, and radiometric detail.” A working knowledge of resolution is essential
1627
+
for understanding both practical and conceptual aspects of […] sensing. 68
1628
+
Our understanding, or lack of understanding, of resolution may be the limiting factor in our efforts to use data
1629
+
and understand its underlying value systems. In photography, for instance, the higher the aperture, the shallower
1630
+
the depth of field, the closer the lens needs to come to the object. This also introduces new possibilities for
1631
+
failure: if the diaphragm does not afford an appropriate setting for a particular equation, it might not be possible
1632
+
to resolve the image at all –– the imaging technology might simply refuse or even state an ‘unsupported setting’
1633
+
error message; in which case the technological assemblage will refuse to resolve an image entirely –– the
1634
+
foreclosure of an abnormal option rather than an impossibility. Thus, the properties of the technological
1635
+
assemblage that the user handles, the affordances, add complexity to the setting of a resolution.
1636
+
1637
+
A
1638
+
1639
+
spect Ratio, Resolution, and Resolving Power
1640
+
1641
+
In optical systems, the quality of the rendered image depends on the resolving power and acutance of the
1642
+
technological assemblage that renders the image; the (reflected) light of the source or subject that is captured;
1643
+
and the context and conditions in which the image is recorded. Consider, for instance, how different objects
1644
+
(lens, film, image sensor, compression algorithm) have to weigh (or dispute) between standard settings (frame
1645
+
rate, aperture, ISO, number of pixels and pixel aspect ratio, color encoding scheme, or weight in mbps), while
1646
+
66
1647
+
1648
+
Gibson, James J. The Ecological Approach to Visual Perception. Hillsdale, NJ: Lawrence Erlbaum, 1986.
1649
+
see: http://smithiesdesign.com/wp-content/uploads/2016/09/05-Gibson-Theory-of-affordances.pdf.
1650
+
67 Estes, John E., and David S. Simonett. 'Fundamentals of image interpretation - Manual of remote sensing,’ in:
1651
+
1652
+
American Society of Photogrammetry (1975): p. 879.
1653
+
68
1654
+
1655
+
Campbell, James B. and Randolph H. Wynne. Introduction to Remote Sensing. New York: Guilford Press, 2011: p. 285.
1656
+
VIII
1657
+
1658
+
Supercritical (2019) by UCNV: Video displayed on a monitor with high resolving power. Due to the down scaling of the source
1659
+
material, heavy interpolation artifacts and as a result, moire appeared. What is finally resolved is a HD, high acutance video with low
1660
+
fidelity.
1661
+
1662
+
also having to evaluate the technologies’ possible affordances: the possible settings the mediating technological
1663
+
architecture offers when connecting or composing these objects and settings. Finally, the resolving power is an
1664
+
objective measure of resolution, which can, for instance, be measured in horizontal lines (horizontal resolution)
1665
+
and vertical lines (vertical resolution), line pairs or cycles per millimeter. The image acutance refers to a
1666
+
measure of sharpness of the edge contrast of the image and is measured following a gradient. A high acutance
1667
+
means a cleaner edge between two details while a low acutance means a soft or blurry edge.
1668
+
Following this definition of optical resolution, digital resolution should –– in theory –– also refer to the
1669
+
pixel density of the image on display, written as the number of pixels per area (in ppi or ppcm) and maybe
1670
+
extended to consider the apparatus, its affordances, and settings (such as pixel aspect ratio or color encoding
1671
+
1672
+
IX
1673
+
1674
+
schemes).69 However, in an everyday use of the term, the meaning of digital resolution is constantly confused
1675
+
or conflated to simply refer to a display’s standardized output or graphics display resolution: the number of
1676
+
distinct pixels the display features in each dimension (width and height). As a result, resolution has become an
1677
+
ambiguous term that no longer reflects the quality of the content that is on display. The use of the word
1678
+
‘resolution’ in this context is a misnomer, since the number of pixels in each dimension of the display (e.g. 1920
1679
+
× 1080) says absolutely nothing about the actual pixel density: the pixels per unit or the quality of the content
1680
+
on display, which may in fact be zoomed, stretched, letter-boxed or incorrectly color encoded, to fit the standard
1681
+
display resolution.
1682
+
Moreover, by following a genealogy of the screen, one will find that this quantitative measurement of the
1683
+
quadrilateral screen has lost all references to its origins: early cathode ray tubes (CRTs) were circular. This was
1684
+
because the electron guns that shoot electron beams at the screen are stronger when made in a round shape. As
1685
+
to fit the picture better to the rectangular grid of the raster scan, television tubes gradually became rectangular. 70
1686
+
While projection mapping hardware and software can be used to turn irregularly shaped objects into a display
1687
+
or controlled surface for video projection –– regardless of the size or shape of their content –– 99% of all digital
1688
+
screens are still quadrilateral and follow a standard shape and aspect ratio.
1689
+
Besides the historical lineage of the cathode ray tube screen, another reason for the now consistent
1690
+
quadrilateral shape is that most screens are produced in China where Plasma, LCD, LED and OLED screens are
1691
+
cut from large sheets in a line and en masse. Only recently exceptions to the quadrilateral frame have slowly
1692
+
been invading the market. For instance, the flexible LED screen, the holographic led fan display, or the LED
1693
+
screens that are constructed out of multiple panels or strips, which can be clicked and connected together
1694
+
irregularly.
1695
+
But we are not entirely stuck in the box: with the release of the iPhone X in late 2017, a new future for the
1696
+
(non-quadrilateral) screen came to the market: the iPhone “Super Retina display” features not just rounded
1697
+
corners, but a “notch”: a cut-out for the housing of its sensors. Funny enough, in Apple's Human Interface
1698
+
Guidelines, the brand advises its developers to establish the notch trademark as follows: “Don’t mask or call
1699
+
special attention to key display features. Don't attempt to hide the device's rounded corners, sensor housing, or
1700
+
indicator for accessing the Home screen by placing black bars at the top and bottom of the screen. Don't use
1701
+
69 The importance of the resolution of color encoding has been described in detail by Carolyn Kane in her book:
1702
+
1703
+
Kane, Carolyn. Chromatic Algorithms: Synthetic Color, Computer Art, and Aesthetics after Code. Chicago: University of
1704
+
Chicago Press, 2014.
1705
+
70
1706
+
1707
+
On a side-note, a recent wave of children's Sci-Fi movies re-introduced circular and elliptical (or non quadrilateral)
1708
+
screens; a resolution that could be read as a reference to the now often forgotten electronic history of the (round) screen,
1709
+
such as in spy and radar technologies.
1710
+
Examples of this can be found in my Lexicon of Glitch Affect (2019)
1711
+
see: https://beyondresolution.info/A-Lexicon-of-Glitch-Affect (20.02.20)
1712
+
X
1713
+
1714
+
visual adornments like brackets, bezels, shapes, or instructional text to call special attention to these areas,
1715
+
either.”71 A story of how compromise became a feature...
1716
+
Even so, the shape and proportions of the digital screen or the frame in which the screen rests, are still
1717
+
most often quadrilateral. Although there are many possible different aspect ratios and resolutions, there are only
1718
+
a few standard aspect ratios and resolutions. The aspect ratio of a standard home screen in 2018 was16:9, while
1719
+
its predecessor (before 2010) was 4:3. Despite experimentation within the fields of cinema and video, generally,
1720
+
screens are confined to these two aspect ratios. When content does not fit the aspect ratio of the screen, a
1721
+
technique called letterboxing or "stylized pillarboxing" may be used, adding black or blurred bars on both sides,
1722
+
to center and make the content fit or fill the screen. Only in 2018, did the YouTube Player window give up its
1723
+
rigid –– and quite artificial –– default aspect ratio. From then on videos would no longer be scaled into a 16:9
1724
+
playback mask, instead adapting to the dimensions of a clip. A video in 4:3 is no longer played with black bars
1725
+
on the side, but rather uses the full available width or height. This might have been to give way for the growing
1726
+
popularity of vertical video, or the come back of the nostalgic 4:3 format.
1727
+
We can see from this example of Youtube pillar-boxing, how often historical settings either ossify as
1728
+
requirements or de facto norms, or are notated as de jure, legally binding, standards by organizations such as
1729
+
the International Organization for Standardization (ISO). While this makes the process of resolving an image
1730
+
less complex –– systematizing parts of the process –– ultimately it also makes the process less transparent and
1731
+
more black-boxed. And it is not only institutions such as ISO that program, encode, and regulate (standardize)
1732
+
the flows of data in and between our technologies, or that arrange the data in our machines following systems
1733
+
that underline efficiency or functionality. In fact, data is organized following either protocol or proprietary
1734
+
standards developed by technological oligarchs to include all kinds of inefficiencies that the user is not
1735
+
conditioned or even supposed to see, think, or question. These proprietary standards function as a form of
1736
+
control. By re-encapsulating our information inside various wrappers that (re-)encode, edit, and even deform
1737
+
our data we are subjected to a type of corporate nepotism, and (sometimes) covert cartel operations who impose
1738
+
insidious data collection policies and even lock users into their proprietary software.
1739
+
Just as in the realm of optics, a resolution does not just mean a final rendition of the data on the screen,
1740
+
but also involves the processes and affordances involved during its rendition –– the trade-offs inside the
1741
+
technological assemblage which record, produce, and display the image (or other media, such as video, sound,
1742
+
or 3D data). The current conflation of the meaning of resolution within the digital –– as a result of which
1743
+
resolution only refers to the final dimensions the image is displayed at or in –– obscures the complexities and
1744
+
politics at stake in the process of resolving. In consequence, limits are imposed on our understanding of the use,
1745
+
compilation and reading of (imaging) data. In short, further theoretical refinements that elaborate on the usage
1746
+
and development of the term resolution have been missing from debates on resolution from the moment of its
1747
+
71
1748
+
1749
+
‘Human Interface Guidelines,’ Apple Developer website.
1750
+
See: https://developer.apple.com/ios/human-interface-guidelines/overview/iphone-x/ (20.02.20)
1751
+
XI
1752
+
1753
+
importation from the field of optics, where it has been in use for two centuries. As such, to garner a better
1754
+
understanding of our imaging technologies, the word resolution itself needs to be resolved; or rather, it needs to
1755
+
be disentangled to refer not just to a final output, but to a more procedural construct.
1756
+
1757
+
U
1758
+
1759
+
ntie&&Dis/Solve: Digital Resolutions
1760
+
1761
+
Resolutions are man-made. Constrained by the procedural trade-offs between the affordances of hardware and
1762
+
software settings; the more complex an image-processing technology is, the more actors its rendering entails ––
1763
+
each following their own rules or standards to resolve an image. However, these actors and their inherent
1764
+
complexities are increasingly positioned beyond the fold of everyday settings, outside the options represented
1765
+
inside the interface. As such, resolutions are not just an interface effect but also a hyperopic lens, obfuscating
1766
+
possible alternative forms to resolve data –– whether it be an image, a video or a sound.
1767
+
Unknowingly, the user and audience suffer from technological hyperopia: a state of farsightedness that
1768
+
prevents the user from seeing the processes taking place directly under their nose. Hyperopia is the result of
1769
+
one's exclusive focus on the final end product. This is a consequence of the shift from user created resolutions,
1770
+
to the setting of resolutions and, finally, to the imposition of resolutions as standard settings. Every time we
1771
+
press print, run, record, publish, render, we also press “compromise.” Unfortunately, however, what we
1772
+
compromise –– settings between or beyond our standards, and which deliver other, maybe unwanted, but
1773
+
maybe also desirable outcomes –– is often obfuscated. We hence need to shift our understanding of resolutions,
1774
+
seeing them as disputable norms, habitual compromises and expandable limits. By challenging the actors
1775
+
involved in establishing resolution settings, the user can scale actively between increments of hyperopia and
1776
+
myopia. The question is: has the user become unable to construct their own settings, or has the user simply
1777
+
become oblivious to resolutions and their inherent compromises? It is also to ask how the user became this
1778
+
blind?
1779
+
One answer to this question can be found in a redefinition of the screen. Today, the term “screen” may
1780
+
still refer to its old configuration: a two-dimensional material surface or threshold that partitions one space from
1781
+
the next, or functions as a shield. As curator and theorist Christiane Paul writes, the screen acts as a mediator of
1782
+
(digital) light.72 However, over the past decades, technological innovations have transformed the notion of the
1783
+
screen into a wide variety of complex amalgamations.
1784
+
In The Language of New Media (2002), Lev Manovich offers a short genealogy of the screen,
1785
+
distinguishing three different types. The first screen he describes as the “classical screen”: a static, pictorial
1786
+
screen or painting. The second type of screen Manovich calls the “dynamic screen”: television, moving images,
1787
+
video. A third and more complex screen that Manovich recognizes is the interactive “screen of real time.” This
1788
+
72
1789
+
1790
+
Paul, Christiane. ‘Mediations of Light: Screens as Information Surfaces,’ in: Digital Light, ed. Sean Cubitt, Daniel
1791
+
Palmer, and Nathaniel Tkacz. London: Open Humanities Press, 2015: p. 179–92.
1792
+
XII
1793
+
1794
+
type of screen is still in development and includes radar, computer, telephone and VR screen technologies.73 In
1795
+
“Elements of Screenology,” Erkki Huhtamo writes:
1796
+
in spite of their ubiquitous presence screens are strangely evasive, hard to grasp. They are constantly
1797
+
metamorphosing, appearing in new places and new forms. There are “Big Screens” and “Small
1798
+
Screens.” Some are flat, some fat, attached to a box. Some are like the sun –– active, radiating “life” of
1799
+
their own –– while others are like the moon, passive, reflecting light projected at them. There are
1800
+
screens observed from a distance, and others touched and interacted with, held in one's hand. How to
1801
+
formulate a definition that would embrace them all? Does it even make sense to ask such a question?”
1802
+
he moves on writing: “Screens should not be studied in isolation of the apparatus they are part of. The
1803
+
notion of apparatus comes from cinema studies: it comprises not only the technical system, but also the
1804
+
elements of the viewing situation, including the relationship between the screen and the viewer, which is
1805
+
both physical and imaginary.”74
1806
+
While some screens are still volatile and “should not be touched” or at least “handled with care,” modern
1807
+
screens generally consist of a layer that can be touched, and are thus considered “touch screen.” These now
1808
+
pervasive interactive screens, with two-finger-touch or multi-touch sensitivity, allow the user to zoom in and
1809
+
out on information. Yet this is not the only way the smartphone's touch screen has changed screen technology.
1810
+
Today, almost the entire mobile phone is covered by the screen, transforming the experience of the phone into
1811
+
an entirely screen based, “mobile device.” These direct, responsive qualities –– further advanced by built-in
1812
+
accelerometers –– have ignited qualitative shifts. In the iPhone, the accelerometer is responsible for lighting up
1813
+
the screen when the phone is picked up, or rotating it toward the users point of view. As a result, the screen no
1814
+
longer has a standard orientation –– requirements for content scale automatically and are “responsive.” The
1815
+
mobile device is now a black mirror when inactive, and a space of representation when active.
1816
+
Further, the screen has transformed into a navigational plane, rendering it similar to an interface or GUI.
1817
+
Huhtamo dabbles with the possibility of describing the contemporary screen as a framed surface or container
1818
+
for displaying visual information that is controlled by the user and therefore not permanently part of the frame.
1819
+
He finally argues that the screen exists as a constantly changing, temporally constructed interface between the
1820
+
user and information.75 The screen of the mobile device consists of different elements –– such as widgets, apps
1821
+
73
1822
+
1823
+
Manovich, Lev. The Language of New Media. MIT press, 2001: p. 95–103.
1824
+
1825
+
74 Huhtamo, Erkki. ‘Elements of Screenology,’ in: ICONICS: International Studies of the Modern Image, (Vol.7, 2004): p.
1826
+
1827
+
31–82.
1828
+
75
1829
+
1830
+
Huhtamo, Erkki. ‘The Pleasures of the Peephole: An Archaeological Exploration of Peep Media,’ in: Book of Imaginary
1831
+
Media: Excavating the Dream of the Ultimate Communication Medium. NAi Publishers, 2007: p. 74–155.
1832
+
XIII
1833
+
1834
+
and shortcuts –– and also spatial and temporal units. As Galloway explains in The Interface Effect,76 the
1835
+
interface is part of the processes of understanding, reading and translating our mediated experiences: it operates
1836
+
as an effect. In his book The Interface Envelope, James Ash writes:
1837
+
within digital interfaces, the specific mode of resolution of an object is carefully designed and
1838
+
tested in order to be as optimal as possible […]. In a digital interface, resolution is the outcome
1839
+
of transductions between a variety of objects including screened images, software, hardware, and
1840
+
physical input devices, all of which are centrally involved in the design of particular objects
1841
+
within an interface.77
1842
+
Not only has the screen morphed from a flat surface to an interactive plane of navigation, the technologies that
1843
+
shape its content have developed into extremely complex systems. As Manovich wrote back in 1995, “Rather
1844
+
than being a neutral medium of presenting information, the screen is aggressive. It functions to filter, to screen
1845
+
out, to take over, rendering nonexistent whatever is outside its frame.”78 The screen is thus not simply a
1846
+
boundary plane. It has become an autonomous object that affects what is being displayed; a threshold mediating
1847
+
different systems or a process oscillating between states. The mobile screen itself is located in-between different
1848
+
applications and uses.
1849
+
In the computer, most of the interactions with our interfaces are mediated by software applications that act
1850
+
like platforms. These platforms do not take up the full screen, but instead exist within a window. While they all
1851
+
exist in the same screen, these windows follow their own sets of rules and standards; they exist next to and on
1852
+
top of each other like walled gardens. In a sense, these platforms are a modern version a framework: offering a
1853
+
simulacrum of freedom and possibility. In the case of the platform Instagram, for example, content is
1854
+
reformatted and deformed. Instagram recompresses and reformats any posted data, text, sound, or images, while
1855
+
it has rules for the number of characters, as well as what characters and compressions can be used or uploaded.
1856
+
On the Instagram platform, one can only post a URL in the profile section of the platform. Moreover, it saves
1857
+
content on its own server, making it accessible only via the platform. In short, Instagram enforces its own
1858
+
resolutions that range from formatting to accessibility. It is important to realize that the screen is in a constant
1859
+
state of assemblage: delimiting and demarcating our ways of seeing, while expanding the axial and lateral
1860
+
resolution to layers that are usually obfuscated or uncharted.
1861
+
It is hence imperative to rethink the definition of resolution and expand it from a simple measure of
1862
+
acutance. What is resolved on the screen and what is not depends not just on the material qualities of the screen
1863
+
76
1864
+
1865
+
Galloway, Alexander R. The Interface Effect. Cambridge: Polity Press, 2012.
1866
+
1867
+
77 Ash, James. The Interface Envelope: Gaming, Technology, Power. Bloomsbury USA, 2015: p. 35.
1868
+
78
1869
+
1870
+
Manovich, Lev. ‘An Archeology of a Computer Screen,’ in: Kunstforum International (v.3, 1995): p. 124–135.
1871
+
XIV
1872
+
1873
+
or display –– or the signal it receives –– but also on the processes, platforms, standards, and interfaces involved
1874
+
in setting these measures; behind or in the depths beyond the screen or display.
1875
+
So while in the digital realm, the term resolution is often simplified to just mean a number – signifying
1876
+
the ability of a system to record and display fine spatial, spectral and radiometric detail or the width and height
1877
+
of a screen –– the critical understanding of the term resolution I propose also considers depth. By depth I
1878
+
understand factors beyond the screen, which concern: audible technologies, protocols, and other (proprietary)
1879
+
standards, together with the technological interfaces and the objects’ materialities and affordances, which
1880
+
together form a final resolution.
1881
+
Depth thus conveys the position that resolution is never a neutral settlement, but an outcome that carries
1882
+
historical, economic and political ideologies, which were once implemented by choice. While resolutions
1883
+
compromise, obfuscate and obscure particular visual outcomes, the processes of standardization and upgrade
1884
+
culture as a whole also compromise particular technological affordances –– creating new ways of seeing or
1885
+
perceiving –– altogether. These alternative technologies of sight also need to be considered as part of
1886
+
Resolution Studies.
1887
+
1888
+
A
1889
+
1890
+
Rheology of Data
1891
+
1892
+
In 1941, the Argentinian writer Jorge Luis Borges published El Jardín de senderos que se bifurcan (The Garden
1893
+
of Forking Paths), which contains the short story ‘The Library of Babel.’ In this story, Borges describes a
1894
+
universe in the form of a vast library that contains all possible books which are generated following a few
1895
+
simple rules: every book consists of 410 pages, each page displays 40 lines and each line contains
1896
+
approximately 80 letters. Each book features any combination possible of the 25 orthographic symbols: 22
1897
+
letters, a period (full stop), a comma, and a space. While the exact number of books in the Library of Babel can
1898
+
be calculated, Borges says the library is “indefinite and perhaps infinite.”
1899
+
The fascinating part of the story is Borges' description of the behavior of the library visitors. In
1900
+
particular, the “Purifiers,” who arbitrarily destroy books that do not follow their rules of language and decoding.
1901
+
The word “arbitrarily” is important here, because it references the fluidity of the library: its openness to
1902
+
different languages and other systems of interpretation. One book may, for instance, offer an index to the next
1903
+
book, or a system of decoding: a “bridge” to read the next. This provokes the question: how do the Purifiers
1904
+
know they did not just read the books in the wrong order? How can they be certain that they were not just
1905
+
lacking an index or a codex that would help them access the books to be purified (burned)?
1906
+
When I learned about NASA’s use of sonification: the process of displaying any type of data or
1907
+
measurement as sound; I realized that with the right listening device, anything can be heard –– even a rainbow.
1908
+
This does not always mean it makes sense to the listener, but rather, it is significant for the willingness of
1909
+
XV
1910
+
1911
+
contemporary space scientists to build bridges between different domains. I would later conceive of such
1912
+
bridges through the concept of a rheology of data.
1913
+
Rheology is a term from the realm of physics, or, to be more precise, from mechanics used to describe
1914
+
the flow of matter –– primarily in a liquid state, but also soft solids, or solids that respond in a plastic way;
1915
+
rather than deforming elastically in response to an applied force. By rheology of data I thus mean a study of
1916
+
how data can be read and displayed in completely different forms, depending on the context, software or
1917
+
interface. A rheology of data facilitates a deformation and flow of the matter of data; it demonstrates the
1918
+
possibility of pushing data into different applications and the presentation of data in different forms. In a sense,
1919
+
Limenia, the little antenna we used on the Brocken, is a hardware for rheology, because it transcodes one part of
1920
+
the electromagnetic spectrum (e.g. bluetooth or GPS) to the audible spectrum.
1921
+
Thinking about the rheology of data became meaningful to me when I first ran the open source Mac OS
1922
+
X plugin technology Syphon (first released in 2010 as open video tap, but later developed by Anton Marini in
1923
+
collaboration with Tom Butterworth as Syphon). With the help of Syphon, I could suddenly make certain
1924
+
applications (Adobe After Effects, Modul8, Processing, or Open Frameworks) share information –– such as full
1925
+
frame rate video or stills –– with one another, in real time. Syphon allowed me to project my slides and video as
1926
+
XVI
1927
+
1928
+
textures on top of 3D objects (from Modul8 to Unity). The plugin taught me that my thinking in software
1929
+
environments ( as walled gardens) was flawed, or at least limiting. Software is much more interesting when it
1930
+
allows me to leak and push my content through its walls, which otherwise operate as closed architectures.
1931
+
Syphon hence showed me that data is more fluid than how we are conditioned to perceive and use it.
1932
+
In the realm of computation, though, there is still very little fluidity. The ‘Library of Babel’ remains an
1933
+
asynchronous metaphor for our contemporary reality of computation. Computer programs only function when
1934
+
definite forms of formatted data are inserted: data for which the machine has codecs installed. (The value of)
1935
+
RAW, non-formatted or unprocessed data is easily dismissed because it is often hard to open or read. There
1936
+
seems to be hardly any freedom in transgressing this. The insights I gained from reading about NASA opened a
1937
+
new approach in my computational thinking: I started teaching my students not just about the electromagnetic
1938
+
spectrum, but also about how NASA, through sonification and other transcoding techniques, could listen to the
1939
+
weather. With such information I hoped they would understand the freedom they can take in the processes of
1940
+
perceiving and interpreting data.
1941
+
Only the contemporary Purifiers –– software, its users, and computer systems in general –– enforce the
1942
+
rule that illegible data is invalid data. Take, for instance, Satrom’s QTlets, which have, after a lifespan of a little
1943
+
over five years –– at least in my OS –– become completely obsolete and unplayable. In reality, it simply means
1944
+
that I do not have the right decoder, which is no longer available and supported. In general, it means that the
1945
+
string of data is not run through the right program or read in the right language, which would translate its data
1946
+
into a legible form of information. Data is not solid: it can flow from one context or environment to the next,
1947
+
changing both its resolution and its meaning. Such a capacity can be both a danger and a blessing in disguise.
1948
+
1949
+
R
1950
+
1951
+
esolution Studies
1952
+
1953
+
Resolution theory is a study of literacy: literacy of the machines, the people, the people (engineers) creating the
1954
+
machines as well as the people being created by the machines. Resolution Studies is distinct in that it does not
1955
+
only involve the study of the effects of technological progress or the consequences of scaling –– e.g., the habits
1956
+
and violences the come with particular settings of resolution –– which have already been theorized in books by
1957
+
Alex Galloway79 or Wendy Hui Kyong Chun80. Resolution studies also involves research on alternative settings
1958
+
that could have been in place, but that are not; as the affordances of technology have positioned certain settings
1959
+
outside of the domain of compatibility, these settings have been compromised.
1960
+
Key to the study of resolutions is the realization that the condition of data is fluid. Every string of data is
1961
+
ambiguously promiscuous and has the potential to be manipulated into anything. This is how a rheology of data
1962
+
79 Galloway, Alexander R. The Interface Effect. Cambridge: Polity Press, 2012.
1963
+
80
1964
+
1965
+
Chun, Wendy Hui Kyong. Updating to Remain the Same: Habitual New Media. Cambridge: MIT Press, 2016.
1966
+
XVII
1967
+
1968
+
can support fluidity in data transactions: every piece of information functions within an environment that
1969
+
encodes and decodes, contextualizes and embeds data. In doing so, data gains meaning and becomes
1970
+
information. Different forms of ossification slither into every crevice of private life, while unresolved,
1971
+
ungoverned free space seems to be slipping away. Here we find the double edged sword of standardization: the
1972
+
power and dangers of standardization. The question then becomes how we move more easily beside or beyond
1973
+
these generally imposed flows of data? How, in other words, do we break away from these imposed, even
1974
+
imperialist, standards of resolution?
1975
+
First of all, we need to realize that a resolution is more than a formal solution. Resolutions involve a
1976
+
technological (dis)entanglement and also an inherent compromise: if something is resolved one way, it is not
1977
+
resolved or rendered in another way. Determinations, like standard resolutions, are as dangerous as any other
1978
+
presumption: they preclude alternatives and perpetuate harmful or merely kludged and kippled procedures.
1979
+
A resolution is the lens through which constituted materialities become signifiers in their own right.
1980
+
They resonate through the hive mind, constantly transforming our technologies into informed material
1981
+
vernaculars. As technology is evolving faster than we as a culture can come to terms with, determinations ––
1982
+
such as standards –– become threats in precluding choices we didn't even know existed. The radical digital
1983
+
materialist thus believes in “informed materials.”81 A term developed by B. Bensaude-Vincent and Isabelle
1984
+
Stengers and adopted and reworked by Susan Schuppli in her lectures on the “Material Witness.” Matter is not
1985
+
just reshaped mechanically through chemical research and development, but is transformed into informed
1986
+
material: “Instead of imposing a shape on the mass of material, one develops an 'informed material' in the sense
1987
+
that the material structure becomes richer and richer in information. Accomplishing this requires a detailed
1988
+
comprehension of the microscopic structure of materials, because it is playing with these molecular, atomic,
1989
+
and even subatomic structures that one can invent materials adapted to industrial demands.”
1990
+
Schuppli uses the term informed material in congruence with the “material witness.” According to
1991
+
Schuppli, material witness is a legal term that refers to someone who has knowledge pertinent to a criminal act
1992
+
or event that could be significant to the outcome of a trial. Schuppli goes on, “In my work, I poach the term
1993
+
‘material witness’ to express the ways in which matter carries trace evidence of external events. But the
1994
+
material witness also performs a twofold operation; it is a double agent. The material witness does not only
1995
+
refer to the evidence of the event but also the event of evidence.” 82 Schuppli goes on:
1996
+
It is insufficient to say that this specific type of material records or registers external events, because all
1997
+
material does that: with the right kind of analysis one can determine that my hand had been on the table,
1998
+
81 Bensaude-Vincent, B. and I. Stengers, A History of Chemistry. Harvard University Press, 1996: p. 206.
1999
+
82
2000
+
2001
+
Menkman, Rosa and Lucas van der Velden, ‘Dark Matters interview with Susan Schuppli,’ in: SONIC ACTS
2002
+
RESEARCH SERIES #21: Nov 24, 2015. See: http://sonicacts.com/portal/dark-matters-an-interview-with-susan-schuppli
2003
+
XVIII
2004
+
2005
+
but this does not make a material witness of the table. A material witness has to disclose the kinds of
2006
+
institutional frameworks and practices that are able to render the material witness significant. So in
2007
+
considering the material witness as both the evidence of the event and the event of evidence, it allows
2008
+
me to understand why certain events are deemed to be worthy of our attention, and other things are
2009
+
disregarded.83
2010
+
Along similar lines then, it is important to understand what events have resolved, but also the event of resolve;
2011
+
it is not enough to understand what standards are used in a final resolution, but also how the standards have
2012
+
come into being and been resolved previously. The compression, software, platform, interface, and finally
2013
+
hardware such as the screen or the projector, all inform how a string of data is resolved; its presence, legibility
2014
+
and its meaning. They enforce and deform data into formatted or informed materials. The infrastructures
2015
+
through which we activate or access our data always engender distortions of our perception. Yet at the same
2016
+
time, because there is some freedom for the trickster using data's fluidity, the rightful meaning or “factuality” of
2017
+
data should always be open for debate.
2018
+
Resolution Studies is not only about the effects of technological progress or the consequences of scaling.
2019
+
Resolution Studies is the study of how resolution as a material witness embeds tonalities into culture in more
2020
+
than just its technological facets. Moreover, from this point Resolution Studies can pivot on researching the
2021
+
standards that could have been in place, but are not. As a form of vernacular resistance, based on the concept of
2022
+
providing ambiguous resolutions, resolution studies can employ the liminal resolution of the screen as a looking
2023
+
glass. Here, “technological hyperopia” –– i.e. the condition of a user being unable to see clearly understand the
2024
+
processes that take place during the production of an image –– is fractured and gives space to myopia and vice
2025
+
versa.
2026
+
2027
+
83
2028
+
2029
+
Menkman, Rosa and Lucas van der Velden, ‘Dark Matters interview with Susan Schuppli,’ in: SONIC ACTS
2030
+
RESEARCH SERIES #21: Nov 24, 2015. See: http://sonicacts.com/portal/dark-matters-an-interview-with-susan-schuppli
2031
+
XIX
2032
+
2033
+
THE GUY BEHIND THE GUY BEHIND THE GUY84
2034
+
2035
+
A Discrete Cosine Transform or 64 basis functions of the
2036
+
JPEG compression (Joint Photographic Experts Group)
2037
+
consisting of 8 x 8 pixel macroblocks.
2038
+
2039
+
84
2040
+
2041
+
Arcangel, Cory. On Compression, 2007.
2042
+
see: http://www.coryarcangel.com/things-i-made/2007-007-on-c/
2043
+
I
2044
+
2045
+
R
2046
+
2047
+
evealing the surface and structure of the image85
2048
+
2049
+
A side effect of the JPEG compression is that the limits of the images’ resolution – which involve not just the images’
2050
+
number of pixels in length and width, but also the luma and chroma values, stored in the form of 8 x 8 pixel
2051
+
macroblocks – are visible as artifacts when zooming in beyond the resolution of the JPEG.
2052
+
Because the RGB color values of JPEG images are transcoded into Y’CbCr macroblocks, accidental or random data
2053
+
replacements can result into dramatic discoloration or image displacement. Several types of artifacts can appear; for
2054
+
instance ringing, ghosting, blocking, and staircase artifacts. The relative size of these artifacts demonstrates the
2055
+
limitations of the JPEGs informed data: a highly compressed JPEG will show relatively larger, block-sized artifacts.
2056
+
85 Ted Davis: ffd8, 2012.
2057
+
2058
+
II
2059
+
2060
+
J
2061
+
2062
+
PEG compression consists of six steps:
2063
+
2064
+
1. Color space transformation. Initially, the image has to be transformed from the RGB colorspace to Y′CbCr.
2065
+
This color space consists of three components that are handled separately; the Y’ (luma or brightness) and the
2066
+
Cb and Cr values; the blue-difference and red-difference chroma components.
2067
+
2. Downsampling. Because the human eye doesn’t perceive small differences within the Cb and Cr space very
2068
+
well, these elements are down-sampled, a process that reduces data dramatically.
2069
+
3. Block splitting. After the color space transformation and down-sampling steps, the image is split into 8 x 8
2070
+
pixel tiles or macroblocks, which are transformed and encoded separately.
2071
+
4. Discrete Cosine Transform. Every Y’CbCr macroblock is compared to all 64 basis functions (base cosines)
2072
+
of a Discreet Cosine Transform. A value of resemblance per macroblock per base function is saved in a matrix,
2073
+
which goes through a process of reordering.
2074
+
5. Quantization. The JPEG compression employs quantization, a process that discards coefficients with values
2075
+
that are deemed irrelevant (or too detailed) visual information. The process of quantization is optimized for the
2076
+
human eye, tried and tested on the Caucasian Lena color test card.
2077
+
III
2078
+
2079
+
Effectively, during the quantization step, the JPEG compression discards most of the information within areas of
2080
+
high frequency changes in color (chrominance) and light (luminance), also known as high contrast areas, while
2081
+
it flattens areas with low frequency (low contrasts) to average values, by re-encoding and deleting these parts of
2082
+
the image data. This is how the rendered image stays visually similar to the original and least similar to human
2083
+
perception. But while the resulting image may look similar to the original, the JPEG image compression is
2084
+
lossy, which means that the original image can never be reconstructed.
2085
+
6. Entropy coding. Finally, a special form of lossless compression arranges the macroblocks in a zigzag order.
2086
+
A Run-Length Encoding (RLE) algorithm groups similar frequencies together while Huffman coding organizes
2087
+
what is left.
2088
+
2089
+
E
2090
+
2091
+
rrors depend on resolution. They are visible only at a certain scale or scope.
2092
+
2093
+
Mesmerized by the screen, focusing on the moves of the next superhero, I was conditioned to ignore the dust
2094
+
imprinted on the celluloid or floating around in the theatre, touching the light of the projection and mingling
2095
+
itself with the movie.
2096
+
The dust –– a micro-universe of plant, human and animal fibres; particles of burnt meteorites, volcanic
2097
+
ashes, and soil from the desert –– could have told me stories that reached beyond my imagination; stories
2098
+
deeper and more complex than what was resolved in front of me, reflected on the movie screen.
2099
+
… But I never paid attention. I focused my attention to where I was conditioned to look: to the movie,
2100
+
reflecting off the screen. All I saw were the images. I did not see the physical qualities of the light nor the
2101
+
materials making up its resolution; before, behind, and beyond the screen.
2102
+
Decennia of conditioning the user to ignore these visual artifacts and to pay attention only to the overall image
2103
+
has changed these artifacts into the ultimate camouflage for secret messaging. Keeping this in mind, I
2104
+
developed DCT (2015).
2105
+
2106
+
IV
2107
+
2108
+
H
2109
+
2110
+
ow Not to Be Read86 :: DCT
2111
+
2112
+
DCT is a font that can be used on any .TTF (TrueType Font) supporting device and uses methods of
2113
+
cryptography and steganography; secretly hidden, the message is transcoded and embedded on the surface of
2114
+
the image where it looks like a JPEG artifact (glitch).
2115
+
The premise of DCT began with the realization that the legibility of an encrypted message does not just depend
2116
+
on the complexity of the encryption algorithm, but also on the placement of the message. This encrypted
2117
+
message, hidden on the surface of the image, is only legible by the ones in the know, anyone else will ignore it.
2118
+
Like dust on celluloid, DCT mimics JPEG errors. It appropriates the algorithmic aesthetics of JPEG
2119
+
macroblocks to stenographically mask a secret message, mimicking the error. The encrypted message, hidden
2120
+
on the surface of the image, is only recognizable to the ones who know where to look for it.
2121
+
During the most important step of the JPEG algorithm, the compression technology employs a mathematical
2122
+
technique known as Discreet Cosine Transform (DCT), to compress the amount of image data needed to
2123
+
transport, store, and present the image. DCT consists of a set of 64 patterns, called macroblocks or basis
2124
+
functions. If an image is compressed correctly, these macroblocks are invisible. The incidental trace of the
2125
+
macroblocks is generally ignored as an artifact, an impurity, or error.
2126
+
DCT is titled after Discrete Cosine Transform, the algorithm at the core of JPEG compression. The
2127
+
encryption uses the 64 macroblocks that form the ‘visual alphabet’ for all JPEG compressed images. These 64
2128
+
macroblocks of the JPEG DCT are directly translated to a set of characters; a set of 64 glyphs starting from
2129
+
binary conversion number 010 0000 to 101 1111 or in decimal 32 to 95, arranged in a zigzag order (reminiscent
2130
+
of Entropy coding step in the JPEG compression). The emerging glyph set can then stenographically be used to
2131
+
articulate a message on top of the JPEG image. DCT thus uses the macroblocks to form their own alphabet, in
2132
+
the form of a simple font, existing on the edges of what the reader recognizes as noise (or error) and the
2133
+
computer as compressed data.
2134
+
2135
+
86
2136
+
2137
+
Steyerl, Hito. How Not to Be Seen. Art Forum, 2013.
2138
+
see: http://artforum.com/video/id=51651&mode=large&page_id=0
2139
+
V
2140
+
2141
+
P
2142
+
2143
+
atch with key to the institutions (2015)
2144
+
2145
+
Embroidered black on black patch, 5 corners, features the logo of the i.R.D.
2146
+
The i.R.D. patch is inspired by one of the Symbology (2007) patches uncovered by Trevor Paglen.
2147
+
It consists of the logo of the i.R.D., embroidered in black on a black patch, providing a key to decipher anything
2148
+
written in DCT: 010 0000 – 101 1111; the binary conversion numbers or decimal 32 to 95 of the 64 most used
2149
+
glyphs.
2150
+
VI
2151
+
2152
+
VII
2153
+
2154
+
- Choose a lofi JPEG base image on which macroblocking
2155
+
artifacts are slightly apparent.
2156
+
- If necessary, you can scale the image up via nearest
2157
+
neighbour interpolation, to preserve hard macroblock edges
2158
+
of the base image.
2159
+
- Download and install the DCT font.
2160
+
- Position your secret message on the surface of the JPEG.
2161
+
- Make sure the font has the same size as the macroblock
2162
+
artifacts in the image.
2163
+
- Flatten the layers (image and font) back to a JPEG. This will
2164
+
make the text no longer selectable and readable as copy-andpaste data.
2165
+
2166
+
DCT
2167
+
2168
+
Recipe
2169
+
2170
+
“A recipe using DCT” was first released in the #Additivism cookbook. 87
2171
+
2172
+
- To prepare the JPEG that you want to write your secret message on:
2173
+
- Choose a lo-fi JPEG image that has slightly apparent macroblocking artifacts. If necessary, you can scale the
2174
+
image up via the nearest neighbor interpolation, to preserve hard macroblock edges of the base image.
2175
+
- Download and install the DCT font.
2176
+
- Position your secret message on the surface of the JPEG. Make sure the font has the same size as the
2177
+
macroblock artifacts in the image.
2178
+
- Flatten the layers (image and font) back to a JPEG. This will make the text no longer selectable and readable
2179
+
as copy-and-paste data. Et voila!
2180
+
87
2181
+
2182
+
Allahyari, Morehshin and Daniel Rourke. The 3D Additivist Cookbook. 2015. see: www.additivism.org
2183
+
VIII
2184
+
2185
+
IX
2186
+
2187
+
DCT
2188
+
2189
+
:SYPHONING. The 1000000th (64th) interval (2015-2017)
2190
+
2191
+
by A. Macroblock (partitioned matrix)
2192
+
DCT:SYPHONING was first commissioned by the Photographers Gallery in London, for the show “Power
2193
+
Point Polemics,” where it was shown as .ppt (Jan–Apr, 2016).
2194
+
A 3 channel video installation was conceived for the 2016 Transfer Gallery's show "Transfer Download", first
2195
+
installed at Minnesota Street Project in San Francisco (July–September, 2016).
2196
+
The final form of DCT:SYPHONING was released as VR, as part of DiMoDA’s “Morphe Presence” (2017).
2197
+
DCT:SYPHONING is downloadable as a stand alone here
2198
+
In this contemporary translation of the 1884 Edwin Abbott Abbott Flatland, A. Macroblock, an
2199
+
anthropomorphized DCT, observes some of the complexities at work in digital image compression. A.
2200
+
Macroblock (Senior) narrates its first syphon (data transfer) together with DCT Junior.
2201
+
As the two DCTs translate data from one compression to the next (aka the “realms of complexity”).
2202
+
Senior introduces Junior to the different levels of image plane complexity. Senior and Junior start their journey
2203
+
in the realm of the blocks (the realm in which they normally resonate) and move to dither, lines, to then traverse
2204
+
into the more complex realms of wavelets and vectors. Junior does not only react to the old compression
2205
+
technologies (they are nostalgic and boring), but also the newer, more complex ones –– they scare Junior,
2206
+
because of their “illegibility.”
2207
+
X
2208
+
2209
+
With the coming of VR and Augmented reality goggles, the concept of looking at an interface is slowly
2210
+
disappearing. The viewer is no longer looking through a window or a platform, but instead their experience is
2211
+
moved into the screen: the screen has become part of the display, presenting the user with a Z-access, a new
2212
+
navigational complexity. From this perspective, and as a DCT, we cannot approach compression as passive.
2213
+
This is why in DCT:SYPHONING the digital “image” is not just the outcome of the process, it is the
2214
+
process. The elements in a composition represent process. Technically, the VR version exploits 3D (and 2D)
2215
+
image processing artifacts, such as Z-fighting, Gimbal lock, View frustum, clipping planes, no flag, collision,
2216
+
boundary walls, aliasing and ringing, jitter, chewing, posterization and quantization. In DCT:SYPHONING,
2217
+
geometry is not just passive points and planes.
2218
+
XI
2219
+
2220
+
XII
2221
+
2222
+
XIII
2223
+
2224
+
XIV
2225
+
2226
+
XV
2227
+
2228
+
XVI
2229
+
2230
+
XVII
2231
+
2232
+
XVIII
2233
+
2234
+
E
2235
+
2236
+
cology of Compression Complexities (2017)
2237
+
2238
+
Archival print, black and white, 2x1,5 meters.
2239
+
The more heavily encoded our world becomes, the more opaque its subliminal messages.
2240
+
A map of the different complexities of compression artifacts featuring the realms of:
2241
+
⦁
2242
+
—
2243
+
2244
+
〜
2245
+
2246
+
Dots (pixels, dither, coordinates);
2247
+
Lines (interlacing, interleaving, scan line, border, beam);
2248
+
Blocks (macroblocks, cluster);
2249
+
Wavelets (JPEG2000)
2250
+
2251
+
⟗
2252
+
2253
+
Vectors (3D obj, time encoding in MPEG4)
2254
+
2255
+
As technology becomes more ubiquitous and our relationships with digital devices ever more seamless, our
2256
+
technical infrastructures are rendered more intangible. An Ecology of Compression Complexities displays the
2257
+
topological qualities of compressed data (its constant deforming) on a mesh. It offers a chart or prism to enable
2258
+
us to see what we normally cannot see.
2259
+
The i.R.D. host classic resolutions and their inherent (often invisible) artifacts, such as dots, lines, blocks, and
2260
+
wavelets, inside an Ecology of Compression Complexities (2015-2017). The Ecology of Compression
2261
+
Complexities is an illustration of a transmission ecology in which different signals connect to each other, while
2262
+
in reality these different compression technologies are not always compatible to compress or transcode to other
2263
+
complexities.
2264
+
In an Ecology of Compression Complexities, compressions can visit each other and have an exchange.
2265
+
The map is a study of compression artifacts and their qualities and ways of diversion and dispersion.
2266
+
2267
+
XIX
2268
+
2269
+
XX
2270
+
2271
+
S
2272
+
2273
+
pomenik (2017)
2274
+
2275
+
A Monument for Resolutions that will Never Be
2276
+
4x3 Meter, non-quadrilateral, extruding and video-mapped sculpture
2277
+
A four by three-meter Spomenik for resolutions that will never be is a non-quadrilateral, extruding and videomapped sculpture, that reflects videos shot from within DCT:SYPHONING.
2278
+
Historically, a Spomenik is a piece of abstract, brutalist, and monumental anti-fascist architecture from former
2279
+
Yugoslavia, commemorating or meaning “many different things to many people.”88
2280
+
The Spomenik for resolutions that will never be is dedicated to resolutions that will never exist and screen
2281
+
objects (shards) that were never implemented, such as the non-quadrilateral screen. Technically, the Spomenik
2282
+
functions as an oddly shaped screen with mapped video, consisting of 3D vectors extruding in space.
2283
+
The installed shard is three meters high and obscures a compartment in the back of the Spomenik: a small
2284
+
room hiding a VR installation that runs DCT:SYPHONING, while the projection on the Spomenik features video
2285
+
footage from within the VR. In doing so, the Spomenik reflects literal light on the issues surrounding image
2286
+
processing technologies and addresses some of the hegemonic conventions that continuously obscure our view.
2287
+
Recently a movement called Post Digital described the experience of the display as “post screen,” which means
2288
+
that anything born digital –– which includes screen based interfaces and aesthetics –– influences our everyday
2289
+
(visual) language so much, that they have become part of the ‘real’ world and its offline, off screen, vernaculars.
2290
+
Screen based language, aesthetics and resolutions –– and thus its inherent politics and ethics –– are no longer
2291
+
reserved or resolved for the LEDs, but are now adopted on all planes, digitally and non-digitally. The Spomenik
2292
+
reflects on this by bringing the digital material to the front of its monumental façade.
2293
+
2294
+
88
2295
+
2296
+
‘Introduction: What are Spomeniks*?,’ in: Spomenik Database.
2297
+
See: https://www.spomenikdatabase.org/what-are-spomeniks (20.02.20)
2298
+
XXI
2299
+
2300
+
I
2301
+
2302
+
Everywhere we imagined ourselves standing turned into a cliche beneath our feet.
2303
+
– Naomi Klein, No Logo: 1999.
2304
+
2305
+
II
2306
+
2307
+
.
2308
+
2309
+
III
2310
+
2311
+
i
2312
+
2313
+
nstitutions of Resolution Disputes (i.R.D.)
2314
+
2315
+
20’’ Acrylics; 5 institutions of the i.R.D., encrypted in DCT (2015)
2316
+
2317
+
i
2318
+
2319
+
nstitutions of Resolution Disputes [i.R.D.] 89
2320
+
What an institution is or is not today, is hard to define. Some describe institutions as “stable, valued, recurring
2321
+
patterns of behavior.” 90 Others conjure up the image of bricks and mortar; physically present buildings that
2322
+
harbor formally organized organs that facilitate certain events or produce certain materials. An institution can
2323
+
be an organization founded for religious, educational, professional, or social purposes. Think for instance about
2324
+
a church, a library or an art collection.
2325
+
Institutionalization is the process by which organizations and procedures acquire value and stability. But it is
2326
+
89 Solo show at Transfer Gallery, NYC, March 28-April 15, 2015.
2327
+
90
2328
+
2329
+
Huntington, Samuel P.. Political Order in Changing Societies. Yale University Press: 2006: p. 12.
2330
+
IV
2331
+
2332
+
important to understand institutions not as a single entity but as an interconnected object in an networked field.
2333
+
Institutions are not isolated –– their task is to mediate between employees, assets and audience or users. This is
2334
+
why there are at least two sides to any institution: the infrastructural and “content” part. The outwards facing
2335
+
part that produces knowledge, exhibitions or facilitates a sense of cohesiveness within the (sub)cultures it caters
2336
+
to and the framework or backend that keeps the institution running.
2337
+
The back end of the institution determines the conditions of the front end –– it develops conditions for
2338
+
creation and leverages the ways in which the institution interacts with other institutions, audiences and
2339
+
solicitors. This is why institutions need, at least to some degree, to be able to facilitate institutional
2340
+
recalibration: the capacity to rethink and remould the output and the logistical forms of the field. But some
2341
+
institutions are so encumbered with their status, that they cannot change, reform or recalibrate (quickly). They
2342
+
have to answer to rules that provoke a particular content, action or behavior rather than serve the cause they
2343
+
were actually erected for in the first place.
2344
+
Between March 28th and April 15th 2015 I opened the i.R.D., in the form of a solo show at Transfer Gallery
2345
+
New York. The i.R.D. were created during a time in my life when I felt especially let down by institutions. I felt
2346
+
taken advantage of, and realized that institutions often work to keep their institute alive rather than to serve the
2347
+
purpose they have been erected for in the first place, ie. to serve and help the people that need them. During this
2348
+
time in limbo I developed the i.R.D, that consist of five institutions, through which I voice my critique on
2349
+
institutions: institutions should not consist of a group of people hiding themselves behind the title of
2350
+
“institution.”
2351
+
Though the i.R.D. mimic an institution, in reality they are not classic, institutional organs. Instead, the
2352
+
i.R.D. multiplex the term institution by revisiting its usage in the late 1970s. In this context, formulated by
2353
+
Joseph Goguen and Rod Burstall, institutions refer to a more compound framework that deals with growing
2354
+
complexities, connecting different logical systems (such as databases and programming languages) within the
2355
+
computer sciences. 91 The main result of these non-logical institutions is that different logical systems can be
2356
+
glued together at a substrata level, forming illogical frameworks through which computation can also take
2357
+
place: this type of institutions work for functionality, not for bureaucracy.
2358
+
Inspired by the idea of hyperfunctional, yet illogical frameworks, the i.R.D. are dedicated to researching
2359
+
the interests of anti-utopic, obfuscated, lost and unseen, or simply too good to be implemented resolutions.
2360
+
They ask: what do the hegemonic conventions of sight obscure? Is there any space for ‘fiat standards,’
2361
+
standards that are not backed by a reasonable intrinsic value, or standards that are maybe illogical? In doing so,
2362
+
the i.R.D. intend to shed a light on the processes behind cultural production: the systems of affordances, the
2363
+
politics of integration and imposition of templates.
2364
+
91
2365
+
2366
+
Goguen, Joseph A., and Rod M. Burstall. ‘Institutions: Abstract model theory for specification and programming,’ in:
2367
+
Journal of the ACM (JACM) 39.1 (1992): p. 95-146.
2368
+
V
2369
+
2370
+
But while the i.R.D. call attention to resolutions, they do not just wish to aestheticize their formal
2371
+
qualities and put them on display: the i.R.D refuse to take the format of a Wunderkammer. This was already
2372
+
done by YoHa in their Evil Media Distribution Centre. 92 Institutions that intend to host disputes cannot get
2373
+
away with simply displaying objects of contention. Disputes involve discussions and debate. In other words: the
2374
+
objects need to be unmuted –– or be given –– a voice. A dilemma that informs some key questions: how can
2375
+
objects be displayed in an active way? How can the i.R.D. exhibit the (normally) invisible?
2376
+
2377
+
T
2378
+
2379
+
he institutions
2380
+
2381
+
T
2382
+
2383
+
he institutions of Resolution Disputes (i.R.D.) call attention
2384
+
to media resolutions. While a ’resolution’ generally simply
2385
+
refers to a standard (measurement) embedded in the
2386
+
technological domain, the iRD reflect on the fact that a
2387
+
resolution is indeed a settlement (solution), but at the same time
2388
+
a space of compromise between different actors (objects,
2389
+
materialities and protocols) who dispute their stakes
2390
+
(framerate, number of pixels etc.) within the growing digital
2391
+
territories.
2392
+
Common settings can ossify as generally accepted requirements or
2393
+
de facto standards, while other standards are notated as norms by
2394
+
standardizing organizations such as the International Organization
2395
+
for Standardization. We call this progress.
2396
+
Moreover, resolutions are non-neutral standard settings that involve political, economical, technological and
2397
+
cultural values and ideologies; embedded in the genealogies and ecologies of our media. In an uncompromising
2398
+
fashion, quality (fidelity) speed (governed by efficiency) volume (generally encapsulated in tiny-ness for
2399
+
hardware and big when it comes to data) and profit (economic or ownership) have been responsible for plotting
2400
+
the vector of progress. This dogmatic configuration of belief x action has made upgrade culture one of the great
2401
+
legitimizers of scaled violence, putting ‘insufficient’ technological resolutions to rest. While a resolution can be
2402
+
understood as a manifold assemblage of common, but contestable standards, they should also be considered in
2403
+
terms of other options: those that are unknown and unseen, obsolete and unsupported.
2404
+
92
2405
+
2406
+
Yoha. Evil Media Distribution Centre (2013)
2407
+
see: http://yoha.co.uk/evilmedia (20.02.20)
2408
+
VI
2409
+
2410
+
R
2411
+
2412
+
esolutions inform both machine vision and human modes of
2413
+
2414
+
perception. They ubiquitously shape the material of everyday life.
2415
+
They shape the material of everyday life ubiquitously. They do this not
2416
+
just as an “interface effect” but as hyperopic lens, obfuscating any
2417
+
other possible alternative resolution from the users media literacy.
2418
+
As the media landscape becomes more and more compound: a
2419
+
heterogenous assemblage in which one technology never functions on its
2420
+
own; its complexities have moved beyond the fold of everyday settings.
2421
+
Technological standards have compiled into resolution clusters: media
2422
+
platforms that form resolutions like tablelands, flanked by steep cliffs and
2423
+
precipices looking out over obscure, incremental abysses that seem to
2424
+
harbor a mist of unsupported, obsolete norms. The platforms of resolution now organize perspective; they are
2425
+
the legitimizers of both inclusion and exclusion; of what cannot be seen or what should be done; while the fog,
2426
+
the other possibilities, become more and more obscure.
2427
+
Yet, it is important to understand that resolution platforms are not inherently evil.93 They can be impartial.
2428
+
It is important that we unpack these resolutions and note how they are conditioning our perception. A culture
2429
+
that adheres to only one or few platforms of resolutions supports nepotism amongst standards. These clusters
2430
+
actively impose notions of simplicity, masking the issues at stake and in turn savouring stupidity, which is
2431
+
bound to escalate into a glutinous techno-fascism.
2432
+
2433
+
T
2434
+
2435
+
he question is, have we become unable to construct our own
2436
+
resolutions, or have we become oblivious to them? Either way we
2437
+
are in need for another re-“(Re-)Distribution of the Sensible”.
2438
+
Resolutions work not just as interface effects but as a hyperopic lens,
2439
+
which obfuscates any other possible alternative resolutions from the
2440
+
users’ screens and media literacy. When we speak about video, we
2441
+
only ever refer to a four-cornered moving image. Why do we not
2442
+
consider video with more or fewer corners, timelines, or soundtracks.
2443
+
Fonts are monochrome: they do not come with their own textures,
2444
+
gradients, or chrominance and luminance mapping. Text editors still
2445
+
follow the layout of paper: there is hardly any modularity within
2446
+
93
2447
+
2448
+
Andrew Goffey and Matthew Fuller, Evil Media (2012)
2449
+
VII
2450
+
2451
+
written-word technologies. Even ghosts, the figments of our imagination, have been conditioned to
2452
+
communicate exclusively through analogue forms of noise (the uncanny by default), while aliens communicate
2453
+
through blocks and lines (the more intelligent forms of noise).
2454
+
We comfortably navigate resolution platforms; unknowingly suffering from technological hyperopia, we
2455
+
have lost track of the compromises that are at stake inside our resolutions and stare at screens, which show us
2456
+
mirage after mirage.
2457
+
2458
+
i
2459
+
2460
+
RD intend to impose methods of “creative problem
2461
+
2462
+
creation” to bring authorship back to the layer of setting a
2463
+
'resolution'. The radical digital materialist believes in an
2464
+
“informed materiality”; while every string of data is
2465
+
ambiguously fluid and has the potential to be manipulated into
2466
+
anything, every piece of infor-mation functions within /
2467
+
adhesive*/ encoding, contextualization and embedding (etc).
2468
+
A resolution is the lens through which constituted materialities
2469
+
become signifiers in their own right. resolutions resonate the
2470
+
tonality zeitgeist and constantly transform our technologies into
2471
+
informed material vernaculars. Technology is evolving faster than
2472
+
we as a culture can come to terms with. This is why determinations
2473
+
such as standards are dangerous; they can preclude the alternative.
2474
+
2475
+
T
2476
+
2477
+
hrough iRDs tactics beyond resolution, the otherwise grey mundane
2478
+
objects of everyday-life show their colors. iRD are not a wunderkabinet of
2479
+
dead media, but a foggy bootleg trails for vernacular resistance.
2480
+
Progress has fathered many dead technologies. A Wunderkammer, or curiosity
2481
+
cabinet of media resolutions, would celebrate these dead objects by trapping
2482
+
them inside a glass bell, relieving them indefinitely of their action radius.
2483
+
While the i.R.D. adhere to the settlements of governing media resolutions, it
2484
+
also welcomes ventures along the bootleg trails of the tactical undead. These
2485
+
undead move beyond resolution, through the literacies of the governing
2486
+
techno-cultures, into liminal spaces. They follow the wild and uncanny desire paths that cut through sensitive
2487
+
forms and off-limit areas into speculative materialities. They threaten the status quo of secure media forms and
2488
+
provide the ambiguity that is so necessary for inspiration, action and curiosity.
2489
+
VIII
2490
+
2491
+
W
2492
+
2493
+
e are in need of a re-(Re-)Distribution of the Sensible.94
2494
+
2495
+
The i.R.D. offers a liminal space for Resolution Studies. Resolution studies is not only about the effects of
2496
+
technological progress or about the the consequences of scaling. Resolution Studies analyzes how resolution
2497
+
embeds the tonalities of culture in more than just its technological facets. Resolution Studies researches the
2498
+
standards that could have been in place, but are not. As a form of vernacular resistance based on the concept of
2499
+
providing ambiguous resolutions, the i.R.D. employ the liminal resolution of the screen as a looking-glass.
2500
+
Here, hyperopia is fractured and gives space to myopia and visa versa. This is how i.R.D. expose the colors
2501
+
hidden inside the grey mundane objects of everyday life. 95
2502
+
2503
+
T
2504
+
2505
+
he i.R.D. believe that methods of creative problem creation can bring authorship back to the layer of
2506
+
resolution setting.96
2507
+
Resolution theory moves against what sometimes seems like an unsolvable compulsion to flatten reality. The
2508
+
i.R.D. might seem like a one way trail into the sea of fog and toward the abyss of techno-norms. It could also be
2509
+
a modular framework that opens and expands standards through inspection and reflection. As any good theory
2510
+
of media, resolution theory is a theory of literacy. Literacy of the machines, the people, the people creating the
2511
+
machines and the people being created by the machines. Through challenging the platforms of resolution, it can
2512
+
help the wanderer to scale actively between these states of hyperopia and myopia. It can uncover crystal cities
2513
+
of fog as well as shine a light on soon-to-be-distributed futures.
2514
+
Here we can mine for the timonds. 97
2515
+
2516
+
94 Ranciere, Jacques. The Politics of Aesthetics. London: Continuum International Publishing, 2004.
2517
+
95
2518
+
2519
+
Certeau, Michel de. The Practice of Everyday Life. Berkley: University of California Press, 1984.
2520
+
2521
+
96 jon.satrom, Creative Problem Creation, 2013.
2522
+
97
2523
+
2524
+
Menkman, Rosa. Glitch Timond, 2014.
2525
+
see: https://beyondresolution.info/Glitch-Timond
2526
+
IX
2527
+
2528
+
No objects, spaces, or bodies are sacred in themselves; any component can be interfaced with any other if the
2529
+
proper standard, the proper code, can be constructed for processing signals in a common language.
2530
+
- Donna Haraway, A Cyborg Manifesto, 1985
2531
+
2532
+
X
2533
+
2534
+
B
2535
+
2536
+
eyond Resolution. In which pattern recognition lost its resolution (2015)
2537
+
2538
+
15:30 min Live AV performance registration from Syndrom 3.X @ Static Gallery, Liverpool, January, 2015.
2539
+
Featuring remixed video images by Alexandra Gorczynski and my remixed sounds from the track Professional
2540
+
Grin by Knalpot. Sound mastering by Sandor Caron.
2541
+
Features Ryan Maguires algorithm for MP3 compression, “Ghost in the MP3.”
2542
+
Beyond Resolution is about the ghostly presence of absence, or the traces of a past captured in the present. The
2543
+
soundtrack Beyond Resolution features the Professional Grin and an inverted riff of Alvin Luciers’ “Sitting in a
2544
+
Room” experiment; one that only shows the “generation loss” instead of the generation left over, which is what
2545
+
we usually get to see or hear in art projects. In doing so it shows what sounds the MP3 compression normally
2546
+
cuts out as irrelevant –– in a sense inverting the compression by putting “irrelevant” or deleted data on display.
2547
+
2548
+
XI
2549
+
2550
+
M
2551
+
2552
+
yopia (2015)
2553
+
2554
+
12x4 meter wall vinyl installation with extruding vectors.
2555
+
Resolutions are the determination of what is run, read, and seen, and what is not. In a way, resolutions form a
2556
+
lens of (p)reprogrammed truths. Every time a new way of seeing; a new resolution is developed; a new prehistory of the history of the image is made; this new resolution forms a new lens of truth –– a force that is
2557
+
revealed only retrospectively. But these actions and qualities take place beyond the fold of our perception,
2558
+
which is why we have gradually become blind to the politics of these congealed and hardened compromises.
2559
+
We are collectively suffering from technological hyperopia.
2560
+
Myopia is a giant vinyl wall installation (12 x 4 meters), plus extruding vectors that present a zoomed-in
2561
+
perspective of JPEG2000 wavelet compression artifacts. These artifacts were the aesthetic result of a glitch,
2562
+
made when I added a line of another language into the data of a high res JPEG2000 image –– a compression
2563
+
standard used and developed for medical imaging, supporting zoom without block distortion. This action
2564
+
revealed both the surface and structure of the image.
2565
+
The title Myopia hints at a proposal for healing our collective suffering from technological hyperopia: a
2566
+
broad condition of farsightedness. With Myopia I build a place that disintegrates the architecture of zooming
2567
+
and endows the public with qualities of short-sightedness. The scope exists in the depth of the image or even
2568
+
behind the image. This was echoed in the conclusion of the installation the day before the i.R.D. show closed,
2569
+
when visitors were invited to bring an exacto blade and to cut their own resolution of Myopia to mount on any
2570
+
institution of choice: a book, computer or other rigid surface.
2571
+
2572
+
XII
2573
+
2574
+
LIII
2575
+
2576
+
T
2577
+
2578
+
acit:Blue (2015)
2579
+
2580
+
2 min. single channel video
2581
+
A discussion between a Masonic pigpen and DCT
2582
+
hardware used: NovaDrone by Casper Electronics w/ custom patching.
2583
+
Cipher:
2584
+
#
2585
+
X
2586
+
#
2587
+
・ X
2588
+
・
2589
+
Tacit:Blue is a reference to the old Northrop Tacit Blue stealth surveillance aircraft, which was developed
2590
+
against passive radar detection. The Tacit Blue aircraft is now decommissioned, as is the masonic pigpen
2591
+
encryption, in which the video carries an encrypted message. Every flash shows the next line of my “secret
2592
+
message.” As Daniel Rourke comments, “These technologies were designed to exist beneath, or parallel to,
2593
+
optic thresholds, but now these thresholds are not optic as much as they are about digital standards and
2594
+
II
2595
+
2596
+
resolution densities.” 98
2597
+
In Tacit:Blue small interruptions appear in an otherwise smooth, blue, video document of a conversation
2598
+
between two cryptography technologies: a Masonic Pigpen or Freemasons cipher (a basic, archaic, and
2599
+
geometric simple substitution cipher) and the Discrete Cosine Transform encryption DCT (Menkman, 2015).
2600
+
The sound and light that make up the blue surface are generated by transcoding the same electric signals using
2601
+
different components; what you see is what you hear.
2602
+
The technology responsible for the audiovisual piece is the NovaDrone (Pete Edwards/Casper Electronics,
2603
+
2012), a small AV synthesizer designed by Casper Electronics. In essence, the NovaDrone is a noise machine
2604
+
with a flickering military RGB LED on top. The synthesizer is easy to play with; it offers three channels of
2605
+
sound and light (RGB) and the board has twelve potentiometers and ten switches to control the six oscillators
2606
+
routed through a 1/4-inch sound output, with which you can create densely textured drones, or in the case of
2607
+
Tacit:Blue, a rather monotonous, single AV color/frequency distortion.
2608
+
The video images have been created using the more exciting functions of the NovaDrone. Placing the
2609
+
active camera of an iPhone against the LED on top of the NovaDrone, which turns the screen of the phone into
2610
+
a wildly moving suprematist collage of color bars, revealing the NovaDrone’s second practical usage as a light
2611
+
synthesizer.
2612
+
In this process the NovaDrone exploits the iPhone's CMOS (Complimentary Metal-OxideSemiconductor) image sensor, a technology that is part of most commercial cameras, and is responsible for the
2613
+
transcoding of captured light into image data. When the camera function on the phone is activated, the CMOS
2614
+
moves down the sensor capturing pixel values one row at a time. However, because the flicker frequency of the
2615
+
military RGB LED is changed by the user and higher than the writing speed of the phone's CMOS, the iPhone
2616
+
camera is unable to synch up with the LED. What appears on the screen of the iPhone is an interpretation of its
2617
+
input, riddled with aliasing known as rolling shutter artifact; a resolution dispute between the CMOS and the
2618
+
RGB LED. Technology and its inherent resolutions are never neutral; every time a new way of seeing is
2619
+
created, a new pre-history is written.
2620
+
2621
+
98
2622
+
2623
+
Rourke, Daniel. ‘Resolution Disputes: A Conversation Between Rosa Menkman and Daniel Rourke.’ in: Furtherfield,
2624
+
2015. see: https://www.furtherfield.org/resolution-disputes-a-conversation-between-rosa-menkman-and-daniel-rourke/
2625
+
III
2626
+
2627
+
LXVII
2628
+
2629
+
We shape our tools and, thereafter our tools shape us.
2630
+
— John Culkin99
2631
+
2632
+
C
2633
+
2634
+
onclusion: a need for Shadow Knowledge
2635
+
2636
+
Our (digital) cultures are in a state of hypertrophy: faster, better, shinier, deadlier; more awful and overexposed.
2637
+
It is hard to stay on top of the dialogue and keep an eye out for the newest problem, or even to find the
2638
+
contemporary discourse. What is really happening now? How does the flow and functioning of contemporary
2639
+
culture –– the culture of now –– construct itself?
2640
+
2641
+
Just like maps, images have the power to not just describe the world; they are partially responsible for creating
2642
+
it. They conjure a lens through which we look and perceive. And guide us in our understanding, our thoughts
2643
+
and actions. But in a far more obscure and violent way, the technologies (its protocols, interfaces and
2644
+
infrastructures) through which we access and create our images (e.g. image processing technologies or image
2645
+
data capture devices) also engender distortions of our perception. As Anil Dash states: “technology isn’t an
2646
+
industry, it’s a method of transforming the culture and economics of existing systems and institutions.”100 This
2647
+
is why technology should always be the subject of scrutiny and debate.
2648
+
However, in comparison to other fields such as law or medicine, ethics has never been (a big) part of
2649
+
technological education. Given that the fields of software engineering are devoid of such tradition, it has long
2650
+
been a free for all. And this has shown time and time again in the choices developers (hardware, software and
2651
+
UIX designers) have made. While every choice they make is engrained with consequences, we are still far away
2652
+
from fully understanding or pro-actively challenging for instance the profound privacy, security and racial
2653
+
biases that are embedded within the products they deliver us. 101
2654
+
Technology has been biased for decades, if not centuries and quite possibly there is no such thing as
2655
+
unbiased technology. When I write technology I mean not just our digital imaging technologies: these histories
2656
+
99
2657
+
2658
+
Culkin, J.. ‘A schoolman’s guide to Marshall McLuhan,’ in: Saturday Review, 1967: p. 70.
2659
+
2660
+
100 Dash, Anil. ‘There is no 'technology industry,’ in: Medium, Aug 19, 2016.
2661
+
2662
+
see: https://medium.com/humane-tech/there-is-no-technology-industry-44774dfb3ed7
2663
+
101
2664
+
2665
+
Dash, Anil. ‘There is no 'technology industry,’ in: Medium, Aug 19, 2016.
2666
+
see: https://medium.com/humane-tech/there-is-no-technology-industry-44774dfb3ed7
2667
+
IV
2668
+
2669
+
go as deep as the electricity grid, the sewer or our system for public transportation. Researchers such as Ingrid
2670
+
Burrington102 have shown how different forms of infrastructure can be recognized, mapped and uncovered,
2671
+
even in the space of our day to day neighbourhoods. There is a huge value to this kind of infrastructural
2672
+
research and its consequential tourism: it is a way to train ourselves at recognizing and unpacking the
2673
+
increments of obfuscation and the different scales at which these infrastructures function and exist. It can help
2674
+
us with expanding and questioning our resolutions, in the fight to regain our agency.
2675
+
At the very basis of such fight lies the understanding that any software, hardware or interface has
2676
+
(habitual) values engrained inside of it. As Safiya Umoja Noble states: “Infrastructure is created by people and
2677
+
therefore embeds and reflects the values of the people who create it.”103 To be able to ignore or unsee these
2678
+
(infrastructural) resolutions that govern our digital technologies, and thus our daily realities, or to presume these
2679
+
infrastructures are “hidden” or “magic,” is an act reserved only to the digitally illiterate. To formulate questions
2680
+
around how our technologies work, is a prerequisite for actively taking part and harnessing agency in our
2681
+
contemporary society.
2682
+
Still, compromises and biases are often waved away as ‘accidents;’ they are the result of an uninformed
2683
+
engineer or an unfortunate byproduct of a financial calculation. A main point of Beyond Resolution is that we
2684
+
can no longer ignore these unfortunate ‘accidents’. To develop technology in an uninformed matter and
2685
+
consequently perpetuate biased engagement is not ok: we are at a point when we can choose what biases to
2686
+
implement; good, bad or neutral. We can choose our compromises. But of course before we can challenge these
2687
+
imposed biases and compromises - we need the tools not just to recognise and understand them, but to speculate
2688
+
on other possibilities.
2689
+
And this is what Beyond Resolution has tried to insist on: an extended formulation of resolution. A
2690
+
resolution is not just a trade-off between settings that manage speed and functionality (bandwidth, control,
2691
+
power, efficiency, fidelity), while at the same time considering claims in the realms of value vs. storage,
2692
+
processing or transmission. A resolution also always involves an inherent compromise of other ways of
2693
+
rendering, and it is in these other ways that we need to train to see, run and formulate our alternatives.
2694
+
With the example of the genealogy of the color test cards I offered one way to make such latent and biased
2695
+
power structures more apparent: we need to insist that these standard images, trapped in the histories of our
2696
+
technologies, become part of the public domain. In order to illuminate the white shadows that govern the
2697
+
outcomes of our image processing technologies, we must document the genealogies of standardization. These
2698
+
genealogies belong in high school textbooks: the latent violence coded within such norms should be studied as
2699
+
102 Burrington, Ingrid. Networks of New York: An illustrated field guide to urban internet infrastructure. Melville House,
2700
+
2701
+
2016.
2702
+
103
2703
+
2704
+
Noble, Safiya Umoja. Algorithms of oppression: How search engines reinforce racism. NYU Press, 2018.
2705
+
V
2706
+
2707
+
part of standard curricula to inform a future generation of engineers of compromises made in the past.
2708
+
It is often said that all we have is the past to train with. I believe there is truth to that. But some
2709
+
importance also lies in our contemporary state. Art schools, academies, universities and other institutions that
2710
+
study and teach new media or engineering play an important role here. They need to keep training their students
2711
+
but also the institution itself to not only critically engage with newly developed materialities, protocols and
2712
+
languages, but also –– and I believe that this is where the ‘new’ in new media comes in –– to develop a certain
2713
+
fluid literacy around these constantly developing and mutating materialities. Because they impose constraints
2714
+
and qualities on their content, or often even form the content themselves.
2715
+
But how can we study new media, its materialities, resolutions and compromises, when they are not fixed but
2716
+
rather constructed as a constant “interplay between a text’s physical characteristics and its signifying
2717
+
strategies”?104 To engage with digital culture means to be able to formulate a critical point of view, which
2718
+
involves analysis and active change through critical thought processes, such as speculation. Uncovering and
2719
+
studying these spaces of speculation is of vital importance and it is here where we find a growing need for
2720
+
players such as artists, theorists, designers and many others, who continue to engage critically with
2721
+
contemporary cultures. We need a fundamental acceptance of Shadow Knowledge: in the shadows, things lack
2722
+
definition. It is where we can find objects of unsupported dimension and scale, ambiguous and fluid. The
2723
+
shadows are blurry and liminal, but ultimately potent spaces that can exist between what is enlightened and
2724
+
opaque (or black boxed). Shadows offer shady outlines, that can function either as a vector of progress or a
2725
+
paint by numbers. In the shadows we can rest, heal and recalibrate. Now is not the time to hope or fear. It is the
2726
+
time to look for new weapons. The future lies in the shadows of our present.
2727
+
2728
+
104
2729
+
2730
+
Hayles, Katherine. ‘Print is flat, code is deep: The importance of media-specific analysis,’ in: Poetics Today (25, no. 1
2731
+
2004): p. 67–90.
2732
+
VI
2733
+
2734
+
On October 18th,
2735
+
AKA international
2736
+
standards day, the
2737
+
i.R.D. laid flowers
2738
+
to commemorate all
2739
+
unaccepted and
2740
+
depreciated
2741
+
standards at the ISO
2742
+
offices in Geneva.
2743
+
2744
+
X
2745
+
2746
+
XV
2747
+
2748
+
+2200
system/public/assets/papers/readings/text/Menkman-Glitch-Momentum-2011.txt
+2200
system/public/assets/papers/readings/text/Menkman-Glitch-Momentum-2011.txt
···
1
+
m) The
2
+
Glitch
3
+
nt Moment
4
+
h
5
+
e Glitch
6
+
nt(um)
7
+
The
8
+
Glitch
9
+
Moment(um)
10
+
11
+
Rosa Menkman
12
+
13
+
04
14
+
15
+
The
16
+
Glitch
17
+
Moment(um)
18
+
19
+
04
20
+
21
+
Contents
22
+
Acknowledgements
23
+
24
+
5
25
+
26
+
Introduction
27
+
28
+
7
29
+
30
+
Glitch Studies Manifesto
31
+
32
+
11
33
+
34
+
a Technological Approach To Noise
35
+
36
+
12
37
+
12
38
+
15
39
+
15
40
+
17
41
+
26
42
+
26
43
+
44
+
Linear Progression and the Myth of Perfect Transmission
45
+
Noise Artifacts
46
+
Encoding And Decoding: Compression Artifacts
47
+
A Vernacular Of File Formats
48
+
Orderly Chaos: Feedback Artifacts
49
+
The Other Noise Artifact: Glitch
50
+
The Perception Of Glitch
51
+
52
+
The Meaning Of Noise
53
+
The Glitch Moment(um): A Void In Techno-Culture
54
+
Technorealism And the Accident Of Art
55
+
56
+
28
57
+
28
58
+
29
59
+
31
60
+
61
+
A Phenomenology Of Glitch Art
62
+
63
+
33
64
+
The Predicaments Of Defining Glitch Art
65
+
33
66
+
Categorical Precursors: A Binary Approach To Glitch Art?
67
+
35
68
+
From Passive Appropriation Or ‘Pure Glitch Art’ To Active, ‘Post-Procedural Glitch Art’ 36
69
+
Post-procedural Glitch Art Or the Intentional Faux Pas
70
+
37
71
+
The Concept And Technique Of Ruin
72
+
40
73
+
Creating the ‘Perfect Glitch’ Using Critical Media Aesthetics
74
+
43
75
+
The Tipping Point of Cool: Critical Media Aesthetics' Becoming Commodities
76
+
44
77
+
78
+
From Artifact To Commodity
79
+
80
+
From Circuitbending to Simulation
81
+
From Databending to Transcoding
82
+
From Enchanting Affect To Filtered Effect
83
+
The Glitch Art Genre: Between the Void And Commoditized Form
84
+
The Genre Paradox
85
+
Organizing Glitch Spheres
86
+
87
+
46
88
+
46
89
+
49
90
+
53
91
+
55
92
+
57
93
+
94
+
Glitch Art Networked
95
+
Glitch Sphere Relations
96
+
Some Final Reflections On The Glitch Spheres
97
+
98
+
59
99
+
62
100
+
63
101
+
64
102
+
103
+
The Emancipation of Dissonance Glitch
104
+
105
+
65
106
+
107
+
Bibliography
108
+
109
+
67
110
+
111
+
Colophon
112
+
Network Notebooks editors: Geert Lovink and Sabine Niederer
113
+
Producer of this publication: Margreet Riphagen
114
+
Copy editing: Rachel O'Reilly
115
+
Design: Studio Léon&Loes, Rotterdam http://www.leon-loes.nl
116
+
Printer: Printvisie Rotterdam
117
+
Publisher: Institute of Network Cultures, Amsterdam
118
+
Supported by: School for Communication and Design at the Amsterdam University of Applied Sciences
119
+
(Hogeschool van Amsterdam, Domein Media, Creatie en Informatie) and Stichting Democratie en Media.
120
+
The glitch moment(um) is composed of texts that have been extended and reworked by Rosa Menkman, 2006–2011.
121
+
If you want to order copies please contact:
122
+
Institute of Network Cultures
123
+
Kenniscentrum Create-IT
124
+
Singelgrachtgebouw
125
+
Rhijnspoorplein 1
126
+
1091 GC Amsterdam
127
+
The Netherlands
128
+
http://www.networkcultures.org
129
+
books@networkcultures.org
130
+
t:+31 (0)20 59 51 866 – f: +31 (0)20 59 51 840
131
+
A pdf of this publication can be freely downloaded at:
132
+
http://www.networkcultures.org/networknotebooks
133
+
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.
134
+
Amsterdam, October 2011.
135
+
ISBN/EAN 978-90-816021-6-7
136
+
137
+
Network Notebook Series
138
+
The Network Notebooks series presents new media research commissioned by the INC.
139
+
PREVIOUSLY PUBLISHED NETWORK NOTEBOOKS
140
+
Network Notebooks 03:
141
+
Dymtri Kleiner, The Telekommunist, 2010.
142
+
ISBN: 978-90-816021-2-9.
143
+
Network Notebooks 02:
144
+
Rob van Kranenburg, The Internet of Things: A critique of ambient technology and the all- seeing network of RFID, 2008.
145
+
ISBN: 978-90-78146-06-3.
146
+
Network Notebooks 01:
147
+
Rosalind Gill, Technobohemians of the new Cybertariat? New media work in Amsterdam a decade after the Web, 2007.
148
+
ISBN: 978-90-78146-02-5.
149
+
150
+
4
151
+
152
+
Acknowledgements
153
+
I am very thankful to the Institute of Network Cultures for supporting me and my love for
154
+
glitch, and for devoting one of their publications to this subject. I am especially grateful
155
+
to Geert Lovink, not just for his endless insights into and support towards all fringes of
156
+
digital culture, but specifically for pushing me into unexpected territories. I have to thank
157
+
Jodi for opening my eyelids and Goto80 for opening my earlids, and for the journey we
158
+
shared. Besides this, I have Karl Klomp to thank for his technical support.
159
+
Annet Dekker and Josephine Bosma wrote the first texts that showed me a way into theorizing glitch art – I have to thank them for writing those texts and also Matthew Fuller for
160
+
the conversation that we had the day he welcomed me into his office, which I consider the
161
+
starting point for this text. Special thanks to my editor, Rachel, who worked long hours
162
+
reviewing my glitches.
163
+
Finally, I would like to make a shout out to The gli.tc/h/bots and all broken executables
164
+
everywhere, for bringing me late night Unicode barf, all the organizers of festivals I have
165
+
attended and all the people who have supported me in the last years, artistically, theoretically and technically. Finally, my family and close friends – may good spam be with you.
166
+
167
+
5
168
+
169
+
Introduction
170
+
Glitch first came into my life in 2005, when I visited the world wide wrong exhibition
171
+
by the Dutch/Belgium artist collective Jodi (Joan Heemskerk and Dirk Paesmans) at MonteVideo/Time Based Arts in Amsterdam (now known as the NIMK, Netherlands Media Art
172
+
Institute). An introductory text on the work of the artists by Annet Dekker went a long
173
+
way in articulating the artists’ deconstructive methods. However, the work that made the
174
+
biggest impression on me, untitled game (1996-2001), which was a modification of the
175
+
videogame quake 1, seemed the most incomprehensible. I could only understand it as
176
+
irrational and void of meaning, and so I walked away from it, confused and titillated. In
177
+
hindsight, I learned about myself in that moment – about my expectations and conceptions of how a videogame should work. The strange game seemed only to return me to my
178
+
own perspectives and expectations around the medium that it was failing to be. A second
179
+
text by Josephine Bosma usefully outlined Jodi’s active deprogramming of computers,
180
+
and the paradoxes and tensions inherent to their working method. Even still, untitled
181
+
game in particular remained for me under-articulated in theory, which increased my
182
+
curiosity about this kind of art practice. I did not realize it then, but my taste for glitch,
183
+
and for its potential to interrogate conventions through crashes, bugs, errors and viruses,
184
+
was spawned by that initial and persistent critical evasion of untitled game from my
185
+
theoretical grasp.
186
+
At the end of my master thesis in 2006, which focused on Jodi’s work untitled game,
187
+
I had still not yet referred to Jodi’s art work as ‘glitch art’ – I only mentioned the words
188
+
‘glitch’ and ‘buggy’. This is probably because the notion of glitch art was just crossing over
189
+
from sound culture, and leaking into visual art cultures only sporadically.01 Glitch more
190
+
fully entered my vocabulary for visuals and networks when I began an artistic collaboration with the musician Goto80 (Anders Carlsson) in 2007. He explained to me how he exploited the Commodore 64 sound chip (the SID chip) for the creation of music. The bugs
191
+
Goto80 used gave a very specific texture to the sound (the result of noise artifacts) and I
192
+
began to develop and recognize visual equivalents to this process. I found more and more
193
+
artifact-based correspondences between audio and visual technologies, such as compressions, feedback and glitches, in my at that time mostly online art practice. Then in early
194
+
2008, Geert Lovink invited me to the Video Vortex conference for a visual live performance
195
+
(which was quite a challenge since I had never stood onstage before) and in 2009 put
196
+
me in touch with Matthew Fuller, an artist, author and lecturer in London, which later
197
+
turned out to be two key turning points, artistically and theoretically. I began performing and more strongly theorizing what I was then calling my acousmatic videoscapes.02
198
+
I explained to Fuller my observations of compressions, feedback and glitches in sound
199
+
200
+
01| Around this time, there were only a few people using the term ‘glitch art’ in the context of the visual arts: Ant Scott had been working on his
201
+
‘glitch art’ since July 2001 and was also one of the key performers at a Glitch festival that took place in Norway in 2002. Besides this, Iman
202
+
Moradi just finished his Glitch Aesthetics - dissertation (2004) in which he used the terms ‘glitch art’ and ‘glitch design’ interchangeably. The
203
+
term ‘glitch art’ first entered Wikipedia in 2007, where it was explained as manipulated B-movies and erotic art, also known as a subset of
204
+
Rape Art. This description changed only in 2009. In conversation with Moradi, we agreed that the term only permeated visual art theory and a
205
+
general vocabulary after 2005, if not a couple of years later.
206
+
207
+
02 | In reference to Pierre Schaeffer, I called these videos ‘videoscapes’. These videos followed the logic of acousmatics; I refused the audience
208
+
knowledge over the instrument and thus denied them their inherent cultural conditioning that would otherwise help them in their process of
209
+
making meaning. In doing so, I put in front of them the ‘unseen visual artifacts’, from behind Pythagoras’ curtain - the shrouded, black veil of
210
+
technology, http://videoscapes.blogspot.com/
211
+
212
+
7
213
+
214
+
and their correspondences to the visual sphere, who pointed me to the early information
215
+
theory of Claude Elwood Shannon and Warren Weaver. Their work proved most useful
216
+
to my project of developing a technology-driven framework for theorizing these usually
217
+
unwelcome, increasingly exploited noise artifacts in which my practice was so invested.
218
+
In the beginning of 2010, I developed and distributed the glitch studies manifesto
219
+
(2010) in which I argued in favor of more critical attention to glitch’s increasing ossification in standardized design.03 I also wrote a vernacular of file formats (2010), a
220
+
work in the genre of a handbook, intended to more rigorously communicate the technical specifications of all (or most-used) digital compression artifacts that could be created
221
+
through random data insertion at that historical moment.04 This comprehensive PDF
222
+
guide to compression artifacts was positively received by the glitch community and also
223
+
spawned my next project, the co-development of a piece of generative glitch design software together with Johan Larsby, called monglot (2011). monglot made it possible for
224
+
anybody, without extensive data corruption skills, to technically interrogate and learn
225
+
about the development of specific glitch formations.05
226
+
More recently I have merged my technical, narrative and historical comprehension of digital glitch culture into an audiovisual performance called the collapse of pal (2010), in
227
+
which the (Paul Klee’s) Angel of History, as narrated by Walter Benjamin, reflects specifically on the ending of PAL, the analog Phase Alternate Line television signal. By introducing a critical and melodramatic narrative to a work of glitch art, I tried to underline that
228
+
there is more to glitch art, and more at stake, than just design and aesthetics. The work
229
+
addresses themes such as planned obsolescence, built-in nostalgia, critical media aesthetics and the gentrification and continuing development of a glitch art genre. Finally, I
230
+
have been participating in critical community building around glitch, in my work as co-organizer and co-curator of the GLI.TC/H festival alongside Jon Satrom, Nick Briz and Evan
231
+
Meaney. The first installment of GLI.TC/H took place in Chicago in 2010 and was hugely
232
+
successful. In 2011 the GLI.TC/H festival will spread to Amsterdam and Birmingham.
233
+
Every form of glitch, whether breaking a flow or designed to look like it breaks a flow, will
234
+
eventually become a new fashion. That is fate. This is because of glitch’s inherently critical moment(um) – a concept I use throughout my work to indicate the potential any glitch
235
+
has to modulate or productively damage the norms of techno-culture, in the moment at
236
+
which this potential is first grasped. In this publication for the Institute of Network Cultures, I have consolidated my efforts at writing into the silences, under-articulated theories and assumed madness of digital glitch art. The book makes sense of recent glitch art,
237
+
technically, culturally, critically, aesthetically and finally as a genre. I bring in the early information theorists not usually studied as theoretical foundations for digital art practice
238
+
(Shannon and Weaver) to consider and refine a signal and informational framework appropriate to glitch’s technological origins and orientations. I go on in later parts to build
239
+
03 | Rosa Menkman, ‘Glitch Studies Manifesto’ in Geert Lovink and Rachel Somers Miles (eds) Video Vortex Reader II: moving images beyond
240
+
YouTube, Amsterdam: Institute of Network Cultures, 2011, pp. 336-347, http://dl.dropbox.com/u/17713740/Glitch%20Studies%20Manifesto%20rewrite%20for%20Video%20Vortex%202%20reader.pdf.
241
+
04 | Rosa Menkman, a vernacular of file formats, August 2010. http://dl.dropbox.com/u/9054743/lofi%20Rosa%20Menkman%20-%20A%20
242
+
Vernacular%20of%20File%20Formats.pdf.
243
+
05 | Johan Larsby and Rosa Menkman, monglot, 2011. < http://rosa-menkman.blogspot.com/search/label/Monglot.
244
+
245
+
8
246
+
247
+
the term glitch more broadly than this technological beginning. I describe the ‘glitch’ as a
248
+
(actual and/or simulated) break from an expected or conventional flow of information or
249
+
meaning within (digital) communication systems that results in a perceived accident or
250
+
error. A glitch occurs on the occasion where there is an absence of (expected) functionality, whether understood in a technical or social sense. Therefore, a glitch, as I see it, is not
251
+
always strictly a result of a technical malfunction.
252
+
At the same time, theorists need to be more clear about the relationship between technical
253
+
and metaphorical or cultural dimensions of glitch culture. Focusing on the glitch within
254
+
this broader perspective makes it possible to think through some of the more interesting
255
+
political and social uses of the glitch within the field of digital art. Glitch makes sense
256
+
differently in terms of noise, failure and accident. Moreover, glitch transitions between
257
+
artifact and filter, or, in other words, between radical breakages and commodification
258
+
processes. Finally, glitch could be said to exist, in all of its tensions and through all kinds
259
+
of cultural feedback, as a recognizable genre of art. Finally, I finish up with a relational
260
+
visualization of the glitch cultural communities and scenes that this book attempts to
261
+
make sense of.
262
+
263
+
9
264
+
265
+
10
266
+
267
+
Glitch Studies Manifesto
268
+
1. T
269
+
he dominant, continuing search for a noiseless channel has been – and will always
270
+
be – no more than a regrettable, ill-fated dogma.
271
+
Acknowledge that although the constant search for complete transparency brings newer,
272
+
‘better’ media, every one of these improved techniques will always possess their own inherent fingerprints of imperfection.
273
+
2. D
274
+
ispute the operating templates of creative practice. Fight genres, interfaces and
275
+
expectations!
276
+
Refuse to stay locked into one medium or between contradictions like real vs. virtual, obsolete vs. up-to-date, open vs. proprietary or digital vs. analog. Surf the vortex of technology, the in-between, the art of artifacts!
277
+
3. G
278
+
et away from the established action scripts and join the avant-garde of the unknown.
279
+
Become a nomad of noise artifacts!
280
+
The static, linear notion of information-transmission can be interrupted on three occasions: during encoding-decoding (compression), feedback or when a glitch (an unexpected break within the flow of technology) occurs. Noise artists must exploit these noise
281
+
artifacts and explore the new opportunities they provide.
282
+
4. Employ bends and breaks as metaphors for différance. Use the glitch as an exoskeleton for progress.
283
+
Find catharsis in disintegration, ruptures and cracks; manipulate, bend and break any
284
+
medium towards the point where it becomes something new; create glitch art.
285
+
5. Realize that the gospel of glitch art also tells about new standards implemented by
286
+
corruption.
287
+
Not all glitch art is progressive or something new. The popularization and cultivation of
288
+
the avant-garde of mishaps has become predestined and unavoidable. Be aware of easily
289
+
reproducible glitch effects automated by softwares and plug-ins. What is now a glitch will
290
+
become a fashion.
291
+
6. Force the audience to voyage through the acousmatic videoscape.
292
+
Create conceptually synaesthetic artworks that exploit both visual and aural glitch (or
293
+
other noise) artifacts at the same time. Employ these noise artifacts as a nebula to shroud
294
+
the technology and its inner workings and to compel an audience to listen and watch
295
+
more exhaustively.
296
+
7. R
297
+
ejoice in the critical trans-media aesthetics of glitch artifacts.
298
+
Utilize glitches to bring any medium into a critical state of hypertrophy, to (subsequently)
299
+
criticize its inherent politics.
300
+
8. E
301
+
mploy Glitchspeak (as opposed to Newspeak) and study what is outside of knowledge. Glitch theory is what you can just get away with!
302
+
Flow cannot be understood without interruption, nor function without glitching. This is
303
+
why glitch studies is necessary.
304
+
305
+
11
306
+
307
+
A Technological Approach To Noise
308
+
Since, ordinarily, channels have a certain amount of noise, and therefore a finite
309
+
capacity, exact transmission is impossible.01
310
+
- Shannon Weaver
311
+
312
+
Linear Progression and the Myth of Perfect Transmission
313
+
In 1948, Claude Shannon, today known as the founder of information theory, developed a
314
+
basic mathematical theory of communication while working in the Bell Telephone laboratories in the US during the Second World War. Shannon’s main concern was to work out
315
+
a way in which the channels of communication could be used most efficiently.02 In the
316
+
model, Shannon reduced communication to a process of ‘transmitting information’, and
317
+
distinguished information from the category of a message. He wrote:
318
+
The fundamental problem of communication is that of reproducing at one point
319
+
either exactly or approximately a message selected at another point. Frequently
320
+
the messages have meaning; that is they refer to or are correlated according to
321
+
some system with certain physical or conceptual entities. These semantic aspects
322
+
of communication are irrelevant to the engineering problem. The significant aspect is that the actual message is one selected from a set of possible messages. The
323
+
system must be designed to operate for each possible selection, not just the one
324
+
that will actually be chosen.03
325
+
Shannon based his mathematical theory of communication on the fundament that information does not change when its context changes. He also suggested that communication systems could roughly be divided into three main categories – discrete, continuous
326
+
and mixed – all following a basic model for communication. In these systems, information can be understood as a quantity, ‘a yes or no decision’, a bit.04 The model thus ‘has
327
+
applications not only in communication theory, but also in the theory of computing
328
+
machines, the design of telephone exchanges and other fields’.05 Building on the prior
329
+
work of, among others, Harry Nyquist and Ralph Hartley, Shannon developed this linear
330
+
model to calculate and optimise the signal to noise ratio (a measure that compares the
331
+
level of a desired signal to the level of background noise). Because Shannon focused on
332
+
the transmission of information between machines and not on the transmission of meaning between human beings, the model makes it possible to consider noise from a purely
333
+
mathematical level, while bracketing to one side the influence of culture, linguistics or
334
+
other contextual factors that bring communication into the realm of interpretation and
335
+
336
+
01 | Claude Elwood Shannon, ‘A Mathematical Theory of Communication’, Reprinted with corrections from The Bell System Technical Journal,
337
+
Vol. 27 (July, October, 1948): p. 48.
338
+
02 | Susan Ballard, ‘Information, Noise and et al’, M/C Journal, 10.5 (October, 2007), http://journal.media-culture.org.au/0710/02-ballard.php.
339
+
03 | Claude Elwood Shannon, ‘A Mathematical Theory of Communication’, Reprinted with corrections from The Bell System Technical Journal 27
340
+
(July, October, 1948): p. 1.
341
+
04 | Susan Ballard, ‘Information, Noise and et al’, M/C Journal, 10.5 (October, 2007), http://journal.media-culture.org.au/0710/02-ballard.php.
342
+
05 | Claude Elwood Shannon, ‘A Mathematical Theory of Communication’, reprinted with corrections from The Bell System Technical Journal,
343
+
Vol. 27 (July, October, 1948): p. 3.
344
+
345
+
12
346
+
347
+
meaning. This makes his work considerably abstract, while opening up possibilities for
348
+
considering non-human, machinic communication more cogently, as the starting point
349
+
for theorizing noise.
350
+
351
+
Visualization of the communication model as outlined by Shannon and Weaver.
352
+
353
+
Shannon’s modelling of communication, in terms of signal and noise, consists of five
354
+
basic steps. The transmission of information begins at the information source, which
355
+
produces the message. A transmitter encodes the messages in signals made suitable for
356
+
transmission, which it then sends through a channel. A receiver decodes the message from
357
+
the signal, to finally deliver the message in its proper form to the destination, the machine
358
+
for which the message is intended or where the message arrives.06 The model ‘information source-> encoder-> channel-> decoder-> destination’ that Shannon constructed also
359
+
includes an additional arrow inserting noise into the channel, which is a sixth, disruptive,
360
+
external factor.07 In signal processing theory (which has existed in different forms since
361
+
the 17th century), noise is generally considered in terms of the mechanical imprecision of
362
+
instrumentation.08 Shannon’s adaptation of signal processing to communication theory
363
+
accounts for external noise being introduced to the signal while it is in transmission, to
364
+
obscure the purity of the signal. This kind of external noise has a particular materiality
365
+
and enters into the equation as unexplained variation and random error.09
366
+
In addition to this first kind of noise, Shannon also described a second kind of noise
367
+
called entropy, which is encoded within the message itself. Entropy, taken from the field
368
+
and theories of thermodynamics, is the measure of disorder of a system at a given time.
369
+
According to thermodynamics, it is inherent to any system of information, natural or technological, to tend towards disorder or to fall apart completely. This entropic orientation
370
+
is essential and in some ways, positive, because it can tell something about the relationships between the material bodies, representations and spaces connected together for
371
+
372
+
06 | Claude Elwood Shannon, ‘A Mathematical Theory of Communication’, Reprinted with corrections from The Bell System Technical Journal,
373
+
Vol. 27 (July, October, 1948): p. 2.
374
+
07 | Claude Elwood Shannon, ‘A Mathematical Theory of Communication’, Reprinted with corrections from The Bell System Technical Journal,
375
+
Vol. 27 (July, October, 1948): p. 2.
376
+
08 | Paolo Prandoni and Martin Vetterli, Signal Processing for Communications, Lausanne: EPFL Press, 2008, http://www.sp4comm.org/webversion.html.
377
+
09 | Susan Ballard, ‘Information, Noise and et al’, M/C Journal, 10.5 (October, 2007), http://journal.media-culture.org.au/0710/02-ballard.php.
378
+
379
+
13
380
+
381
+
the purposes of transmission. Following this, it is important to realise that in Shannon’s
382
+
communication model, information is not only obfuscated by noise, it is also dependent
383
+
upon it for correct transmission. Without noise, either encoded within the original message, or present from sources outside the channel, there cannot be a functioning channel.
384
+
Noise serves to contextualize information; information needs noise to be transmitted successfully. Consequently, without noise there is no information.10 Shannon eventually with
385
+
Weaver adapted this mathematical model into ‘The Shannon and Weaver model of communication’, bringing machine communication theory to the consideration of human
386
+
communication, by incorporating Norbert Wiener’s cybernetic concept of feedback.11
387
+
In optimising signal to noise ratios, Shannon was working very much in line with what
388
+
has become the dominant, modernist and even twentieth century ideal for technology:
389
+
the notion of the optimally transparent channel. (I use 'transparent' throughout the book
390
+
to describe the assumption that technology can be "see-through", or does not intervene
391
+
into the process of sending or perceiving information.) Within media design and development cultures, the pursuit of ultimate, noise-free and hi-fi channels and supposed highest
392
+
levels of ‘reality’ has tended to be the Holy Grail (epitomized for example as media dreams
393
+
in the Holodeck of Star Trek, or the direct brain cinema of Bigelow’s Strange Days, and so
394
+
on).12 While the ideal is always unreachable, innovation is nevertheless still assumed to
395
+
lie in finding an interface that is as non-interfering as possible, enabling the audience to
396
+
forget about the presence of the medium and believe in the presence and directness of
397
+
immediate transmission. As Bolter and Grusin note in Remediation (2001), ‘our culture
398
+
wants to multiply its media and to erase all traces of mediation: ideally, it wants to erase
399
+
its media in the very act of multiplying them’. It is the very ‘logic of immediacy’, according to Bolter and Grusin, which ‘dictates that the medium itself should disappear’.13 An
400
+
example of this is the computers’ Graphical User Interface, which was developed to let users interact with multiple electronic devices using graphics rather than complicated text
401
+
commands. This development made these technologies more accessible and widespread,
402
+
yet more obfuscated in their functionalities. Indeed, what makes any medium specific is
403
+
how it fails to disappear – as techné. To study media-specific artifacts is to take interest in
404
+
the failure of media to disappear, or in other words, in noise artifacts.
405
+
406
+
10 | Susan Ballard, ‘Information, Noise and et al’, M/C Journal, 10.5 (October, 2007), http://journal.media-culture.org.au/0710/02-ballard.php.
407
+
11 | The fact that Shannon and Weaver approached human communication with a model developed for the transmission of information and
408
+
thus did not consider the difference of the human process of making meaning resulted in heavy criticism from other human communication
409
+
theorists, who eventually developed modified and alternative models. These models often emphasised the fact that communication doesn’t
410
+
mean the transmission of information, but rather that meaning is actively constructed by both the initiators and interpreters. In 1954, Wilber
411
+
Schramm adjusted Shannon and Weaver’s model, putting greater emphasis on the process of encoding and decoding. This alteration was later
412
+
adopted by Stuart Hall, who wrote about encoding and decoding from the perspective of mass communication (principally in television).
413
+
The only other mediated communication model that has gained a wide usage since Shannon and Weaver is the communication model by
414
+
McLuhan. McLuhan essentially argues that mediation and communication is what we all live inside of, therefore cannot be caught in one
415
+
transparent model. McLuhan drops the source and the sender that enclose Shannon’s communication model. He also rejects the notion
416
+
of mathematics and mostly focuses on the influence of the medium over the content of the message. According to McLuhan, the medium
417
+
shapes the content of the message: the medium for instance changes the scale, pace or pattern of the message. Although I am aware that
418
+
aspects of both McLuhan’s and Shannon and Weaver’s work can be applied to glitch in different ways, I believe that for the purposes of my
419
+
research the model of Shannon and Weaver is most clearly useful as a basic, informational approach to noise.
420
+
12 | Michael Heim, The Metaphysics of Virtual Reality, New York: Oxford University Press, 1993. p. 122.
421
+
13 | Jay David Bolter and Richard Grusin, Remediation: Understanding New Media, Massachusetts: MIT Press, 1999. pp. 5-6.
422
+
423
+
14
424
+
425
+
Noise Artifacts
426
+
While the linear communication process described by Shannon is reasonably deterministic, its predictability is undermined by the overall addition of noise, which reveals itself
427
+
on the surface of the information and is categorizable as ‘noise artifacts’. Types of noise
428
+
artifacts depend on the form of the information, including how it is shaped by encoding/
429
+
decoding (in the digital realm this process materializes de/compression artifacts) or misshaped by signal corruption, whilst in transmission. Feedback is another differentiation
430
+
of a signal from the linear transmission model that can also lead to particular, mediumspecific artifacts. Once their cause is known, different forms of noise artifacts can be
431
+
named according to these three categories. Each category of interruption involves its own
432
+
technical aesthetics, shaped through media specificity.
433
+
434
+
Encoding And Decoding: Compression Artifacts
435
+
Today’s communication strives to become exponentially faster and (partly as a consequence) to become more transparent. In the present pursuit of immediacy, signal speed
436
+
has been increasingly prioritized. In contrast, earlier developments in audio and video
437
+
technologies focussed on the reduction of noise in order to improve media experiential
438
+
‘quality’. Today quality seems to be of secondary importance; recent technological developments appear to reverse or downplay the focus on signal quality as a genuine digitalcultural concern.
439
+
With the help of more powerful (transfer) protocols such as en/decoding or compression
440
+
algorithms, information can travel faster and further. A compression reorganises information, the time and space through which the elements of sound and images are communicated, by scaling, reordering and decomposing.14 Compression can be quite complex.
441
+
Consider a single file. First there is a file format, which can be a ‘container’ of, for example, sound and image (examples include MPEG, AVI or MOV). This container possesses
442
+
the meta-information about what type(s) of compression-decompression protocol(s) or
443
+
‘codecs’ are needed to store and transfer the information or to view the data object. The
444
+
container thus does not carry the compression algorithm itself. Instead these are installed
445
+
on the computer in the form of codecs. Besides this, there are two different kinds of compression protocols: lossless and lossy.
446
+
Lossless and especially lossy compressions have become almost ubiquitous, whereas
447
+
original RAW (uncompressed) information is considered now rare and relatively unwieldy, especially in the realms of digital music, photography and cinema. Lossless compressed files can be rebuilt exactly the way they were before being compressed, and retain
448
+
all information during the process. Contrarily, a good example of new lossy data compression technologies is the mp3 data format, which has made it possible to distribute
449
+
music easily, but in lower quality than the CD (which uses an uncompressed linear PCM
450
+
organisation). Lossy data compression takes a pragmatic, versioned distance from the
451
+
original file. This compression focuses only on the data that is important for the eye and/
452
+
453
+
14 | Adrian Mackenzie, ‘Codecs’, in Matthew Fuller (ed.) Software Studies, Massachusetts: MIT Press, 2008, pp. 48-55. p. 36.
454
+
455
+
15
456
+
457
+
or the ear to perceive and discards the information that is believed to be of lesser importance. For video images for instance, perception depends on the thresholds of luminance
458
+
(brightness) and chrominance (colouring) in space and time. Therefore, video codecs are
459
+
designed around the transmission of these two values as efficiently as possible. The rise
460
+
of lossy files has resulted in contemporary media consumption practices in which noise is
461
+
increased rather than decreased.
462
+
Most compressions are relatively concealed. They rarely come to the surface to explicitly reveal their language, or system of rules. They have been built, debugged and tested
463
+
many times in order for them to seem negligible, or indeed, to recede into transparency.
464
+
However, every de/encoding (or compression) technology has its visible and less visible
465
+
artifacts, that will be able to come to the surface when either an error corrupts the image
466
+
information, or the encoding/decoding process malfunctions.
467
+
468
+
16
469
+
470
+
A Vernacular Of File Formats15
471
+
The only resolution to the problem of non-communication was to incorporate it
472
+
within the system.16
473
+
- Friedrich Kittler
474
+
475
+
A Photoshop RAW image (.RAW). | This is the original, uncompressed (RAW) source image, which I will glitch
476
+
throughout this chapter, in order to outline a condensed version of my Vernacular Of File Formats. The image is a
477
+
video still, which is why it involves (nearly invisible) scan lines. These lines are from the original video.
478
+
479
+
Raw image files contain minimally processed data (pixels) from the image sensor of, for
480
+
instance, a digital camera or image scanner. The file header of a RAW image typically contains information concerning the byte-ordering of the file; the camera sensor information
481
+
and other image metadata like exposure settings; the camera, scanner or lens model; the
482
+
date (and, optionally, place) of shooting or scanning; the format, size and number of colors; as well as other information needed to display the image.
483
+
484
+
15 | This section is in fact a condensed adaptation of my 2010 artwork, Vernacular of File Formats.
485
+
16 | Friedrich Kittler, Draculas Vermächtnis: Technische Schriften, Leipzig: Reclam Verlag Leipzig, 1993. p. 242.
486
+
487
+
17
488
+
489
+
A databent Photoshop RAW (h=0) image (.RAW). | This image was constructed by opening the original three channel
490
+
interleaved RAW image as a single channel (single color/greyscale) interleaved document. The image generated is
491
+
a reversible databend. This image is uncompressed.
492
+
493
+
It is possible to save a RAW image file without a header (when you open the image in
494
+
Photoshop, for example, you can choose header=0 in a pop-up box). When the RAW image is saved without a header the computer doesn’t know the dimensions or any other
495
+
crucial information that is needed to reconstruct the image out of the image data. This
496
+
opens up creative possibilities. It is for instance possible to input new dimensions for the
497
+
image, change the amount of color channels or choose whether or not the image will be
498
+
displayed as ‘interleaved’ or ‘non-interleaved’. In the case of a RAW image file, interleaving and non-interleaving refer to the order in which the RGB (Red, Green and Blue) color
499
+
values of every pixel are stored. In an interleaved RAW image, the data is stored in a RGBRGBRGB sequence. When the image is saved in non-interleaved order, the RGB values
500
+
are not ordered sequentially but have their own ‘layers’. By deviating from the values of the
501
+
originally recorded image, the image can be displayed in a distorted way and the structure
502
+
of the file becomes visible.
503
+
504
+
18
505
+
506
+
A Databent Bitmap image (.BMP). | This BMP image was databent by copy-pasting a selection of random image data
507
+
over and over into the original file. The image generated is an irreversible databend. This image is uncompressed.
508
+
509
+
The BMP file format is uncompressed. Every bit that indexes a bitmap pixel value is
510
+
packed within a linear row and processed in a reversed order to the normal image raster
511
+
scan order, starting in the lower right corner, advancing row by row from the bottom to the
512
+
top. This is why, when you copy-paste just some parts of the image data, the lower part of
513
+
the image data and the image itself will remain intact, while the upper part of the image
514
+
shifts horizontally.
515
+
In BMP files, and many other bitmap file formats, the color palette consists of a block of
516
+
bytes (a table or palette) listing the colors available for use in a particular indexed-color
517
+
image. Each pixel in the image is described by a number of bits (1-32 bit color depth) that
518
+
index a single color from the color palette, which is described right after the header. The
519
+
BMP color palette uses the interleaved RGB color model. In this model, a color depends on
520
+
different intensities (from 0 to 255) of the primary RGB colors. A color is thus defined by
521
+
the final intensities of R+G+B. When you copy-paste the image data, certain shifts within
522
+
the RGB values may take place; the intensity of the data from B can (for instance) shift to
523
+
R, creating sudden discolored blocks.
524
+
525
+
19
526
+
527
+
528
+
Graphics Interchange Format images (.GIF). | Left: A GIF image featuring truncation (involving quantization
529
+
error). Right: A GIF image in which quantization error is minimized through the use of dither artifacts. The GIF
530
+
compression is lossless.
531
+
532
+
The Graphics Interchange Format is a bitmap image format that supports 8 bits per pixel.
533
+
This compression can therefore consist of no more then 256 colors. The format supports
534
+
animation and employs dither (a grain or block artifact), which can be intentionally applied as a form of noise to ‘randomize quantization error(s)’. Quantization refers to the
535
+
procedure of constraining information from a relatively large or continuous set of values
536
+
(such as real numbers) to a relatively small discrete set difference between the actual analog value and quantized digital value of color. ‘Quantization error’ is thus an error often
537
+
impacting upon color, caused by truncation (the discarding of less significant color information).’
538
+
Dither helps to prevent images from displaying or transforming into large-scale patterns
539
+
such as ‘banding’ (a stepped process of rendering smooth gradations in brightness or
540
+
hue). Moreover, because the human eye perceives the diffusion caused by dither as a
541
+
mixture of the colors, unavailable (cut out or uncodable) colors are approximated. This
542
+
creates the illusion of color depth.
543
+
544
+
20
545
+
546
+
Graphics Interchange Format images (.GIF).
547
+
This interlaced GIF image was databent by the introduction of a random error to the information data. The image
548
+
generated is an irreversible databend.
549
+
550
+
The gif format uses a four pass dimensional interlacing strategy. This basically means
551
+
that the image, consisting of different rows of pixels, decodes some rows of pixels before
552
+
other rows. The example image shows the displacement of the different rows during weaving (the putting together of the two layers), resulting in ‘combing artifacts’ with ‘jagged
553
+
edges’.
554
+
555
+
21
556
+
557
+
A databent progressive Joint Photographic Experts Group image (.JPEG). This baseline JPEG image was databent by
558
+
the introduction of a random error to the information data. The image generated is an irreversible databend. The
559
+
JPEG compression is lossy.
560
+
561
+
A JPEG compression consists of 6 subsequent steps:
562
+
1. C
563
+
olor space transformation. Initially, images have to be transformed from the RGB color space to another color space (called Y´CbCr), that consists of three components that
564
+
are handled separately: the Y (luma or brightness) and the Cb and Cr values (chroma or
565
+
color values, which are divided into hue and saturation).
566
+
2. D
567
+
ownsampling. Because the human eye doesn’t perceive small differences within the
568
+
Cb and Cr space very well, these elements are ‘downsampled’ (their information is reduced).
569
+
3. Blocksplitting. After the color space transformation, the image is split into tiles or
570
+
‘macroblocks’, which are rectangular regions of the image that are transformed and
571
+
encoded separately.
572
+
573
+
22
574
+
575
+
An 8 × 8 DCT basis function of a JPEG with differentiated macroblocks (.JPEG).
576
+
577
+
4. Discrete Cosine Transform. Next, a discrete cosine transform (which works similar
578
+
to the Fourier Transform function exploited in ‘datamoshing’ and ‘macroblock’ experiments to which I will later refer) is used to create a frequency spectrum, to transform the
579
+
8×8 blocks to a combination of 64 two-dimensional DCT basis functions or patterns (as
580
+
mapped out by the lines).
581
+
5. Quantization. During the quantization step, the highest brightness-frequency variations become a base line (or 0-value), while small positive and negative frequency differentiations are given a value that starts from this baseline, which takes many fewer bits to
582
+
represent.
583
+
6. Entropy coding. Finally, entropy coding is applied. Entropy coding is a special form
584
+
of lossless data compression that involves arranging the image components in a ‘zigzag’
585
+
order. This allows the quantized coefficient table to be rewritten in a zigzag order to a sequence of frequencies. A ‘run-length encoding’ (RLE) algorithm groups similar frequencies together and after that, via ‘Huffman coding’ organizes what is left.
586
+
23
587
+
588
+
A databent baseline Joint Photographic Experts Group image (.JPEG).
589
+
This baseline JPEG image was databent by the introduction of a random error to the information data. The image
590
+
generated is an irreversible databend. The JPEG compression is lossy.
591
+
592
+
Because the RGB color values of JPEG images are described in such a complex algorithm,
593
+
random data replacements can also result into dramatic discoloration and displacement.
594
+
The very high compression ratio of a JPEG affects the quality of the image and the size of
595
+
the artifacts. When using quantization with block-based coding, as I have done in these
596
+
JPEG-compressed images, several types of artifacts can appear, for instance ‘ringing’ or
597
+
‘ghosting’, ‘blocking’ and ‘jaggies’. The image shows ‘blocking’ or ‘staircase’ artifacts appearing most clearly along the curving edges as a result of the 8×8 JPEG blocks. Blockiness
598
+
in ‘busy’ regions is sometimes also referred to as ‘quilting’ or ‘checkerboarding’. ‘Jaggies’
599
+
is the informal name for artifacts in raster images. They are often the result of poor aliasing, which happens when a JPEG signal reconstruction after downsampling has produced
600
+
only high frequency outcomes.
601
+
602
+
24
603
+
604
+
A databent Joint Photographic Experts Group Committee 2000 image (.JPF). | This JPEG 2000 image was databent by
605
+
the introduction of a random error to the information data. The image generated is an irreversible databend. The
606
+
JPEG 2000 compression is lossy.
607
+
608
+
The JPEG 2000 standard is a compression standard developed especially for the medical
609
+
imaging industry because of the many edge and blocking artifacts possible for the JPEG
610
+
format, which can cause catastrophic misreadings in medical pictures. JPEG 2000 has improved scalability and edit-ability. In JPEG 2000, after the color transformation step, the
611
+
image is split into so-called tiles, rectangular regions of the image that are transformed
612
+
and encoded separately. Tiles can be any size and it is also possible to consider the whole
613
+
image as one single tile. This tiling process turns the image into a collection of ‘sub-bands’,
614
+
which represent several approximation scales. A sub-band is a set of coefficients that represent aspects of the image associated with a certain frequency range as well as a spatial
615
+
area of the image. The quantized sub-bands are split further into ‘precincts’; rectangular
616
+
regions in the wavelet domain. A ‘wavelet’ is a wave-like oscillation with amplitude that
617
+
starts out at zero, increases, and then decreases back to zero. It can typically be visualized
618
+
as a ‘brief oscillation’ like one might see recorded by a seismograph or heart monitor. Precincts are split further into code blocks, which are located in a single sub-band and have
619
+
equal sizes. The chrominance components (of JPEG 2000) can be (but do not necessarily
620
+
need to be) downscaled in resolution; in fact, since the wavelet transformation already
621
+
separates images into scales, downscaling or downsampling is more effectively handled
622
+
by dropping the finest wavelet scale.
623
+
25
624
+
625
+
Orderly Chaos: Feedback Artifacts
626
+
Feedback, a category of noise, is a circular process in which a part of the process’s output
627
+
is returned (fed back) to the input, influencing the future behaviour of the process. The
628
+
addition of feedback changes the communication model considerably, from a linear to a
629
+
non-linear model of transmission, which also opens the model up to new forms of noise.
630
+
Feedback then is both an artifact within digital technologies and a generative quality
631
+
available in many communication media.
632
+
Norbert Wiener established the foundations of cybernetics (in 1948) by describing the
633
+
principle of cybernetic feedback. He wrote that in a feedback-oriented system, the factual
634
+
outcome has to be compared with the intended outcome. If the intended condition is not
635
+
reached, it is pursued through feedback, either by increasing or decreasing the difference.
636
+
An example of such a system is for instance a thermostat, but more complex examples
637
+
include signal (for instance audio) equalizers or the stock market.17
638
+
639
+
The Other Noise Artifact: Glitch
640
+
Malfunction and failure are not signs of improper production. On the contrary,
641
+
they indicate the active production of the "accidental potential" in any product.
642
+
The invention of the ship implies its wreckage, the steam engine and the locomotive discover the derailment.18
643
+
- Paul Virilio
644
+
645
+
When the cause of a noise artifact is known, the artifact is often not referred to as noise, but
646
+
instead is named after its technical cause, for instance as a compression artifact (jaggies,
647
+
macroblocking, checkerboarding) or feedback artifact. The difference between compression and feedback artifacts is thus not always strict. When the source of the noise artifact
648
+
is not (yet) known, the noise becomes puzzling. In the digital realm, this kind of noise is
649
+
often referred to as ‘glitch’. Glitch, an unexpected occurrence, unintended result, or break
650
+
or disruption in a system, cannot be singularly codified, which is precisely its conceptual
651
+
strength and dynamical contribution to media theory. From an informational (or technological) perspective, the glitch is best considered as a break from (one of) the protocolized
652
+
data flows within a technological system. According to The American Heritage® Dictionary
653
+
of the English Language, the word glitch was first recorded in English in 1962 in the writings
654
+
of astronaut John Glenn, who describes glitch as a term adopted by his team ‘to describe
655
+
some of our problems’. Glenn gives the technical sense of the word that the astronauts had
656
+
adopted, as ‘literally… a spike or change in voltage in an electrical current’ only to note how
657
+
that more specific technical definition was soon applied to a range of ‘other, not-yet-specified’ problems. Close to the moment of its inception then, ‘glitch’ already passes beyond
658
+
specific technical use to describe a wide variety of malfunctions and mishaps.19
659
+
660
+
17 | Norbert Wiener, Cybernetics: Or Control and Communication in the Animal and the Machine, Paris: Hermann & Cie & Camb, 1948.
661
+
18 | Sylvere Lotringer and Paul Virilio, The Accident of Art, Semiotext(e): New York, 2005. p. 2.
662
+
19 | The American Heritage® Dictionary of the English Language, Fourth Edition, Houghton Mifflin Company, 2004. http://dictionary.reference.
663
+
com/browse/glitch.
664
+
665
+
26
666
+
667
+
The result of a glitch can range significantly, from a catastrophe to just a minor hiccup
668
+
or slip. In the case of minor glitches, the informational inputs, encoding or decoding or
669
+
other technological protocols are revealed to be at some point ‘erroneous’, while the rest
670
+
of the system or the parts processing the data flows within that system (the hardware, software or for instance the monitor interface) continue to function, and display the errorridden output unscrutinized. When the glitch is more disastrous, the system might not
671
+
function properly at all, or ever return to its normal mode of operation.
672
+
From a media culture perspective then, the term glitch refers to a not yet defined break
673
+
from a procedural flow, fostering a critical potential. Here I use ‘procedural’ as a moniker
674
+
from ‘procedural programming’ (or ‘imperative programming’), to reference series' of
675
+
computational steps that must be carried out in order for a program to reach a desired
676
+
state. Once a procedural flow is broken, there are two possible ways in which the glitch
677
+
tends to move. If the cause of the machine’s erratic behavior becomes known, the glitch
678
+
tips and becomes a simple bug report of a failure, in which it will be described under its
679
+
technological name (which at that point is often a compression artifact). However, if the
680
+
cause of the glitch remains unknown, the glitch can either be ignored and forgotten, or
681
+
transformed into an interpretation or reflection on a phenomenon (or the memory thereof) defined by a social or cultural context (conventions, histories, perspectives) and the
682
+
technology that is malfunctioning. In short, failure is a phenomenon to overcome, while a
683
+
glitch is incorporated further into technological or interpretive processes. Accordingly, when
684
+
the glitch opens up to the realm of symbolic or metaphorical connotations, the interruption shifts from being a strictly informational or technological actuality, into a more complex post-procedural phenomenon to be reckoned with.
685
+
686
+
27
687
+
688
+
The Perception Of Glitch
689
+
Our taverns and our metropolitan streets, our offices and furnished rooms, our
690
+
railroad stations and our factories appeared to have us locked up hopelessly.
691
+
Then came the film and burst this prison-world asunder by the dynamite of the
692
+
tenth of the second, so that now, in the midst of its far-flung ruins and debris, we
693
+
calmly and adventurously go traveling.01
694
+
- Walter Benjamin
695
+
696
+
The Meaning Of Noise
697
+
To develop a categorization of noise for contemporary audio-visual media theory, I have
698
+
used Claude Shannon’s mathematical theory of communication. In his definition of informational noise, Shannon conveniently focused on the transfer of information between
699
+
machines, leaving human elements and context out of the equation. Drawing on Shannon’s model, I was able to divide digital noise into three basic categories of noise artifacts:
700
+
encoding/decoding artifacts (which are most often referred to as compression artifacts),
701
+
feedback artifacts and the ‘other’ corruptions known as glitch artifacts – artifacts for which
702
+
the causes are not (yet) known. It is important to realize that the difference between each
703
+
of these artifacts is not rigid, as the description of a glitch artifact can be understood as
704
+
a de/compression or feedback artifact (and visa versa), depending on the viewer’s knowledge of the technology. In the context of human-computer communication, I also deviate
705
+
from Shannon and Weaver and believe that the concept of noise becomes more complex
706
+
as it connotes meaning and translation. Consequently, human-computer definitions of
707
+
noise must also include social parameters and become more complex, inevitably negotiating questions of context, perception and aesthetics.
708
+
The etymological definition of noise refers to states of aggression, alarm and powerful
709
+
sound phenomena in nature (‘rauschen’)02. When the concept of noise is approached
710
+
within a social context, noise does not exist independently, but only in relation to what
711
+
it is not. However complex or inclusive noise appears as a signifier, it is always a kind
712
+
of negativity: it stands for unaccepted sound, not music, invalid information or the absence of a message. Noise is unwanted, other and unordered. Accordingly, there is also
713
+
no unequivocal cultural definition of noise, because in the end, what noise is and what
714
+
noise is not, is a social matter. As James Brady Cranfield-Rose writes, ‘noise is a “cipher”,
715
+
a question mark, forever eluding fixed definitions’.03 Furthermore, whichever way noise
716
+
is defined, its negative orientation also has positive, critical dimensions. Noise tends to
717
+
reflexively stage a reconsideration or re-view its opposite – the world of meaning, norms
718
+
and regulations, goodness, or beauty.04
719
+
720
+
01 | Walter Benjamin, ‘The Work of Art in the Age of Mechanical Reproduction’, in Hannah Arendt (ed.) Illuminations, New York: Schocken, 1968,
721
+
pp. 219-254. p. 236.
722
+
02 |Torben Sangild, The Aesthetics of Noise, Copenhagen: Datanom, 2002. www.ubu.com/papers/noise. p. 5-8.
723
+
03 | James Brady Cranfield-Rose, Tick-tick-tick-tick-tick… Oval, the glitch and the utopian politics of noise, unpublished master thesis, Burnaby,
724
+
Canada: Simon Fraser University, 2004. p. 13, http://lib-ir.lib.sfu.ca/handle/1892/8961.
725
+
04 | Paul Hegarty, Noise/Music: A History, London and New York: Continuum, 2007. p. 5.
726
+
727
+
28
728
+
729
+
The Glitch Moment(um): A Void In Techno-Culture
730
+
Noise aesthetics pose both a technological and perceptual challenge to habitual or ideological conventions. While media developers design their technologies in order that the user will
731
+
forget about the presence of the medium, following the ideal logic of transparent immediacy,
732
+
in reality, the complexity of the user’s inherently aesthetic and perceptual responses to the
733
+
human computer interface requires a more nuanced approach. As Ernst Gombrich declared:
734
+
‘However we analyse the difference between the regular and the irregular, we must ultimately
735
+
be able to account for the most basic fact of aesthetic experience, the fact that delight lies
736
+
somewhere between boredom and confusion’.05 Situations of either extreme immediacy or
737
+
extreme reliability do not contribute as might be expected to the actual richness of a media
738
+
experience. Most people need some kind of interplay between surprise and uniformity to
739
+
keep them actively involved.06 Expanding on this important, indeed integral role of irregularity and surprise in human perception, Gombrich quotes Adelbert Ames, Jr, who explains:
740
+
the organism is continually comparing the prognosis of the continually changing
741
+
new external events with his determined frame of significance. If they conform, i.e.
742
+
‘work’, he is no longer interested; but in so far as they do not, he has to take stock of
743
+
the situation. There are three possibilities – either his frame of significances may
744
+
be wrong, or his immediate sense response may be wrong, or both. In any case, he
745
+
has a problem to solve.07
746
+
The first encounter with a glitch comes hand in hand with a feeling of shock, with being lost and in awe. The glitch is a powerful interruption that shifts an object away from
747
+
its flow and ordinary discourse, towards the ruins of destructed meaning. This concept
748
+
of flow I emphasize as both a trait within the machine as well as a feature of society as
749
+
a whole. DeLanda distinguishes between chaotic disconnected flows and stable flows of
750
+
matter that move in continuous variations, conveying singularities.08 DeLanda draws here
751
+
on Deleuze and Guattari, who describe flow in terms of the beliefs and desires that both
752
+
stimulate and maintain society. They write that a flow is something that comes into existence over long periods of time. Within these periods, conventions are established, while
753
+
deviations tend to become rare occurrences and are often (mis)understood as accidents
754
+
(or glitches). Although meaningful aspects of every day life might in fact be disclosed
755
+
within these rare fluctuations, their impact or relevance is often likely to be ruled out,
756
+
because of social tendencies to put emphasis on the norm.09
757
+
A glitch is the most puzzling, difficult to define and enchanting noise artifact; it reveals itself
758
+
to perception as accident, chaos or laceration and gives a glimpse into normally obfuscated
759
+
760
+
05 | Ernst Hans Josef Gombrich, The Sense of Order: A Study in the Psychology of Decorative Art, London: Phaidon Press, 1984. p. 9.
761
+
06 | Robert Pepperell, ‘Computer aided creativity: practical experience and theoretical concerns’, in Proceedings of the 4th conference on Creativity & cognition, Loughborough, UK: ACM, 2002. pp. 50-56, http://portal.acm.org/citation.cfm?id=581710.581720&type=series.
762
+
07 | Ames, Jr. Adelbert, ‘The morning Notes’, in Ernst Hans Josef Gombrich, The Sense of Order: A Study in the Psychology of Decorative Art,
763
+
London: Phaidon Press, 1984. p. 117.
764
+
08 | Manuel DeLanda, War in the Age of Intelligent Machines, New York: Zone Books, 1991. p. 20.
765
+
09 | Gilles Deleuze and Pierre-Félix Guattari, A Thousand Plateaus: Capitalism and Schizophrenia, Trans. B. Massumi, Londen: The Athlone
766
+
Press, 1988. p. 219.
767
+
768
+
29
769
+
770
+
machine language. Rather than creating the illusion of a transparent, well-working interface to information, the glitch captures the machine revealing itself. Television is arguably
771
+
one of the more flow-centric, ideologically ‘transparent’ media forms. In Television: Technology and Cultural Form (1974), Williams describes a viewer frequently caught up in a flow of
772
+
technology and its contents. He emphasizes that the process of this flow seems natural, but
773
+
is in fact strictly guided by larger corporations and powers. When a (televisual) flow breaks,
774
+
the user comes to witness only shreds of the flow through which the message is normally
775
+
transmitted, while the machinic functions that are conventionally relied upon – as obfuscated – are revealed.10 When a supposedly transparent interface is damaged in this way, the
776
+
viewer is momentarily relocated to a void of meaning. Interruptions like these are often
777
+
perceived as disastrous, threatening and uncanny. Sometimes they create a moment where
778
+
seemingly any sense that could be made of a situation is eliminated from thought or possibility. On other occasions, the metaphorical impact of the unspeakable mediatic disaster
779
+
also brings with it the tendency to reflect (on for instance what the differentiation from the
780
+
flow means). Eric Kluitenberg describes how this was the case on September 11, 2001, when
781
+
the CNN website temporarily went down and a black screen repeatedly interrupted the flow
782
+
of the television broadcast. He refers to these moments in time as
783
+
the rupture of professional media codes, which signaled complete panic and disarray […], the infinity of possible alternative discourses, of other possible modes of
784
+
explanation and interpretation.11
785
+
What is challenged or brought forward in the case of the void is the idea of authorship
786
+
itself, which, prior to this supposedly voiding moment, was in fact neutralized from
787
+
media-cultural experience. It is possible to realize at this point – and only belatedly – that
788
+
the conventions of ‘the seamless surface of the networked media spectacle itself, and its
789
+
illusion of stability’12 tend to foreclose any sense of authorship whatsoever. In media accidents like these, the void involves the unknown – that which cannot be described or
790
+
planned for. These empty spaces of non-understanding trigger a horror vacui: a fear of
791
+
voids to which nothing else can be compared and that is beyond all possibilities of calculation, measurement or imitation.13 However, these terrifying voids also create a form of
792
+
counter-experience, a negative pleasure that is not so different from the proto-modern,
793
+
aesthetic conception of the sublime (described as early as 1693 in John Dennis’s writings
794
+
on the Alps), as contradictory and immense ‘delight that is consistent with reason’ but yet,
795
+
‘mingled with Horrors, and sometimes almost with despair’.14
796
+
Like in this ‘nature’-generated sublime, the glitch is an uncanny or overwhelming experience of unforeseen incomprehension. Experiencing a glitch is often like perceiving a stun-
797
+
798
+
10 | Raymond Williams, Television: Technology and Cultural Form, Hanover: University Press of New England, 1974.
799
+
11 | Eric Kluitenberg, Delusive Spaces. Essays on Culture, Media and Technology, Rotterdam: NAi Publishers and Amsterdam: Institute of
800
+
Network Cultures, 2008. p. 357.
801
+
12 | Eric Kluitenberg, Transfiguration of the Avant-Garde/The Negative Dialectics of the Net, posting to nettime mailing list, 23 January, 2002,
802
+
http://www.nettime.org/Lists-Archives/nettime-l-0201/msg00104.html.
803
+
13 | Eric Kluitenberg, Delusive Spaces. Essays on Culture, Media and Technology, Rotterdam: NAi Publishers and Amsterdam: Institute of
804
+
Network Cultures, 2008. p. 333.
805
+
14 | Jeffrey Barnouw, ‘The Morality of the Sublime: To John Dennis’, Comparative Literature, Vol. 35, No. 1 (Winter, 1983): p. 21-42.
806
+
807
+
30
808
+
809
+
ningly beautiful, brightly colored complex landscape of unexplainable, unfathomable and
810
+
otherworldly images and data structures. A glitch represents a loss of control. The ‘world’ or
811
+
the interface does the unexpected. It goes beyond the borders of its known and programmed
812
+
territories, changing viewers' assumptions about technology and its assumed functions
813
+
(as was for instance the case during the September 11 broadcast), and comes to seem profoundly irrational in its ‘behavior’. The glitch makes the computer itself suddenly appear
814
+
unconventionally deep, in contrast to the more banal, predictable surface-level behaviors of
815
+
‘normal’ machines and systems. In this way, glitches announce a crazy and dangerous kind
816
+
of moment(um) instantiated and dictated by the machine itself.
817
+
The concept of moment(um) is twofold: first of all there is the moment, which is experienced as the uncanny, threatening loss of control, throwing the spectator into the void (of
818
+
meaning). This moment then itself becomes a catalyst, with a certain momentum. Noise
819
+
turns to glitch when it passes a momentary tipping point, at which it could tip away into
820
+
a failure, or instead force new knowledge about the glitch’s techné, and actual and presumed media flows, onto the viewer.
821
+
Through the distorted images and behaviors of machinic outputs, the viewer is thrown
822
+
into a more risky realm of image and non-image, meaning and non-meaning, truth and
823
+
interpretation. The machine no longer behaves in the way the technology was supposed
824
+
to. Its glitching interface, strange sounds and broken behavioral patterns introduce tension into user intentions; an astonishing image (or sound) must be somehow negotiated
825
+
amidst a normally much more boring masquerade of human computer relations. Though
826
+
at first the viewer reacts with shock and perceives the experience as a loss, the glitch cannot be subdued as a solid state of perception. Just as the understanding of a glitch changes
827
+
once it is named, so does the notion of transparency or systemic equilibrium supposedly
828
+
damaged by the glitch itself. The ‘original’ experience of rupture is moved beyond its sublime moment(um) and vanishes into a realm of new conditions. The glitch has become
829
+
a new mode; and its previous uncanny encounter has come to register as an ephemeral,
830
+
personal experience of a machine.
831
+
832
+
Technorealism And the Accident Of Art
833
+
I can no longer use the figure without destroying it, so I’d rather be abstract.15
834
+
- Mark Rothko
835
+
836
+
Notions of disaster, aesthetics of failure and accidental events have been integral to modern
837
+
and contemporary art, Avant-Garde progressions and turnings. With the growing importance of technology, especially so in the modern century, it is the accident that becomes
838
+
immanent to culture, as Virilio has emphasized most strongly among media theorists:
839
+
To invent the sailing ship or steamer is to invent the shipwreck. To invent the train
840
+
is to invent the rail accident of derailment. To invent the family automobile is to
841
+
produce the pile-up on the highway. To get what is heavier than air to take off in the
842
+
843
+
15 | Sylvere Lotringer and Paul Virilio, The Accident of Art, Semiotext(e): New York, 2005. p. 22.
844
+
845
+
31
846
+
847
+
form of an aeroplane or dirigible is to invent the crash, the air disaster. As for the
848
+
space shuttle, Challenger, its blowing up in flight in the same year that the tragedy
849
+
of Chernobyl occurred is the original accident of a new motor, the equivalent of the
850
+
first ship-wreck of the very first ship.16
851
+
In correlation with Gombrich, Virilio argues that although many people encounter accidents as negative experiences, an accident can also have positive consequences. The
852
+
accident doesn’t only equal failure, but can also ‘reveal something absolutely necessary
853
+
to knowledge’.17 To Virillio, the accident resides beyond the classical opposition of functional versus dysfunctional. In the introduction to the Deaf ’98 festival Reader, which was
854
+
largely dedicated to Virilio’s theories on the accident, the accident is even described as
855
+
hyper-functional. The accident (and thus the glitch) shows a system in a state of entropy
856
+
and so aids towards an understanding of the ultimate functioning of a system. This opens
857
+
up space for research and practice, and the arts are a special domain for this.18
858
+
In The Accident of Art (2005), Virilio argued that art itself has been terrorized by the last
859
+
century; it has been devastated consecutively by the two World Wars, the Holocaust and
860
+
nuclear power. Dadaists and Surrealists cannot be understood without World War 1; they
861
+
are its casualties, the ‘broken faces’ or war victims that used automatic writing as their
862
+
machine-gun.19 Virilio explains how WW1 blew reality into pieces and how the cubist
863
+
painter Georges Braque collected those pieces and put them back together, not just as a
864
+
formalist experiment or as a destruction of perspective but as an artistic realism appropriate to the techno-cultural present. For Virilio, while figurative work retreats, this category
865
+
of Abstract art is ‘not really abstract’.20 Because the war disfigured, destroyed and mutilated reality, as much as it did human bodies and outdoor spaces, realist conventions (formerly/formally understood) were no longer reproducible. Thus, many artists could only
866
+
use some (destroyed or mutilated) form of figuration. This understanding leads Virilio to
867
+
conclude that in the art of the accident, there should be a differentiation between nonfigurative and disfigured art.21 Such a ‘formal’ comprehension of technological realisms
868
+
makes for all kinds of disaster or accident related art. In the digital realm, what has come
869
+
to be known as glitch art deals with the digital dimension of error, accident and disaster
870
+
from different angles, within a larger context of cultural meaning.
871
+
872
+
16 | Paul Virilio and Julie Rose, The Original Accident, Cambridge: Polity Press, 2007. p. 10.
873
+
17 | Sylvere Lotringer and Paul Virilio, The Accident of Art, Semiotext(e): New York, 2005. p. 63.
874
+
18 | Andreas Broeckmann, Joke Brouwer, Bart Lootsma, Arjen Mulder and Lars Spuybroek, The Art of the Accident, NAI Publishers/V2_Organisatie: Rotterdam, 1998. p. 3.
875
+
19 | Andreas Broeckmann, Joke Brouwer, Bart Lootsma, Arjen Mulder and Lars Spuybroek, The Art of the Accident, NAI Publishers/V2_Organisatie: Rotterdam, 1998. p. 3.
876
+
20 | Sylvere Lotringer and Paul Virilio, The Accident of Art, Semiotext(e): New York, 2005. p. 19-21.
877
+
21 | Sylvere Lotringer and Paul Virilio, The Accident of Art, Semiotext(e): New York, 2005. p. 19-21.
878
+
879
+
32
880
+
881
+
A Phenomenology Of Glitch Art
882
+
“Failure” has become a prominent aesthetic in many of the arts in the late 20th century, reminding us that our control of technology is an illusion, and revealing digital tools to be only as perfect, precise, and efficient as the humans who build them.01
883
+
- Kim Cascone
884
+
885
+
The Predicaments Of Defining Glitch Art
886
+
Artists often find themselves on a frontline, reflecting on the cultures, politics and technologies of their time. Over the last decades, audiovisual media and computers have
887
+
gradually gained more and more importance in an art field that is still fundamentally
888
+
ruled by classical media forms and genres. Noise itself is of course not new; similarly,
889
+
contemporary glitch art relates to a long history of noise art and artists battling in different ways against media forms and their flows and conventions, including especially what
890
+
I have outlined as the convention of transparent immediacy.
891
+
While not being new, noise art arises unpredictably in new forms across different technologies and cultural scenes. Over time, noise artists have migrated from exploring the
892
+
grain, the scratching and burning of celluloid (for example, a colour box by Len Lye,
893
+
1937) to the magnetic distortion and scanning lines of the cathode ray tube (a significant
894
+
work being Nam June Paik in magnetTV in 1965). Subsequently, glitch artists wandered
895
+
the planes of phosphor burn-in, as Cory Arcangel did in panasonic TH-42PWD8UK plasma screen burn, in 2007. With the arrival of LCD (liquid crystal display) technologies,
896
+
dead pixels were rubbed, bugs were trapped between liquid crystals or plastic displays
897
+
and violent screen cracking LCD performances took place (of which my favorite is %SCR2,
898
+
by Jodi, under the Pseudonym webcrash2800 in 2009).
899
+
To some artists, myself included, it has become a personal matter to break the assured
900
+
informatic flows of media. While normally, transparent media screens generate conventional impressions of immediacy, there is a desire to force the viewer to think beyond
901
+
his comfort zones. Glitch artists make use of the accident to ‘disfigure’ flow, image and
902
+
information, or they exploit the void – a lack of information that creates space for deciphering or interpreting the process of creating (new kinds of) meaning. Through these
903
+
tactics, glitch artists reveal the machine’s techné and enable critical sensory experience
904
+
to take place around materials, ideologies and (aesthetic) structures. Their destructive or
905
+
disfiguring processes have no technological name, definition or explanation (yet). For this
906
+
reason, it is necessary to not only define and categorize glitch at technological levels, but
907
+
also to look closely at how specific media are exploited on a more complex techno-cultural
908
+
level. The artists I discuss here include Ant Scott, 5VOLTCORE Gijs Gieskes and Jodi. Of
909
+
course many other artists whose practices are invested in the moment(um) or culture of
910
+
glitch could have been included here. An actual historiography would for instance also
911
+
include signal processing artists like Karl Klomp, Lovid, Morgan Higby-Flowers and Max
912
+
Capacity, aesthetic glitch-tricksters like Jon Satrom, jonCates, fabric artist Melissa Baron,
913
+
and databend generative artists such as stAllio!, glitch-irion Pixelnoizz and Hellocatfood.
914
+
This historiography is still unwritten (partly because it is still in progress).
915
+
01 | Kim Cascone, ‘The Aesthetics of Failure: Post-Digital Tendencies in Contemporary Computer Music’, Computer Music Journal 24.4 (Winter
916
+
2000): p. 13.
917
+
918
+
33
919
+
920
+
As is clear by now, the inherent openness of glitch as a concept makes glitch art difficult,
921
+
if not impossible, to define. Although a glitch can take place strictly within the computational system, the majority of artifacts that are called or referred to as glitches within
922
+
glitch art are not purely informational, but make sense only through a synthesis of agents
923
+
and contexts involved. Glitch is post-procedural (a break from a procedural flow) and so,
924
+
dialectically connects to, while departing from, a linear and informational model of media communication (‘information source-> encoder-> channel-> decoder-> destination’),
925
+
while also incorporating contextual and social processes of interpretation and making
926
+
meaning. Furthermore, it is necessary to recall that the word ‘glitch’ in ‘glitch art’ is often
927
+
used as a metaphorical concept, even by glitch artists, and therefore varies from the standalone technical or informational term ‘glitch’.
928
+
929
+
Ant Scott. SUQQE. Digital screenshot. 2002.
930
+
931
+
The complexities that must be faced by a theorist or researcher when trying to define or
932
+
demarcate some kind of ‘essence of glitch art’ (if this is even possible) come to the foreground upon close engagement with Ant Scott’s (Beflix) work. For years, Ant Scott has been
933
+
a leading figure in the realm of glitch art. From 2001 until 2005 he published hundreds of
934
+
glitch images – static and animated – on his blog, appearing here as the first glitch artist
935
+
actually using the term ‘glitch art’ for his work. These images don’t have a common source;
936
+
further, some of them are ‘found’ glitch artifacts turned into or framed as art, while others are intentionally made from scratch by the artist. Ant Scott describes his series glitch
937
+
(2007), a collection of 25 ‘works’ (small digital renders of lo-fi captured glitches) accessible
938
+
via his home page, as the best of his ‘pure glitch’ phase. The images, which at first might
939
+
appear bewildering, are actually created from computer crashes, software errors, hacked
940
+
games, and megabytes of raw data turned into colored pixels.02 They originate or are con-
941
+
942
+
02 | Ant Scott, GLITCH #12, GLITCH ART, 2007, http://www.Ant Scott.com/works/glitch.php?id=12.
943
+
944
+
34
945
+
946
+
structed from thorough trial and error processes, to which Scott carefully reassigns colours,
947
+
and crops select areas of interest. The result is the works that make up the glitch series.
948
+
Ant Scott’s working process presents all kinds of dilemmas in the quest for a definition
949
+
and categorisation of glitch art. What kind of ‘glitch’ is this ‘glitch art’ exploring? How
950
+
can the glitch be explained as an unexpected, abnormal mode of operation, when the artist’s working process and what he aims for are these abnormalities to begin with? Can the
951
+
intended error be really described as erroneous? On the other hand, Scott’s wide-ranging
952
+
interrogation of glitch aligns with other aspects of glitch that I have outlined. A glitch can
953
+
indeed exist within and across different systems, for instance the system of production
954
+
and the system of reception. Similarly, a glitch can depend on different actors within
955
+
these systems; not just the technological elements that Shannon described, but also the
956
+
ideological and cultural contexts of the technology, which brings aspects of time, place
957
+
and structure (aesthetics) into the art work, all of which differ between different publics,
958
+
involved in the process of making meaning. Despite glitch art having no solid, or single
959
+
definition through time and place, just as Virilio argued that it is helpful to describe a difference between non-figurative and disfigured art, I believe it is useful to make a similar
960
+
distinction between different dimensions of ‘glitch’ in ‘glitch art’. Glitch art then potentially incorporates a range of works that are post-procedural, deconstructive, accidental
961
+
and so on, alongside works more focussed on a final end-product, aesthetic or design.
962
+
963
+
Categorical Precursors:
964
+
A Binary Approach To Glitch Art?
965
+
The post-procedural essence of glitch art is opposed to conservation; the shocking perception and understanding of what a glitch is at one point in time cannot be preserved
966
+
for a future time. The artist tries to somehow demonstrably grasp something that is by
967
+
nature unstable and ungraspable. Their commitments are to an unconventional utopia
968
+
of randomness, chance and idyllic disintegrations that are potentially critical. The core
969
+
of a work of glitch art is therefore best understood as the momentary culmination of a
970
+
history of technological and cultural movements, and as the articulation of an attitude
971
+
of destructive generativity. In short, glitch art practices are invested in processes of nonconforming, ambiguous re-formations.
972
+
At the same time, however, many works of glitch art have developed into archetypes and
973
+
even stereotypical models, and some artists do not focus on the post-procedural dialectics
974
+
and complexity of glitch at all. They skip the process of creation through destruction of
975
+
a flow and focus only, directly, on the creation of new formal designs for glitch, either
976
+
by creating the final imagistic (or sonic) product, or by developing shortcuts to recreate
977
+
the latest-circulated glitch re-formation. Purposive, design-driven efforts at glitch can be
978
+
created in plug-ins, filters or ‘glitching software’ that automatically emulate, simulate or
979
+
mimic a particular glitching method. These tools tend to surrender ‘affect’ (the shocking
980
+
moment(um) of glitch) in favor of ‘effect’.
981
+
Design-driven glitch art has tended to be referred to as artificial or ‘glitch-alike’. Iman
982
+
Moradi has gone so far as to develop a true-false binary to deal with these matters of glitch
983
+
imitation, which he explains with the following statement and schema:
984
+
35
985
+
986
+
Because of the intrinsic nature of this imagery and its relation to pure glitches,
987
+
both in terms of process and viewer perception, I felt the need to form a word that
988
+
adequately describes this artifact’s similarity with actual glitches and present it
989
+
as an obviously separate entity. Thus the term “Glitch-alike” came about to fulfil
990
+
this role. […] Glitch-alikes are a collection of digital artefacts that resemble visual
991
+
aspects of real glitches found in their original habitat.03
992
+
Pure GlitchGlitch-alike
993
+
AccidentalDeliberate
994
+
Coincidental
995
+
Planned
996
+
Appropriated Created
997
+
FoundDesigned
998
+
RealArtificial
999
+
While Moradi’s scheme can be a useful starting point for consideration, I also see a lot of issues with it. The creation of a binary opposition within glitch art seems not only too simple,
1000
+
but also in conflict with a genre that so often scrutinizes and aims to violate binary oppositions. The glitch genre is primarily about breaking categories open, uncovering what is
1001
+
in-between and beyond. The ‘glitch’ in ‘glitch art’ does not only depend on technology, but
1002
+
also involves ideologies and visual structures (aesthetics) including the artist’s individual
1003
+
perspective, and the context of viewing. Instead of denouncing a non-informational glitch
1004
+
(or glitch practice) as artificial or false, I think it is more interesting to research why and how
1005
+
a particular investment in glitch is actually understood as glitch art within a larger media
1006
+
culture. This can be done by describing existing cultural instantiations of, and relations
1007
+
between, a range of differently spawned glitch art practices in context.
1008
+
1009
+
From Passive Appropriation Or ‘Pure Glitch Art’
1010
+
To Active, ‘Post-procedural Glitch Art’
1011
+
When all is said, what remains to be said is the disaster. Ruin of words, demise
1012
+
of writing, faintness faintly murmuring: what remains without remains (the
1013
+
fragmentary).04
1014
+
- Maurice Blanchot
1015
+
1016
+
At a most basic level, glitch artists can challenge the standard mode of operation of a system
1017
+
by appropriating glitches that are spawned (partially or completely) by production processes.
1018
+
Typically, these glitches are encountered accidentally and often unstable (both in their
1019
+
process and in terms of results), which means that the artist has to somehow capture the
1020
+
glitch, in order to appropriate and present it to his audience. An example of this first kind of
1021
+
glitch art is an image by Greg J. Smith. The image shows a Mac interface going haywire for no
1022
+
understandable reason. Although the image can be described as compelling or titillating in
1023
+
terms of aesthetics, the work does not signify more than what was captured in the first place:
1024
+
a broken computer interface. It is a passive capture of failure, sent off to an audience.
1025
+
1026
+
03 | Iman Moradi, Glitch Aesthetics, unpublished bachelor thesis, Huddersfield, UK: University of Huddersfield, 2004. http://www.oculasm.org/
1027
+
glitch/download/Glitch_dissertation_print_with_pics.pdf, p. 10.
1028
+
04 | Maurice Blanchot, The writing of the disaster, Nebraska: University of Nebraska Press, 1995. p. 33.
1029
+
1030
+
36
1031
+
1032
+
Another form of glitch art relies on errors within the production system that the artist actively
1033
+
triggers. These purposively triggered breaks from the flow are at least partially understood
1034
+
and can often be debugged. In this case the artist chooses to exploit a production system (the
1035
+
protocols built into the machine's hard- and software), or the input that makes a system’s
1036
+
protocols behave in a particular way, or both. An example of this second category of glitch art
1037
+
is 5VOLTCORE. During their live performances, the men of 5VOLTCORE attack the computer
1038
+
with power interruptions from an audio signal, which produces short circuits that generate
1039
+
unexpected signals.05 This process tortures the machine and makes it scream out shreds of
1040
+
powerfully colored images, until the computer eventually dies, which ends the performance.
1041
+
In their performances, 5VOLTCORE take issue with the governing charge of the computer.
1042
+
Working in direct opposition to the computer’s procedural flow, actively overturning it, their
1043
+
aggressive glitches lead always to one fatal endpoint, rather than breaking open the future;
1044
+
they are not so invested in the generative qualities of post-procedural glitch.
1045
+
1046
+
Post-procedural Glitch Art Or the Intentional Faux Pas
1047
+
1048
+
Gijs Gieskes. Circuitbend Sega Megadrive2.2. Modified Sega. 2007.
1049
+
1050
+
A less aggressive and more ‘positive’ example of an intervention in machinic flow can be
1051
+
found in Gijs Gieskes’ work. Gieskes takes machines apart and changes their circuitry.
1052
+
Through circuitbending, he redefines the technology and its contents, penetrating and
1053
+
exploring the machine from the inside. First, he dismantles the system and then he deconstructs and re-appropriates it. One of his circuitbent machines, the circuitbend
1054
+
sega megadrive2.2 (2007), consists of a Sega console with a modified circuit, actively
1055
+
transforming the videogame console into an autonomous video synthesising machine.
1056
+
1057
+
05 | 5VOLTCORE, 5VOLTCORE ||| SHOW, 2006, http://5voltcore.com/typolight/typolight257/index.php/show.html.
1058
+
1059
+
37
1060
+
1061
+
Gieskes did not add any code to the chips or the videogame; he only changed the circuitry of
1062
+
the console. This means that the glitches that appear on the television screen were already
1063
+
part of the videogame’s software (the ROM); the generated visuals are readymade, manipulated appropriations of mass-produced objects. The look and feel of these videographic
1064
+
utterances is dependent on the technology inside the original machine. This introduces
1065
+
questions around the built-in aesthetics and conventional usage of the circuitbend sega
1066
+
megadrive2.2.06 Gieskes’ work perverts a classical sense of aura, which according to Walter Benjamin, would be built upon unicity and authenticity. Contrarily, the circuitbend
1067
+
sega megadrive2.2 doesn’t possess one particular ‘here and now’.07 Instead, the artwork is
1068
+
generated every time the machine is activated. Therefore, the aura is situated within the interpretations and context of the user or viewer and the changed technology of the machine.
1069
+
Another example of the intentional faux-pas, or glitch art that is in violation of accepted
1070
+
social norms and rules, is untitled game (1996-2001), a combined series of 11 modifications of the first person shooter game (FPS) quake 1 by the Dutch/Belgium art duo Jodi. Jodi
1071
+
makes subversive glitch art that battles against the hegemonic flows of proprietary media
1072
+
systems. They work to reframe users’ or consumers’ perception of these systems. The duo’s
1073
+
work is often simultaneously politically provocative and confusing. This is partly because
1074
+
Jodi originally never prioritized attaching explanations to their work, but also because of
1075
+
the way in which their practice itself overturns generic expectations. They challenge the
1076
+
ideological aspects of proprietary design by misrepresenting existing relationships between
1077
+
specific media functionalities and the aesthetic experiences normally associated with them.
1078
+
1079
+
1080
+
Jodi. Untitled Game. 11 Quake modifications for PC Mac. 1999. MODS: E1M1AP and Ctrl-F6.
1081
+
1082
+
06 | Gijs Gieskes, circuitbend sega megadrive2.2, 2007, http://gieskes.nl/circuitbending/?file=segamegadrive2.
1083
+
07 | Walter Benjamin, ‘The Work of Art in the Age of Mechanical Reproduction’, in Hannah Arendt (ed.), Illuminations, New York: Schocken,
1084
+
1968, pp. 219-254. p. II.
1085
+
1086
+
38
1087
+
1088
+
In an online interview in 2006 I encouraged Dirk (di from Jodi) to break the duo’s silence
1089
+
around the description of their art. About the work untitled game, Dirk said:
1090
+
Our point was to erase and make this other version of Quake and then deny [the
1091
+
Quake game] the name. […] to call it untitled game (meant) that it was just a prototype of any of these games that (consists of) these kind of standard construction
1092
+
elements and things you can do as a user.08
1093
+
In untitled game, Jodi critically exploited errors in the source code of the original game.
1094
+
The glitches created by these modifications destabilize and alter the normal laws of physics, so that steering and shooting becomes unpredictable and illogically geared, while
1095
+
the sounds and designs of the game itself are also modified to surprise. By changing the
1096
+
algorithms that define the videogame’s playability, the game becomes seemingly ‘unplayable’, at least, according to what is expected as normal game-play. The game itself is not
1097
+
totally ruined; it actually functions quite well, albeit in a wholly non-Newtonian, visually
1098
+
nonsensical way that the FPS-player is not trained to be aware of, or competent with. In
1099
+
E1M1AP for instance, one of the 11 mods making up untitled game, Jodi used the gravity algorithm to create unsettling vortex effects, while in Ctrl-F6 the collective exploited
1100
+
anti-aliasing to create cubes filled with beautifully evolving moiré patterns.09
1101
+
untitled game is an intentionally ruined videogame that questions conventional and
1102
+
normative videogame goals, for example ‘self-improvement’, ‘competition’, and ‘winning’, all of which are naturally embedded in the software design codes of the games that
1103
+
dominate the videogame battlefield. The modified algorithms, visuals and sounds of untitled game generate a new ensemble of conventions, aims and feelings, in which visual
1104
+
and dimensional experimentation takes hold over competitive logic, and the outcome of
1105
+
the game is no longer a score but a colorful, disconcerting experience.
1106
+
In this way, untitled game rebels against the techno-social determinism of (game)
1107
+
technology and consumption, and frames this particular medium of ‘play’ as a taken for
1108
+
granted technique of enculturation. When read through McLuhan – who as early as the
1109
+
1960s identified media technological developments as the most important (and at that
1110
+
time, under-acknowledged) sites of social cultivation – Jodi seem to indicate that not only
1111
+
media content and socially determining genres (game conventions), but also specific material forms (interfaces) and techné (the game's operational elements) are important to
1112
+
interrogate as objects of study. Recall McLuhan’s own words here:
1113
+
“the medium is the message” because it is the medium that shapes and controls
1114
+
the scale and form of human association and action. The content or uses of such
1115
+
media are as diverse as they are ineffectual in shaping the form of human association. Indeed, it is only too typical that the “content” of any medium blinds us to the
1116
+
character of the medium.10
1117
+
1118
+
08 | Rosa Menkman, Beauty in the Age of Digital Art; aesthetic, poetic or rhetoric, June 2006. http://rosa-menkman.blogspot.com/2006/05/beautyin-age-of-digital-art.html.
1119
+
09 | Rosa Menkman, Jodi op de Pijnbank, unpublished master thesis, Amsterdam: University of Amsterdam, 2006, http://home.student.uva.nl/
1120
+
rosa.menkman/Jodi%20op%20de%20pijnbank.pdf.
1121
+
10 | Marshall Mcluhan, Understanding Media: The Extensions of Man, New York: McGraw Hill, 1964.
1122
+
1123
+
39
1124
+
1125
+
In digital glitch art like untitled game, the medium is redefined as a platform that
1126
+
doesn’t follow its genre, form or technique. This triggers the user to reflect upon her conventional frames of reference for the particular game and perhaps even the commercial
1127
+
game in general. The work criticizes the flow of a specific medium, its interface and its
1128
+
inherent conventions, but does not necessarily break it (as opposed to 5VOLTCORE’s performance). The fact that the game still ‘works’ while being programmed to glitch, makes
1129
+
it all the more critically challenging as media experience. Jodi shows that software is more
1130
+
than just a preprogrammed tool: it is a materialization of social modalities, which can
1131
+
furthermore be endlessly re-modified to different interpretive or social conclusions.
1132
+
The irrational and conceptual glitches within untitled game, its voiding of original and
1133
+
received meanings, forces the viewer to make active sense of the work. The structures of original meaning are intentionally ruined. But in this case, ‘ruin’ is both a conceptual orientation
1134
+
and a technique that underlines the constructedness of media (art), forcing the viewer to consider the computer as no longer just a device of standardization but instead as a technology
1135
+
that functions within a social reality. Only after reflecting on this new form of the work, can
1136
+
the user see that what the glitch does is not just destroy the old videogame, but in fact modify
1137
+
its existing denotations and exchanges, entangling it within new lines or architectures of
1138
+
meaning. The ‘techniques of the void’ – the systematic distortion of communication – helps
1139
+
to open media up for discussions of their internal politics. This is how, through the tactics
1140
+
used within these glitched games, users can re-territorialize these techniques.
1141
+
1142
+
The Concept And Technique Of Ruin
1143
+
You cannot prohibit the catastrophe, you must surf it!11
1144
+
- Paul Virilio
1145
+
1146
+
Today news and current affairs is generated and spread not only through rich and powerful press monopolies and infrastructures, but at the same time through smaller, more independent and autonomous agents that do not require a great capital outlay to contribute
1147
+
to debate online. This is why social blogging softwares like Blogger are often described
1148
+
as democracy-enhancing tools; they are celebrated as an ideal medium supporting the
1149
+
political mythology of ‘freedom of speech’.12
1150
+
During 2006 and 2007, Jodi made the work <$blogtitle$>, based on the social publishing
1151
+
tool Blogger, from Google.13 <$blogtitle$> looks like a Blogger page in a broken state. The
1152
+
pages generated by Jodi’s (mis)usage of the tool are either filled with gibberish or in ruins.
1153
+
It’s hard to say: perhaps you are looking at back-end code, broken on to the surface of the
1154
+
site, or perhaps it is just nonsense that was never part of any codified language system?
1155
+
1156
+
11 | Andreas Broeckmann, Joke Brouwer, Bart Lootsma, Arjen Mulder and Lars Spuybroek, The Art of the Accident, NAI Publishers/V2_Organisatie: Rotterdam, 1998. p. 30-32.
1157
+
12 | Donald Matheson, ‘Weblogs and the Epistemology of the news: some Trends in Online Journalism’, Sage Journals, London: SAGE Publications, Thousand Oaks CA and New Delhi Vol 6.4 (2004): pp. 443-468. p. 445.
1158
+
13 | Jodi, <$blogtitle$>, 2006-2007, http://blogspot.jodi.org/.
1159
+
1160
+
40
1161
+
1162
+
In these works, Jodi indeed plays with different language systems, for instance the visual
1163
+
and the non-visual source (code) of the Blogger software. Template formats such as the
1164
+
title of the blog, the post headers and certain blog addresses in the link list appear all in
1165
+
ruins, while Blogger-specific images like comment-icons, dates and additional otherwise
1166
+
functional visual elements are now reduced to theatrical objects. What is normally invisible as the infrastructure of the blog – snippets of code and interface commands like “S
1167
+
= Publish, D = Draft»” or “Allow New Comments on This PostYes No”14 – are moved to the
1168
+
front of the site, where normally only a ‘human discourse’ would be visible.
1169
+
Jodi’s <$blogtitle$> partially exposed the mythical notion of 'democracy enhancing' social blogging tools, when Blogger blocked 7 of its 22 blog pages. In this case, the process
1170
+
of ‘free online publishing’ resulted in censorious destruction. This unforeseen eventuality made it clear that Blogger-users (any blog users) answer to a built-in (political) system
1171
+
and don’t operate completely under their own authority. Moreover, the system is governed
1172
+
by the belief (shared by both the creators of the technology, the conventional users, and
1173
+
the audience) that the software will be used to distribute only conventionally formatted
1174
+
knowledge. Bloggers that do not subscribe to the conventions risk the possibility of being
1175
+
blocked or having their blogs completely deleted.
1176
+
1177
+
Jodi. My Blog is blocked. blogspot.jodi.org. 2007.
1178
+
1179
+
14 | Jodi, <$blogtitle$>, 2006-2007. http://blogspot.jodi.org/.
1180
+
1181
+
41
1182
+
1183
+
<$blogtitle$> stands apart as a purposeful artifact that captures what Deleuze and
1184
+
Guattari have described as a ‘line of flight’: an elusive, divergent, inherently political
1185
+
moment(um) through which axioms are questioned, genres are broken open and categories are created.15 Jodi uses the glitch to emphasize a rejection of what can be referred to
1186
+
as ‘software-determinism’ or in the case of blogger, ‘platform-determinism’. In an interview with Tilman Baumgartel, Jodi states: ‘It is obvious that our work fights against high
1187
+
tech. We also battle with the computer on a graphical level. […] We explore the computer
1188
+
from inside, and mirror this on the net’.16 <$blogtitle$>, as an example of this working
1189
+
method, enacts this battle at the border between system and entropy, standardization and
1190
+
corruption, expression and code, meaning and non-meaning, thwarting the user and the
1191
+
viewer’s expectations and understandings.
1192
+
<$blogtitle$> is generated within the system of Blogger, but does not follow the rules,
1193
+
the language or the syntax of that blogging system. On the one hand, the work can be
1194
+
understood as a social criticism towards Blogger and other celebrated ‘direct’ read/write
1195
+
web 2.0 platforms or as a blog that entails a (re-shuffled) sign system through which the
1196
+
viewer can navigate and glean her own select fragments of meaning.
1197
+
In <$blogtitle$>, artistic negation has become a generative and creative force. In a
1198
+
seeming void of meaning, the spectator is forced to use his imagination while reflecting
1199
+
on the work. The glitch’s formal fragmentation signifies that the work is ‘open’ to interpretation and meaningful engagement. This new text is no longer a work that displays or
1200
+
retells conventions, but a writerly software where meaning can be actively (re)constructed.
1201
+
By ruining the Blogger medium, Jodi’s use of formal fragmentation opens the platform
1202
+
itself up to deconstruction, interpretation and further active engagement. As a result, the
1203
+
meaning of the ruined work is never finished, whole or complete. Instead of being static
1204
+
it differs from reading to reading, or with each fragmented element of the syntax. In this
1205
+
sense, the work has become a virtual space where the audience can actualize an infinite
1206
+
amount of potential meanings. However, for the reader to actually give meaning to the
1207
+
ruins, they must take the initiative of imposing (their own select) new constraints, new
1208
+
frameworks of analysis and limitations on other possibilities. The viewer becomes aware
1209
+
that every act of creating meaning is also just as strongly an act of destruction (of more
1210
+
infinite possibilities).17
1211
+
Moreover, in the case of <$blogtitle$>, this openness also had a negative consequence:
1212
+
Blogger interpreted the blog as a malicious spamblog and consequently blocked it. This
1213
+
act could be described as a rather rigorous ‘death of the author’, in which the meaning
1214
+
of the work is not negotiated, but instead dismissed and deleted. In fact this could be
1215
+
understood as a second death. The author ‘dies’ in a Barthesian sense at the moment of
1216
+
(web) ‘publication’ when the viewer’s interpretation takes over from authorial intention,
1217
+
but also in a second and more violent way when the corrupted, ‘writerly’ text is totally
1218
+
eliminated from the blogosphere altogether.
1219
+
1220
+
15 | Gilles Deleuze and Pierre-Félix Guattari, A Thousand Plateaus: Capitalism and Schizophrenia, Trans. B. Massumi, Londen: The Athlone
1221
+
Press, 1988. p. 213.
1222
+
16 | Tilman Baumgärtel, ‘TP: Interview with Jodi. We love your computer’, Telepolis. May 2006, http://www.heise.de/tp/r4/artikel/6/6187/1.html.
1223
+
17 | Gilles Deleuze and Claire Parnet, Dialogues II, London and New York: Continuum, 2006. p. 112.
1224
+
1225
+
42
1226
+
1227
+
<$blogtitle$> opens up and intervenes into the normally inter-locked relations between conventional information, a possible message, and the back-end coding of Blogger,
1228
+
and treats these relation as a system that can be modified or expanded towards new possibilities through ‘glitching’. Here glitches articulate an alternative language that blends
1229
+
systems into a form that nobody can read (yet). The ‘voided’ <$blogtitle$> shows the
1230
+
conventions by which the user/reader navigates online, and the norms that help him to
1231
+
operate these daily technologies transparently. The constructedness of such discourse, in
1232
+
terms of locked down proprietary software is not necessarily negative in itself, but sometimes (as <$blogtitle$> suggests) leads to generalized assumptions and the under- or
1233
+
non-acknowledgement of invisible political forces in the form of underlying conventions.
1234
+
The glitch can help us uncover these obfuscated political dimensions as well as create
1235
+
strategies to see through them. In <$blogtitle$>, Jodi shows that a glitch can be completely constructed (by the artist), but also that such constructs can in turn reveal the constructedness of software-generated knowledge and expression. Jodi’s investment in glitch
1236
+
shows that Blogger can, like quake 1, be used in many more ways than users pacified by
1237
+
convention might assume.18
1238
+
1239
+
Creating the ‘Perfect Glitch’ Using Critical Media
1240
+
Aesthetics
1241
+
[The] absence of meaning is in this case the presence of all meanings, absolute
1242
+
ambiguity, a construction outside meaning.19
1243
+
- Jacques Attali
1244
+
1245
+
Within the constructed ruins of glitch, new possibilities and new meanings arise. There
1246
+
is something more than just destruction: new understandings lie just beyond the tipping
1247
+
point. The glitch generates new understandings of techno-culture through the gestations
1248
+
of Glitchspeak, glitch’s constantly growing vocabulary of new expressions.
1249
+
I use the term ‘Glitchspeak’ in opposition to George Orwell’s ‘Newspeak’. For Orwell,
1250
+
Newspeak is a language whose political goal it is to shrink its vocabulary and grammatical nuance over time, so as to render any alternative thinking – which he referred to as
1251
+
‘thoughtcrime’, or ‘crimethink’ – impossible. The final goal of Newspeak is to construct
1252
+
a society in which only politically approved (dominant and conventional) statements can
1253
+
be articulated, at the expense of the possibility of free expression, rebellion, and so on.20
1254
+
Fighting Newspeak, Glitchspeak contests the obfuscated limitations of language created
1255
+
by proprietary technology, to capture the constant transformation and growing wealth of
1256
+
glitch artifacts and their meanings.
1257
+
Most glitch artists are always, directly or indirectly, trying to answer one question: How
1258
+
much agency should I provide to my systems of destruction? Their post-utopian strategies aim
1259
+
to identify where the ‘tipping point’ is: When and how can a glitch be found and transition
1260
+
18 | Michael Truscello, Behind the Blip: Essays on the Culture of Software (review), Cultural Critique, no. 63, (2006): pp. 182-187.
1261
+
19 | Jacques Attali and Brian Massumi, Noise: The Political Economy of Music, Manchester: Manchester University Press, 1985. p. 33.
1262
+
20 | George Orwell, Nineteen Eighty-Four, London: Secker and Warburg, 1949. p. 372.
1263
+
1264
+
43
1265
+
1266
+
into something new? The perfect glitch exists, momentarily, at the shocking tipping point
1267
+
between (potential) failure and a movement towards the creation of a new understanding.
1268
+
The glitch’s inherent moment(um), the power it needs or has to pass through an existing
1269
+
membrane or semblance of understanding, helps the utterance to become an unstable articulation of counter-aesthetics, a destructive generativity. As an exoskeleton for such (postutopian) progress however, the glitch does not just take place on a critically ruined surface.
1270
+
The choice to accept the glitch, to welcome it as an aesthetic form, means to accept a new
1271
+
critical dialectic that makes room for error within the histories of ‘progress’.
1272
+
Following this dialectic of critical media aesthetics, the glitch can obtain a place within
1273
+
larger media cultural scenarios of political productivity and evolution. The role of glitch
1274
+
artifacts as (instances of) critical media aesthetics is, again, twofold. On the one hand,
1275
+
these aesthetics show a medium in a critical state: a ruined, unwanted, unrecognized, accidental and horrendous moment. This transforms the way the consumer perceives its
1276
+
normal operation (every accident transforms the normal) and registers the passing of a
1277
+
tipping point after which it is possible for the medium to be critically revealed at greater
1278
+
depth. On the other hand, these aesthetics critique the medium itself, as a genre, interface and expectation. They radically challenge the technological, social or ideological
1279
+
constructedness of all media cultural formations while producing a theory of reflection.
1280
+
1281
+
The Tipping Point of Cool:
1282
+
Critical Media Aesthetics' Becoming Commodities
1283
+
In The Laws of Cool, Alan Liu asks himself: ‘What is “Cool”?’ He describes that cool is
1284
+
the ellipsis of ‘knowing what is cool and of withholding that idea’. ‘Cool is information
1285
+
designed to resist its status as information, a paradoxical “gesture” through which the
1286
+
unknown struggles to arise (or resists arising) in the midst of the economies of knowledge
1287
+
work’.21 Liu concludes that those who insist on asking what is cool are definitely uncool.
1288
+
Keeping Liu’s statement in mind and thus paradoxically over-theorizing cool glitches, I
1289
+
suggest that the cool glitch can be found at the moment of its preliminary non-definition;
1290
+
when it is still denied its existence – before its tipping point – where errors are deleted, or
1291
+
remain ignored, blocked or unaccepted, unwilled.
1292
+
Liu-cool glitches only exist during the moment(um) of glitch – before the glitch is overcome as failure or has become a new established form. To think a glitch is ‘cool’ is to acknowledge that the glitch is still actively reflected upon and has not yet been established.
1293
+
Indeed the coolest work of glitch art is denied existence at the same time as it incorporates this very dismissal into its momen(tum), so as to implicitly say something about this
1294
+
action: the deletion of <$blogtitle$> gave Jodi the opportunity to exploit this Liu-cool
1295
+
logic and incorporate it into the informational lure of their work.
1296
+
Cool is in a constant state of flux, as is ‘cool glitch art’. The latter exists as an assemblage
1297
+
relying on, on the one hand, the construction, operation and content of the technology
1298
+
(the medium) and on the other hand the work, the writer/artist, the interpretation by the
1299
+
21 | Liu, Alan. ‘What’s cool?’, in The Laws of Cool: Knowledge Work and the Culture of Information, Chicago: University of Chicago Press, 2004,
1300
+
pp. 176-179.
1301
+
1302
+
44
1303
+
1304
+
viewer and/or user (the social meaning) and the work’s aesthetics. The tipping point,
1305
+
the application of (aesthetic) meaning or value, can move the glitch from the realm of
1306
+
cool glitch art to hot, established or even commodified. It is however important to realize
1307
+
that not all glitch art is ‘cool’, or progressive or something new. The popularization and
1308
+
cultivation of an avant-garde of mishaps and breakages has indeed become predestined
1309
+
and unavoidable. What is now (or next) a glitch will become a hot fashion soon enough reproducible, standardized, automated by softwares and plug-ins. This movement is an
1310
+
integral part of a movement that should be theorized as the genre of glitch art.
1311
+
1312
+
45
1313
+
1314
+
From Artifact To Commodity
1315
+
I woke up one morning in March to a flood of emails telling me to look at some
1316
+
video on YouTube. Seconds later I saw Kanye West strutting around in a field of
1317
+
digital glitches that looked exactly like my work. It fucked my show up… the very
1318
+
language I was using to critique pop content from the outside was now itself a
1319
+
mainstream cultural reference.01
1320
+
- Paul B. Davis
1321
+
1322
+
From CircuitBending to Simulation
1323
+
The ‘debuggers’ of the digital realm, the technicians and engineers, are tasked with controlling the erratic reactions and behaviors of files, hardwares and softwares. Complex
1324
+
compression algorithms developed for mobile phones, dvds and mp3s for example,
1325
+
are designed to minimize the amount of artifacts a device will show at the surface of its
1326
+
interface, to make the medium seem as transparent as possible. In the culture of glitch
1327
+
art, technicians and engineers are more like ‘re-buggers’; they engage themselves with an
1328
+
almost opposite practice, prompting and amplifying glitch artifacts on purpose and even
1329
+
writing softwares or building blatantly non-standard machines with the intent to spawn
1330
+
more new artifacts for the user/viewer’s development of a rich Glitchspeak vocabulary.
1331
+
1332
+
Jeff Donaldson. noteNdo. miniDV capture from Circuit bent NES. 2007.
1333
+
1334
+
01 | Paul B. Davis, Define Your Terms (Or Kanye West Fucked Up My Solo Show), 28th May 2009, http://www.seventeengallery.com/index.
1335
+
php?p=3&id=42.
1336
+
1337
+
46
1338
+
1339
+
Don Miller/NO CARRIER. glitchNES 0.2 screenshot. 6502 ASM. August 2010.
1340
+
1341
+
For more than 20 years now, glitch sounds have been consumed and standardized in
1342
+
contemporary music culture. Only recently a similar trend is crystalizing itself around
1343
+
visual glitch-generation tools. An example of this tendency towards generative glitch design and its consumption can be found within the 8bit scene. noteNdo (Jeff Donaldson
1344
+
from New York) has been performing his live visuals generated with a circuitbent Nintendo Entertainment System (nes) since 2001. He has played at many festivals, while his
1345
+
broken sprites – the two-dimensional images generated in his console and triggered by
1346
+
his custom-made bends – have inspired other people to start circuitbending their own
1347
+
nes devices.
1348
+
Because circuitbending involves a certain threshold of knowledge and expertise (a basic
1349
+
understanding of electronics and its tools), programmers have apprehended opportunities in emulating, rather than manually repeating, the physical process of circuitbending. The artist no carrier (Don Miller, also from New York) for example, whose work
1350
+
arose slightly later in the visual glitch scene, has written glitchnes, a software that
1351
+
47
1352
+
1353
+
emulates some parts of the technological process of circuitbending an nes console.02
1354
+
no carrier describes his work in this way:
1355
+
glitchnes is an open source software project for nes. This software causes
1356
+
graphical glitches similar to hardware circuitbending. The images produced are
1357
+
caused by deliberate ram corruption due to overloading the ppu, or Picture Processing Unit, of the nes. The result is random flashes of colour and patterns that
1358
+
change with each button press. The open source nature of this project allows
1359
+
users to create their own glitchnes roms and cartridges with unique tile sets
1360
+
and effects by altering the source code to their liking.03
1361
+
The difference between noteNdo’s hardware glitching and no carrier’s emulated,
1362
+
software glitching are many. But as one example, noteNdo never open-sourced his
1363
+
knowledge and strategies for circuitbending the hardware of a nes, while no carrier
1364
+
aims to keep his glitchnes and all the research it involves open and accessible. This
1365
+
means that while noteNdo kept his research closed and in so doing, retained some form
1366
+
of authorship of his work, no carrier open sourced his work, giving users the opportunities to change parameters (and in doing so, the looks of the generated glitches) or
1367
+
even build a similar work from scratch.
1368
+
1369
+
Vade. What a Horrible Night To Have A Curse - Casltevania. Glitched System Emulation via OpenEmu. 2009.
1370
+
1371
+
02 | Don Miller, nocarrier glitchnes, no carrier.com, 2009, http://www.no carrier.com/glitchnes.html.
1372
+
03 | Don Miller, biography for the Playlist exhibition, 2010, http://www.imal.org/playlist/artworks/17.
1373
+
1374
+
48
1375
+
1376
+
The idea that a glitch can be designed or distributed by knowledge alone, or glitch
1377
+
software, seems at first maybe a-typical, but throughout the development of the glitch
1378
+
genre, this has become a more and more common tendency or even a tradition. In this
1379
+
tradition, no carrier’s glitchnes was followed by the open emu project for Quartz
1380
+
Composer. open emu is an open source game emulation tool, a program that allows the
1381
+
computer to virtually run a certain console’s operating systems on another machine for
1382
+
which it was not designed. open emu also makes it possible for users to modify game
1383
+
engines and make use of ‘existing’ game architectures and scripts for all kinds of creative purposes. It even ‘includes a separate plugin just for the Nestopia engine (the nes
1384
+
emulation program), which supports extended features, such as ROM glitching, cheat
1385
+
codes and game rewinding’.04 This particular form of glitching does not effect the game
1386
+
itself; instead it randomly shuffles the sprites by a process referred to as ‘name table
1387
+
ram disordering’.
1388
+
The open emu extended plugins made it extremely easy for ‘anyone (to) software ‘bend’ a
1389
+
virtual nes, in real-time’.05 But whereas the original noteNdo console and no carrier’s
1390
+
glitchnes were technically invested projects, the difference of the open emu rom lies
1391
+
in the fact that it puts an emphasis on aesthetics. The simulation of medium-specific or
1392
+
maybe even medium-branded game artifacts generates images that imitate or mimic the
1393
+
process of glitching. Finally, a difference between emulation and simulation lies within
1394
+
the space offered to users to learn about the technology and its exploited protocols.
1395
+
Today more and more ‘new’ glitch art is being modeled after original glitches within older
1396
+
media, perpetuating a shift in glitch culture from destabilizing breaks within technology
1397
+
or information-based processes towards generic and associative displays of more and
1398
+
less ‘retro’ effects. With the help of these slowly standardized, commodified, institutionalizing effects, any user can handle a broad range of data types and technologies in
1399
+
predetermined, often retro-nostalgic ways, and create what can best be described as an
1400
+
approximation of what originally would have been the materialization of a destabilizing
1401
+
break of machine technology.
1402
+
1403
+
From DataBending to Transcoding
1404
+
Another striking example of the evolution of glitch can be found in the realm of databending. In 2005, Beflix (Ant Scott), Organised (Iman Moradi) and Dmtr (Dimitre Lima) developed the glitchbrowser (2005, taken offline in 2008). Whereas normally browsers are
1405
+
designed to make websites accessible, standardizing their outputs into easily understandable, uniformly structured content that appears the same across different computers, the
1406
+
glitchbrowser was designed to ‘read’ a web-source and replace all its inherent images
1407
+
with glitched or disfigured versions.
1408
+
Today, if you go to the url where the glitchbrowser was once available, the browser is
1409
+
commemorated by a text that explains the concept as
1410
+
04 | Today, open emu has a designed plug-in emulation tool for many different game engines like Nintendo, Sega and Game Boy.
1411
+
05 | Dan Winkler and Anton Marini, open emu, 14 Jan 2009. http://openemu.sourceforge.net/.
1412
+
1413
+
49
1414
+
1415
+
a deliberate attempt to subvert the usual course of conformity and signal perfection. Information packets that are communicated with integrity are intentionally lost in transit or otherwise misplaced and rearranged. The consequences of
1416
+
such subversion are seen in the surprisingly beautiful readymade visual glitches
1417
+
provoked by the glitchbrowser and displayed through our forgiving and unsuspecting web browsers.06
1418
+
The glitchbrowser specifically re-encoded embedded jpegs and gifs and returned
1419
+
them to the user as ‘damaged’ pictures, imitating transmission errors.
1420
+
glitchbrowser as a browser was an autonomous work of art, but the browser also
1421
+
made it possible to download the generated glitch images, and to present them as independent, stand alone ‘works’. This is how the browser took on a dual role as both a
1422
+
conceptual art piece and as a glitch-image generation tool. The glitchbrowser was an
1423
+
early version of a still growing number of glitch image generation softwares and plugins
1424
+
that include projects like corrupt™ (Recyclism, 2006), glitchmonkey (Youpy, 2007)
1425
+
and bytemolester (Károly Kiripolszky, 2008). These scripts, plugins and softwares all
1426
+
generate results similar to the glitchbrowser, while breaking and disfiguring jpegs
1427
+
and / or gifs (the most commonly used image compressions) in different ways and different environments (for instance within a browser, or as an app for the pc).
1428
+
A couple of years later, in 2010, reflecting on these trends and their resulting outputs,
1429
+
I wrote a vernacular of file formats: an edit guide for databend compression
1430
+
design. The Vernacular was an attempt to voice my concerns regarding the growing
1431
+
popularity of designed glitching in favor of informational or process-oriented glitch research. It was a performative, playfully meant intervention that criticized the simplicity
1432
+
that glitch had become, while at the same time aimed to demystify the last remaining
1433
+
‘cool’ elements of file format-based glitching. In the introduction I wrote:
1434
+
Glitches are hot. It is clear from what we can see on mtv, Flickr, in the club or
1435
+
the bookstore. While the “Glitch: designing imperfection” coffee table book introduces the glitch design aesthetic to the world of latte drinking designers, and
1436
+
Kanye West uses glitches to sing about his imperfect love life, the awkward, shy
1437
+
and physically ugly celebrate under the header “Glitched: Nerdcore for life”.07
1438
+
The logic of these developments seemed to reduce the glitch to an imagistic slogan: ‘No
1439
+
Content – Just Imperfection’. I wanted to move glitch artists beyond these burgeoning
1440
+
conservative impulses into rethinking and expanding out from the standardizing and
1441
+
only aesthetically engaging forms of glitch. In the Vernacular, I showed a selection of
1442
+
the most commonly used file formats in their corresponding (glitched) states, in order
1443
+
to categorize and compare prominent examples of what has come to be heralded as the
1444
+
results of hot glitch design.
1445
+
1446
+
06 | Dimitre Lima, Iman Moradi and Ant Scott, glitchbrowser 02.12.2005 – 18.03.2009, 2009. http://glitchbrowser.com/.
1447
+
07 | Rosa Menkman, a vernacular of file formats, August 2010. http://dl.dropbox.com/u/9054743/lofi%20Rosa%20Menkman%20-%20A%20
1448
+
Vernacular%20of%20Fil%20Formats.pdf.
1449
+
1450
+
50
1451
+
1452
+
Larsby, Johan and Rosa Menkman. MONGLOT. Glitch Software Interface. 2011.
1453
+
1454
+
Building from the Vernacular I went on to create monglot, a piece of glitch software,
1455
+
together with Johan Larsby, in 2011. monglot allows the user to easily transcode completely identical informational disturbances in different file formats (for instance from
1456
+
jpeg to PNG), in order to uncover and understand the inherent structures of an encoding
1457
+
(or the file format). It is ironically in this way a software of reproduction and repetition
1458
+
(non-creativity), invested in coming into standardizing contact with the non-standardized
1459
+
moment(um) of glitch artifacts.
1460
+
1461
+
The monglot software generates glitch images by mashing together the visual language of an image with the imagedata encoded in the language of the compression,
1462
+
the latter erupting over the surface of the former as precisely ruined artifacts. It can
1463
+
generate common glitch aesthetics like ‘fragmentation’, ‘grain’, ‘ghosting’, ‘interlacing’, ‘jitter’, ‘jaggies’, (…) ‘posterization’, ‘pixelating’, ‘quantization error’, ‘ringing’,
1464
+
‘staircase noise’ and so on. The software enables the user to increase their knowledge
1465
+
about what kinds of compressions can generate certain kinds of aesthetic outcomes.
1466
+
Moreover, monglot factored in the ambivalence and double articulation (encoding
1467
+
vs. image data; artifact vs. filter) of file format-based Glitch Art and design. The images
1468
+
generated by the software are ironically standardized (by enabling the repetition of any
1469
+
glitch) and as such exist as a compromise, in-between ‘cool’ (unknown, or under-articulated) glitch and known glitch design. Unlike monglot, most other glitch art softwares
1470
+
up until then had only accepted one or two different file formats for experimentation.
1471
+
monglot focuses on opening up glitched outcomes to experimentation, modulation
1472
+
and generative inquiries. At the same time, glitch art (as progressive and against the
1473
+
grain) becomes a virtual entity through the software (a concept that is only referenced).
1474
+
monglot imposes breaks strategically, as a norm, to reflexively think through the dialectical and forking paths that inform the makings of glitch art.
1475
+
51
1476
+
1477
+
Finally, the research into file formats and the opportunities they provide for artistic and
1478
+
technological experimentation was taken one-step further in extrafile, an application
1479
+
written by Kim Asendorf in 2011. Asendorf describes the purpose of his software as ‘a project for developing new image file formats for artistic purposes’. The extrafile software
1480
+
presents the concept of an image file format as a work of art, bringing the filename extension into the scope of the artwork itself, as a form of image exclusivity.08
1481
+
1482
+
Asendorf, Kim and Rosa Menkman. Extrafile vs. Monglot. BLINX. 2011.
1483
+
1484
+
Asendorf, Kim and Rosa Menkman. Extrafile vs. Monglot. BASCII. 2011.
1485
+
1486
+
08 | Kim Asendorf, extrafile, 2011, http://extrafile.org/.
1487
+
1488
+
52
1489
+
1490
+
The extrafile software allows a user to transcode any image (e.g. a jpeg) into a nonproprietary format invented by the artist, Asendorf, such as blinx or bascii. While the
1491
+
outcome or look of the file depends on the image chosen by the user, the encoding or
1492
+
compression itself is the primary work of art. Some of the compressions developed by
1493
+
Asendorf only restructure the image data and are at first sight invisible in the image; they
1494
+
only show within the encoding. Other compressions such as bascii involve the transcoding of the image into very different building blocks and color palettes, changing the look
1495
+
of the file quite dramatically. All of the different, artistically generated file formats take
1496
+
the work of art away from mainstream standards. The process and the resulting bytes, regardless of content, become the artwork itself. extrafile thus offers an escape from the
1497
+
licensed image file formats (standards) and the proprietary protocols that are under the
1498
+
rule of the International Organization for Standardization (ISO), which has authored standardization laws for information processing since 1947. extrafile is best understood as
1499
+
a critique against the governing codes of ict culture that involve an overly complex and
1500
+
entangled web of proprietary licensed protocols.
1501
+
Both monglot and extrafile are open sourced image generation tools, inviting artists to
1502
+
play with the code, the software and the concept. Whereas extrafile itself is not a glitch
1503
+
software, the functionality of the tool is best (if not only) appreciated by glitching its outputs.
1504
+
When Asendorf launched extrafile online, he did so without showcasing any images generated by himself. Instead, he relayed part of the authorship on to other glitch artists like
1505
+
Jose Irion Neto, Benjamin Gaulon, Bit Synthesis and myself. Asendorf invited us to produce
1506
+
glitched images using extrafile just prior to the launch of the software on the internet.
1507
+
While monglot both transcodes image files from one conventional standard to another
1508
+
and databends images on an informational level, extrafile transcodes image files into
1509
+
new, rogue standards. Both applications underline a critique of the standardization of file
1510
+
formats and the commodification of their corresponding glitches. At the same time, they
1511
+
contribute to the growing indifference between glitch art as informational or processbased research, and glitch design, which focuses on the aesthetics of an end product.
1512
+
1513
+
From Enchanting Affect To Filtered Effect
1514
+
If it works, it’s obsolete.09
1515
+
- Marshall Mcluhan
1516
+
1517
+
We already know too much for noise to exist.10
1518
+
- Douglas Kahn
1519
+
1520
+
As the popularization and cultivation of glitch artifacts is now spreading more widely, it is
1521
+
interesting to track the development of these processes in specific case studies. One case
1522
+
study of a compression artifact, recently referred to as ‘datamoshing’, tells an especially
1523
+
interesting account of glitch cultivation. The datamosh artifact is located in a realm where
1524
+
1525
+
09 | Meenakshi Gigi Durham and Douglas Kellner, Media and Cultural Studies, New Jersey: Wiley-Blackwell, 2006. p. 110.
1526
+
10 | Douglas Kahn, Noise, Water, Meat: A History of Sound in the Arts, Massachusetts: MIT Press, 1999. p. 21.
1527
+
1528
+
53
1529
+
1530
+
compression artifacts and glitch artifacts collide. The artifact caused by compression is
1531
+
stable and reproducible, as it is the effective outcome of keyframes being deleted. The
1532
+
outcome of this deletion is the visualisation of the indexed movement of macroblocks,
1533
+
smearing over the surface of an old keyframe. This makes the video morph in unexpected
1534
+
colours and forms.11
1535
+
In 2005, Sven König embarked on his exploration into the politics of file standards,
1536
+
through this particular datamoshing effect, and in relation to the free codec Xvid. Xvid
1537
+
is a primary competitor of the proprietary DivX Pro Codec (note that Xvid is DivX spelled
1538
+
backwards), which is often used for speedy online video distribution through peer-to-peer
1539
+
networks. In aPpRoPiRaTe! (Sweden: 2005) König used the codec to manipulate and appropriate ‘complete video files found in file sharing networks’.12 His work included an
1540
+
open source software script that could be used to trigger the compression-effect in realtime. Through the use of the Xvid codec and copyrighted material, König tried to pinpoint
1541
+
the tension between the usage of non-proprietary compression codecs and their uptake in
1542
+
DRM (Digital Rights Management) remix-strategies.
1543
+
In his next project, download finished! (2007), König explored how the codec could be
1544
+
used to transform and republish found footage from p2p networks and online archives. The
1545
+
result became the rough material for his online transformation software, which translated
1546
+
‘the underlying data structures of the films onto the surface of the screen’. With the help of
1547
+
the software, file sharers could become ‘authors by re-interpreting their most beloved films’.13
1548
+
A swift maturation of the datamoshing effect took place in 2009 at the same time as Paul
1549
+
B. Davis was preparing for his solo show at the Seventeen Gallery in London. Davis’ show
1550
+
was partially based on a formal and aesthetic exploration of the artifact. While the show
1551
+
was intended to critique popular culture by way of datamosh interventions, this culture
1552
+
caught up with him overnight, when the effect penetrated the mainstream just prior to the
1553
+
opening of his show. Davis’ reaction to the fate of appropriation plays out as the opening
1554
+
quote of this chapter: ‘It fucked my show up… the very language I was using to critique pop
1555
+
content from the outside was now itself a mainstream cultural reference’.14
1556
+
1557
+
11 | The lossy compressed video image is framed fundamentally differently from analogue or RAW video footage. The frames no longer rely upon
1558
+
RAW pixels. Instead, macroblocks become one of the elementary components of the lossy compressed moving image (at least under current
1559
+
standard codecs of the Moving Pictures Experts Group ‘MPEG’ and others). Lossy compressed video often depends on luminance (brightness) and chrominance (coloring) thresholds arranged within 16x16 pixel (more or less) macroblocks within the keyframes (the I-frames)
1560
+
of an image sequence. The thresholds (or frequencies) of chrominance and luminance depend on an oscillating cosine function (following
1561
+
Fourier Transform).
1562
+
Moreover, the material of the digital film is no longer based on a linear series of discrete images (a sequence); instead the video consists of
1563
+
different kinds of frames (I-frames or reference/key frames, P-frames or forward-predicted frames and B-frames or bi- directional frames),
1564
+
of which only the keyframe possesses a complete matrix of macroblocks. The frames between the keyframes (the P- and B-frames) consist
1565
+
of motion vectors that index only the difference in position (the offset) of the macroblocks between the original and the next frame. The
1566
+
handling of space and time within the video technologies is thus significantly different between the linear analog or RAW footage and lossy
1567
+
compressed footage.
1568
+
A recently popularized wave of video artworks was based on the deletion of keyframes and the exploitation of the vector motion of P-frames.
1569
+
This is currently dubbed datamoshing, pixel bleeding or just compression art.
1570
+
12 | Sven König, aPpRoPiRaTe!, 2005, http://www.popmodernism.org/appropirate/.
1571
+
13 | Sven König, download finished, the art of filesharing - make p2p cinema, 2007, http://www.download-finished.com/.
1572
+
14 | Paul B. Davis, Define Your Terms (Or Kanye West Fucked Up My Solo Show), 28th May 2009, http://www.seventeengallery.com/index.
1573
+
php?p=3&id=42.
1574
+
1575
+
54
1576
+
1577
+
Prominent music videos, including Kanye West’s welcome to heartbreak (2009, directed by Nabil Elderkin) and Chairlift’s evident utensil (2009, Ray Tintori) indeed had
1578
+
popped up, bringing the datamoshing effect into the mainstream via mtv.15 The new wave
1579
+
of interest in the effect generated by these clips lead to a Youtube tutorial on datamoshing,
1580
+
followed by an explosion of datamosh videos and the creation of different datamosh plugins, developed by for instance the Japanese artist ucnv. In the 2010 gli.tc/h festival in
1581
+
Chicago, thirty percent of the entries were based on the datamoshing technique (around
1582
+
80 of a total 240). The technique that was used to critique popular culture, by artists like
1583
+
König or Davis, was now used to generate live visuals for the masses.16 Datamoshing had
1584
+
become a controlled, consumed and established effect. The aesthetic institutionalization
1585
+
of the datamoshing artifact became more evident when Murata’s video art work monster
1586
+
movie (2005), which used datamoshing as a form of animation, entered the Museum of
1587
+
Modern Art in New York in an exhibition in 2010.
1588
+
This ‘new’ form of conservative glitch art puts an emphasis on design and end products,
1589
+
rather than on the post-procedural and political breaking of flows. There is an obvious
1590
+
critique here: to design a glitch means to domesticate it. When the glitch becomes domesticated into a desired process, controlled by a tool, or technology – essentially cultivated
1591
+
– it has lost the radical basis of its enchantment and becomes predictable. It is no longer a
1592
+
break from a flow within a technology, but instead a form of craft. For many critical artists,
1593
+
it is considered no longer a glitch, but a filter that consists of a preset and/or a default:
1594
+
what was once a glitch is now a new commodity.
1595
+
1596
+
The Glitch Art Genre:
1597
+
Between the Void and Commoditized Form
1598
+
The fatal manner of glitch, its orientation towards the destruction of what is, can present
1599
+
a problem to those who want to describe old and new culture as a continuum of different
1600
+
discrete practices. One way to deal with this problem is to repeatedly coin new terms and
1601
+
concepts to make room for splinter practices within the expanding media cultural field.
1602
+
An abundance of designations such as databending, datamoshing and circuitbending
1603
+
have come into existence to name and bracket varieties of glitch practices, but all in fact
1604
+
refer to similar practices of breaking flows within different technologies or platforms.
1605
+
While technological glitch is primarily a process of shock requiring investigation and cognition, glitch art is best described as a collection of forms and events that oscillate between
1606
+
extremes: the fragile, technologically-based moment(um) of a material break, the conceptual
1607
+
or techno-cultural investigation of breakages, and the accepted and standardized commodity
1608
+
that a glitch can become. To encapsulate a whole range of unstable processes and sometimes
1609
+
almost contradictory intentions of glitch artists, it is useful to consider glitch art as a genre.
1610
+
In thinking about a genre that encompasses both the most rebellious and the most stable or
1611
+
commoditized works of glitch, the first question that arises is whether there can even be any
1612
+
common denominator in these works. What does saying ‘glitch is a genre’ actually say?
1613
+
15 | See quote, beginning of this chapter.
1614
+
16 | Peter Kirn, ‘Live Glitching with MIA at Coachella: Glotchy-Glithcy Videos, Pictures, Live Gig Report’, Create Digital Motion, 1 May 2009, http://
1615
+
createdigitalmotion.com/2009/05/01/live-glitching-with-mia-at-coachella-glotchy-glithcy-videos-pictures-live-gig-report/#more-3750.
1616
+
1617
+
55
1618
+
1619
+
To consider glitch art as a genre is to emphasize that genres are social and consensusbased constructs, rather than definitive categories.17 Steve Neale has suggested that
1620
+
genres are best understood as processes:
1621
+
The process-like nature of genres manifests itself as an interaction between three
1622
+
levels: the level of expectation, the level of the generic corpus, and the level of the
1623
+
‘rules’ or ‘norms’ that govern both. […] the elements and conventions of a genre
1624
+
are always in play rather than being, simply re-played; and any generic corpus is
1625
+
always being expanded.18
1626
+
While genres are always ‘in play’, they also – by definition – have some sort of organized
1627
+
and perceived unity. This unity models both how a viewer perceives any work in the genre
1628
+
and how she comes to associate new works within it. Mary Ann Doane suggests that ‘the
1629
+
unity of a genre is generally attributed to consistent patterns in thematic content, iconography, and narrative structure’.19 In glitch art, this ‘thematic content’ can be found within
1630
+
the work’s language and design, while iconographic and narrative themes are positioned
1631
+
within glitch art’s investment in the rupture of procedures and technique, the break from
1632
+
a flow or the void of meaning in the social understanding and the esthetical references.
1633
+
To call glitch a genre also means to suggest that it is intelligible as a tendency: to exploit
1634
+
medium-reflexivity and to take on the rhetorical questioning of the perfect use and function of technologies, their conventions and expectations. Paradoxically then, out of its
1635
+
instantiation in error and breakages, Glitch art can, through its play with conventions and
1636
+
expectations, be described as a genre that fulfills certain expectations. This reflexive approach to materiality in glitch tends to, as Katherine Hayles would assert, re-conceptualize
1637
+
materiality itself as ‘the interplay between a text’s physical characteristics and its signifying strategies’. Rather than suggesting media materiality as fixed in physicality, Hayles’
1638
+
re-definition is useful because it
1639
+
opens the possibility of considering texts as embodied entities while still maintaining a central focus on interpretation. In this view of materiality, it is not merely
1640
+
an inert collection of physical properties but a dynamic quality that emerges from
1641
+
the interplay between the text as a physical artifact, its conceptual content, and the
1642
+
interpretive activities of readers and writers.20
1643
+
Glitch genres perform reflections on materiality not just on a technological level, but also
1644
+
by playing off the physical medium and its non-physical, interpretative or conceptual
1645
+
characteristics. To understand a work from the genre of glitch art completely, each level
1646
+
of this notion of (glitch) materiality should be studied: the text as a physical artifact, its
1647
+
conceptual content, and the interpretive activities of artists and audiences.
1648
+
1649
+
17 | Rick Altman, Film/Genre, London: British Film Institute, 1999.
1650
+
18 | Steve Neale, ‘The Question Of Genre’, Screen, vol. 31.1 (1990): pp. 45-66. p. 56.
1651
+
19 | Mary Ann Doane, The Desire to Desire: The Woman’s Film of the 1940s, Bloomington and Indianapolis: Indiana University Press, 1987. p. 34.
1652
+
20 | N. Katherine Hayles, ‘Print is flat, code is deep: The importance of media-specific analysis’, Poetics Today 25, no. 1 (2004): pp. 67-90.
1653
+
1654
+
56
1655
+
1656
+
The Genre Paradox
1657
+
Obsolescence never meant the end of anything, it’s just the beginning.21
1658
+
- McLuhan
1659
+
1660
+
There is another factor to consider in this pursuit of materiality around glitch art
1661
+
genres. As I have described in the opening chapters, software engineering paradigms
1662
+
are fixated upon the development of better, faster and stronger technologies, while
1663
+
the ideal transparent technology will never be achieved and remains a mythological
1664
+
holy grail. On top of this dominant cultural comprehension of media technological
1665
+
‘progression’ however, it is also the case that engineers are economically driven to
1666
+
strive for built-in obsolescence. Paradoxically, while designing for great perfection, it
1667
+
is a basic economic condition for the media engineer of our time to always save room
1668
+
for improvement. This ‘planned obsolescence’ results in the proprietary capitalist
1669
+
scheming for the limited usage of each new purchased technology, which will manipulate the consumer into future investments on (sooner) improving his technologies. I would like to argue that this economical reasoning is very much connected to
1670
+
the growing fetishization of nostalgic imperfection in (glitch) art, which over the last
1671
+
decades has become a kind of conceptual virus. Today it is completely normal to pay
1672
+
extra money for aesthetically appealing plugins like Hipstamatic or Instagram, that
1673
+
imitate (analogue) imperfections or nostalgic errors, like ‘faux vintage’ lens flare and
1674
+
lomographic discolorations.
1675
+
Built-in obsolescence and built-in nostalgia have made the gap between new and old
1676
+
technologies both smaller and more dialectical. While the obsolescence and nostalgic revival of imperfect media used to be closely connected to the factor of (linear)
1677
+
time, this factor is now more disorganized, transforming the uncanny anachronism
1678
+
or avant-garde tendencies of post-procedural glitch into a fetish: something that is
1679
+
(‘now’) understood as a sign of (any ‘cool’) time. This apparent coming together of the
1680
+
hype cycle (the arrival, adoption and social distribution of specific technologies) with
1681
+
new technologies’ designed-for obsolescence, results in glitch itself being increasingly understood as retro-nostalgic artifacts. Given that the radical moment(um) and
1682
+
conceptual utility of glitch was at least initially a way for artists to penetrate and experience economical and political drives (and their critique) within the development
1683
+
of new technologies, this nostalgic hovering around glitch sets up very strong contradictions and tensions within the glitch art genre. If I am to describe glitch art as a
1684
+
genre then (which I argue is quite a useful way to comprehend the inter-influencing
1685
+
forms, reflexive materialities and expectations generated around glitch practices),
1686
+
it is important to bare in mind Rick Altman’s warning (paraphrasing Wittgenstein)
1687
+
about categorical ‘genre’ interpretations. He suggests to the media theorist:
1688
+
Don’t say: “There must be something common…” but look and see whether
1689
+
there is anything common to all. In the past, it has simply been taken for
1690
+
granted that genres are broadly shared categories […]. When we look more
1691
+
closely at generic communication, however, it is not sharing and understand21 | Régine Debatty, ‘Playlist, it’s not (just) about nostalgia’, Make Money Not Art, 15 January, 2010, http://we-make-money-not-art.com/
1692
+
archives/2010/01/previously-playlist-playing-ga.php.
1693
+
1694
+
57
1695
+
1696
+
ing that appear, but competing meanings, engineered misunderstanding and
1697
+
a desire for domination rather than communication.22
1698
+
Altman implies that classification by genre is neither an objective nor a clear activity, since
1699
+
the predication of meaning always precedes the act of classification. In order to place an
1700
+
item in one category, it must first be interpreted as being such-or-such. This interpretation is always and inevitably an act of classification and thus involves the domination of
1701
+
certain iconographic structures. This occurs for example with the work 404 error by Jodi,
1702
+
which has become not just about an error or non-place, but has been erected as an iconographic work standing for a ‘desired destination’, and spawned a cult of broken link art
1703
+
works. Such works, at the same time, insist that their spectators establish new conceptual
1704
+
paradigms for approaching these particular works of glitch art.23
1705
+
The genre of glitch art draws heavily upon spectator literacy (references to media technology texts, aesthetics and machinic processes) as well as on knowledge of more ‘conventional’ canons of media-reflexive modern art. Accordingly, glitch art prompts the
1706
+
spectator to engage not only with complex themes, but also with complex subcultural and
1707
+
meta-cultural narratives or gestures, presenting considerable cognitive challenges. Users
1708
+
do not consume but instead become prosumers, active participants in a culture invested
1709
+
in constant re-definition.
1710
+
1711
+
22 | Rick Altman, Film/Genre, London: British Film Institute, 1999. p. 99. The citation is from Wittgenstein’s Philosophical Investigations (posthumously published in 1953), section 66, where the philosopher attempts to establish common features of games – a project that is indeed
1712
+
very much related to the establishing of genre definitions.
1713
+
23 | White, Michele, ‘The Aesthetics of Failure: Confusing Spectators with Net Art Gone Wrong’, in The Body and the Screen: Theories of Internet
1714
+
Spectatorship, Cambridge, MA: MIT Press, 2006, pp. 85-113. p. 99.
1715
+
1716
+
58
1717
+
1718
+
59
1719
+
1720
+
Of 313 returns only the bookmarks that list ”glitch” as top-tag were inserted into the issuecrawler
1721
+
1722
+
The size of the nodes depends on the amount of in-links within the network.
1723
+
1724
+
The color of the nodes depends on the amount of out-links within the network.
1725
+
1726
+
what spheres are part
1727
+
1728
+
of this glitch-artist
1729
+
1730
+
bookmark network?
1731
+
1732
+
Crawl Depth: 2
1733
+
1734
+
Iterations: 1
1735
+
1736
+
Co-link Analysis Mode: page
1737
+
1738
+
Source: Delicious. Query: ”glitch” + ”blog” + ”artist” ( = 313 results).
1739
+
1740
+
web sphere(s) and
1741
+
1742
+
Issuecrawler.net/ settings
1743
+
Privilege starting points: off
1744
+
1745
+
Method:
1746
+
1747
+
form (a) networked
1748
+
1749
+
-tagged bookmarks
1750
+
1751
+
glitch + blog + art
1752
+
1753
+
what extend do the
1754
+
1755
+
cluding) of links. To
1756
+
1757
+
including (or ex-
1758
+
1759
+
organizing and
1760
+
1761
+
of curating - the
1762
+
1763
+
delicious is an act
1764
+
1765
+
Bookmarking on
1766
+
1767
+
the bookmarkers perspective
1768
+
1769
+
glitch blogosphere
1770
+
1771
+
Organizing Glitch Spheres
1772
+
1773
+
60
1774
+
1775
+
Of 505 returns only the bookmarks that list ”glitch” as top-tag were inserted into the issuecrawler
1776
+
1777
+
The size of the nodes depends on the amount of in-links within the network.
1778
+
1779
+
The color of the nodes depends on the amount of out-links within the network.
1780
+
1781
+
what spheres are part
1782
+
1783
+
of this glitch-artist
1784
+
1785
+
bookmark network?
1786
+
1787
+
Crawl Depth: 2
1788
+
1789
+
Iterations: 2
1790
+
1791
+
Co-link Analysis Mode: page
1792
+
1793
+
Source: Delicious. Query: ”glitch” + ”artist” ( = 505 results).
1794
+
1795
+
web sphere(s) and
1796
+
1797
+
Issuecrawler.net/ settings
1798
+
Privilege starting points: off
1799
+
1800
+
Method:
1801
+
1802
+
form (a) networked
1803
+
1804
+
-tagged bookmarks
1805
+
1806
+
the glitch + artist
1807
+
1808
+
To what extend do
1809
+
1810
+
excluding) of links.
1811
+
1812
+
including (or
1813
+
1814
+
organizing and
1815
+
1816
+
of curating - the
1817
+
1818
+
delicious is an act
1819
+
1820
+
Bookmarking on
1821
+
1822
+
the bookmarkers perspective
1823
+
1824
+
an organization of glitch artists
1825
+
1826
+
61
1827
+
1828
+
Copy homepages of actors that are indexed in at least 2 lists (56). Insert urls into the issuecrawler
1829
+
1830
+
The size of the nodes depends on the amount of in-links within the network.
1831
+
1832
+
The color of the nodes depends on the amount of out-links within the network.
1833
+
1834
+
sphere(s) and if so,
1835
+
1836
+
what sphere(s) can be
1837
+
1838
+
distinguished?
1839
+
1840
+
Crawl Depth: 2
1841
+
1842
+
Iterations: 1
1843
+
1844
+
Co-link Analysis Mode: page
1845
+
1846
+
Source: query google for inurl:http://twitter.com inurl: glitch inurl:members
1847
+
1848
+
(a) networked web
1849
+
1850
+
Issuecrawler.net/ settings
1851
+
Privilege starting points: off
1852
+
1853
+
Method:
1854
+
1855
+
their websites form
1856
+
1857
+
listed actors and
1858
+
1859
+
But do the glitch-
1860
+
1861
+
stream or network.
1862
+
1863
+
within a particular
1864
+
1865
+
ing) of contacts
1866
+
1867
+
cluding (or exclud-
1868
+
1869
+
entails the active in-
1870
+
1871
+
organizing - it
1872
+
1873
+
is an act of social
1874
+
1875
+
Making Twitter lists
1876
+
1877
+
the bookmarkers perspective
1878
+
1879
+
glitch actors organized
1880
+
1881
+
Glitch art networked
1882
+
Since the last gli.tc/h festival in Chicago, I have been noticing glitch networks growing
1883
+
more and more tight. This observation was reinforced for me after May 18, 2011, when
1884
+
JamesBWatson deleted the ‘Glitch Art’ article on Wikipedia. Some strong reactions to this
1885
+
deletion within the online glitch scene were aroused when I (re-)posted an image of this
1886
+
moment on my website. Does it say something about ‘glitch’ culture that a very obviously
1887
+
partial and incomplete definition of the field, with links to just a few select artists, would
1888
+
sit unedited in such an ‘encyclopedic’ space, and then be removed for not being representative enough? Of course. At the same time though, through this incident it became obvious that the glitch art scene does actively know itself, define itself and relate to itself as a
1889
+
tangible community of actors in certain ways. I reflected more deeply upon the implicit
1890
+
organization of glitch artists on the internet, as a complex community of specific interinfluencing actors and objects. This notion inspired me to attempt to map the difficult-torepresent online existence and associations of glitch art practices and culture.
1891
+
I invited Esther Weltevrede, a PhD candidate at the humanities department of the University of Amsterdam working on internet sphere mapping and analysis, to assist me in
1892
+
some modest experiments towards this end. There are a number of problems that immediately arise in mapping web spheres. First of all, we had to choose a bias – some starting
1893
+
points from which to ‘search’ – because there is no way to create the ‘integral map of glitch
1894
+
art’ without starting points (and also no way of representing all possible starting points).
1895
+
There is bias in the tools used to do the mapping, in the web platforms that the researcher
1896
+
chooses to focus upon (where tags are scraped from), and in the depth and level of mapping assumed to glean useful levels of detail and degrees of understanding from the larger
1897
+
data set acquired by the process.
1898
+
Esther and I chose to use the Issuecrawler, a tool that indexes the web following set instructions by crawling from particular chosen starting points. We focused on the bookmarking
1899
+
web service delicious and social micro blogging software Twitter as the focus of the data
1900
+
scrape. In delicious, we only scraped bookmarks that used glitch as a ‘top-tag’. The maps
1901
+
were then organized around three research questions. First of all: 1) ‘Bookmarking on delicious is an act of curating – the organizing and including (or excluding) of links. To what
1902
+
extend do the ‘glitch’ + ‘blog’ + ‘art’-tagged bookmarks form (a) networked web sphere(s)
1903
+
and what spheres are part of this glitch-artist bookmark network?’ 2) ‘To what extent do
1904
+
the glitch + artist -tagged bookmarks form (a) networked web sphere(s) and what spheres
1905
+
are part of this glitch-artist bookmark network?’ And: 3) ‘Making Twitter lists is an act
1906
+
of social organizing – it entails the active including (or excluding) of contacts within a
1907
+
particular stream or network. But do the ‘glitch’-listed actors and their websites form (a)
1908
+
networked web sphere(s) and if so, what spheres(s) can be distinguished?’
1909
+
Through a productive orientation around tool biases, the different maps could potentially
1910
+
give insight into glitch scene(s). The maps are generated specifically from the tags: ‘glitch’,
1911
+
‘blog’ and ‘art’ or ‘glitch’ and ‘artist’. We excluded generic social networks and media sharing platforms (YouTube, Vimeo) so as to focus on active artist and organizational websites.
1912
+
The maps reveal the actors and agents of glitch discovered through this process (in the form
1913
+
of websites), while the size and color of the nodes shows the quantitative in- and out-linking
1914
+
behavior of these actors. A node in the map only appears if there are at least two in- or out62
1915
+
1916
+
links to the site.01 Very basically, the more in-links and out-links that show up, the greater
1917
+
the artist or group’s online activity or authority (at least) in the network. Of course, glitch
1918
+
artists and organizations that do not explicitly propagate the tag ‘glitch’ will not easily be
1919
+
included in these maps. This is an acknowledged blind spot of the maps and of course does
1920
+
not mean that those that escape the grasp of the map are not part of any glitch sphere.02
1921
+
1922
+
Glitch Sphere Relations
1923
+
The Delicious map based on the ‘glitch’ + ‘artist’-tags and the Delicious map based on the
1924
+
‘glitch’ + ‘blog’ + ‘artist’-tags show two very similar, complex networks of inter-influencing
1925
+
actors within the glitch scene. These two maps enable a modest assessment of the location
1926
+
of the glitch art communities in relation to other web-based/digital arts communities. It
1927
+
appears that the glitch art sphere is closely connected to the spheres organized around digital tools, platforms and softwares (e.g. Processing, Openframeworks, Puredata, Arduino).
1928
+
Besides this, the glitch spheres are organized through some hubs or blog nodes. These can
1929
+
be divided into digital art blogs proximate to the New York City community of new-media art
1930
+
(such as Rhizome, 319scholes and the New Museum); blogs with a European geo-location
1931
+
(note for example Neural and Furtherfield); and nodes dedicated to festivals (note Transmediale, Bent festival and gli.tc/h). Vagueterrain, a digital arts blog, seems to function as a
1932
+
pre-eminent center-hub that is greatly responsible for bringing the different communities
1933
+
together, while being disproportionately small; notably it does not receive many in-links,
1934
+
whereas Rhizome does. This could be a matter of authority.
1935
+
Of all three maps, the ‘glitch’ + ‘artist’ map seems to offer up a most interesting map of
1936
+
glitch art nodes, with some anomalies. I had expected the Chicago school of glitch to have
1937
+
a more pronounced, heavily interlinked presence. While there is definitely a disproportionate amount of Chicago-based artists indexed in all of the maps, they do seem to be
1938
+
equally connected to nodes located geographically elsewhere and thus the rest of the
1939
+
world’s glitch artists. Besides the Chicago school of glitch, the presence of a Dutch community (Gieskes, Klomp, Menkman, Jodi, Impakt, mu, v2 and wormweb) is striking.
1940
+
It is a problem generated by the tool’s orientation around text search terms that other
1941
+
glitch actors, including for instance the Japanese visual school of glitch that includes
1942
+
ucnv, Youpy and Shusaku Hariya, are missing from the maps. For these artists, the use
1943
+
of Japanese over English means that they are less linked with other artists that are represented and this is why they just don’t ‘show up’.03
1944
+
01 | For the construction of the map, we set the Issuecrawler to crawl two iterations deep (to index the links not only from the starting point, but
1945
+
also from the first list of indexed sites) in order to partially overcome nodes created by, for instance, ‘the individual tagging himself’, and to
1946
+
be able to make more legible the differences between influential and less influential nodes.
1947
+
02 | We additionally deleted nodes like MySpace, Twitter, Flickr, Vimeo and creative commons because they tend to take a lot of space in the final
1948
+
visual outcome of a query invested otherwise in mapping independent (non-proprietary) art practices in relation. Platform nodes (of tags) would
1949
+
however be interesting for more platform-focused research. In the last map, we decided not to index the Twitter lists that were based on the computer gamed titled ‘Glitch’ since these lists are completely trivial to glitch art. As a full disclosure, I have to admit that some bugs in the mapping
1950
+
software led to the double appearance of some nodes. After some failed attempts to merge these ‘double’ nodes automatically or by hand, I deleted
1951
+
the smallest one of each. This led to a little discrepancy in the final maps, which I believe otherwise strongly calibrate an overview of the spheres.
1952
+
03 | In this sense, while the maps seem to ‘argue’ (that is, in their biases, rather than passively show) that glitch art is still a very western discourse
1953
+
and culture, responsive as it is to Euro-American techno-capitalisms and specific traditions of media/machine philosophy, more research
1954
+
would be required across language groups, and in to sub-cultural languages, and national webs, and local art scenes, to confirm that.
1955
+
1956
+
63
1957
+
1958
+
At the same time there is a contemporaneity to the maps that is quite informative. For
1959
+
example, in the summer of 2011 (during the time the research maps were constructed)
1960
+
IDN magazine featured numerous glitch artists and projects, including Quayola, Kim
1961
+
Asendorf and Clement Valla who are new to the glitch scene. They have already been captured in the delicious networks and are visible here. In this sense the maps created can be
1962
+
assumed to be useful snapshots of a still mutating field that is full of fresh data.
1963
+
The fact that I have published these results on my own rosa-menkman website (which
1964
+
appears quite large and colorful in the results), and that the crawler indexes both in-links
1965
+
and out-links to my blog, is a further matter for consideration. As a research authority, artistic agent, and active discursive hub within these spheres that I am researching, the rosamenkman blog could be considered to be itself shaping the mapped community. This is an
1966
+
important problem to keep in mind and an ongoing consideration for my research work.
1967
+
In contrast, personal blogs and artists pages such as Goto80, designingimperfection,
1968
+
beigerecords, Gieskes, Jonsatrom and jodi seem not to link-out into the glitch community, while receiving in-links from their peers, which tends to reveal matters or even
1969
+
vectors of status and authority.04 Notice however how Vade’s many websites are mapped.
1970
+
Because this artist has many different web-located nodes (the URLs syphon, Vade.info,
1971
+
v002, abstrakt.vade.info all refer to his sites) his appearance within the mapped community seems both hyper-present and diffused (given the many different nodes all governed
1972
+
by Vade). Both Vade and Pixelnoizz take up an interesting place within the map, forming
1973
+
active hubs between the generative or design driven side of the map (right) and the more
1974
+
post-procedural driven side of glitch art (on the left side of the glitch map).
1975
+
The third Twitter glitch-listed actor map consists of two very clearly separated spheres.
1976
+
The first sphere is centered around glitch.fm, the glitchhopforum and a couple of other
1977
+
glitch music related sites. The second is a bigger, diffused sphere that locates and maps
1978
+
the genre of glitch art. Within the latter, sites like 8bitpeoples (an chiptune/8bit/lo-fi community), gli.tc/h (the glitch art festivals website), Vagueterrain (the Digital Art/Culture/
1979
+
Technology blog), slowelectronics (where the music label slowelectronics is located) and
1980
+
personal blogs act as hubs in between the different communities. It is striking to me how
1981
+
many of the personal sites are connected with multiple hubs, indicating a larger trend of
1982
+
cross-fertilization and networking between the different digital art communities.
1983
+
1984
+
Some Final Reflections On The Glitch Spheres
1985
+
The circuitbending ‘community’ seems under-represented. Possible reasons for their lack
1986
+
of nodes could be that this community is not as ‘social’ (in terms of in- and out-linking)
1987
+
as other glitch artists working more directly online and with code. Or perhaps they are
1988
+
not tagged as ‘glitch’ or as ‘art’, or both. The maps leave out under-tagged and anti-social
1989
+
communities in this way – what appears mapped are the parts of the (glitch) web sphere
1990
+
built most strongly on link connections and common (English) labels.
1991
+
1992
+
04 | While this should in most cases indeed be understood as a sign of authority (or status), it could in some cases also be a problem with the
1993
+
crawler encountering a coding obstruction when fetching out-links.
1994
+
1995
+
64
1996
+
1997
+
Interestingly, the Glitch.fm community (glitch music) is not tagged as ‘art’ or ‘artists’ in
1998
+
delicious (they have no delicious presence), while the click and cut/microsound musicians don’t seem to be active bloggers. They are not present in the ‘glitch’ + ‘blog’ + ‘art’
1999
+
sphere, but are otherwise tagged as glitch artists in the map that does not include the
2000
+
‘blog’ tag. The 8bit/chiptune music scene, on the other hand, is notably connected to
2001
+
the glitch scene in all three maps. Key players in this inter-linkage are: Goto80, noteNdo,
2002
+
Nullsleep, NO CARRIER and the 8bitpeoples music label.
2003
+
Glitch art communities of course do not start or end at the ‘borders’ of these maps, and
2004
+
many more could be created which would enable additional insights into the varied communities and ‘techno-sociality’ of glitch agents and practices. Link relations are, complexly, ‘influence’ relations but like the glitch itself they do not initially or easily give out
2005
+
knowledge. You have to embed yourself a little and trace the moment(um).
2006
+
2007
+
The Emancipation
2008
+
of Dissonance Glitch
2009
+
I don’t use the accident. I deny the accident.
2010
+
There is no accident, just as there is no beginning and no end.05
2011
+
- Jackson Pollock
2012
+
2013
+
The noiseless channel doesn’t exist. What makes every medium specific is how it fails to
2014
+
reach a state of complete transparent immediacy. These failures are embodied by noise
2015
+
artifacts; categorizable as either compressions, or feedback, or the not (yet) technologically defined break of a (computational) flow, named glitch. Moving from information
2016
+
theory into the art and culture of noise and noise artifacts, glitch art proliferates in a spectrum of disturbances that traverse both the sonic and visual, technical and socio-cultural
2017
+
realm. Here the difference between failure and glitch becomes important: while failure
2018
+
is a phenomenon to overcome, the glitch is a phenomenon that will be incorporated into
2019
+
new processes and conditions of technological design or cultural meaning.
2020
+
Contemporary glitch artists exploit the inherent moment(um) of glitch in different ways.
2021
+
A threefold categorization of glitches addressed a continuum for thinking about glitch:
2022
+
from complete machine ‘spontaneity’ in the accident form, to controlled, debuggable or
2023
+
conceptual glitching; to a more conventional realm of glitch design and aesthetics. The
2024
+
perfect glitch only exists for a spectator at the tipping point between destruction and the
2025
+
creation of something new; this is more a dialectical relation than a linear trajectory of
2026
+
possibility. Glitch reveals but also bridges gaps between the functioning and the malfunctioning of systems.
2027
+
In the end, the glitch is a subjective phenomenon. There is no unequivocal cultural definition of glitch, as there is none for noise, because in the end, what glitch is and what glitch
2028
+
2029
+
05 | Monica Bohm-Duchen, The private life of a masterpiece, California: University of California Press, 2001. p. 230.
2030
+
2031
+
65
2032
+
2033
+
is not is a subjective matter. Further, as a sub-genre that participates in larger media cultures of distributed authorship, this subjective experience of glitch is paradoxically shared
2034
+
by many, which makes glitch theory difficult to practice, accessible to many, contestable
2035
+
and necessary. An intended or designed error can still rightfully be called glitch art; and
2036
+
glitch art is not always just a personal experience of shock, but can be a metaphorical
2037
+
expression, dependent upon multiple agents for interpretation. Accordingly, it is less interesting for theory to police the difference between true or false glitch art, than to understand how and through which technological systems and cultural fabrics any particular
2038
+
work of glitch art comes to be understood and experienced as glitch.
2039
+
At the same time, some recent shifts within the realm of glitch art are important to keep
2040
+
track of. It seems that increasingly, glitch art practices downplay the technological dimension of glitch, and that the concept of glitch has changed. As the error itself has been
2041
+
increasingly gentrified, the glitch is already being supposedly ‘upgraded’ to more static
2042
+
and imagistic values (minus the radical moment(um) of glitch). Glitch is also becoming a
2043
+
prominent area of study, and archive of thought, for the media culture intellectual. Academics politicize their work through the solid cultural and technological understanding
2044
+
of digital society developed in and by glitch culture, while glitch risks being reduced to
2045
+
just another theory for thinking the subjective experience of media. Perhaps, since glitch
2046
+
art is full of paradoxes, describing glitch art as a genre, institutionalizing, is yet another
2047
+
paradox that could be in line with the corrupting and damaging future potentials of glitch.
2048
+
To think with glitch is to straddle a gap between non-sense and knowledge. It is to search
2049
+
for the unfamiliar while at the same time to tenaciously de-familiarize oneself from what
2050
+
might be taken for granted of software, hardware and signal realities by less critical media
2051
+
theoreticians and artists. To embrace and account for glitch is therefore to be potentially
2052
+
open to new critical modes of thought and action. When these notions of glitch’s radical
2053
+
difference become (paradoxically) standardized, the actual agents of glitch culture adapt
2054
+
and move to take on and mine other technologies, protocolized flows, and discourses
2055
+
elsewhere. Glitch work is a kind of corrupting investigative work, followed by a vision that
2056
+
destroys itself by its own purposive modes of inquiry. Like the best ideas, glitch practices
2057
+
are dangerous because they generate awareness.
2058
+
Some consider glitches as solely technological phenomena, while others perceive them
2059
+
as social constructions reactive to technological expectations or aesthetics. Glitchspeak
2060
+
explains the utterances that do not fail to be heard, yet at the same time exist outside of
2061
+
knowledge. At the same time, cultural and technological flows and functions, designed to
2062
+
be taken for granted, cannot be understood without such interruptions. This is why the
2063
+
study of glitch is necessary. Study what is outside of knowledge, start and continue with
2064
+
glitch studies. The glitch is what you can just get away with!
2065
+
2066
+
66
2067
+
2068
+
Bibliography
2069
+
Adelbert, Ames, Jr. ‘The Morning Notes’, in Ernst Hans Josef Gombrich, The Sense of Order: A Study in the
2070
+
Psychology of Decorative Art, London: Phaidon Press, 1984. p. 117.
2071
+
Altman, Rick. Film/Genre, London: British Film Institute, 1999.
2072
+
Attali, Jacques and Brian Massumi. Noise: The Political Economy of Music, Manchester: Manchester
2073
+
University Press, 1985. p. 33. George Orwell, Nineteen Eighty-Four, London: Secker and Warburg, 1949.
2074
+
Ballard, Susan. ‘Information, Noise and et al’, M/C Journal, 10.5 (October, 2007).
2075
+
http://journal.media-culture.org.au/0710/02-ballard.php.
2076
+
Barnouw, Jeffrey. ‘The Morality of the Sublime: To John Dennis’, Comparative Literature, Vol. 35, No. 1
2077
+
(Winter, 1983): p. 21-42.
2078
+
Baumgärtel, Tilman. ‘TP: Interview with Jodi. "We love your computer"', Telepolis. May 2006,
2079
+
http://www.heise.de/tp/r4/artikel/6/6187/1.html.
2080
+
Benjamin, Walter. ‘The Work of Art in the Age of Mechanical Reproduction’, in Hannah Arendt (ed.)
2081
+
Illuminations, New York: Schocken, 1968, pp. 219-254.
2082
+
Blanchot, Maurice. The Writing of the Disaster, Nebraska: University of Nebraska Press, 1995.
2083
+
Bolter, Jay David and Richard Grusin. Remediation: Understanding New Media, Massachusetts: MIT Press, 1999.
2084
+
2085
+
Bohm-Duchen, Monica. The Private Life of a Masterpiece, California: University of California Press, 2001.
2086
+
Broeckmann, Andreas, Joke Brouwer, Bart Lootsma, Arjen Mulder and Lars Spuybroek. The Art of the
2087
+
Accident, NAI Publishers/V2_Organisatie: Rotterdam, 1998.
2088
+
Cascone, Kim. ‘The Aesthetics of Failure: Post-Digital Tendencies in Contemporary Computer Music’,
2089
+
Computer Music Journal 24.4 (Winter 2000).
2090
+
Cranfield-Rose, James Brady. Tick-tick-tick-tick-tick… Oval, the glitch and the utopian politics of noise,
2091
+
unpublished master thesis, Burnaby, Canada: Simon Fraser University, 2004.
2092
+
http://lib-ir.lib.sfu.ca/handle/1892/8961.
2093
+
Davis. Paul B. Define Your Terms (Or Kanye West Fucked Up My Solo Show), 28th May 2009.
2094
+
http://www.seventeengallery.com/index.php?p=3&id=42.
2095
+
Debatty, Régine. ‘Playlist, it’s not (just) about nostalgia’, Make Money Not Art, 15 January, 2010.
2096
+
http://we-make-money-not-art.com/archives/2010/01/previously-playlist-playing-ga.php.
2097
+
DeLanda, Manuel. War in the Age of Intelligent Machines, New York: Zone Books, 1991.
2098
+
Deleuze, Gilles and Claire Parnet. Dialogues II, London and New York: Continuum, 2006.
2099
+
Deleuze, Gilles and Pierre-Félix Guattari. A Thousand Plateaus: Capitalism and Schizophrenia, Trans. B.
2100
+
Massumi, Londen: The Athlone Press, 1988.
2101
+
Doane, Mary Ann. The Desire to Desire: The Woman’s Film of the 1940s, Bloomington and Indianapolis:
2102
+
Indiana University Press, 1987.
2103
+
Durham, Meenakshi Gigi and Douglas Kellner. Media and Cultural Studies, New Jersey: Wiley-Blackwell, 2006.
2104
+
2105
+
Gombrich, Ernst Hans Josef. The Sense of Order: A Study in the Psychology of Decorative Art, London:
2106
+
Phaidon Press, 1984.
2107
+
Hayles, N. Katherine. ‘Print is flat, code is deep: The importance of media-specific analysis’, Poetics Today
2108
+
25, no. 1 (2004): pp. 67-90.
2109
+
Hegarty, Paul. Noise/Music: A History, London and New York: Continuum, 2007.
2110
+
Heim, Michael. The Metaphysics of Virtual Reality, New York: Oxford University Press, 1993.
2111
+
Kahn, Douglas. Noise, Water, Meat: A History of Sound in the Arts, Massachusetts: MIT Press, 1999.
2112
+
Kirn, Peter. ‘Live Glitching with MIA at Coachella: Glotchy-Glithcy Videos, Pictures, Live Gig Report’,
2113
+
Create Digital Motion, 1 May 2009. http://createdigitalmotion.com/2009/05/01/live-glitching-with-mia-atcoachella-glotchy-glithcy-videos-pictures-live-gig-report/#more-3750.
2114
+
Kittler, Friedrich. Draculas Vermächtnis: Technische Schriften, Leipzig: Reclam Verlag Leipzig, 1993.
2115
+
2116
+
67
2117
+
2118
+
Kluitenberg, Eric. Delusive Spaces. Essays on Culture, Media and Technology, Rotterdam: NAi Publishers
2119
+
and Amsterdam: Institute of Network Cultures, 2008.
2120
+
Kluitenberg, Eric. Transfiguration of the Avant-Garde/The Negative Dialectics of the Net, posting to nettime
2121
+
mailing list, 23 January, 2002. http://www.nettime.org/Lists-Archives/nettime-l-0201/msg00104.html.
2122
+
Larsby, Johan and Rosa Menkman. Monglot, 2011. http://rosa-menkman.blogspot.com/search/label/Monglot.
2123
+
Liu, Alan. ‘What’s cool?’, in The Laws of Cool: Knowledge Work and the Culture of Information, Chicago:
2124
+
University of Chicago Press, 2004, pp. 176-179.
2125
+
Lotringer, Sylvere and Paul Virilio, The Accident of Art, Semiotext(e): New York, 2005.
2126
+
Mackenzie, Adrian. ‘Codecs’, in Matthew Fuller (ed.) Software Studies, Massachusetts: MIT Press, 2008.
2127
+
Matheson, Donald. ‘Weblogs and the Epistemology of the news: some Trends in Online Journalism’, Sage
2128
+
Journals, London: SAGE Publications, Thousand Oaks CA and New Delhi Vol 6.4 (2004): pp. 443-468.
2129
+
McLuhan, Marshall. Understanding Media: The Extensions of Man, New York: McGraw Hill, 1964.
2130
+
Menkman, Rosa. Beauty in the Age of Digital Art; aesthetic, poetic or rhetoric, June 2006.
2131
+
http://rosa-menkman.blogspot.com/2006/05/beauty-in-age-of-digital-art.html.
2132
+
Menkman, Rosa. ‘Glitch Studies Manifesto’ in Geert Lovink and Rachel Somers Miles (eds) Video Vortex
2133
+
Reader II: moving images beyond YouTube, Amsterdam: Institute of Network Cultures, 2011, pp. 336-347.
2134
+
http://dl.dropbox.com/u/17713740/Glitch%20Studies%20Manifesto%20rewrite%20for%20Video%20
2135
+
Vortex%202%20reader.pdf.
2136
+
Menkman, Rosa. Jodi op de Pijnbank, unpublished master thesis, Amsterdam: University of Amsterdam, 2006.
2137
+
http://home.student.uva.nl/rosa.menkman/Jodi%20op%20de%20pijnbank.pdf.
2138
+
2139
+
Michele, Michele. ‘The Aesthetics of Failure: Confusing Spectators with Net Art Gone Wrong’, in The Body
2140
+
and the Screen: Theories of Internet Spectatorship, Cambridge, MA: MIT Press, 2006, pp. 85-113.
2141
+
Miller, Don. Biography for the Playlist exhibition, 2010, http://www.imal.org/playlist/artworks/17.
2142
+
Moradi, Iman. Glitch Aesthetics, Unpublished Bachelor Thesis, Huddersfield, UK: University of
2143
+
Huddersfield, 2004. http://www.oculasm.org/glitch/download/Glitch_dissertation_print_with_pics.pdf.
2144
+
Neale, Steve. ‘The Question Of Genre’, Screen, vol. 31.1 (1990): pp. 45-66.
2145
+
Pepperell, Robert. ‘Computer aided creativity: practical experience and theoretical concerns’, in
2146
+
Proceedings of the 4th conference on Creativity & cognition, Loughborough, UK: ACM, 2002, pp. 50-56.
2147
+
http://portal.acm.org/citation.cfm?id=581710.581720&type=series.
2148
+
Prandoni, Paolo and Martin Vetterli. Signal Processing for Communications, Lausanne: EPFL Press, 2008.
2149
+
http://www.sp4comm.org/webversion.html.
2150
+
Shannon, Claude Elwood. ‘A Mathematical Theory of Communication’, Reprinted with corrections from
2151
+
The Bell System Technical Journal, Vol. 27 (July, October, 1948).
2152
+
The American Heritage® Dictionary of the English Language, Fourth Edition, Houghton Mifflin Company, 2004.
2153
+
http://dictionary.reference.com/browse/glitch.
2154
+
2155
+
Truscello, Michael. Behind the Blip: Essays on the Culture of Software (review), Cultural Critique, no. 63,
2156
+
(2006): pp. 182-187.
2157
+
Virilio, Paul and Julie Rose. The Original Accident, Cambridge: Polity Press, 2007.
2158
+
Wiener, Norbert. Cybernetics: Or Control and Communication in the Animal and the Machine, Paris:
2159
+
Hermann & Cie & Camb, 1948.
2160
+
Williams, Raymond. Television: Technology and Cultural Form, Hanover: University Press of New England, 1974.
2161
+
2162
+
Winkler, Dan and Anton Marini. Open Emu, 14 Jan 2009.
2163
+
http://openemu.sourceforge.net/.
2164
+
2165
+
68
2166
+
2167
+
ment(um
2168
+
Glitch
2169
+
Momen
2170
+
(um)
2171
+
Glitc
2172
+
Mo- The
2173
+
Momen
2174
+
Glitch culture organizes itself around the investigation
2175
+
and aestheticization of breaks in the conventional flow
2176
+
of information or meaning within (digital) communication systems.
2177
+
2178
+
In this book, Rosa Menkman brings in early information theorists not usually encountered in glitch’s theoretical foundations to refine a signal and informational vocabulary appropriate to glitch’s technological
2179
+
moment(um) and orientations. The book makes sense
2180
+
of recent glitch art and culture: technically, culturally,
2181
+
critically, aesthetically and finally as a genre.
2182
+
The glitch takes on a different form in relation to noise,
2183
+
failure or the accident. It transitions between artifact
2184
+
and filter; between radical breakages and commodification processes. Menkman shows how we need to be
2185
+
clearer about the relationship between the technical
2186
+
and cultural dimensions of glitch culture. Honing in
2187
+
on the specificities of glitch artifacts within this broader perspective makes it possible to think through some
2188
+
of the more interesting implications of glitched media
2189
+
experience. Using a critical media aesthetic orientation, Menkman addresses the ongoing definitional tensions, paradoxes, and debates that any notion of glitch
2190
+
art as a genre must negotiate, rather than elude.
2191
+
2192
+
Rosa Menkman is a Dutch visualist, theorist and curator, working
2193
+
with glitches, compressions, feedback and other forms of noise
2194
+
artifacts, aiming to contribute to the development of a discourse
2195
+
for glitch art and culture.
2196
+
2197
+
Amsterdam, The Netherlands 2011.
2198
+
ISBN/EAN 978-90-816021-6-7
2199
+
2200
+
+1299
system/public/assets/papers/readings/text/Menkman-Vernacular-of-File-Formats-2010.txt
+1299
system/public/assets/papers/readings/text/Menkman-Vernacular-of-File-Formats-2010.txt
···
1
+
A
2
+
VERNACULAR
3
+
OF FILE
4
+
FORMATS
5
+
6
+
1
7
+
8
+
AN EDIT GUIDE FOR COMPRESSION DESIGN
9
+
10
+
Type to enter text
11
+
12
+
Type to enter text
13
+
14
+
Tango between a corrupt format and its user.
15
+
!
16
+
!
17
+
!
18
+
!
19
+
Glitch art is a practice that studies
20
+
and researches
21
+
Contents
22
+
! !the
23
+
vernacular of file formats in exploitative
24
+
! Static! manners
25
+
! ! to
26
+
There is not sufficient data. Please enter data
27
+
deconstruct and create new, brutalist
28
+
works.
29
+
! ! (audio)visual
30
+
Uncompressed!!
31
+
You have too
32
+
However, glitch artists often go! ! beyond
33
+
this
34
+
Lossless! formal
35
+
! !
36
+
m u c h Noise Art > Filter art > when
37
+
approach;
38
+
Cool becomes
39
+
they realize
40
+
Hot >>>>>>>
41
+
that the glitch
42
+
does not !exists
43
+
! ! Lossy!
44
+
!
45
+
character!
46
+
without human perception and therefore
47
+
have
48
+
a
49
+
more
50
+
! Moving image!
51
+
!
52
+
A p p l y Glitches are hot. It is clear
53
+
inclusive
54
+
from what
55
+
approach
56
+
we can
57
+
to digital
58
+
see onmaterial.
59
+
MTV, ! ! MOV !
60
+
! !
61
+
g a u s s i a n Flickr, in the club or the
62
+
Thebookstore.
63
+
materiality of
64
+
While
65
+
glitchthe
66
+
art is
67
+
"Glitch:
68
+
constantly
69
+
mutating;
70
+
it !exists
71
+
! ! DV
72
+
!
73
+
!
74
+
blur.
75
+
designing imperfection" coffee
76
+
as an unstable
77
+
table book
78
+
assemblage
79
+
introduces
80
+
thatthe
81
+
relies
82
+
the one! hand the
83
+
! ! on
84
+
WMV!
85
+
glitch design aesthetic construction,
86
+
to the world
87
+
operation
88
+
of latteanddrinking
89
+
content! !of AVI!
90
+
the apparatus
91
+
(the
92
+
!
93
+
You still have designers, and Kanye West
94
+
medium)
95
+
uses glitches
96
+
and on the
97
+
to sing
98
+
otherabout
99
+
handhis
100
+
the! !work,
101
+
! the writer/artist,
102
+
!
103
+
!
104
+
quirks! Clone imperfect love life, the awkward,
105
+
and the interpretation
106
+
shy and physically
107
+
by theugly
108
+
reader and/or user (the
109
+
s t a m p f o r celebrate under the headermeaning).
110
+
"Glitched: Nerdcore for life".
111
+
s m o o t h Glitch has become hot. A brightly colored bubblegum
112
+
surface
113
+
wrapper that doesn't ask for much involvement, or offers any
114
+
stimulus. Inside I find gum that I keep chewing - hoping for
115
+
Your file has some new explosion of good taste - but the more I chew it,
116
+
i n v a l i d the less tasty and more rubbery it gets. Glitch design fulfills
117
+
m a r k e r s . an average, imperfect stereotype, a filter or commodity that
118
+
E n t e r n e w echoes a stabilized "medium is the message" standard.
119
+
markers
120
+
Thus, the materiality of the glitch art is not (just) the digital
121
+
Naturally, the "No Contentmaterial
122
+
- Just Imperfection"
123
+
that follows the
124
+
slogan
125
+
vernacular
126
+
of this of file formats, nor the
127
+
kind of hot glitch design is machine
128
+
complimented
129
+
it appears
130
+
by cool glitches.
131
+
upon, but a constantly changing
132
+
Y o u r In "The Laws of Cool", Alan
133
+
construct
134
+
Liu asksthat
135
+
himself
136
+
depends
137
+
What on
138
+
is "Cool"?
139
+
the interactions between text,
140
+
d i m e n s i o n s He describes that cool is social,
141
+
the ellipsis
142
+
esthetical,
143
+
of knowing
144
+
political
145
+
whats
146
+
and economic
147
+
cool
148
+
dynamics and the
149
+
do
150
+
n o t and withholding that idea.point
151
+
Thoseof who
152
+
viewinsist
153
+
fromon which
154
+
asking,the
155
+
are different actors make
156
+
correspond. definitely uncool.
157
+
meaning.
158
+
C h a n g e To answer the question that
159
+
Digital
160
+
we doartists
161
+
not dear
162
+
exploit
163
+
to asktheir
164
+
and come
165
+
digital materials thus also
166
+
dimensions
167
+
once more full round in themetaphorically
168
+
ellipsis, I will give
169
+
or my
170
+
critically
171
+
take on(showing
172
+
cool
173
+
the medium in a
174
+
ERROR
175
+
glitches. Cool glitches are critical
176
+
the glitches
177
+
statethat
178
+
or do
179
+
criticizing
180
+
not (only)the
181
+
focus
182
+
medium and its inherent
183
+
on a static end product, but
184
+
norms)
185
+
(also)and
186
+
onnot
187
+
a process,
188
+
just formally.
189
+
personal
190
+
G o t o d a t a exploration or a narrative element
191
+
more and
192
+
(that
193
+
more
194
+
often
195
+
people
196
+
reflects
197
+
arecritically
198
+
starting to use the term "glitch
199
+
therapy and on a medium). This is whygenre",
200
+
cool isI in
201
+
think
202
+
a constant
203
+
it has become
204
+
state ofapparent
205
+
flux,
206
+
that the question of
207
+
repair your as is the genre of "cool glitch
208
+
whatart",
209
+
constitutes
210
+
which finally
211
+
a genre,
212
+
existsand
213
+
as an
214
+
how a genre should be
215
+
registry
216
+
assemblage that relies onstudied
217
+
on the needs
218
+
one hand
219
+
to be
220
+
theincluded
221
+
construction,
222
+
in Glitch Studies. Also, in the
223
+
operation and content of the
224
+
case
225
+
apparatus
226
+
of a "glitch
227
+
(thegenre",
228
+
medium)
229
+
I think
230
+
andthere
231
+
on is a need to research
232
+
the other hand the work,
233
+
the process
234
+
the writer/artist,
235
+
of stylizationand
236
+
of glitch
237
+
the - the point where the
238
+
Y o u r interpretation by the reader
239
+
formal
240
+
and/orcreation
241
+
user (the of
242
+
meaning).
243
+
glitchesSoare
244
+
in not unknown, new
245
+
k e y f r a m e s the end, there is no one definition
246
+
utterances
247
+
of cool
248
+
but glitch
249
+
are becoming
250
+
art.
251
+
stabilized, new commodities
252
+
are missing.
253
+
and even filters.
254
+
Your codecs In an effort to make whatThis
255
+
waskind
256
+
once
257
+
of study
258
+
cool now
259
+
involves
260
+
hot, more
261
+
or visa
262
+
then just a vernacular of
263
+
are
264
+
n o t versa, and to take what
265
+
file formats
266
+
happened
267
+
or a in
268
+
research
269
+
the designing
270
+
into technology but also includes
271
+
supported.
272
+
imperfection book a step culture,
273
+
further, I individuals,
274
+
made this Vernacular
275
+
politics and
276
+
of the history of the
277
+
File Formats, in whichtechnology.
278
+
I study ways to exploit and
279
+
deconstructed the organizations of file formats into new,
280
+
brutalist designs.
281
+
But I still wonder if there is really anything consistent within
282
+
the glitch art "genre". If so, then I think it is the critical use
283
+
…I am waiting for the first of
284
+
"Glitchs
285
+
error, perceived
286
+
not dead" or
287
+
hoodie
288
+
non perceived,
289
+
in H&M. real or designed. And
290
+
And because "fans are asI bad
291
+
thinkaswhen
292
+
the ignorant",
293
+
I watch afor
294
+
glitched
295
+
the sake
296
+
video, or any other glitch
297
+
of being bad, I will definitely
298
+
work,
299
+
wearthis
300
+
the ishoodie.
301
+
what IHoera!
302
+
find most interesting to look for: what
303
+
Please
304
+
critical elements play a role in the work - does the work
305
+
respect the software. Now it is too little too
306
+
criticize
307
+
late - you
308
+
something,
309
+
did not or
310
+
try does
311
+
hard it show the technology in a
312
+
enough - you are just not good enough. [system
313
+
criticalshut
314
+
state?
315
+
down indefinitely]
316
+
2
317
+
Dear mr compression I write a 1000 poems to you Is this what they call
318
+
progress? Warmly yours, the noxious angel of history
319
+
320
+
UNCOMPRESSED
321
+
RAW
322
+
323
+
<-- Photoshop .RAW
324
+
3channel, 2020x1138. (with
325
+
scan lines from the original
326
+
video)
327
+
Raw image files contain
328
+
minimally processed data
329
+
(pixels) from the image
330
+
sensor of either a digital
331
+
camera, image scanner, or
332
+
motion picture film scanner.
333
+
The file header of a RAW
334
+
image typically contains
335
+
information concerning the
336
+
byte-ordering of the file, the
337
+
camera sensor information
338
+
and other image metadata like
339
+
exposure setting, camera/
340
+
scanner/lens model, date (and,
341
+
optionally, place) of shoot/
342
+
scan, format, size, number of
343
+
colors, and other information
344
+
needed to display the image.
345
+
It is possible to save a
346
+
RAW image file without a
347
+
header (choose header=0).
348
+
When the interleaved RAW
349
+
image is saved without a
350
+
header the computer
351
+
doesnʼt know the
352
+
dimensions or any other
353
+
crucial information that is
354
+
needed to reconstruct the
355
+
image out of the image
356
+
data.
357
+
<-- Photoshop .RAW, (h=0)
358
+
I opened a 3 channel
359
+
interleaved RAW document
360
+
as a 1 channel (1 color)
361
+
interleaved document.
362
+
(reversible databend)
363
+
364
+
When you open the image,
365
+
softwares like for instance
366
+
photoshop will ask you for this
367
+
data, giving you the
368
+
opportunity to
369
+
“bend” (reversible) the image.
370
+
At this moment, you will be
371
+
able to choose the dimensions,
372
+
the amount of channels and if
373
+
the image will be displayed
374
+
interleaved or non-interleaved.
375
+
By entering another value then
376
+
the original, the image will be
377
+
displayed in a distorted way.
378
+
<-- Photoshop .RAW, (h=0)
379
+
interleaved, I opened
380
+
a 3 channel interleaved
381
+
document
382
+
3
383
+
but entered a
384
+
slightly smaller
385
+
value for the width.
386
+
(reversible databend)
387
+
388
+
In the case of a RAW image
389
+
file, interleaving and noninterleaving refer to the order in
390
+
which the RGB color values of
391
+
every pixel are stored.
392
+
In an interleaved Raw
393
+
image, the data is stored in
394
+
a RGBRGBRGB sequence.
395
+
<-- Photoshop .RAW, (h=0)
396
+
I opened the 3 channel
397
+
interleaved RAW document
398
+
as a 3 channel noninterleaved 8 bit document.
399
+
(reversible databend)
400
+
401
+
<-- Photoshop .RAW, (h=0)
402
+
I opened a 3 channel
403
+
interleaved RAW image in
404
+
Microsoft Word (Convert to
405
+
Text Only) and saved it. This
406
+
technique reformats the
407
+
image data into a Microsoft
408
+
Word document, changing
409
+
some of the values and
410
+
adding / deleting some extra
411
+
data. In the image this shows
412
+
by abrupt discolorations, and
413
+
image general image shifts.
414
+
(Stallioʼs Wordpad effect)
415
+
(irreversible databend)
416
+
417
+
<-- Photoshop .RAW, (h=0)
418
+
I opened a 3 channel noninterleaved raw document
419
+
with smaller hight
420
+
(databend, reversible)
421
+
When the image is saved in
422
+
non-interleaved array, the RGB
423
+
values are not ordered
424
+
sequentially but have their own
425
+
ʻlayersʼ with
426
+
427
+
4
428
+
429
+
<-- Photoshop .RAW, (h=0)
430
+
I opened a 3 channel noninterleaved document but
431
+
entered a slightly smaller
432
+
value for the width (1 pixel)
433
+
(reversible databend)
434
+
435
+
<-- Photoshop .RAW, (h=0)
436
+
I opened a 3 channel noninterleaved RAW document
437
+
with a much smaller width
438
+
(reversible databend)
439
+
440
+
<-- Photoshop .RAW, (h=0)
441
+
I opened a 3 channel noninterleaved RAW image in
442
+
Microsoft Word (Convert to
443
+
Text Only) and saved it
444
+
(wordpad, textpad and
445
+
other text editors could also
446
+
work). This technique
447
+
reformats the image data
448
+
into a Microsoft Word
449
+
document, changing some
450
+
of the values and adding /
451
+
deleting some extra data.
452
+
In the image this shows by
453
+
abrupt discolorations, and
454
+
image general image shifts.
455
+
(Stallioʼs Wordpad effect)
456
+
(irreversible databend)
457
+
458
+
5
459
+
460
+
BMP
461
+
462
+
<-- Bitmap (.bmp)
463
+
bend by copy pasting a bit of
464
+
the image data over and over.
465
+
(irreversible databend)
466
+
The BMP file format is
467
+
uncompressed; every bit that
468
+
indexes a bitmap pixel value
469
+
is packed within a linear row,
470
+
"upside-down" with respect to
471
+
normal image raster scan
472
+
order, starting in the lower
473
+
right corner, advancing row
474
+
by row from the bottom to the
475
+
top.
476
+
477
+
<-- Low quality Bitmap (.bmp)
478
+
Random data replacement
479
+
changed the values of the
480
+
indexed colors in the color
481
+
palette (irreversible databend)
482
+
This is why, when you copypaste some of the image data
483
+
the lower part of the image
484
+
will still be intact, while the
485
+
upper part of the image only
486
+
shifts (and sometimes
487
+
discolors)
488
+
In BMP files, and many other
489
+
bitmap file formats the color
490
+
palette consists of a block of
491
+
bytes (a table or palette)
492
+
listing the colors available for
493
+
use in a particular indexedcolor image.
494
+
Each pixel in the image is
495
+
described by a number of bits
496
+
(1-32 bit color depth) that
497
+
index a single color from the
498
+
color palette, that is described
499
+
right after the header.
500
+
The BMP color palette uses
501
+
the interleaved RGB color
502
+
model. In this model, a color
503
+
depends on different
504
+
intensities (from 0 to 255) of
505
+
the primary RGB colors. A
506
+
color is thus defined by the
507
+
final intensities of R+G+B.
508
+
When you copy-paste the
509
+
image data, the intensity data
510
+
From B can (for instance)
511
+
shift to the R, creating sudden
512
+
discoloration.
513
+
514
+
<-- .BMP Wordpad effect
515
+
In comparison to the RAW
516
+
image, the image now waves
517
+
towards the right. This is
518
+
because the BMP
519
+
6
520
+
raster format is
521
+
saved from left to
522
+
right, top to bottom.
523
+
(irreversible databend)
524
+
525
+
LOSSLESS
526
+
GIF
527
+
528
+
<-- Graphics Interchange
529
+
Format (.gif), 8 colors
530
+
restricted pattern
531
+
(with dither)
532
+
Graphics Interchange Format
533
+
is a bitmap image format that
534
+
supports 8 bits per pixel and
535
+
can therefore consist of no
536
+
more then 256 colors.
537
+
The format supports
538
+
animation.
539
+
Dither (the grainy blocky
540
+
artifacts) is an intentionally
541
+
applied form of noise used to
542
+
543
+
“randomize quantization
544
+
error”; the difference between
545
+
the actual analog value and
546
+
quantized digital value. This
547
+
error is caused by truncation
548
+
(the discarding of less
549
+
significant information).
550
+
Dither thus helps to prevent
551
+
from large-scale patterns
552
+
such as "banding" (stepwise
553
+
rendering of smooth
554
+
gradations in brightness or
555
+
hue). Moreover, the not
556
+
available colors are
557
+
approximated because the
558
+
human eye perceives the
559
+
diffusion as a mixture of the
560
+
colors. This creates the
561
+
illusion of color depth.
562
+
<-- Graphics Interchange
563
+
Format (.gif), non-interlaced,
564
+
8 colors, restricted pattern
565
+
(with dither)
566
+
(irreversible databend)
567
+
<-- Graphics Interchange
568
+
Format (.gif), interlaced,
569
+
animated 8 colors, restricted
570
+
pattern (with dither)
571
+
(irreversible databend)
572
+
The gif format uses a 4 pass
573
+
one dimensional interlacing
574
+
strategy. This means that one
575
+
half of the image, consisting of
576
+
every other row of pixels is
577
+
rendered after the other half.
578
+
In the image on the left this
579
+
shows through a gradual
580
+
displacement during weaving
581
+
(the putting together of the two
582
+
layers), which resulted in a
583
+
second “ghost image” (or
584
+
combing artifacts with jagged
585
+
edges).
586
+
587
+
7
588
+
589
+
PNG
590
+
591
+
<-- Portable Network
592
+
Graphics (PNG). interlaced
593
+
(irreversible databend)
594
+
pre-compression: filtering
595
+
(prediction)
596
+
1-2 of the 7 stages before the
597
+
image is reconstructed.
598
+
(irreversible databend)
599
+
Like GIF PNG is a lossless
600
+
compressed raster format,
601
+
which means that it
602
+
represents an image as a
603
+
two-dimensional array of
604
+
colored pixels.
605
+
606
+
PNG is a bitmapped image
607
+
format that employs lossless
608
+
data compression and offers
609
+
a 7-pass 2-dimensional
610
+
interlacing scheme—the
611
+
Adam7 algorithm.
612
+
This is more sophisticated
613
+
than GIF's 1-dimensional, 4pass scheme, and often
614
+
allows for a clearer lowresolution image to be
615
+
visible earlier in the transfer.
616
+
This is visible in image 1
617
+
which just passed its first
618
+
stage of the 7 part
619
+
interlacing scheme. In this
620
+
stage a part of the image is
621
+
rendered almost flawless,
622
+
while the further it gets
623
+
rendered, the more the
624
+
corrupted data becomes
625
+
visible.
626
+
<-- PNG. interlaced. stage
627
+
3/4. (irreversible databend)
628
+
629
+
8
630
+
<-- PNG. interlaced. stage
631
+
5/6.(irreversible databend)
632
+
633
+
<-- PNG. interlaced
634
+
(irreversible databend)
635
+
The 7-pass de-interlacing
636
+
of the corrupted image
637
+
sometimes stops early (as
638
+
early as the first pass).
639
+
640
+
<-- PNG. interlaced
641
+
(irreversible databend)
642
+
643
+
<-- PNG. non-interlaced
644
+
(irreversible databend)
645
+
646
+
9
647
+
648
+
TARGA
649
+
650
+
<-- Truevision Advanced
651
+
Raster Graphics Adapter;
652
+
(.TGA) is an initialism for
653
+
Truevision Graphics Adapter.
654
+
(can be lossy or lossless)
655
+
16 bits Lossless RLE
656
+
compressed
657
+
(irreversible databend)
658
+
Targa recognizes
659
+
over half a dozen image file
660
+
formats, some of which are
661
+
more widely used than
662
+
others.
663
+
I have databend a Tagra
664
+
compressed file by searching
665
+
and replacing + adding some
666
+
data. Apparent are shifts of
667
+
some blocks and some small
668
+
color shifts.
669
+
To understand how these
670
+
shifts went the way they went,
671
+
is however to complex and I
672
+
am not going to try to
673
+
understand or explain this in
674
+
this pdf.
675
+
676
+
<-- Targa (.tga)
677
+
24 bits Lossless RLE
678
+
compressed
679
+
(irreversible databend)
680
+
681
+
<-- Targa (.tga)
682
+
32 bits Lossless RLE
683
+
compressed
684
+
(irreversible databend)
685
+
686
+
10
687
+
688
+
TIFF
689
+
690
+
<-- Targa (.tga)
691
+
32 bits Lossless RLE
692
+
compressed
693
+
(irreversible databend)
694
+
695
+
<-- Tagged Image File
696
+
Format (.TIF) or in short
697
+
TIFF, with white
698
+
background layer
699
+
(irreversible databend)
700
+
Just like Targa, TIFF is a
701
+
very complex compression.
702
+
I have had some really
703
+
interesting diverse
704
+
experiences with this file
705
+
format, but I find it very had
706
+
to get grips on the reason
707
+
why they come to the
708
+
surface the way they do.
709
+
Thats why these are real
710
+
glitch bends to me.
711
+
712
+
<-- TIFF
713
+
With white background
714
+
layer
715
+
(irreversible databend)
716
+
717
+
11
718
+
719
+
<-- TIFF
720
+
interleaved macintosh Save
721
+
Image Pyramid
722
+
(irreversible databend)
723
+
724
+
<-- TIFF
725
+
interleaved macintosh Save
726
+
Image Pyramid
727
+
(irreversible databend)
728
+
729
+
<-- TIFF
730
+
interleaved macintosh Save
731
+
Image Pyramid
732
+
(irreversible databend)
733
+
734
+
12
735
+
736
+
PSD
737
+
738
+
<-- PSD document
739
+
(irreversible databend)
740
+
Color channel shift
741
+
displacement
742
+
743
+
<-- PSD document
744
+
(irreversible databend)
745
+
746
+
<-- PSD document
747
+
(irreversible databend)
748
+
749
+
13
750
+
751
+
LOSSY JPEG
752
+
753
+
<-- Joint Photographic
754
+
Experts Group (.JPG)
755
+
(lossy)
756
+
severely downsampled so
757
+
that the 8x8 macroblocks
758
+
(and quantization error)
759
+
are apparent.
760
+
(irreversible databend)
761
+
A JPG compression
762
+
consists of 6 subsequent
763
+
steps:
764
+
1. Color space
765
+
transformation
766
+
2. Downsampling
767
+
3. Block splitting
768
+
4. Discrete cosine
769
+
transform
770
+
5. Quantization
771
+
6. Entropy coding
772
+
1. Initially, images have to
773
+
be transformed from the
774
+
RGB color space to
775
+
another color space (called
776
+
Y′CbCr), that consists of
777
+
three components that are
778
+
handled separately; the Y
779
+
(luma or brightness) and
780
+
the Cb and Cr values
781
+
(chroma or color values,
782
+
which are divided into hue
783
+
and saturation).
784
+
2. Because the human eye
785
+
doesnʼt perceives small
786
+
differences within the Cb
787
+
and Cr space very well,
788
+
these elements are
789
+
downsampled.
790
+
3. After the color space
791
+
transformation, the image
792
+
is split into tiles or
793
+
macroblocks. Rectangular
794
+
regions of the image that
795
+
are transformed and
796
+
encoded separately.
797
+
<-- 8 × 8 DCT basis
798
+
patterns of a JPG.
799
+
800
+
4. Next, a Discrete Cosine
801
+
Transform (which works
802
+
similar to the Fourier
803
+
Transform function,
804
+
exploited in datamoshing
805
+
and macroblock studies) is
806
+
used to create a frequency
807
+
spectrum, to transform the
808
+
8×8 blocks to a
809
+
combination of the 64 twodimensional DCT basis
810
+
functions or patterns (as
811
+
differentiated by the red
812
+
lines).
813
+
814
+
14
815
+
816
+
5. During the Quantization
817
+
step, the highest brightnessfrequency variations become
818
+
a base line (or 0-value), while
819
+
small positive and negative
820
+
frequency differentiations get
821
+
a value, which take many
822
+
fewer bits to represent.
823
+
824
+
<-- high frequency mapping
825
+
from which basic values are
826
+
derived.
827
+
6. finally, entropy coding is
828
+
applied. Entropy coding is a
829
+
special form of lossless data
830
+
compression that involves
831
+
arranging the image
832
+
components in a "zigzag"
833
+
order. This allows the
834
+
quantized coefficient table to
835
+
be rewritten in a zigzag order
836
+
to a sequence of frequencies.
837
+
A run-length encoding (RLE)
838
+
algorithm groups similar
839
+
frequencies together and after
840
+
that, via "Huffman coding"
841
+
organizes what is left.
842
+
Because the RGB color
843
+
values are described in such
844
+
a complex algorithms, some
845
+
random data replacement
846
+
often results into dramatic
847
+
discoloration and other
848
+
effects.
849
+
<-- low res JPG, Baseline
850
+
standard.
851
+
(irreversible databend)
852
+
<-- low res JPG, Progressive
853
+
(irreversible databend)
854
+
The very high compression
855
+
ratio of this jpg effects the
856
+
quality of the image and the
857
+
size of the artifacts.
858
+
When using quantization with
859
+
block-based coding, as in
860
+
these JPEG-compressed
861
+
images, several types of
862
+
often unwanted artifacts can
863
+
appear, for instance ringing or
864
+
ghosting. In the bend image to
865
+
the left, the low quality and
866
+
corruption have made these
867
+
artifacts more apparent.
868
+
869
+
15
870
+
871
+
<-- low res JPG, Progressive
872
+
(irreversible databend)
873
+
Ringing is (often) the result of
874
+
the loss of high frequency
875
+
components, but can also be
876
+
used to “enhance” the image,
877
+
because this artifact put an
878
+
emphasis on the edges (in
879
+
moving image these artifacts
880
+
can also be referred to as
881
+
mosquito noise).
882
+
Ghosting is an artifact that
883
+
appear when a part of the
884
+
image is somehow doubled
885
+
(often by refraction of a
886
+
television signal within the
887
+
atmosphere).
888
+
'Blocking' / Staircase artifacts
889
+
appear most clearly along the
890
+
curving edges, as a result of
891
+
the 8×8 jpg blocks.
892
+
Blockiness in "busy" regions is
893
+
sometimes also referred to as
894
+
quilting or checker-boarding.
895
+
The JPG to the left shows
896
+
some typical color distortion
897
+
(color mismatching) after
898
+
random data replacement.
899
+
It also shows some less
900
+
typical JPG artifacts called
901
+
jaggies.
902
+
"Jaggies" is the informal
903
+
name for artifacts in raster
904
+
images. They are often the
905
+
result from poor aliasing,
906
+
which happens when a JPG
907
+
signal reconstruction after
908
+
downsampling has
909
+
produced only high
910
+
frequency outcomes.
911
+
912
+
<-- JPG, Progressive
913
+
(irreversible databend)
914
+
915
+
<-- JPG, Progessive
916
+
(irreversible databend)
917
+
918
+
16
919
+
920
+
<-- JPG, Baseline standard
921
+
(irreversible databend)
922
+
923
+
<-- JPG, Baseline
924
+
(irreversible databend)
925
+
926
+
JPEG 2000
927
+
928
+
<-- Joint Photographic
929
+
Experts Group committee in
930
+
2000; low res JPEG 2000.
931
+
(irreversible databend)
932
+
The JPEG 2000 standard was
933
+
mainly developed because of
934
+
the many edge and blocking
935
+
artifacts of the JPG format.
936
+
JPEG 2000 has improved
937
+
scalability and edit-ability.
938
+
In JPG 2000, after the color
939
+
transformation step, the
940
+
image is split into so-called
941
+
tiles, rectangular regions of
942
+
the image that are
943
+
transformed and encoded
944
+
separately.
945
+
Tiles can be any size, and it is
946
+
also possible to consider the
947
+
whole image as one single
948
+
tile. This results into
949
+
a collection of
950
+
17
951
+
sub-bands which
952
+
represent several
953
+
approximation scales.
954
+
955
+
<-- JPEG 2000.
956
+
(irreversible databend)
957
+
A sub-band is a set of
958
+
coefficients that represent
959
+
aspects of the image
960
+
associated with a certain
961
+
frequency range as well as a
962
+
spatial area of the image.
963
+
The quantized sub-bands are
964
+
split further into precincts,
965
+
rectangular regions in the
966
+
wavelet domain.
967
+
A wavelet is a wave-like
968
+
oscillation with an amplitude
969
+
that starts out at zero,
970
+
increases, and then
971
+
decreases back to zero. It
972
+
can typically be visualized as
973
+
a "brief oscillation" like one
974
+
might see recorded by a
975
+
seismograph or heart
976
+
monitor.
977
+
Precincts are split further into
978
+
code blocks. Code blocks are
979
+
located in a single sub-band
980
+
and have equal sizes.
981
+
The chrominance
982
+
components can be, but do
983
+
not necessarily have to be,
984
+
down-scaled in resolution;
985
+
in fact, since the wavelet
986
+
transformation already
987
+
separates images into
988
+
scales, downsampling is
989
+
more effectively handled by
990
+
dropping the finest wavelet
991
+
scale.
992
+
993
+
<-- JPEG 2000
994
+
(irreversible databend)
995
+
<-- Low quality JPEG 2000
996
+
(irreversible databend)
997
+
998
+
18
999
+
1000
+
MOVING IMAGE
1001
+
MOV
1002
+
1003
+
<-- Quicktime movie (.mov)
1004
+
Compression: none
1005
+
256 colors,
1006
+
least quality
1007
+
The low quality and the
1008
+
compression to fewer colors
1009
+
introduced posterization
1010
+
artifacts.
1011
+
Posterization of an image
1012
+
means the conversion of a
1013
+
continuous gradation of
1014
+
tones to several regions of
1015
+
fewer tones, with abrupt
1016
+
changes from one tone to
1017
+
the another.
1018
+
Posterization may be
1019
+
deliberate or may be an
1020
+
unintended artifact of color
1021
+
quantization.
1022
+
1023
+
<-- Quicktime movie (.mov)
1024
+
Compression: Animation
1025
+
256 greys, least quality
1026
+
codec
1027
+
displaced scan lines and
1028
+
posterization artifacts.
1029
+
(I donʼt know why the scan
1030
+
lines displaced themselves)
1031
+
1032
+
<-- Quicktime movie
1033
+
(.mov) Compression:
1034
+
Animation 1000 colors,
1035
+
least quality codec
1036
+
"Banding artifacts," or “false
1037
+
contours” result from color
1038
+
quantization within digitally
1039
+
compressed images.
1040
+
The lines in the image can
1041
+
also be referred to as 10-bit
1042
+
Banding artifacts.
1043
+
1044
+
19
1045
+
1046
+
DV
1047
+
1048
+
<-- DV (Digital Video) tape
1049
+
rewind (shows macroblocks)
1050
+
The artifact to the left shows
1051
+
the macroblocks the footage
1052
+
consists of. This artifact is
1053
+
also referred to as screen
1054
+
tearing (a video artifact that
1055
+
appears when information
1056
+
from two or more different
1057
+
frames is shown a single
1058
+
screen draw).
1059
+
The lossy compressed video
1060
+
image is framed
1061
+
fundamentally different from
1062
+
analog or RAW video footage.
1063
+
First of all, the frames no
1064
+
longer rely upon raw pixels.
1065
+
Instead, macroblocks have
1066
+
become one of the
1067
+
elementary components of the
1068
+
lossy compressed moving
1069
+
image (at least under current
1070
+
standard codecs).
1071
+
1072
+
Lossy compressed video
1073
+
often depends on luminance
1074
+
(brightness) and chrominance
1075
+
(coloring) thresholds arranged
1076
+
within 16x16 pixel (more or
1077
+
less) macroblocks within the
1078
+
keyframes (the I-frames) of an
1079
+
image sequence. The
1080
+
thresholds (or frequencies) of
1081
+
chrominance and luminance
1082
+
depend on an oscillating
1083
+
cosine function (following
1084
+
Fourier Transform).
1085
+
The exploitation/bending of
1086
+
this FFT and its chrominance
1087
+
and luminance values shows
1088
+
in the discolored macroblocks,
1089
+
that now show the otherwise
1090
+
mostly obscured macroblock
1091
+
structures of digital video.
1092
+
<-- DV, pro50 DV,
1093
+
macroblock bend
1094
+
(rearranged macroblock data)
1095
+
“checkerboarding”
1096
+
<-- DV pro50 DV,
1097
+
compression: pro50 DV
1098
+
bend luminance (brightness)
1099
+
and chrominance (coloring)
1100
+
values.
1101
+
Re-organizations of color
1102
+
(chrominance) structures.
1103
+
The matrix of macroblocks is
1104
+
still completely intact but the
1105
+
chrominance values are off.
1106
+
Also referred to as
1107
+
“mosaicking”, “pixelating” or
1108
+
“quilting” artifacts.
1109
+
1110
+
20
1111
+
1112
+
WMV
1113
+
1114
+
<-- Windows Media Video
1115
+
(.wmv) compression: two
1116
+
pass VBR Constrained
1117
+
bitrate: 5kbps (datamoshed)
1118
+
The handling of space and
1119
+
time within video is
1120
+
significantly different between
1121
+
the linear analog or RAW
1122
+
footage and lossy
1123
+
compressed footage. Besides
1124
+
the macroblock structures
1125
+
within digital film material, the
1126
+
footage is also no longer
1127
+
based on a linear series of
1128
+
discrete images (a
1129
+
sequence); instead the video
1130
+
consists of different kinds of
1131
+
frames (I-frames or
1132
+
reference/key frames, Pframes or forward-predicted
1133
+
frames and B-frames or bidirectional frames), of which
1134
+
only the keyframe possesses
1135
+
a complete matrix of
1136
+
macroblocks.
1137
+
1138
+
When a video is encoded,
1139
+
each frame is stored as an I
1140
+
frame (the keyframce) or a P/
1141
+
B frame. An I frame is like a
1142
+
JPEG image, it holds the still
1143
+
image in its entirety. P and B
1144
+
frames are the smart frames
1145
+
that allow videos to be
1146
+
compressed. They store only
1147
+
the differences between the
1148
+
current frame and the last
1149
+
frame.
1150
+
A motion prediction algorithm
1151
+
is used to calculate the
1152
+
difference between the Pand B-frames. These frames
1153
+
consist of motion vectors that
1154
+
index only the difference in
1155
+
position (the offset) of the
1156
+
macroblocks between the
1157
+
original and the next frame.
1158
+
<-- Windows Media Video
1159
+
(.wmv) compression: two
1160
+
pass VBR Constrained
1161
+
bitrate: 5kbps (datamoshed)
1162
+
1163
+
<-- Windows Media Video
1164
+
(.wmv) compression: two
1165
+
pass VBR Constrained
1166
+
bitrate: 5kbps (datamoshed)
1167
+
A recently popularized wave
1168
+
of video artworks was based
1169
+
on the deletion of keyframes
1170
+
and the exploitation of the
1171
+
vector motion of P-frames.
1172
+
Which was also referred to
1173
+
as“datamoshing”, “pixel
1174
+
bleeding” or simply
1175
+
“compression art”.
1176
+
The effect you see in the
1177
+
datamosh videos is what
1178
+
happens when you store only
1179
+
the differences between
1180
+
frames, ie. when all I-Frame
1181
+
or keyframe references
1182
+
have been
1183
+
deleted.
1184
+
21
1185
+
1186
+
<-- Windows Media Video
1187
+
(.wmv) Compression: two
1188
+
pass VBR Constrained
1189
+
bitrate: 5kbps (datamoshed)
1190
+
In this type of compression,
1191
+
artifacts tend to remain on
1192
+
several generations of
1193
+
decompressed frames, and
1194
+
move with the optic flow of
1195
+
the image, leading to a
1196
+
peculiar effect, part way
1197
+
between a painting effect r
1198
+
smear that moves with
1199
+
objects in the scene.
1200
+
1201
+
AVI
1202
+
The DIVX file compression
1203
+
follows a similar encoding
1204
+
as the WMV compression.
1205
+
And is more often used.
1206
+
1207
+
<-- AVI (.avi)
1208
+
Compression: DIVX
1209
+
(datamoshed)
1210
+
1211
+
<-- AVI (.avi)
1212
+
Compression: DIVX
1213
+
(datamoshed)
1214
+
1215
+
22
1216
+
1217
+
<-- AVI (.avi) compression:
1218
+
none, 16 colors, least quality
1219
+
(dither)
1220
+
Color quantization performs a
1221
+
multi-scale analysis on the
1222
+
neighborhood of each pixel, to
1223
+
determine the presence and
1224
+
scale of banding artifacts, and
1225
+
probabilistically dithers the color
1226
+
of the pixel. The overall effect is
1227
+
to "break down" the false
1228
+
contours
1229
+
1230
+
<-- AVI (.avi) compression:
1231
+
Cinepak, 256 colors
1232
+
lowest quality
1233
+
Color quantization
1234
+
decreasing color depth
1235
+
1236
+
<-- AVI (.avi) compression:
1237
+
none, black and white
1238
+
best quality
1239
+
1240
+
The image shows a lot of
1241
+
jaggies.
1242
+
1243
+
!
1244
+
1245
+
23
1246
+
1247
+
<-- AVI (.avi) compression:
1248
+
none, 16 grays
1249
+
least quality
1250
+
1251
+
<-- AVI Cinepak
1252
+
compression 256 grays,
1253
+
lowest quality.
1254
+
Cinepak is my most favorite
1255
+
compression. The
1256
+
compression is based on
1257
+
vector quantization, which
1258
+
results in blocky artifacting
1259
+
at low bitrates.
1260
+
Cinepak divides a movie
1261
+
into key images and intracoded images. Each image
1262
+
is divided into a number of
1263
+
horizontal bands which
1264
+
have individual 256-color
1265
+
palettes transferred in the
1266
+
key images. Each band is
1267
+
subdivided into 4x4 pixel
1268
+
blocks. The compressor
1269
+
uses vector quantization to
1270
+
determine the one or two
1271
+
band palette colors which
1272
+
best match each block and
1273
+
encodes runs of blocks as
1274
+
either one color byte or two
1275
+
color bytes plus a 16-bit
1276
+
vector which determines
1277
+
which pixel gets which color.
1278
+
The cinepak compression is
1279
+
not widely supported
1280
+
anymore, and this is why,
1281
+
when opening the image in
1282
+
a player, the indexed colors
1283
+
of the image are often
1284
+
associated with an incorrect
1285
+
color palette, which results
1286
+
into very unexpected
1287
+
colorations of the image.
1288
+
1289
+
<-- AVI Cinepak
1290
+
compression, 256 grays,
1291
+
lowest quality. Opened in
1292
+
VLC player that
1293
+
associated the
1294
+
24
1295
+
video with a
1296
+
purple/black/turquoise
1297
+
palette.
1298
+
1299
+
+1
-25
system/public/papers.aesthetic.computer/index.html
+1
-25
system/public/papers.aesthetic.computer/index.html
···
684
684
685
685
<!-- papers-end -->
686
686
687
-
<div class="guest-header">guest papers — related reading on the platter</div>
688
-
689
687
<!-- guest-start -->
690
-
691
-
<div class="p guest" data-paper-id="menkman-glitch" data-no-cards="1" data-created="2011-01-01" data-updated="2011-01-01T00:00:00.000Z">
692
-
<div class="title"><a href="/menkman-glitch-momentum-2011.pdf">The Glitch Moment(um)</a></div>
693
-
<div class="detail">Glitch as critical practice — signal, noise, and the politics of failure · Institute of Network Cultures · 70pp</div>
694
-
<div class="abstract">Menkman's book treats glitch as a critical practice rather than an accident. It remains a useful companion for thinking about failure, image politics, and media noise.</div>
695
-
<div class="meta-row"><span class="author">Rosa Menkman</span><span class="created" title="Published">2011</span></div>
696
-
</div>
697
-
698
-
<div class="p guest" data-paper-id="menkman-vernacular" data-no-cards="1" data-created="2010-01-01" data-updated="2010-01-01T00:00:00.000Z">
699
-
<div class="title"><a href="/menkman-vernacular-of-file-formats-2010.pdf">A Vernacular of File Formats</a></div>
700
-
<div class="detail">Compression artifact taxonomy — databending one self-portrait through every codec · 20pp</div>
701
-
<div class="abstract">A tight taxonomy of compression artifacts and databending methods. It works as both a visual glossary and a reminder that file formats have aesthetics.</div>
702
-
<div class="meta-row"><span class="author">Rosa Menkman</span><span class="created" title="Published">2010</span></div>
703
-
</div>
704
-
705
-
<div class="p guest" data-paper-id="menkman-resolution" data-no-cards="1" data-created="2020-01-01" data-updated="2020-01-01T00:00:00.000Z">
706
-
<div class="title"><a href="/menkman-beyond-resolution-2020.pdf">Beyond Resolution</a></div>
707
-
<div class="detail">Resolution as ideology — optics, standards, and the invisible norms of the image pipeline · 2020</div>
708
-
<div class="abstract">Beyond Resolution reframes image resolution as an ideological standard, not just a technical setting. It is a strong companion for thinking about how pipelines shape what counts as clarity.</div>
709
-
<div class="meta-row"><span class="author">Rosa Menkman</span><span class="created" title="Published">2020</span></div>
710
-
</div>
711
-
712
688
<!-- guest-end -->
713
689
714
690
<div class="colophon">
715
691
<p>These papers are sourced from the <a href="/platter">research platter</a>, a living document that tracks the design, history, and direction of <a href="https://aesthetic.computer">Aesthetic Computer</a>. The platter is the raw material. The papers are what gets shaped from it. They are working drafts, written in first person, typeset in LaTeX with XeTeX, and compiled by an oven server that polls the repository every 60 seconds.</p>
716
692
<p>The cards format is a 4×6 inch mobile-friendly version of each paper, designed for reading on a phone. Every paper is translated into Danish, Spanish, and Chinese.</p>
717
-
<p>All papers by <a href="https://prompt.ac/@jeffrey">@jeffrey</a> unless otherwise noted. Guest papers are hosted with permission as related reading.</p>
693
+
<p>All papers by <a href="https://prompt.ac/@jeffrey">@jeffrey</a>. Related readings live on the <a href="/platter">platter</a>.</p>
718
694
</div>
719
695
720
696
<div class="build-status" id="buildStatus">
+3
system/public/papers.aesthetic.computer/platter.html
+3
system/public/papers.aesthetic.computer/platter.html
···
499
499
<a class="item" href="https://aesthetic.computer/assets/papers/readings/text/shklovsky.txt">Viktor Shklovsky — "Art as Device" (1917) <span class="file">txt</span></a>
500
500
<a class="item" href="https://aesthetic.computer/assets/papers/readings/text/Adorno-Theodor-W-Punctuation-Marks.txt">Theodor Adorno — "Punctuation Marks" (1958) <span class="file">txt</span></a>
501
501
<a class="item" href="https://aesthetic.computer/assets/papers/readings/text/harman-1.txt">Graham Harman — "Revenge of the Surface" (~2011) <span class="file">txt</span></a>
502
+
<a class="item" href="https://aesthetic.computer/assets/papers/readings/text/Menkman-Glitch-Momentum-2011.txt">Rosa Menkman — <em>The Glitch Moment(um)</em> (2011) <span class="file">txt · 2199 lines</span></a>
503
+
<a class="item" href="https://aesthetic.computer/assets/papers/readings/text/Menkman-Vernacular-of-File-Formats-2010.txt">Rosa Menkman — <em>A Vernacular of File Formats</em> (2010) <span class="file">txt · 1298 lines</span></a>
504
+
<a class="item" href="https://aesthetic.computer/assets/papers/readings/text/Menkman-Beyond-Resolution-2020.txt">Rosa Menkman — <em>Beyond Resolution</em> (2020) <span class="file">txt · 2747 lines</span></a>
502
505
</div>
503
506
<div class="item-group">
504
507
<div class="item-group-label">Making & Materiality</div>