Vertical AtlasVertical Atlas

In this essay, Cristina Cochior expands on the three workshops of Vertical Atlas - world.orbit, which explored the artistic, theoretical and philosophical possibilities for a non-exploitative use of remote sensing software to witness Earth in all its complexity.

As remote sensors spin around Earth on satellites, enveloping it in membranes of data, they participate in imaginaries that describe and mobilise the world in particular ways.

The distant aesthetics that satellites generate evoke a fascination with the possibility of expanding human perception through remote mechanic and programmatic sensing machines. Remote sensing is the production of data by satellites, that either actively transmit and read back electromagnetic radiation, or passively capture the signals of optical and thermal sensors.

The data layers that are created by satellites frame digital everyday life; we are attuned to the logics that they produce. They indicate the state of the weather, they provide geolocation information, they also inform policies for environmental protection and the management of various natural resources. However, the same software that is used for monitoring environmental change on a planetary scale is also used to determine possible new areas to exploit economically. Through their rendition into data, environments appear as programmable, extractable, a continuation of the colonial project (Gabrys 2016).

Starting from the premise that the ways in which remote sensing software enable us to witness Earth are affected by root assumptions in the data's makeup, thus leaving out important frictions in the process of its generation, how is it possible to repurpose this technology to relate to the world in more affirmative and generative ways? What awareness or data treatments would be needed for that?

Data worlds

Satellite data imagery quite literally creates specific visions of the world by presenting Earth as a single entity. Simultaneously, all the infrastructural elements that participate in this disembodied space view propel processes of world making.

In his article Three Aspects of Data Worlds, Jonathan Gray points towards the world-making capacities of data. Through the idea of "data worlds" it is acknowledged that the capacities of Earth observation and other data go beyond representation. These "shape the way we see and think about things, serve as a common point of connection across situations, and help to conventionalise ways of organising the world."

Gray defines data worlds through three main aspects. The first is that data is capable of speech acts, referring to the capacity of data worlds to "make things up". Their enunciative power comprises both the things that are being said as well as "the background on which things become sayable". Futhermore, data worlds are organised by and in turn organise the meaning-making acts of various collectives. Thirdly, Gray addresses trans-national aspects of data worlds - made possible through data infrastructures that allow for international coordination.

Gray calls for a redistribution of data worlds that takes into consideration for whom and by whom data is being created, and which concerns it is mapping out.

Current status of Earth observation technologies

Earth observation (EO) data is part of a larger data economy that has been undergoing overwhelming change since Sputnik's launch in orbit in 1957. Technological advances, open data policies, reduced launch costs, and the rise of data processing cloud infrastructures have lowered the participation barrier for actors that previously did not have access to the space sector. This in turn has given rise to diverse applications around both commercial and citizen-led engagement with remote sensing data.

Based on the number of spacecraft applications that have been filed with the FCC, ITU, and in some cases from announcements in mainstream media in 2020, a projected 107,000 satellites could be in space by 2029 (AGI Ansys 2020) launched by both state actors and commercial enterprises.

Due to the rapid increase in resolution of newer technologies, older, lower resolution satellite data has become available for greatly reduced and often zero costs, while higher resolution data is highly sought after in a very competitive and rapidly growing market. And with cloud-computing platform such as Google Earth Engine (GEE) enabling computational operations like data management, storage and access to happen on the server side, rather than the user side, many more possibilities for non-experts to engage with the data have been opened up. Government sponsored initiatives are part this effort too: the European Space Agency (ESA) launched the Copernicus project, for example. While it is clear that this will not immediately revolutionise the way satellite data is used, new users can bring new critical perspectives to remote sensing practice.

Witnessing data

Earth observation data is used as a technology for witnessing the state of Earth: it produces information based on which various decision-making processes are shaped. The data serves as evidence for scientific, regulatory and commercial purposes, including resource mapping, geolocalisation, or the monitorisation of ecosystems.

Artist Susan Schuppli calls this form of witnessing material. Material witnessing is "a Möbius-like concept that continually twists between divulging evidence of the event and exposing the event of evidence." Through this term, she points towards the ways in which matter and evidence coalesce into "a particular kind of political project: one that discloses different orders of knowledge and regimes of perceptibility that enable materials to become evidential and bear witness." For Schuppli, the ways in which computational data encode layers of information about material phenomena are similar to the ways in which nonhuman entities such as rocks, or oil spills in the ocean, are a recording of an object's interactions with the world.

The generated data layers should not be considered merely representational, nor are they purely indexical to their object of observation. Instead they produce an excess of information through the added data layers - in themselves inexhaustible objects in the world - which, combined with various nonhuman and machinic assemblages can provide different interpretations of the situatedness of the recording (or self-recording) object. Decoupling data from a representational function opens the possibility to more directly engage with its world-making capacities.

Innocent observers

Witnessing is no innocent act, as media historian Lisa Gitelman states. She traces parallels between the figure of the "innocent observer" as it was enabled by the advent of photography and as it is currently enabled by the misleading, but prevalent oxymoron "raw data". Appearing to be unencumbered by human agency, photography was also presumed to be an instance of mechanical objectivity when it first appeared. Because of this perceived quality, photography started to become a de facto tool of observation. Photography is nowadays understood as a deeply rhetorical practice which generates its own subjectivities.

The historical perception of photography as a medium of truth is not dissimilar to how satellite data is currently regarded. Lisa Gitelman argues that, just like photography, "data too need to be understood as framed and framing". Satellite data are not merely faithful recordings, but are generated by particular understandings of ecology, mathematics, physics, chemistry and engineering, as well as the political and sociological commitments of its makers.

Furthermore, satellite imagery does not use cameras as a capturing technology to create images of Earth. Instead the data collected is processed and analysed by algorithms which decode it and thereby render it intelligible to humans. In certain cases where machine learning algorithms are used, processes that can not be directly witnessed by humans take place. Satellite imagery for this reason is often claimed as a technology which is less susceptible to human biases. But behind this data lie a gamut of decisions determining how a question is translated into measurements forming the data collecting processes. Certain possible meanings are preselected that crop out other possible framings.

Once data is captured, it still undergoes multiple steps of transformation, from the binary information to pixel values, to being composited from different sources into one image, to having gaps removed and conflicts resolved, saved as a file, then uploaded to be matched with other data. At every step, the contingency of the computational translation acts affects the outcome in ways which may be imperceptible to human vision, even if they are presented as having a direct connection with it.

Oceans in Transformation

The work of spatial research organisation Territorial Agency addresses these processes and offers an interesting alternative approach. The process of merging data into one coherent narrative is challenged in their work _Oceans in Transformation which _takes the ocean as its primary focus and works from the premise that it has been irredeemably transformed by anthropogenic infrastructures.

Oceans in Transformation creates dynamic images by bringing multiple datasets together. The aesthetic appeal of the dynamic images serves a rhetorical function: to gather viewers around them and point their attention to meta-narratives which are not directly depicted.

Through the overlap of multiple ocean datasets, patterns become visible which are not captured in the data, but instead emerge from the gaps between different qualities and perspectives that play out within it, potentially in conflict with each other. Rather than working on a unifying narrative of the ocean, the project is built around these incongruences. Multiple narratives and disagreeing perspectives are presented simultaneously, thus countering the allure of apparent neutrality, refusing reconciliation.

Oceans in Transformation challenges the claim that increasing Earth mapping efforts would give a more precise understanding of the world. What emerges from these different juxtaposed ways of perceiving the ocean through data, is that the ocean itself cannot actually be seen: there is no singular agent that is the ocean. Every perspective reveals another one. The possibility emerges that the ocean does not want to be seen, a visibility which in this case would translate into more opportunities for extraction. This goes in direct opposition to the mantra that is often heard in machine learning communities whereby "more data is better data".

Deep Swamp

Tega Brain's work, focuses on a different dimension of data worlds and critically addresses the possibilities and limits of machine learning models to be of help in dealing with the ongoing climate crisis. She questions the perception of the environment as a knowable and controllable system that can be intervened in. Deep Swamp is a tryptich of terraria with semi-inundated environments. Three differently trained artificial intelligence agents were each assigned with taking care of one of the terraria in the gallery environment. They base their rehabilitational interventions on different notions of what an optimal landscape is after being trained on different selections of images from Flickr and Google Art Project. One was trained on what a 'natural wetland' looks like, one was trained on 'landscape art', and the last one was trained on best settings to attract gallery visitors' attention. Based on the images and the model of classification they are given, the algorithms learn to identify patterns which will then be used to make decisions. The spectrum of possible action is constrained by the datasets: the algorithm only changes the room acclimatisation to imitate the wetland categories it has identified in the Flickr photographs, but not to prevent the plants from dying, for example.

The attempt to reduce human involvement in the process makes the particular human biases involved at different stages in the model's training glaringly obvious: starting from the geographical demographics of the platforms' user bases, to the cultural understanding behind the search terms, to the historical preferences in canonised Western art.

Asunder

In Asunder, which Tega Brain developed together with Julian Oliver and Bengt Sjölen, an algorithm trained on Landsat 8 pictures generates fictional landscapes using Google Brain's inpainting technique. The algorithm reconstructs missing parts of an image in such a way that its intervention is unrecognisable. The project combines a conventional Earth model with an atmospheric model, an ocean circulation model and a land use model to simulate Landsat 8 tiles. Asunder starts with a Landsat8 tile of a particular region, uses a GAN algorithm to create multiple geo-engineering proposals, applies one, and then proceeds to interpret the (fictional) development of the tile looking at the land use parameters that are input to a climate model. This leads to realistically visualised absurd and desperate scenarios for geo-engineering.

Both Asunder and Deep Swamp respond to the growing interest in AI in relation to current environmental crises, yet they criticise the techno-solutionist approach often encountered in geo-engineering.

The approaches by Tega Brain and Territorial Agency both address strategies to escape the infrastructural subconscious either by speculating on AI agency or by refusing to normalise data and entertain its discursive capacities. Both move beyond representational expectations of data and develop critical applications that open up possibilities for alternative data practices.

Data infrastructures

The works mentioned above also point towards the mechanisms that are set in motion in order to create the data. From the sensors themselves, to the satellites, the servers needed to run the algorithms that decode the data, and the companies providing the services, each of these elements organises certain relations between technologies, people and things, and perform a particular worldview in doing so.

Seda Gürses and Roel Dobbe use the term "computational infrastructures" to describe the breadth of factors involved in creating and maintaining such infrastructures. Computational infrastructures come "not only with [their] own tools, values and environmental implications, but also with an expansive political economy, propelled by an immense amount of global capital investment."

Earth sensing software relies on computational infrastructures to gather, analyse, render, and make available data assessed on metrics of optimisation and efficiency subservient to venture capital. This in turn is dependent on extractive geo-engineering practices harming Earth. Forefronting the axes of power that make the generation of data possible is a necessary step in the efforts to reconfigure software technoculture otherwise, without being assimilated into it.

Depths and Densities

The Underground Division collective, comprised of Femke Snelting, Helen Pritchard and Jara Rocha, propose a form of thinking-with software that may address this point of tension. Thinking-with software for them relies on "engagement with technological objects that is potentially porous to nontechnical contributions; that is: to those by queers, women, people of colour, non-adult and other less-entitled contributors."

The Underground Division devised the workshop Depths and Densities in which participants were thinking-with GPlates, a free software program and web portal for tectonic plate modelling. They developed "resistant vocabularies, creative misuses and/or plausible f(r)ictions" which would unsettle the extractivist bias present in the software through the figures of disobedient bug reporting and disobedient action research. These methods of doing research attempt to interrupt and transgress the projected production cycle of theory and practice by engaging with reconfigurations of publishing as an epistemic format.

In this case, the group engages with practices of bug reporting as a mode of affirmatively critiquing software "in order to technically equip ourselves with partial and localised repair possibilities, while resisting the production of ever-new and naïve reparative fantasies."

The work of trans*feminist, queer, anti-racist collectives such as The Underground Division critically considers not only the subjectivities produced through "innocent" mechanical observers, but also who has the response-ability in witnessing these acts.

Cosmograms as infrastructures of thinking

Engaging with the poetics and politics of infrastructures points towards similar affective strategies, such as those of cosmograms introduced by Lukáš Likavčan in his book, An Introduction to Comparative Planetology. Cosmograms, he explains, are "diagrams of intrinsic logic of our universe" which "present different images that [show] implicit traces of different relations which might be repeatedly applied through different registers of reality." They are abstract mappings of metaphysical, political and material relations between things which display ontological beliefs about how the universe is organised.

He gives five figures as examples of cosmograms: the Planetary, the Globe, the Terrestrial, Earth-without-us, and Spectral Earth. Currently, commercial mainstream technology, including satellite data infrastructure, operates on the vision of the Globe, where Earth is a passive, domesticated container of goods that can be neatly divided and turned to profit. These perceptions can be observed in the design of the technologies we use, which equally depends on a vision of Earth as an object that can be neatly sliced into data. In the Depths and Densities workshop, participants flagged the extraction-infused vocabulary of the GPlates software manifested by example in the "grabbing" gesture, which could be argued brings up histories of land grabbing.

Cosmograms could be understood as another form of thinking-with the data worlds generated by computational infrastructures. Thinking-with other cosmograms than that of the Globe as a type of tool making for imagination suggests a conceptual direction that is both critical and generative.

Closing remarks

Being able to observe what is happening around the world, whether it is crop monitoring, forestry planning, illegal logging, pipeline monitoring, oil spills detection, iceberg monitoring or handling disasters, makes remote sensing a key technology in addressing the present climate urgencies.

The different artistic and theoretical practices that have been mentioned set possible tactics for highlighting the complexity, uncertainty and ambiguity in EO software. Their interventions and engagements with the data worlds produced by such technologies generate a shift in attention from matters of fact to matters of concern, thus allowing for the response-ability of broader group of people outside the scientific experts and commercial planners currently dominating the field, including people who are subjected to the forms of violence spurred by the currently dominant cosmogram.

And while attempting to repurpose the Globe-imbued logics of EO software for scalable political action without perpetuating its cosmogrammatic insinuations, there is an urgent need to create moments of witnessing the techno-social practices that govern everyday life and to gesture towards anti-colonial, non-extractive forms of relating to Earth which attest to its complex, unknowable assemblage. And for that there is generative potential in the transformative acts of disobedient thinking-with witnessing technologies, the refusal to normalise data processes, the experimentation with non-violent data worlds, or the work through alternative cosmograms.

References

AGI Ansys. 2020. "107,000 Planned Satellites by 2029". Video, 1:11. Available at: https://www.youtube.com/watch?v=oWB7ZySDHg8

Gabrys, Jennifer. 2016. Program Earth. Minneapolis; London: University of Minnesota Press.

Gitelman, Lisa. 2013. "Raw Data" is an Oxymoron. Cambridge, MA: The MIT Press.

Gray, Jonathan. 2018. Three Aspects of Data Worlds. Krisis: Journal for Contemporary Philosophy. Available at: https://archive.krisis.eu/three-aspects-of-data-worlds/

Gürses, Seda, and Dobbe, Roel. 2020. "Seminar on Programmable Infrastructures". Available at: https://www.tudelft.nl/tbm/programmable-infrastructures

Likavčan, Lukáš. 2019. Introduction to Comparative Planetology. London; Moscow: Strelka Press.

Likavčan, Lukáš. 2020. "Vertical Atlas - world.orbit". Video, 53:25. Available at: https://vimeo.com/event/409139/videos/476293641/

Pritchard, Helen; Rocha, Jara and Snelting, Femke. 2021. "We Have Always Been Geohackers". In: Annike Haas; Maximilian Haas; Hanna Magauer and Dennis Pohl, eds. How to Relate: Knowledges, Arts, Practices. Münich: Wilhelm Fink Verlag.

Schuppli, Susan. 2020. Material Witness: Media, Forensics, Evidence. Cambridge, MA: The MIT Press.