1 Introduction

Perhaps every sound technology in human history contains within it some model or script for hearing...

(Sterne, 2012)

Given a specific way of experiencing sound, there is a certain manner in which sound is apprehended. Or rather, there is a certain way that the listener orientates towards what is audible, and there are a number of ways of changing that orientation. Our focus is on the use of contemporary technology and computational research to change this orientation, and potentially reveal that which was not previously a part of the listener’s imagination.

2 3-to-4 Fold Distinction of Sound

Philosopher Norman Sieroka splits sound into what he dubs three manifestations: (i) acoustic (unprocessed), (ii) physiological (processed) and (iii) phenomenal (perceived) (Sieroka, 2015, p. 124). The acoustic is defined as physical and mathematical manifestations: pressure waves with a specific amplitude and frequency profile. The physiological is defined as the interfacing components that constitute an auditory apparatus (e.g., ear, brain). Finally, what is heard or presented during auditory experience – often investigated through phenomenology – is the phenomenal. We do not hear unprocessed soundwaves, nor neurophysiological activity: these are “concepts from physics and neurophysiology” in Sieroka’s view.

Our focus is on the second part of this sequence (the physiological). It is an oversimplification to define this processing step purely in terms of our own physiology. Thus, we will describe this level as a model level in order to facilitate a more generalised description. However, it is important to note that we are not trying to define a neo-functionalism nor overplay the importance of multiple realisability (i.e., that any function can be run on any system). Functionalism goes too far: if function is what is solely emphasised then one assumes that there is a trivial coupling between a function and its substrate (Buechner, 2008). For example, the mapping of mind to brain can easily collapse into either a ramified caricature of Cartesian dualism (mental irreducibility) or a physicalist reductionism, where there is an isomorphic relationship between mental states and their constitutive substrate. Our goal is to perform neither of these theoretical collapses while still being sensitive to both sets of concerns, as well as other similar concerns. We do not wish to homogenise the complexity of this topic, but it will not be discussed in detail here.

In order to initiate a framework that allows for the suggested sensitivity, we turn to Immanuel Kant’s conception of the transcendental. Sieroka’s three distinctions of sound can map onto the Kantian distinctions between object (acoustic), transcendental (model) and subject (phenomenal).⁽¹⁾ Our focus here is on the transcendental level.

Theoretical physicist and philosopher Gabriel Catren defines Kant’s notion of the transcendental as the structuration of a particular intentional (i.e., subject-to-object) relation. The resulting structure is conditioned such that the subject-pole delimits the possible relations to the object-pole, and not vice versa (see Fig. 1). In this case, the object-pole solely means what is aimed at by experience and the subject-pole is the vector that aims at this object-pole (Legrand, 2011). For example: when experiencing a sound the listener becomes the subject-pole, the sound heard by the listener takes the position of the object-pole, and the listener’s transcendental mediates as well as potentiates the resulting relations. The concept of the transcendental in this case enables the articulation of the interplay between a sound and any listener.

However, the transcendental, for Kant, is the conditions for the possibility of experience in its most general sense and is not specific to any particular sensory modality. It constrains experience by specifying a fixed and unchangeable parameter-space within which phenomena can be experienced. Catren names this structuration “transcendental transcendence,” and argues that Kant essentialises and privileges a particular transcendental model or frame (Catren, 2015, p. 67). Out of all possible frames (e.g., insects, animals, aliens), Kant privileges a single universal anthropomorphic schema of the world. To quote Catren:

if we grant that human experience is transcendentally framed, it does not follow that we cannot vary the... transcendental conditions of experience, that we cannot go beyond [Kantian] transcendental transcendence. A critical subject, i.e., a subject capable of performing a critical reflection on its own transcendental structure, can afford the possibility of varying, deforming, or perturbing the very a priori structure of its experience (Catren, 2015, p. 67).

Catren calls the experience of such a deformation a phenoumenodelic experience so as to emphasise that it is enacted at a level that can be differentiated from the strictly phenomenal.⁽²⁾

Catren differentiates transcendental transcendence from what he dubs a speculative transcendence by problematising Kant’s essentialisation. Speculative transcendence defines a more plastic notion of the transcendental where each transcendental delimits the possible subject– and/or object – transformations between the poles of their intentional relation (see Fig. 2). There is no single universal trascendental that applies to any experiencing subject whatsoever, contrary to Kant. Catren’s speculative transcendence allows for one to “escape” the Kantian “claustrophobia” or prohibition that states that we can never “go beyond” our human transcendental structuration – the anthropic manifold⁽³⁾ consisting of physiological, neurophysiological, conceptual-categorical, imaginary, linguistic, historical-cultural-sociological, existential and so on (Catren, 2015, pp. 66–67). Instead, there are multiple, unique transcendental structurations that can be explored via phenoumenodelic variations.

Returning to sound, the Kantian notion of the transcendental affords specific ways of experiencing sound and limits ways in which both the listening subject and the sonic object can be transformed. Catren adds to this the possibility to modulate the very structuration of a sonic experience (i.e., modify the intentional subject-to-object relation itself). The ways in which something is audible multiplies as an effect of there being varied transcendentals. Using this framework, we are able to expand Sieroka’s initial ternary description to include another layer: the speculative. To speak of this speculative framework in terms of the sonic would be to speak of the means for manipulating how, rather than what, we think of sound.

It must be insisted that it is not the case that every perceptual modification can be said to be a phenoumenodelic experience. While the former might only be a perspectival shift, the latter only occurs in the rare instance when the available modifications of an intentional relation are altered. These alterations of experience extend or reduce the intentional space of possibility. For example, octave equivalence is an identity relation specified within a transcendental structure that allows for the experience of A4 (440 Hz) as identical to A5 (880 Hz) with respect to the pitch class of A. Another example of this is when the utterance “dog” is heard as equivalent to the sound “dɒg” as well as “dↄg”.⁽⁴⁾ In terms of these examples, phenoumenodelic variations fundamentally change one’s internal structuring of pitch or phonemes such that the described distinctions disappear or are extended to include unfamiliar elements (e.g., B4 (493 Hz) or “bɒg”). Through this action the horizon of potential phenomenal transformations changes such that new experiences become possible, while others are no longer.

3 Transmutative Tools

Neural networks are computational systems that modify their internal structure in order to learn.⁽⁵⁾ This modification is designed to replicate the types of physical changes that occur in living systems. Deep neural networks expand on this idea but increase the number of layers of neurons between their input – and output – nodes. The number of layers between the input – and output – nodes define the degree of non-linearity and abstraction afforded by the model (Bengio, 2009).

If we imagine the space of possible relations for a given system, then the deep network learns to associate specific features of that space with particular inputs and outputs. This method enables the classification and identification of highly complex objects, although their complexity is usually overlooked. For example, consider a rather simple sonic object: a phoneme or word. The correct identification requires handling many variations in palate techtonics, accent, timing and so on (see Fig. 3). More complex sonic objects such as sentences, music genre and environmental noise tend to far exceed this simple identification task. This is also the case when it comes to other modalities such as vision and kinaesthetics.

The challenge of all neural networks, but especially deep nets is that they are black box algorithms (Bengio, 2013; Coates et al., 2011; Zahavy, 2016). What this means is that no one knows what exactly the network is using to identify, classify or perform other tasks on the desired objects of interest. Even if the tasks of these black box algorithms are theoretically computable by hand, and thus analytically tractable, the dimensionality of the problem space is often so great that these tasks are epistemologically intractable for humans.

In order to ameliorate this intractability, researchers in computer science began developing what can be called transmutative tools that would allow them to explore the internal structure of these black box algorithms (Zeiler & Fergus, 2014). For example, Nguyen et al. (2014) generate “speculative” images using evolutionary algorithms that select visual patterns of a population of images mutated based on the “highest prediction value” of a trained deep net. Similarly, Dosovitskiy & Brox (2015) train their deep networks to predict the pre-image that could have produced a given set of features, “inverting” the standard training procedure. They then perturb the pre-image in an effort to understand the inner workings of the deep net. In both cases, the mechanisms by which the algorithms select their own features for recognition during training were everted and made more legible. In effect, the resulting tools transmute the unknown internal structure of these black boxes into infograms that can be read, or heard. But, a black box is not always easy to hear.

What was most salient about the images resulting from opening the black box were the anomalies (Nguyen et al., 2014). For example, the freight car image in Fig. 4 has no resemblance to what would be typically recognised as a freight car. This suggests, in light of our discussion of Catren’s work, that the underlying “transcendental framework” of this deep net (i.e., its internal black box processes) is different from that of a human transcendental. Anomalies in the image express that salient features for recognition can be entirely different in the context of the machine and of the human. In this sense, the criteria used to recognise these anomalous images within the deep net escape the Kantian frame and require a kind of speculative transcendence. This is not limited to the visual.

There are few examples of transmutative tools used within an audio context. And yet, Kereliuk et al. (2015) note that deep neural networks are increasingly being applied towards music content.⁽⁶⁾ However, the deep neural network trained by Kereliuk et al. (2015) was easily biased such that songs, recognisable as pertaining to a specific genre, were misclassified.⁽⁷⁾ These songs were used to “show how it is relatively easy to fool the same state-of-the-art [deep neural net]-based music content analysis system” (Kereliuk et al., 2015). In other words, the application of these black box algorithms towards music content analysis is quite fragile and much more research needs to be undertaken within the auditory domain.

The difference previously discussed between perspectival shifts and phenoumenodelic variations is useful here. The mischaracterisation of features exemplified by these content analysis deep nets express minute differentiations within a single transcendental – or, more succinctly, a perspectival shift. There is a substantial gap between biasing an algorithm towards a particular mischaracterisation of features and the transmutative tools that we propose to understand what might be the alternate transcendental structures of black box algorithms. It would seem possible to construct situations such that, following the anomalous examples shown in Fig. 4, we are able to explore sonorous structurations and generate what could be called sonic pre-images.⁽⁸⁾

4 Disincepting Misanthroposchematism

In the Summer of 2015, a post entitled “Inceptionism: Going deeper into neural networks” was released on Google’s research blog (Mordvintsev et al., 2015). It popularises Google’s own research into what we have been calling transmutative tools (Szegedy et al., 2015). Their text outlines a deep net process called gradient ascent and the associated system, both of which were engineered to examine and improve black box image processing structures. Google’s blog entry describes this mechanism in some detail under the name “Inceptionism”, which later came to be known as “DeepDream”. Yet, the figures of DeepDream are easily misinterpreted. Unlike the examples above, where the visualisations correspond with the internal structuration of the underlying networks, the popularisation of the images of Google’s DeepDream lost any sense of the implicit transcendental structure that was being visualised. In this way, these images were aestheticised as images rather than infographic data visualisations. DeepDream’s images were not read, they were merely seen.

An aestheticisation of the transmutative output (i.e., a fixation on the images of DeepDream) encourages a misinterpretation of possible speculations. When images are devoid of their infographic legibility the representation of internal features is lost. Without the use of transmutative tools to preserve the infographic context, the DeepDream images can become illusive abstractions of the autonomous creativity of machines. In an extreme case, a rogue algorithm that classifies a song into any genre can be mistakenly construed as a creative listener.⁽⁹⁾

Our claim is that DeepDream, as an idea rather than software, has come to represent a problematic view of the capacities of deep learning in popular culture, especially in terms of creativity. DeepDream describes only what it is trained on: millions of images of animals, eyeballs, places and so on. Gradient ascent, a process used by DeepDream, recursively accentuates existing patterns in the image based on this training. It amounts to a “brute force” search technique of a highly non-linear feature space of images. By “brute force” we do not mean that the system examines every possible permutation before selecting the best result for a given set of criteria. Deep neural nets of this kind force the convergence of prior patterns and the input in a brute manner. The brutality in this approach is circumscribed within a schema of human creativity, which masks the internal mechanisms of the algorithmic process. Yet, a brutal expansion of our scope of perception via phenoumenodelic variations – potentiating greater creativity – is an alternative view of this process. In terms of what was discussed above, the misrecognition of the algorithm’s output as an object of creativity can be seen as confining a non-human transcendental (i.e., the software of DeepDream) within an anthropic manifold (i.e., the schema of creativity). We name this accommodation misanthroposchematism.

Disinceptionism is the systematic disillusionment of the aestheticisation of black box algorithms and their output. It is not our intent to limit the impact of these absorbing images (see Fig. 5); instead it is to unbox the transcendental mechanisms involved in DeepDream. Particular processes of transmutation can be constructed such that researchers and programmers can gain insight into how to further develop deep nets. The pedagogical role of Google’s inceptionism provides the user with a new means towards speculative transcendence. The deep net, by affording a different set of transcendentals, can allow for the exploration of as yet impossible phenomenologies – or what Catren calls phenoumenology. In effect, transmutative tools had to be made to train the trainer to better train the algorithm.

It is in the contrast between our (human) transcendental framework and the speculative (machine) transcendence that inchoate patterns become the focus of investigations beyond our anthroposchematic tendencies. These latter tendencies exclude, confine or delimit the noumena as imperceptible rather than allowing us to gain traction on the infra_perceptible phenoumena.⁽¹⁰⁾ Following philosopher Chris Swoyer, this process of gaining salience via an alternative domain is named “surrogate reasoning” by Sieroka (2015).⁽¹¹⁾ Disinceptionism aims to increase sensitivity to possible transcendental modifications in a similar way, thereby eliciting the possibility of a phenoumenodelic experience (Catren, 2015). It is worth noting that these experiences do not necessarily require the use of technology as per the phonetic example used prior. We merely need to directly or indirectly act on how one models the world.

The situation we have described could be distilled as follows. (1) An object was speculatively registered outside the anthropic manifold. (2) It was necessary to generate a means to perceive this region. (3) Attention is (potentionally mis)directed towards the codified output. (4) Disinceptionism refocuses attention on the process of transmutation, which makes things infra_perceptible (rather than imperceptible) in speculative modes.

5 Conclusion

In the first half of the article, we extended Norman Sieroka’s 3-fold distinction of sound (acoustic-physiological-phenomenal) to encompass a fourth layer: the speculative. We achieved this subtle distinction by integrating Sieroka’s physiological level with Immanuel Kant’s transcendental level and then partitioning it according to Gabriel Catren’s notion of phenoumenadelic speculative transcendence. We closed the section by highlighting some of the important differentiations between perspectival (phenomenal) shifts and transcendental (phenoumenal) shifts with some examples.

In the second half of the article, we provided a brief, intuitive introduction to the notion of neural networks, particularly in their “deep” instantiations. Through this discussion, we outlined how the internal structure of these systems is opaque to the user, which encouraged the development of techniques – transmutative tools – for increasing their transparency. Google’s DeepDream application was then introduced as an illustrative example of both potential speculative transcendence and the inherent inertia of anthroposchematism where a fundamentally alien, black box system is viewed in terms of human creativity and imagination. In an effort to accelerate out of this inertia, we encourage a disinceptionism of the user whereby creativity is achieved by exploring exactly how creativity fails in Google’s deep inception.

In future work, deep neural networks will be used to facilitate a comparative analysis of acoustic or even archaeoacoustic spaces where the algorithm is trained on a variety of built environments. The resulting system could then be used to explore the sonic pre-image produced by these environments. We propose that the audition of the transcendental structurations resulting from these soundscapes could potentiate a sonic field wherein which processes of transmutation become the means for the practice of a new form of auscultation (Barcelos, 2016).⁽¹²⁾ DeepDream is a warning of how easily our tools capture our phenomenal apprehensions, potentially obfuscating the infra_perceptible in the process.

References

• Barcelos, L. (2016). The nuclear sonic: Listening millennial matter. In B. Brits, P. Gibson & A. Ireland (Eds.), Aesthetics after finitude (pp. 71–88). Melbourne: re.press.
• Bengio, Y. (2009). Learning deep architectures for AI. Foundations and Trends® in Machine Learning, 2(1), 1–127.
• Bengio, Y. (2013). Deep learning of representations: Looking forward. In Statistical Language and Speech Processing (pp. 1–37). New York: Springer Publishing Company.
• Buechner, J. (2008). Gödel, Putnam and functionalism. Cambridge, MA: MIT Press.
• Burman, J.T. (2013). Updating the Baldwin Effect: The biological levels behind Piaget’s new theory. New Ideas in Psychology, 31(3), 363–373.
• Carroll, L. (1917). Through the looking-glass and what Alice found there. Illinois: Rand McNally & Company.
• Catren, G. (2015). Pleromatica or Elsinore’s drunkenness. In S.D. Sanctis & A. Longo (Eds.), Breaking the spell: Contemporary realism Under discussion (pp. 63–88). Sesto San Giovanni: Mimesis Edizioni.
• Coates, A., Ng, A.Y. & Lee, H. (2011). An analysis of single-layer networks in unsupervised feature learning. International Conference on Artificial Intelligence and Statistics (pp. 215–223).
• Couroux, M. (2016). Xenochronic dispatches from the domain of phonoegregore. In B. Brits, P. Gibson & A. Ireland (Eds.), Aesthetics after finitude (pp. 29–52). Melbourne: re.press.
• Dieleman, S. & Schrauwen, B. (2014). End-to-end learning for music audio. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2014 (pp. 6964–6968).
• Dosovitskiy, A. & Brox, T. (2015). Inverting visual representations with convolutional networks. arXiv:1506.02753 [cs.NE].
• Eisner, F., Melinger, A. & Weber, A. (2013). Constraints on the transfer of perceptual learning in accented speech. Frontiers in Psychology, 4, 148.
• Hecker, F. (2013). Untitled (f.a.n.n.). Retrieved from: http://kunsthalaarhus.dk/ en/florian-hecker-untitled-fann-sift-flow-slides.
• Hermann, T., Hunt, A. & Neuhoff, J.G. (2011). The sonification handbook. Berlin: Logos Verlag.
• Hinton, G.E., Osindero, S. & Teh, Y.-W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18(7), 1527–1554.
• Ikoniadou, E. (2010). Hypersonic sensation: The non-human in human perception. In N. Hassan & H. Tessler (Eds.), Sounds of the overground. Turku, Finland: International Institute for Popular Culture.
• Kant, I. & Guyer, P. (1998). Critique of pure reason. Cambridge: Cambridge University Press.
• Kereliuk, C., Sturm, B.L. & Larsen, J. (2015). Deep learning, audio adversaries, and music content analysis. Workshop on Applications of Signal Processing to Audio and Acoustics, IEEE (pp. 1–10).
• Krysa, J., Parikka, J. & Huhtamo, E. (2015). Writing and unwriting (media) art history: Erkki Kurenniemi in 2048. Cambridge, MA: MIT Press.
• Laennec, R.T. (1819). De l’auscultation médiate: ou traité du diagnostic des maladies des poumons et du coeur (Vol. 2). Paris: J.A. Brosson and J.S. Chaude.
• Legrand, D. (2011). Ex-nihilo: Forming a body out of nothing. In R. Mackay & R. Negarestani (Eds.), Collapse: Philosophical research and development, vol. VII: Culinary materialis (pp. 499–558). Falmouth: Urbanomic.
• Mordvintsev, A., Olah, C. & Tyka, M. (2015). Inceptionism: Going deeper into neural networks. Retrieved from: http://googleresearch.blogspot.com/2015/ 06/inceptionism-going-deeper-into-neural.html.
• Nguyen, A., Yosinski, J. & Clune, J. (2014). Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. arXiv:1412.1897 [cs.CV].
• Olazaran, M. (1996). A sociological study of the official history of the perceptrons controversy. Social Studies of Science, 26(3), 611–659.
• Sieroka, N. (2015). Leibniz, Husserl and the brain. Basingstoke and New York: Palgrave Macmillan.
• Sigtia, S. & Dixon, S. (2014). Improved music feature learning with deep neural networks. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2014 (pp. 6959–6963).
• Stern, J. & Akiyama, M. (2011). The recording that never wanted to be heard and other stories of sonification. In T. Pinch & K. Bijsterveld (Eds.), Oxford handbook of sound studies (pp. 544–560). Oxford: Oxford University Press.
• Sterne, J. (2012). Mp3: The meaning of a format. North Carolina: Duke University Press.
• Supper, A. (2011). The search for the “killer application”: Drawing the boundaries around the sonification of scientific data. In T. Pinch & K. Bijsterveld (Eds.), Oxford handbook of sound studies (pp. 249–270). Oxford: Oxford University Press.
• Szegedy, C., Lio, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. & Rabinovich, A. (2015). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–9.
• Tudor, D. (1993). Neural synthesis. Lovely Music.
• Zahavy, T., Zrihem, N.B. & Mannor, S. (2016). Graying the black box: Understanding dqns. arXiv:1602.02658[cs.LG].
• Zeiler, M.D. & Fergus, R. (2014). Visualizing and understanding convolutional networks. Computer Vision–eccv 2014, 818–833. Berlin: Springer.

Footnotes

1. The subtleties of Kant’s more nuanced differentiations (e.g., a transcendental object, transcendental unity of apperception) are beyond the scope of the current article (Kant & Guyer, 1998).
Back to text
2. Catren conjoins the terms phenomenon, that which can be sensed, with noumenon, that which cannot be sensed, and delos, or that which is manifest, to make this portmanteau word (Carroll, 1917). The word connotes that one is operating on that which cannot be directly sensed in order to modify what is possibly sensible.
Back to text
3. For a detailed elaboration of a similar evolutionary system within the context of Piagetian genetic epistemology, see Burman (2013).
Back to text
4. We are using the International Phonetic Alphabet for these examples.
Back to text
5. Frank Rosenblatt unintentionally stifled early adoption of neural networks by erroneously emphasising their successes (Olazaran, 1996). The backlash of the associated hype took multiple decades to overcome. If it were not for the laudable work of Geoffery Hinton, we would probably still be dreaming shallow, deep in the dark (Hinton et al., 2006; Bengio, 2013).
Back to text
6. See Dieleman & Schrauwen (2014); Sigtia & Dixon (2014).
Back to text
7. The results of their study can be listened to at http://www.eecs.qmul.ac.uk/~sturm/research/DNN_adversaries/.
Back to text
8. In future research, we hope to build systems that perturb sonorous structurations in order to sonify black box algorithms. This research will also entail the investigation of audible infograms as well as the exploration of sonic signs (Hermann et al., 2011; Stern & Akiyama, 2011; Supper, 2011).
Back to text
9. For example, a deep neural net was trained by Kereliuk et al. (2015), whose input songs could be mutated such that it was possible to misrecognize them as any genre.
Back to text
10. Infra_perceptible is derived from artist and philosopher Marc Couroux’s “infra_legible”, which has been used to describe variations of a particular sonic phenomenon that become backgrounded and normalised such that they are not necessarily perceived by the listener (Couroux, 2016). An example of infra_perceptible sounds are those in the hypersonic register, which are technically outside our perception, and yet have the capacity to influence the experience of what is heard (Ikoniadou, 2010).
Back to text
11. The work of Tudor (1993), Hecker (2013) and much of Erkki Kurenniemi’s work with neural networks also falls into this category. See Krysa et al. (2015) for an interview of Florian Hecker by Robin MacKay detailing much of this work.
Back to text
12. Auscultation is a listening technique in the field of medicine where sounds become signs or symptoms leading to the diagnosis of ailments of the body (Laennec, 1819).
Back to text

Author contacts

• Lendl Barcelos
• lendl@kingston.ac.uk
• Kingston University
• Audio Culture Research Unit
• Penrhyn Rd, Kingston upon Thames, Surrey KT1 2EE, United Kingdom

• Michael O. Vertolli
• michaelvertolli@gmail.com
• Carleton University
• Institute of Cognitive Science
• 1125 Colonel By Dr, Ottawa, ON, Canada, K1S 5B6