AKA machine learning, aesthetics, & the unconscious
When I heard about the neural nets that copy the styles of famous painters I thought it would be the same old junk.
Academics have been saying forever that they were on the verge of discovering the principles of aesthetics, and that they would soon be able to automate the production of beauty – melody, harmony, proportion, plot.
When I was a kid I was excited to read about this sort of thing. But they always turn out to be fatuous, catastrophically oversimplified and overconfident, written - I’m guessing - by people who are intimidated & resentful of the culture around them. Our technical understanding of what makes something look good is still weak, and I don’t think it’s improving very fast. I learned to, when I come across an article about art written by a scientist, turn the page.
But now I think that maybe the automatic production of beauty will arrive soon. The machine learning algorithms work by extrapolating from existing examples, which means that they can produce new examples that fit some pattern (such as the pattern of beauty) without anyone involved having any explicit understanding of what the pattern is or how it can be defined.
This extrapolation without understanding is what happened in the study of visual perception – i.e. making inferences from images. Our understanding of perception is slowly moving forward, as it has been for centuries, but our ability to automate perception has shot ahead. In the 15th century Leonard da Vinci studied how the light reflected by an object is related to its distance – more distant objects tend to be bluer – these are relationships that we all know unconsciously, but which take a lot of work to dig out, such that we consciously understand them. Psychologists and computer scientists are still discovering things about the physics of light which we all know unconsciously. But computer models which incorporate our explicit knowledge of the physics of light are being thrashed by pure machine-learning models, which are fed a huge databases of pictures and simply extrapolate from what they’ve already seen.[2]
I think the same basic point is true of aesthetic things. We really struggle trying to explain why we like a picture or dislike a melody, because most of the work is done at an unconscious level. The progress in understanding those principles will probably continue to be slow.
But now it seems likely to me that, before long, machines will be able to do all these things on demand – play some brand new Mozart, make elegant little drawings of animals, write a pretty good pop song. And the programmer who implements them could be – probably will be – some bozo who has no clue why it works.
[1] 2017-05: SIGGRAPH video with style transfer - https://www.youtube.com/watch?v=HYhzZ-Abku8
[2] 2017-05: Michael Elad “Deep, Deep Trouble: Deep Learning’s Impact on Image Processing, Mathematics, and Humanity” https://sinews.siam.org/Details-Page/deep-deep-trouble-4