Google AI turns paintings into music
The new Google tool "Play a Kandinsky" uses AI to create music based on paintings. In doing so, it mimics the effects of synesthesia, a form of human perception in which two senses are coupled together. The model was the painter and synesthete Wassily Kandinsky, for whom colors were audible and pieces of music were visible.
"An AI analyzes enormous amounts of data and thus recognizes things that a human often cannot perceive. Its use in the field of art is also conceivable. In both the visual and auditory realms, an algorithm can find structures and patterns and thus learn what makes artists' works unique," explains digitization expert Tim Niesen in an interview with news portal Press release.
"Play a Kandinsky" was created in collaboration between Google and the Centre Pompidou in Paris. Users can interact with Kandinsky's 1925 painting "Yellow-Red-Blue" and create their own music by clicking on the colors. The application also explains how a synesthete perceives music and colors.
Training through music and paintings
The developers trained the system using machine learning. The AI has learned both the works of Kandinsky and the music from his creative period. In addition, the algorithm knows which sounds, instruments and emotions the Russian master associated with certain colors.
Play a Kandinsky can create music from the paintings. According to Google, this does not replicate the exact perception through synesthesia. However, it is intended to give users an understanding of how emotions are related to certain sensory impressions. (pte)
