MuseNet

During training time, these composer and instrumentation tokens were prepended to each sample, so the model would learn to use this information in making note predictions. At generation time, we can then condition the model to create samples in a chosen style by starting with a prompt such as a Rachmaninoff piano start:

Or prompted with the band Journey, with piano, bass, guitar, and drums:

We can visualize the embeddings from MuseNet to gain insight into what the model has learned. This long context may be one reason why it is able to remember long term structure in a piece, like in the following sample imitating Chopin:

It can also create musical melodic structures, as in this sample imitating Mozart:

Music generation is a useful domain for testing the Sparse Transformer as it sits on a middle ground between text and images.

Source: openai.com