.png)
Vishruth Veerendranath, Vibha Masti, Utkarsh Gupta, Hrishit Chaudhuri, Gowri Srinivasa
PES University and Carnegie Mellon University
Scene from The Fault in Our Stars
The piano music playing in the background is the music that we have generated using the aforementioned pipeline. EMOPIA’s Compound Word Transformer (EMOPIA-CWT) was used as the conditional music generator
The piano music playing in the background is the music that we have generated using the ScripTONES pipeline. The following are the best results obtained using the aforementioned pipeline. EMOPIA’s Compound Word Transformer (EMOPIA-CWT) was used as the conditional music generator. The scenes are from the movie names mentioned above the video.
We use attribute vector arithmetic to manipulate latent vectors of music and change the sentiment attributes according to the script or user input.
The sentiment conditioned vector $z_{ec}$ is defined as per the equation below
$$ z_{ec} = \begin{cases} |V| * z_{vh} + |A| * z_{ah} & (V \geq 0, A \geq \alpha)\\ |V| * z_{vh} + |A| * z_{al} & (V \geq 0, A < \alpha)\\ |V| * z_{vl} + |A| * z_{ah} & (V < 0, A \geq \alpha)\\ |V| * z_{vl} + |A|* z_{al} & (V < 0, A < \alpha)\\ \end{cases} $$