Compositional bilingual artificial neural networks for predicting hypermetrical structure among interacting flamenco parts

Particularly in traditional, improvisational genres such as flamenco or jazz, much of music can be seen as parallel musical sequences that interact such that each part influences decisions made by other parts. We recently suggested leveraging the formal language theory notion of transduction grammars, in stochastic forms, to model each part as a musical language (Wu 2013). The advantage is that stochastic transductions can be exploited to model the probabilistic, ambiguous, complex structural relationships between interacting parts. Transduction grammar induction techniques can then be used to model unsupervised learning of musical accompaniment and improvisation. We explore an alternative approach carrying many of the same properties, but instead using artificial neural networks to learn compositional distributed vector representations that implicitly encode structural relationships between associated portions of two different musical parts. As with symbolic transduction grammars, these structural association patterns can range from concrete to abstract patterns, and from short to long patterns. Unlike symbolic transduction grammars, a single vector encodes a “soft” set of multiple similar hypotheses in the same neighborhood, because similar vectors tend to be learned for association patterns that are similar—cutting down the combinatorial growth of hypotheses inherent in the symbolic approaches. Since conventional neural networks have difficulty representing compositional structures, we propose to use a bilingual generalization of Pollack’s (1990) recursive auto-associative memory. Whereas Pollack’s RAAM can be seen as a neural approximation of a single compositional language model, our TRAAM (Transduction RAAM) approach is a neural approximation of a bilingual compositional transduction model—a relation between two probabilistically structured musical languages. We discuss empirical analyses of the learning behavior of our new neural approach on the hypermetrical structure prediction problem in flamenco, where meter changes can be rapidly influenced by multiple parts.