Analyzing and synthesizing emotional intent in contemporary piano music through deep learning-based performance style transfer

Project Abstract

This study presents a novel framework for analyzing and synthesizing emotional intent in contemporary piano music through deep learning-based performance style transfer. By modeling the relationship between expressive cues in performer gestures, timing, dynamics, and touch, the project develops a data-driven approach to transfer expressive styles across performances and composers while preserving musical content. A multimodal dataset consisting of high-resolution piano performances, aligned audio, MIDI representations, and performer metadata is created to capture a broad spectrum of emotional expressions, including happiness, sadness, anger, tenderness, and excitement. The methodology integrates audio-based and symbolic representations to learn robust mappings from source performances to target emotional styles using a combination of variational autoencoders, attention-based sequence-to-sequence models, and diffusion-inspired refinements. First, expressive features are extracted from performance data, including dynamic contours, articulation, tempo fluctuations (rubato), pedal usage, and timbral proxies derived from audio spectral characteristics. These features are fused with symbolic representations such as MIDI note trajectories and pedal events to train a conditional generative model capable of translating a neutral or source performance into a desired emotional expression. A novel loss function balances content fidelity with expressive accuracy, ensuring that structural musical content remains intact while expressive timing and dynamics are transformed to reflect the target emotion. The system supports user-defined constraints, enabling performers and composers to specify intensity levels, tempo ranges, and articulation preferences to tailor the synthesized rendition. Evaluation employs both objective metrics and perceptual studies. Objective assessment includes alignment accuracy of transformed performances against ground-truth emotional references, spectral similarity measures, and dynamic range analysis. Perceptual evaluation comprises listening tests with expert pianists and audiences to gauge perceived emotional content, naturalness, and musical coherence. The study also investigates cross-genre and cross-composer generalization, scalability to longer works, and robustness to performance idiosyncrasies. An ablation study isolates the contributions of the symbolic-spectral fusion, the attention mechanism, and the diffusion refinement module. The results demonstrate that the proposed framework can reliably imbue neutral performances with targeted emotional character while preserving essential musical structure and intelligibility. Comparative analyses indicate improvements over baseline style transfer methods in perceived emotional expressiveness and naturalness, with consistent performance across varied repertoires. The research discusses implications for pedagogy, performance practice, and music technology, highlighting potential applications in automated accompaniment, expressive performance augmentation, and restorative music generation for rehabilitation and therapy. Limitations include the need for larger, diverse datasets to capture a broader spectrum of cultural and stylistic nuances and the computational demands of real-time deployment. Future work envisions adaptive user interfaces for real-time emotion control, integration with haptic feedback for pianists, and extension to multi-instrument ensembles to explore inter-arts expressive synchronization.

Project Overview

What This Project Is About

---

A plain-language overview of studying how computer models can understand and imitate the emotional feel of modern piano performances, and then apply those emotions to new piano music.

---

---

The Problem It Addresses

---

Many piano performances convey clear emotions, but creating programs that consistently reproduce or alter those emotions across different pieces is hard. This project explores a way to teach a computer to recognize emotional cues in real piano playing and to reproduce or transform these cues in new performances.

---

---

Objectives of the Project

---

1. Understand basic concepts of how emotion and musical expression are conveyed in piano playing.
2. Learn how a computer can learn from examples of expressive performances.
3. Develop a simple model that can transfer expressive style to new piano pieces.
4. Evaluate whether the synthesized performances feel emotionally consistent and convincing.
5. Discuss potential applications and ethical considerations of expressive AI in music.

---

---

What You Will Do Step by Step

---

1) Gather example recordings with notes and performance labels (emotion tags).

2) Prepare data so the computer can learn from the scores and performances.

3) Train a basic model to map emotions to piano playing nuances (tempo, dynamics, articulation).

4) Apply the model to new piano pieces to create expressive versions.

5) Compare original and transformed performances using listener feedback and simple metrics.

---

---

---

---

Expected Outcome

A working, easy-to-use method that can take a piano piece and generate a version with a chosen emotional character, plus an initial evaluation of how well the emotion was conveyed and areas for improvement.