AI-assisted Music Genre Transfer and Real-time Performance System

Project Abstract

This study presents an AI-driven framework for seamless music genre transfer and real-time performance, aiming to bridge stylistic gaps between traditional and contemporary music forms while preserving musical integrity and expressive intent. The core contribution is a multi-stage system that combines signal processing, deep generative modeling, and interactive performance controls to enable musicians, composers, and producers to morph entries from one genre into another without compromising rhythm, timbre, or emotional contour. At its heart lies a modular architecture that integrates a robust feature extractor, a genre-conditional generative model, and a real-time synthesis engine, all trained on a curated cross-genre corpus spanning classical, jazz, rock, electronic, and world music traditions. The feature extractor captures timbral, rhythmic, harmonic, and dynamic attributes at multiple temporal resolutions, enabling nuanced transfers that respect groove and articulation. A genre-conditioned variational autoencoder and diffusion-based diffusion probabilistic model learn latent representations that encode stylistic fingerprints while allowing precise manipulation along a genre axis. The system supports real-time control via MIDI, touch, and gesture inputs, providing performers with expressive commands to adjust tempo alignment, swing, articulation, harmonic density, and spectral emphasis during live rendering. To ensure perceptual plausibility, a multi-objective optimization framework balances authenticity of the target genre with fidelity to the source material, employing perceptual loss components grounded in psychoacoustic models and a discriminator trained to distinguish genre authenticity. A novel evaluation protocol combines objective metrics—spectral similarity, rhythmic alignment, and timbral distance—with perceptual listening tests across expert musicians and non-expert listeners to validate transfer quality, emotional continuity, and user experience. The system also addresses latency, scalability, and variability issues inherent in real-time synthesis by implementing asynchronous processing, model quantization, and adaptive chunking strategies, enabling stable performance even on resource-constrained hardware. Experimental results demonstrate high-fidelity genre transfer across diverse instrumentations, including piano-led pieces, guitar-led passages, and vocal lines, while maintaining expressive nuances such as microtiming, phrasing, and dynamic swell that characterize the target genre. Comparative studies against baseline genre translation methods reveal superior timbral coherence, rhythmical fidelity, and user satisfaction in tasks including tempo-agnostic and tempo-modulated transfers. The research also explores ethical and creative implications, examining authorship, stylistic originality, and the potential for collaborative composition where AI serves as a co-creator rather than a replacement. Practical applications encompass music production, live performance augmentation, education, and cross-cultural collaboration, with an emphasis on accessibility for independent artists and small studios. The study concludes with a discussion of limitations, including genre coverage scope, training data biases, and the trade-offs between realism and computational efficiency, and outlines future directions to expand multi-modal integration, on-device portability, and adaptive user interfaces to further empower end-users in shaping sonic remixing and live-reinvention of musical works.

Project Overview

What This Project Is About

---

A beginner-friendly overview of how computer tools can change the style of music and how performance settings can adapt in real time. The project explores converting elements of a song from one genre to another (for example, from rock to jazz) while preserving melody and rhythm, and then applying those changes live during a performance using software tools. It focuses on making genre transformation accessible to learners and performers without deep programming knowledge.

---

---

The Problem It Addresses

---

Musicians often want to experiment with different styles, but creating convincing genre changes can be complex and time-consuming. There is a need for a simpler, more intuitive way to explore genre influences and to perform those changes live without relying on costly studio setups.

---

---

Objectives of the Project

---

1. Understand the basic ideas behind music genre characteristics and real-time audio processing.
2. Build a user-friendly system that suggests genre changes while keeping the original melody intact.
3. Implement a simple real-time performance workflow to apply genre changes during a live or simulated show.

---

---

What You Will Do Step by Step

---

1. Learn basic music terms and audio concepts used in the project.
2. Collect and study example tracks from different genres to identify common features.
3. Shape a simple model that maps genre features to audio parameters (e.g., tempo, instrument choices, harmonies).
4. Develop a lightweight interface for selecting target genres during practice or performance.
5. Test the system with sample tracks, evaluate how natural the genre changes sound.
6. Analyze results and adjust settings to improve realism and responsiveness.

---

---

Expected Outcome

An easy-to-use tool that can transform a piece of music into another genre while playing in real time, plus a short report on its usefulness for practice, education, and small performances.