Music Visualization Robot: Transforming Sound into Art
The completed XY plotter robot translating music into visual art
Project Overview
Course
Applied Design Methodology in Mechatronics (ADMM 2023)
Duration
One Semester
Team Size
8 Engineers
Problem Statement
How might we design a robot that interprets music and transforms it into unique, expressive visual art—automatically and in real time?
Conceptual visualization of translating music frequencies into artistic patterns (AI Generated)
Team Members
My Role
Project Manager
- Led sprint planning and Agile methodology implementation
- Coordinated cross-functional team activities and milestone tracking
- Managed project timeline and resource allocation
Algorithm Developer
- Designed music analysis algorithms using Python and FFT
- Developed three unique music-to-motion translation algorithms
- Implemented signal processing techniques for real-time interpretation
Team Coordinator
- Facilitated communication between mechanical, electrical, and software teams
- Ensured integration of subsystems aligned with project requirements
- Led design reviews and technical presentations
Tools & Skills Used
Project Management
- Jira - Task tracking and sprint planning
- Miro - Collaborative design and brainstorming
- Agile Methodology - Iterative development approach
- V-model (VDI 2206) - Development framework
Technical Tools
- Python - FFT and signal processing
- CAD Software - Mechanical design and simulation
- Hardware - Stepper motors, Arduino, paint mechanisms
Project Gant Chart
Design & Engineering Process
We followed the VDI 2206 V-model methodology to systematically develop our music visualization robot:
1. Requirement Definition
Established key system requirements:
- Safety protocols for human-robot interaction
- User-friendly interface for operation
- Artistic expressiveness in visual output
- Cost-effective design within budget constraints
- Real-time music processing capability
Requirements list
2. Black Box Modeling
Created functional abstractions to define system behavior, mapping inputs (music signals) to outputs (robotic movements and painting patterns).
Black box modeling of system inputs and outputs
3. Functional Decomposition
Broke down the system into core functional subsystems:
- Music Signal Acquisition and Processing
- Signal-to-Motion Translation Algorithms
- XY Motion Control System
- Painting Mechanism and Tool Management
- User Interface and Control Panel
Functional Decomposition Diagram
4. Concept Generation
Using structured brainstorming methods, we explored multiple design approaches before selecting the design we wanted to pursue. We used two different techniques: one known as the 6-3-5 rule and another called Synectics. PS: Ours is called 4-2-5 since we adapted the original methods to our team's needs.
6-3-5 Brainstorming Session
Synectics Method
5. System Design & Manufacturing
Finalized a 900×900 mm high-accuracy XY plotter design with:
- Precision stepper motors and belt drive system
- Custom-designed tool holder for multiple painting implements
- Modular frame construction for stability and portability
- Integrated electronics and control system
CAD model and physical assembly of the XY plotter system
6. Algorithm Development
Created three distinct music-to-motion algorithms, each offering unique artistic interpretations:
Firework Algorithm
Colors selected by note frequency, stroke length determined by amplitude. Creates explosive, radial patterns that respond dynamically to music intensity.
Flower Bouquet Algorithm
Generates radial stroke patterns with variations based on octave recognition. Produces organic, flowing forms that evolve with the musical composition.
Shapes Algorithm
Creates geometric patterns based on dominant frequencies in the music. Responds to rhythm and beat patterns with corresponding visual elements.
Visual outputs from the three different algorithms processing different music sample
7. Testing & Verification
Conducted comprehensive testing to validate system performance:
- Movement precision tests: achieved ±0.5 cm accuracy
- Algorithm fidelity testing: confirmed accurate music interpretation
- Painting quality assessment: optimized tool pressure and movement speed
- User experience testing: refined interface and operation workflow
Testing and calibration of the robot
8. Risk Assessment & Final Presentation
Completed safety review, documentation, and prepared for the live demonstration:
- Identified and mitigated potential failure modes
- Created user operation manual and safety protocols
- Prepared demonstration script and backup plans
- Organized live audience participation elements
Results & Outcome
Live Demo Success
Successfully performed in front of an audience, converting real-time music into paintings with three distinct algorithmic styles.
Technical Achievement
Achieved high positional accuracy (±0.5 cm) and responsive real-time music processing with minimal latency.
Artistic Innovation
Created a modular system supporting multiple painting tools (brush, marker, pen) with unique artistic expressions for each algorithm.
User Engagement
Received enthusiastic feedback on the emotional connection between the music input and visual output.
Completed artwork created during the live demonstration
Final Thoughts
This project brought together the rigor of engineering with the expressiveness of art. Leading such a cross-disciplinary team and watching an abstract idea come alive through sound and paint was a deeply rewarding experience.
The Music Visualization Robot demonstrated how technical precision and artistic creativity can work together to create something truly innovative. The project not only fulfilled its technical requirements but also created an emotional connection with its audience—revealing new possibilities for human-machine artistic collaboration.
The ADMM project team with the completed music visualization robot