Bini Park
PRODUCT DESIGNER · MR · APPLE VISION PRO @PLAI
JazzXR
How can we visually memorize the sound of our music by painting our physical wall?
Apple vision Pro Project that lets musicians see their sound come to life by painting their wall!
INFORMATION
This was a short-term prototype project created before Plai pivoted to the piano version.
JazzXR began when the Apple Vision Pro was released in early 2024. The goal was to integrate jazz musicians in New York City and create a tool that professional players could use daily. The idea showed strong potential and gained support from the jazz and saxophone player community in New York. However, we were unable to scale it due to a lack of consistent customer demand.
Our team worked remotely for three weeks, bringing together skilled developers from across the U.S. I led the design and concept development, and also handled integration with the Apple Vision Pro using Xcode. As the lead product designer, I was responsible for everything from user research to product development and concept visualization.
OUTCOMES
Conducted a playtest with renowned saxophonists Ben and Paul
HOW TO PLAY
CONCEPT
Blow your instrument
Create bubble with your music
Bubbles pop on the wall
Paint your room with music
PROBLEM & SOLUTION
STRATEGY
DESIGN QUESTION
How might we create an immersive tool that enhances daily practice and improvisation for jazz musicians using spatial computing?
PROBLEM
Traditional jazz practice tools (backing tracks, sheet music, metronomes) are static and uninspiring.
Jazz musicians, especially in improvisation, benefit from spatial awareness and real-time feedback.
There’s no intuitive, immersive experience that supports creative flow or mimics the feeling of playing with a live band, especially for solo practice.
SOLUTION
We built JazzXR, an immersive spatial tool on Apple Vision Pro that transforms solo jazz practice into an interactive, visually guided experience.
Visualized Pitch & Frequency with colors
Displayed Articulation with shapes
Integrated Volume & Intensity with size
GOAL
STRATEGY
Our main goal is to…
The goal was to explore how spatial computing could genuinely support jazz musicians in their daily practice, not just as a gimmick, but as a tool they'd actually want to use.
I wanted to design something that felt natural to musicians, especially those like Ben and Paul who live and breathe improvisation.
Challenge + Learning
STRATEGY
CHALLENGE
The biggest challenge was designing for a very niche and tactile user group ⇾ professional jazz players
Meanwhile working with bleeding-edge hardware like the Apple Vision Pro, we had to move fast, test ideas with limited access to users, and constantly balance tech novelty with actual musical value.
LEARNING
⇾ So what I learn…
I learned that the best products feel familiar and genuinely useful to the people you're building for.
I realized niche products won’t succeed if the hardware and everyday use cases aren’t aligned.
⇾ Other key learnings from this project..
I realized there’s a formula for success: a niche use case × daily hardware adoption = viable product
--
Design Process
--
LATEST MAIN POSTER & CONCEPT SHOT
Main Concept Poster
Mock Up
USER RESEARCH
USER TARGET
Professional and semi-professional jazz musicians who practice regularly and value creative improvisation
They want….
a practice tool that feels as responsive and dynamic as playing with a live band
visual cues for chords, scales, and rhythm without interrupting musical flow
a seamless setup experience that doesn’t get in the way of creativity
Jazz musicians
Educators
SKETCHES & STORYBOARD
IDEATION SKETCHES
STROYBOARD
GAME FLOW MAP
Key Experiments
Each bubble in JazzXR is unique, shaped by the way the saxophone is played. The system listens in real-time and transforms the sound into colorful, expressive visuals, making each note feel personal and alive.
Here’s how the four key elements define the bubbles:
01 | Pitch & Frequency → Color
Each note has its own color identity, inspired by coloured hearing synesthesia.
02 | Note Length → Duration
Short notes (staccato) create small, quick bursts that fade fast.
Long notes (legato) produce large, flowing bubbles that stretch and linger.
03 | Articulation → Movement
Smooth, connected notes (legato) create soft, drifting bubbles that float seamlessly.
Sharp, detached notes (staccato) generate quick, bouncing bubbles that pop energetically.
04 | Volume & Intensity → Size
Soft, quiet tones create gentle, delicate bubbles that float lightly.
Loud, powerful sounds cause bubbles to expand and burst dramatically.
PlayTest
--
Final Look
--
Design Decision
Design Goal
As a team, we defined three core design goals to guide the experience and ensure it felt intentional, comfortable, and emotionally engaging.
Aesthetic Goal
I aimed for a transparent, bubbly visual style that felt light, playful, and responsive to the music. The interface elements would gently float in space and subtly change size based on the dynamics of the player's performance
Moodboard + Mockup
What's next?
I plan to apply these learnings to more accessible platforms like mobile and desktop, while keeping spatial music interaction in mind.
I’m also exploring how to bridge the gap between professional tools and playful, intuitive interfaces, especially for music learning and expression.
FINAL THOUGHTS!!
While JazzXR didn’t move forward as a full product, it opened valuable insights for designing creative tools in emerging platforms. This prototype helped me:
Understand the gap between niche user needs and hardware accessibility
Explore spatial design patterns for music-based interaction
Validate early-stage ideas with real musicians
WANT TO SEE MORE?