One of the best things to experience on the Apple Vision Pro is an immersive video, such as the impressive collection of videos created by Apple. You sit in a single spot, you look around, and thanks to the 180º video and the spatial sound, you feel as if you have been transported somewhere else. It is incredibly realistic and impressive. Some of these immersive videos feature musicians playing music, and one of the very best examples of that is notable becuase it was not produced by Apple. The Prima Immersive app currently includes one episode of the Sessions immersive video series from The Spatialists, and it features bluegrass band AJ Lee & Blue Summit. When you watch that episode, you feel as if you are sitting in the room with the band as they perform live, just for you.
If there is a downside to an immersive video musical performance, it is that you must stay in one place. You cannot walk around the band. Of course, there is nothing unusual about that limitation—when I attend a live concert or other performance, I typically have to stay in my seat. However, when listening to Episode 42 of the fantastic Vision Pros podcast hosted by Tim Chaten, I learned about a new Apple Vision Pro app called Mahler Chamber Orchestra that removes this limitation. The app allows you to experience a classical music performance, and while you are free to sit in a chair and join the musicians in a circle as they play music, what makes this app stand out is that you can stand up and walk around. For example, if you want to really hear the cello, just walk up in front of the musician playing the cello to get close to that instrument. That is almost certainly something that you would never be able to do in a live performance (unless you are really good friends with the musicians), but in a virtual performance, you can feel free to do so.
The app indicates that more music is coming in the future, but for now, there is just a single performance (which costs $6.99 to purchase): a performance of Mozart’s Quintet in G minor. The musicians are members of the Mahler Chamber Orchestra, which is based in Berlin. The app was created by Reflekt Music, a company that consists of Henrik Oppermann (an expert in 3D sound) and Timothy Summers (a graduate of Harvard and Julliard who plays violin and is one of the musicians in the quintet).
I started by putting on my Vision Pro while I sat in a chair in the room in my house that has my piano and some other instruments, which seemed like the appropriate place in my house for a performance. Once I started the app, I felt like I was witnessing a private concert at my home.
The artists are represented by dots in space. It reminds me of the traditional Wall Street Journal Hedcut representation of a person that is created by dots, except that these dots are in color and the life-size musicians are animated in real-time to reproduce an actual performance. I can turn to one side or the other to face different musicians.
The representation of the artists with dots may be a technical limitation, but it also serves an artistic purpose: this app is not about what the artists look like but instead about the sound of the music that they create. And the spatial sound is really good. Oppermann says that the app uses “point clouds, offering a fully immersive experience with 6 degrees of freedom and spatial audio with precise acoustic modelling.”
What is most impressive is that as I move around the room, I can pay even closer attention to the music coming from each specific instrument. I could even place my head inside of the violin or cello or viola to better hear—and, in some ways, feel—that particular instrument and get a deeper understanding of each specific performance that is a part of the whole. The spatial music adapts in real time to your movements so that no matter where you are positioned, the sound comes from the correct location.
I used this app both using the built-in speakers on the Vision Pro and using my AirPods Pro. I didn’t notice much of a difference; they were both excellent.
Here is a video that, while in 2D, approximates what the experience is like:
Although using the Vision Pro to experience this performance is new, the performance was recorded a few years ago, and there has been an installation that has traveled to different locations in Europe where people could wear earlier VR headsets to experience the performance. This video shows both the original performance that was recorded (along with other performances that have not yet been released for the Apple Vision Pro) and attendees experiencing virtual performances:
After experiencing this quintet performing a composition by Mozart, I now want to hear more. In the podcast episode, Oppermann and Summers talk about recording a full orchestra using spatial sound. I’m not sure how that would work with a Vision Pro—I guess I would have to be standing in a huge room that is big enough to fit an orchestra?—but it would be fascinating to walk around and pay closer attention to individual instruments in a huge orchestra. I would also love a similar experience with other genres of music: perhaps Dixieland jazz, traditional jazz, bluegrass, folk music, or any other type of acoustic music where each instrument or singer provides unique contributions.
I’m still a fan of the immersive videos that Apple and others have been creating for the Apple Vision Pro. But now that I have tried this app, I am eager to see where this type of technology goes in the future as headsets get smaller, lighter, and more powerful, thus making it easier to walk around the virtual musicians.
Click here to get Mahler Chamber Orchestra on the App Store (free, but you must pay to download a performance)