Apple Music Spatial Audio to Get Dynamic Head Tracking in the Fall

[level1807]

People who write these articles clearly haven't even considered whether what they're writing is true. Spatial Audio with videos IS NOT ANCHORED TO YOUR DEVICE. All you need is a kindergarten level of curiosity to devise a simple experiment: turn on a video on the iPhone, keep your head still, and move the iPhone around your head. You won't hear any difference.

Now keep the iPhone in place and just turn your entire body away from it, and keep still for 15 seconds or so. You'll notice that Spatial Audio will automatically re-center itself to your head's new position.

In other words, Spatial Audio knows nothing about the position of your phone relative to your head. It simply "anchors" the 3D audio to the average position of your head over some short period of time.

This also means that enabling this for music would be completely trivial. However, in its current form that wouldn't make sense either: you don't really listen to music in your headphones while sitting still. You walk around, and it would be weird to have your virtual music stage suddenly shift every 10 seconds. Personally I don't see how Head Tracking can make sense with music at all, but perhaps they're planning to come up with something.

 

[jhwalker]

It clearly doesn't - i've no idea what you guys think you're hearing who are saying this. (Unless you're running iOS15 beta in which apparently it does work)

Yes, it works in the iOS15 beta. I'm listening from iPhone to AirPods Max and the sound appears centered when you're looking straight forward - if you turn your head from side to side, the sounds remain fixed in space (which means they move from earpiece to earpiece as needed to maintain the illusion).

 

[ntlman]

How would the source material be an issue for these albums? The remix producer is working from the original multi-track recording tapes, recorded by some of the best bands, producers and engineers in the best studios in the history of popular music. The producer can take each track and place it anywhere they want in the 3d space afforded by Dolby Atmos.

As someone who owns several Atmos music re-mixes on Blu-ray audio, and a full Atmos home theatre with in-ceiling speakers I can say for a fact they do benefit from it. The discs also contain stereo mixes and 5.1 DTS HD mixes all from the same source material and you can freely switch between them. The Atmos track is far more immersive in every case.

In theory, working from original master tapes should allow the 5.1 remixing to great effect. HOWEVER, as anyone who has bought a "remaster" in the last few years can tell you, artists often times just put a high emphasis filter and brickwall the volume on a previously issued CD rip and sell it as a remaster. The Bowie 2015 best of compilation, the U2 remasters, and many other legacy artists are guilty of this. So I don't presume every catalog artist will actually go back to the master recordings as thoroughly as the Beatles/Stones, et al.

Also, we know for a fact that because of Universal Music's imcompetent management, many master tapes to legendary albums were lost in a fire. So there are a lot of artists, based on the article below, whom YOU SHOULD NEVER ALLOW YOURSELF TO BE TRICKED INTO BUYING REMASTERS FROM.

Recordings by Elton John, Nirvana and Thousands More Lost in Fire (Published 2019)

A New York Times investigation has revealed that decades of Universal Music Group treasures burned in 2008.

www.nytimes.com

 

[Kuckuckstein]

I agree with much of what you've posted in this thread, but I don't think this is entirely true. I was under the impression that Spatial Audio is also applying a generalized HRTF to the object-oriented mix, to "trick" the ears into hearing a 3D soundfield. Tom's appears to agree.

Oh absolutely. None of that would work if there would not be an HRTF for each source involved. But as you say, most likely it is a generalized.

What I wanted to say is that the generalized HRTF will work for some, but others will realize that their head would need a slight correction. In the end that is what “Head related” stands for 🙂

I have listened to audio processed through somebody else’s HRTF, and it just makes that the localization of sources is a bit off, that the audio loses the 3d crispness.

 

[bollman]

Thanks for the snide reply. My home theatre consists of a Marantz Atmos receiver with British-made Neat motive speakers, KEF in-ceiling speakers and a Paradigm subwoofer. I also have a headphone listening station with a Schiit Jotunheim headphone amp, Bifrost DAC and a pair of Beyerdynamic 880 500Ω headphones.

I own The Beatles Abbey Road on vinyl, CD, and Blu-ray Audio in high res 2.0 and the aforementioned Dolby Atmos. John Lennon's Plastic Ono Band I own on, vinyl, CD and an original 1970 release on 7.5ips reel to reel (and an Akai reel to reel player to listen to it on). Over the past 25 years I've probably listened to each album hundreds of times.

So I have a pretty good idea what the original recordings sounds like. Listening to them in Atmos was as close as hearing them again for the first time as I can hope for. It isn't better or worse, and it won't replace me listening to them in stereo again. But the Atmos version is there when I want a different and more immersive listening experience.

Well, through the years there have been loads of techniques to "enhance" existing recordings. Atmos goes a bit further and actually put an engineer to work on changing the original recording.

At the time of recording, Beatles had access to 4.0 LP tech, "quadraphonic sound", yet decided not to use that and go for standard 2-channel stereo, even though 4.0 LPs were compatible with 2-channel LP players. I bet they made that choice for a reason.

All this "sound tech" is great for movies and stuff, but there is no way they can make me interested in a "fake" sound stage for music.

And, if we're here for flexing, I've got 2 listening rooms: one with Martin Logans, and one with VonSchweikert 4.5 speakers depending on what to listen to. Yes, big houses are cheap here

 

[sully54]

Well, spatial audio is creating sound as a 3d space around you. In order for audio processing to advance to the point where it eventually makes virtual interactions feel real, the sound will need to change slightly depending on where you are looking, just as if you were in the same room as the other conference participants (or musicians, etc). Whether it's a net positive or not for passive music listening is yet to be seen (although my thoughts, from watching a lot of the music videos that have had spatial audio with tracking for some time now is that it makes sense as it helps trick your brain into thinking the content is "live".)

I understand all that and the concept of dynamic head tracking. And yeah, it makes sense in virtual interactions and I think it will definitely come in handy once AR/VR becomes the norm.

What I dont get is why I need audio from a movie to also dynamically change depending on where I’m looking from. Like say I’m watching a movie on my iPad while washing dishes and I have my iPad on the counter to my right. Why does it make sense to have most of the audio now skewed to the right headphone just because I’m not dead centre in relation to the iPad?

 

[zhenya]

I understand all that and the concept of dynamic head tracking. And yeah, it makes sense in virtual interactions and I think it will definitely come in handy once AR/VR becomes the norm.

What I dont get is why I need audio from a movie to also dynamically change depending on where I’m looking from. Like say I’m watching a movie on my iPad while washing dishes and I have my iPad on the counter to my right. Why does it make sense to have most of the audio now skewed to the right headphone just because I’m not dead centre in relation to the iPad?

Beyond the first few seconds, it won’t. It will slowly pan to center on wherever you happen to be looking for more than ~5 seconds.

 

[dannys1]

No, stereo headphones are not magic, but a Dolby Atmos recording decoded with HTRF into stereo headphones is about as close as it gets in the audio world. There are demos freely available, I suggest you try some out then get back to me for my bank details

The thing is there's two things headphones have never been able to do, which is why mix engineers don't mix on them. You can't hear stuff as if it's "in front" or "behind you". Dolby Atmos, if anything, just makes it sounds like there's a small 360 degree circumferance around each ear. It doesn't' sound like 360 around the head.

You'll notice vocals in headphones sound like they're coming from inside your head. Anything in the center channel or mono field sounds like this - there's depth left and right but none front to back. So you can't have someone sound like they're singing in front of your or behind you - and again, no amount of pseudo audio processing can change that.

 

[dannys1]

If you properly model your specific ears, and use that to derive an appropriate HRTF, you can make the sound "come from behind you". This is how binaural audio works.

Your ears are effectively two point receivers, which is insufficient to localize a sound by time separation alone, so the only way that you as a listener can tell a sound is coming from twenty feet directly in front of you, or twenty feet directly behind you (or indeed twenty feet directly overhead) is by the distortion that your head and ears and body introduce to the incoming sound. If you've accurately modelled this transformation, you can apply it to any sound to convince the brain the sound is coming from somewhere other than two small speakers strapped to your melon (or better yet, inserted was down into your ears).

It doesn't though, as said above

The thing is there's two things headphones have never been able to do, which is why mix engineers don't mix on them. You can't hear stuff as if it's "in front" or "behind you". Dolby Atmos, if anything, just makes it sounds like there's a small 360 degree circumferance around each ear. It doesn't' sound like 360 around the head.

You'll notice vocals in headphones sound like they're coming from inside your head. Anything in the center channel or mono field sounds like this - there's depth left and right but none front to back. So you can't have someone sound like they're singing in front of your or behind you - and again, no amount of pseudo audio processing can change that.

 

[dannys1]

You are wrong. You don’t understand at all how audio waves move in air and how humans process audio.

listen to this and stop the nonsense:

Dolby Atmos - Official Site

Take your music and entertainment to the next level with Dolby Atmos, a revolutionary spatial audio technology for the most immersive sound experience.

www.dolby.com

Believe me, I understand better than you - and years of a mix engineer I understand what my ears actually hear. You pretend you can hear stuff in front or behind you rather than it just being wider.

 

[dannys1]

You don’t really understand how surround sound or Dolby Atmos works. You do understand you have two “hearing holes” in your head and those holes allow you to pinpoint audio in 3D space? it’s not magic…

listen this with headphones and witness how audio comes from behind you… still not magic…

Dolby Atmos - Official Site

Take your music and entertainment to the next level with Dolby Atmos, a revolutionary spatial audio technology for the most immersive sound experience.

www.dolby.com

Absolutely none of this demo sounds like it's coming from behind you in headphones - I mean maybe it's tricking your brain better than mine, but i'm not getting any audio trick from it - it just moves from the left speaker to the right.

Here it's best if I illustrate how it sounds to me - it sounds wider with more depth than pure panning stereo left or right, but I get no perception of sound coming above or below me (at all) and it sounds very narrow in front or behind - as in this diagram the sound moves around my head like this, probably even less from behind than that and more into the actual head again.

 

[dannys1]

Sure they can. When you listen to music in a club or concert hall, the entire space impacts your perception of the sound - even though the source is entirely in front of you. The problem is that reproducing this would require a perfect reproduction of the recording space for you to listen in.

Binaural audio has been able to do what Spatial Audio attempts for decades. Put on a pair of headphones and tell me this video doesn't make it sound like things are behind you.

Yeah again it doesn't for me, it sounds like i've drawn here - perhaps even less "behind" than that - more into the head when it goes behind. There's also not height above or below illusion.

All of this I can get from atmos (to a degree) with in ceiling and speakers behind and to the side. Unfortunately there's no floor option to make things sound like they're below.

 

[Kier-XF]

Perhaps you have an oddly shaped head.

 

[dannys1]

People who write these articles clearly haven't even considered whether what they're writing is true. Spatial Audio with videos IS NOT ANCHORED TO YOUR DEVICE. All you need is a kindergarten level of curiosity to devise a simple experiment: turn on a video on the iPhone, keep your head still, and move the iPhone around your head. You won't hear any difference.

Now keep the iPhone in place and just turn your entire body away from it, and keep still for 15 seconds or so. You'll notice that Spatial Audio will automatically re-center itself to your head's new position.

In other words, Spatial Audio knows nothing about the position of your phone relative to your head. It simply "anchors" the 3D audio to the average position of your head over some short period of time.

This also means that enabling this for music would be completely trivial. However, in its current form that wouldn't make sense either: you don't really listen to music in your headphones while sitting still. You walk around, and it would be weird to have your virtual music stage suddenly shift every 10 seconds. Personally I don't see how Head Tracking can make sense with music at all, but perhaps they're planning to come up with something.

I didn't even realised it worked like this (as I spent about 10 minutes messing with Apple TV programs in Dolby Atmos to see and wasn't that fussed about it) but you're right. Nothing to do with the screen, just centres to the middle of your head - it obviously "feels" like the screen when you're watching the footage directly on it, quite good for centre channel information.

Even head tracking in movies is basically useless to me - it's mildly impressive panning around the vocal track as you turn your head, but...not really useful - and much less so in music.

 

[applesque]

You can run it on 14.6

is that just spatial audio, or lossless as well?

 

[Kier-XF]

is that just spatial audio, or lossless as well?

Both

 

[applesque]

I agree. I wish you could turn off spatial audio in the homepods. Seems to playback the Atmos version by default if available.

Would there only be a supposed advantage to listening to spatial audio for music on HomePods with more than one speaker, or even one speaker would sound better. I only have one HomePod mini.

 

[applesque]

Both

And obviously this only works on a Mac or something wired? Because lossless doesn't work over bluetooth.

 

[dannys1]

Perhaps you have an oddly shaped head.

Haha, even if I did, in-ears should sort that out!

Funnily enough someone posted the binaural video above and I heard a bit more "behind" effect, the problem then is the front affect was totally lost - cars going past in front of the view sounded like they were slightly behind.

Ultimately i'm not sure how good it is for music to be processed in binaural audio. There's a weird synthetic edge to some of these mixes regardless of how wide/deep or behind the pseudo audio effect creates. I'm also not sure these are being mastered - so you are getting an increase in dynamic range but also a loss in volume - but for me the glue that "holds" the stereo master together is one of the nice full things about it. It doesn't appear these are being run through compression and limiting as obviously there is no 2-buss anymore.

It's certainly a very different take on mixing and mix engineers will get better at it - some of these early attempts are pretty woeful.

 

[xyz01]

I bet I have the same CD, but I have no idea how spatial audio would benefit a recording made in 1959.
Sure, multichannel audio is fun, but by gosh it is tiresome to listen to audio engineers go wild on the controls.
No thanks, I'll stick to listening to that famous recording on my Martin Logans in glorious 2-channel.

Well, in this case they've not gone overboard... It's just a subtle change so that it feels more like a band on a stage, it's easier to pinpoint where the individual instruments are. I don't think they use the rear speakers at all, just using the centre speaker to augment the sound stage together with L+R.

Plus a conscious decision on how to use the .1, rather than whatever passover that is set by default.

 

[ErikGrim]

Currently listening to When Doves Cry (2015 Paisley Park Remix) by Prince & the Revolution on AirPods Pro, from iOS 15.

I have 'Spatial Audio' turned on and obviously 'Spatialise Stereo' is not available and I am damn near certain there's head tracking happening. The vocals in the first verse (00:29 to 01:00) are clearly moving from centre, to left and right as I turn my head. It's interesting because it's only the vocals that are moving, and not the entire mix.

You are absolutely correct. Can confirm this is happening.

EDIT: This is now the case for all spatial audio, so it must have been turned on server side as this was not the case yesterday. Abbey Road is an absolute revelation listening like this.

 

[ErikGrim]

No all the mix will stay in a stereo mix in both ears - the centre channel (which be just the lead vocal in almost every case) will stay anchored to the phone so when you turn your head the music moves with you the singer stays as if they're coming out of the phone.

Still meh, but that's how it works.

No, the singer stays right in front of you. Not anchored to your phone. As you're not watching anything on your phone, this (and the premise of the article) makes no sense.

 

[ErikGrim]

The thing is there's two things headphones have never been able to do, which is why mix engineers don't mix on them. You can't hear stuff as if it's "in front" or "behind you". Dolby Atmos, if anything, just makes it sounds like there's a small 360 degree circumferance around each ear. It doesn't' sound like 360 around the head.

It absolutely sounds like 360º around the head.

 

[camsoft]

Can't say i'm a big fan of the spatial audio mixes so far.

The drums lose all punch and weight - they sound pushed back in the mix on the drum heavy things i've tried out - the vocals are too much of the focus for me and everything is swimming in too much reverb. It's like giving someone a new tool for the first time and they go a bit nuts with it.

The other irony of course is that whilst it's mixed in a multi-channel setup we're only hearing it in stereo anyway - you can't defy physics, Airpods have two speakers one on the left and one on the right - you're not hearing Dolby Atmos - you're hearing Dolby Atmos mixed down to Stereo.

It's an interesting concept but generally I think i'll have it turned off. I certainly wouldn't want head tracking enabled for the centre channel either - so if they ever do mixes I prefer hopefully there's an option to enable or disable that.

You do realise our ears are stereo? Doesn’t matter how many speakers we have it’s all mixed down to stereo either artificially by software like this or just by the airwaves in a multiple speaker setup.

 

[dannys1]

No, the singer stays right in front of you. Not anchored to your phone. As you're not watching anything on your phone, this (and the premise of the article) makes no sense.

Yes I realise that now i've done some more testing - still a useless premise as you walk around.

Further more a lot of these Dolby mixes lack mastering and mastering chain compression as it's not possible - they lose the glue that holds the track together. It really doesn't work well with some music styles.

As for the full 360 thing I still disagree. It just doesn't trick my ears like it apparently does yours and a few other peoples - my brain distinctively just hears the sound swap from the left to the right speaker like it's hard panned with a touch of out of phase stereo widening.