I've been trying to implement a real-time visualization (spectrogram) for my audio player app which is written in Swift 3 and uses the AVAudioEngine framework. But, it suffers from a serious time lag problem - roughly half a second, which makes it infeasible (real-time visualization needs to be ... umm ... real-time). I'm trying to understand why this is happening. I elaborate below.
Now, in general, the way to capture any audio data from an audio processing graph is to install a tap on a node in the chain to receive the data as it is sent through the processing chain (and specifically, that node), and then analyze it as it is received, hopefully in real time.
And, this is exactly the way I've implemented my spectrogram visualization. I've installed a tap on the AVAudioEngine's main mixer node, reduced the buffer size to increase the frequency at which data is received, and am performing FFT analysis using the Accelerate framework's vDSP functions. But, when the spectrogram is rendered, I notice a significant lag (~ 0.5 sec), which makes it infeasible for any end user.
P.S. I thoroughly checked (and timed) my code to ensure that my code was not the culprit. My FFT computations and graphics rendering were performed speedily enough not to cause issues.
Is AVAudioEngine the problem ?
I've read rumors online that AVAudioEngine is slow and not meant to be used for any real-time analysis. But, contradicting that claim is the fact that AudioKit uses AVAudioEngine and they've implemented a spectrogram. I can't say that it works because I've not been able to understand how to install AudioKit.
Is Swift the problem ?
In other places, I've read that no such real-time analysis is realistically possible with Swift, and that real-time analysis is best done with C code.
Here is one thread I read about the issue: https://stackoverflow.com/questions/24383080/realtime-audio-with-avaudioengine
Does anyone know what the deal is ?
Now, in general, the way to capture any audio data from an audio processing graph is to install a tap on a node in the chain to receive the data as it is sent through the processing chain (and specifically, that node), and then analyze it as it is received, hopefully in real time.
And, this is exactly the way I've implemented my spectrogram visualization. I've installed a tap on the AVAudioEngine's main mixer node, reduced the buffer size to increase the frequency at which data is received, and am performing FFT analysis using the Accelerate framework's vDSP functions. But, when the spectrogram is rendered, I notice a significant lag (~ 0.5 sec), which makes it infeasible for any end user.
P.S. I thoroughly checked (and timed) my code to ensure that my code was not the culprit. My FFT computations and graphics rendering were performed speedily enough not to cause issues.
Is AVAudioEngine the problem ?
I've read rumors online that AVAudioEngine is slow and not meant to be used for any real-time analysis. But, contradicting that claim is the fact that AudioKit uses AVAudioEngine and they've implemented a spectrogram. I can't say that it works because I've not been able to understand how to install AudioKit.
Is Swift the problem ?
In other places, I've read that no such real-time analysis is realistically possible with Swift, and that real-time analysis is best done with C code.
Here is one thread I read about the issue: https://stackoverflow.com/questions/24383080/realtime-audio-with-avaudioengine
Does anyone know what the deal is ?