macOS Audio questions

[fernandovalente]

Hey,

I'm playing with some audio stuff. There are 3 things I'd like to create:

-A visualizer like the iTunes ones.
-An audio editor like GarageBand's crop bar
-A bar that will react to the audio beats. Something like this one:

Does anyone knows how I can create stuff like these ones?

Thanks!

 

[larswik]

Do you have ANY programming experience?

-Lars

 

[fernandovalente]

Do you have ANY programming experience?

-Lars

Yeah, I'm great at Objective-C, but I know almost nothing about audio

 

[SidBala]

Do you have ANY signal processing experience?

There is a lot of discrete signal processing(FFTs, convolutions, correlations, etc) done under the hood for you to get those cool looking visualizations. I would only recommend you try this if you know what the above terms mean.

That spectrum analyzer bar in the screenshot is basically a FFT that has been plotted as a bar chart.

Beat detection, you detect changes in loudness. Where this exceeds a threshold, you mark that as a beat and update your visual accordingly. More complex algorithms require you to keep a running history of the music and detect the point where there is a repeat of the beat.

The itunes visualization is a bit more complicated. it usually blends a bunch of different effects together to create the final visualization. This is a bit harder to do.

All this is doable. But you need to have a solid understanding of signal processing. A lot of math is involved. It is not something that you can just use common sense to do.

 

[fernandovalente]

Do you have ANY signal processing experience?

There is a lot of discrete signal processing(FFTs, convolutions, correlations, etc) done under the hood for you to get those cool looking visualizations.

That spectrum analyzer bar in the screenshot is basically a FFT that has been plotted as a bar chart.

Beat detection, you detect changes in loudness. Where this exceeds a threshold, you mark that as a beat and update your visual accordingly. More complex algorithms require you to keep a running history of the music and detect the point where there is a repeat of the beat.

The itunes visualization is a bit more complicated. it usually blends a bunch of different effects together to create the final visualization. This is a bit harder to do.

All this is fairly doable. But you need to have a good solid understanding of signal processing.

I don't have any experience with signal processing. Is there any high-level lib to do that(one that would return the float values of the beats)?

 

[ender land]

That spectrum analyzer bar in the screenshot is basically a FFT that has been plotted as a bar chart.

huh, I never realized this - good point

 

[mfram]

Google for "FFT C library". You'll get a whole bunch of hits. Choose a library and go for it!

 

[firewood]

I don't have any experience with signal processing. Is there any high-level lib to do that(one that would return the float values of the beats)?

You need to learn enough signal processing in order to use more descriptive terminology than "float value of the beats". Most spectrum analyzers do nothing of the sort. Using an FFT to do spectral analysis without understanding some signal processing can get you into all sorts of trouble (e.g. a spectrum analyzer that produces garbage results, or unattractive results).

The mid-level library built into Mac OS X is the Accelerate framework.

 

[mfram]

I was looking at the documentation yesterday and noticed that Apple has already done a lot of work for you if you want to write an iTunes Visualizer plug-in. The doc is here.

If you can figure out how to get it working, then iTunes gives you everything you need. When you get the kVisualPluginRenderMessage message, it gives you the audio sample data and even the spectrum data because it has already done the FFT for you. You just have to do something interesting with the data!