Image Image Image Image Image
Scroll to Top

To Top

Software

14

Dec
2013

5 Comments

In Software
Sound Design
Work

By Varun Nair

Unity: Controlling game elements with sound

On 14, Dec 2013 | 5 Comments | In Software, Sound Design, Work | By Varun Nair

I’ve been playing around with Unity 3D in my spare time. Verdict? Lots of fun! Thankfully I’ve found it easy to understand because of the many years I spent as a teenager watching my brother work in 3D Studio Max and Maya.

The audio side of Unity is relatively easy (and therefore limited) if you have previous experience in game audio. Getting both the visuals and audio to work is straightforward if you have any experience in object oriented programming. Thankfully C# (I’m no good at Java Script) is similar to Java and C++, both of which I have been getting familiar with over the past year.

With game audio we often come across a one sided process: the game engine feeds the audio engine data and the audio engine outputs sound. It isn’t very often that we see the opposite happening. In Unity (Pro ONLY), from version 3.5 onwards this was made easier with the OnAudioFilterRead callback. OnAudioFilterRead is meant for creating custom filters, but can very well be used to control other game elements. If you don’t have Unity Pro, it is worth downloading the trial and giving it a go.

This post is a quick and simple recipe to control the intensity of a light with sound, but the principles can very easily be expanded to anything else in game.

Unity

Step1: Setup a scene in Unity
Step2: Attach a sound and light source to an object. Attach a sound to the audio source component.
Step3: Create a new script for this object
Step4: Use a smoothening filter to analyse the amplitude of the signal
Step5: Map the amplitude value to the intensity of the light Step6: TA-DA!

Step1

(If you are familiar with Unity you can skip to Step3)

Create a new Unity project. Create a cube object to use as the floor of the scene (Game Object > Create Other > Cube). With the cube selected, use the inspector (the tab on the right, by default) to scale the dimensions of the cube. X: 30 Y: 0.1  Z: 30

Create a sphere (Game Object > Create Other > Sphere). Change its Y position value to 2 in the inspector.

Step2

In the Project tab (at the bottom of the screen by default), right click on “Assets” and choose Create > Folder. Name this new folder as “Audio”. Drag the audio file you want to use from the Finder (or Explorer) into this folder in Unity. MAKE SURE you always import assets through Unity or it will make a fuss.

Scroll down the inspector pane (with the sphere still selected) and click on “Add Component”. Choose Rendering > Light. Alter the range and colour of the light to your liking.

With the sphere still selected, click on “Add Component” again. Choose Audio > Audio Source. Click on the little dot next to “Audio Clip” under the “Audio Source” properties and choose your file.

Step3

Make sure the sphere is selected and click on “Add Component” in the inspector pane again and click on “New Script”. Name it whatever you want and make sure the language is set to CSharp.

Step4

Now the fun begins. Add the following method within the class in the script:

22
23
24
25
26
27
void OnAudioFilterRead (float[] data, int channels)
	{		
		for (var i = 0; i < data.Length; i = i + channels) {
			// DSP Magic happens here
		}
	}

OnAudioFilterRead is called every time a chunk of audio is routed to your hardware interface. The float array “data” is the audio data that is passed into this function. To put it crudely, manipulating “data” will result in a change in the sound. This is not what we are interested in doing here. We want to analyse the contents of “data”, determine the amplitude and use that value elsewhere. “data” is a float array (in this case, a serial collection of  numbers), where each element of the array corresponds to a sample and it’s value. For a signal at 0dBFS, these numbers would range from 1 to -1. To determine the peak amplitude of the signal, we can convert all the negative numbers to positive numbers using a programming function called absolute.

22
23
24
25
26
27
28
void OnAudioFilterRead (float[] data, int channels)
	{		
		for (var i = 0; i < data.Length; i = i + channels) {
			// the absolute value of every sample
			float absInput = Mathf.Abs(data[i]);
		}
	}

If you directly map these absolute values to a light, you’ll find that it flickers too much, too quickly. Our ears don’t perceive such fine changes. To make these changes gradual, we can use a smoothening filter and assign the output value to a global variable which can then affect the light. If you look at the code below, you will notice that I’ve multiplied the value by 7 to exaggerate the effect.

22
23
24
25
26
27
28
29
30
31
32
33
34
 void OnAudioFilterRead (float[] data, int channels)
	{		
		for (var i = 0; i < data.Length; i = i + channels) {
			// the absolute value of every sample
			float absInput = Mathf.Abs(data[i]);
			// smoothening filter doing its thing
			smooth[0] = ((0.01f * absInput) + (0.99f * smooth[1]));
			// exaggerating the amplitude
			amp = smooth[0]*7;
			// it is a recursive filter, so it is doing its recursive thing
			smooth[1] = smooth[0];
		}
	}

Step5 & 6:

Here’s the end product (you need the Unity web plugin), followed by the full script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
using UnityEngine;
using System.Collections;
 
public class VolDisplay : MonoBehaviour {
 
	private float amp;
	private float[] smooth = new float[2];
 
	void Start () {
		// initalising the filter
		for (int i = 0; i < 2; i++) {
			smooth [i] = 0.0f;	
		}
	}
 
	// Update is called once per frame
	void Update () {
		//intensity of light, controlled by the amplitude of the sound
		light.intensity = amp;
	}
 
	void OnAudioFilterRead (float[] data, int channels)
	{		
		for (var i = 0; i < data.Length; i = i + channels) {
			// the absolute value of every sample
			float absInput = Mathf.Abs(data[i]);
			// smoothening filter doing its thing
			smooth[0] = ((0.01f * absInput) + (0.99f * smooth[1]));
			// exaggerating the amplitude
			amp = smooth[0]*7;
			// it is a recursive filter, so it is doing its recursive thing
			smooth[1] = smooth[0];
		}
	}
}

P.S: Thanks to @lostlab for inspiring this post.

Tags | , ,

Comments

  1. Hi,
    Thanks for share,

    Your code return amplitude,how can i get decible?

    thanks.

    • Varun Nair

      dB = 20 * log(linearAmplitude)

      So,

      float db = 20 * Mathf.Log(amp);

      If you calculate the decibel value after the smoothening filter, it won’t be as accurate as a peak meter since the smoothening filter has been written specifically for controlling the light. You would get a more accurate representation right after the absolute calculation, which will give you the peak level of the signal in dBFS.

  2. So thanks for reply,

    Your mean is this?

    float absInput = Mathf.Abs(data[i]);
    float db = 20 * Mathf.Log(absInput);

    right?

    • Varun Nair

      Yep, that would be the simplest way to do it

  3. Hi again,

    I got that formula is not right to get audio decibel.

    I found a code in java (android) that get decibel from microphone voice and tested by me.i think this work true.but after porting to unity,result is not true.if you can please help me to make it work.

    //java code
    static mRmsSmoothed = 0;
    static mAlpha = 0.9;
    static mGain = 2500.0 / Math.pow(10.0, 90.0 / 20.0);

    static double calcDb()
    {
    double rms = 0;

    for (int i = 0; i < buffer20ms.length; i++)
    rms += buffer20ms[i] * buffer20ms[i];

    rms = Math.sqrt(rms / buffer20ms.length);

    // Compute a smoothed version for less flickering of the display.
    mRmsSmoothed = mRmsSmoothed * mAlpha + (1 – mAlpha) * rms;
    double rmsdB = 20.0 * Math.log10(mGain * mRmsSmoothed);
    return (rmsdB + 20);
    }

    //unity code
    private double mAlpha = 0.9; // Coefficient of IIR smoothing filter for RMS.
    private double mRmsSmoothed; // Temporally filtered version of RMS.
    private double mGain = 2500.0/Mathf.Pow(10.0f, 90.0f/20.0f);
    public float[] data;

    private void ProccessSound()
    {
    double rms = 0;
    //data = new float[clip.samples*clip.channels];
    data = new float[clip.samples / 50];

    clip.GetData(data, 0);
    for (int i = 0; i < data.Length; i++)
    {
    rms += data[i]*data[i];
    }
    rms = Mathf.Sqrt((float) (rms/data.Length));

    mRmsSmoothed = mRmsSmoothed*mAlpha + (1 – mAlpha)*rms;
    double rmsdB = 20.0*Mathf.Log10((float) (mGain*mRmsSmoothed));
    db = rmsdB;
    print("rmsdB : " + rmsdB);
    }

Submit a Comment