Immersive applications often take advantage of sound to enhance suspension of disbelief. VR Juggler application programmers have many options available to them for adding sound to their virtual environments. In this section, we describe the library Sonix that ships with VR Juggler. Sonix provides a layer of abstraction for third-party audio libraries including OpenAL, Audiere, and AudioWorks. In VR Juggler 2.0, there is one Sound Manager implemented using Sonix. As such, Sonix is the easiest tool to use for adding sound to VR Juggler applications, but it is not the only option.
Sonix provides simple audio sound objects on top of several audio APIs. The interface to Sonix is kept very simple in order to get people up and running with sound as fast as possible. Sonix has the following features:
Simple access to spatialized sound triggering.
Small learning curve withsimple interface and usage.
Abstracts several well-known audio systems to provide enhanced applicatin portability.
Supports reconfigurability at runtime.
Changing a sound resource reflects properly in all other handles to the same resource (resources allow multiple users).
Reconfigurations of sound resources are protected (i.e. reconfiguration does not break application).
Supports features needed by 3D virtual environments including spatialized audio, ambient audio, one-shot sounds, and looping sounds.
VR Juggler application programmers can use the full Sonix
API directly, or they can take advantage of the VR Juggler Sound
Manager to reduce the amount of coding required. We will present
both approaches. Example applications can be found in the
directory
$VJ_BASE_DIR/share/vrjuggler/samples/sound/simple.
We begin our explanation of adding sound to a VR Juggler application by presenting the direct use of the Sonix API. This requires more programming than using the VR Juggler Sound Manager, but it will be useful to understand how Sonix operates in order to take full advantage of the Sound Manager. In particular, reading this section is a necessary part of understanding general Sonix usage, and readers are encouraged to read both this section and the next before deciding on which approach to use in their VR Juggler applications.
In Figure 6.1, “Basic Sonix Interface”, we see
the API for Sonix (see Figure 6.2, “The Sonix Design” for the complete
software architecture). The main parts that a sound
programmer will use are the classes
snx::SoundHandle and
snx::sonix (a singleton). The
snx::SoundHandle class is used to
manipulate individual sounds. The
snx::sonix singleton class is used to
start, stop, and reconfigure the sound system. Both classes
must be used for any sound to be heard.
Starting the Sonix system is easy. Basically, the only
call needed to start the system is
snx::sonix::changeAPI(). The
backend audio software is loaded into Sonix through a
plug-in system. The plug-in to use is identified by a string
name, one of “OpenAL”,
“AudioWorks”, or “Audiere”, as
shown in Example 6.12, “Initializing the Sonix Sound APi”. The choice
of which to use depends on availability and compatibility.
For example, AudioWorks is available on the IRIX operating
system, but it can only be used with a build of VR Juggler
that uses SPROC threads. On the other hand, OpenAL and
Audiere on IRIX can only be used with a pthreads build of VR
Juggler.
Setting up a sound is designed to be simple, and an
example of doing so is shown in Example 6.13, “Setting Up a Sonix Sound Handle”. Here, we use an
instance of snx::SoundInfo to
configure the sound object, which is in turn accessed by an
instance of snx::SoundHandle.
Example 6.13. Setting Up a Sonix Sound Handle
snx::SoundInfo info;
info.filename = "crack.wav";
info.datasource = snx::SoundInfo::FILESYTEM;
snx::SoundHandle crack_sound("crack");
crack_sound.configure(info);Periodically, an update function must be called to
keep Sonix running. The method is snx::sonix::step(), and it
takes a single argument of type float
representing the time since the last time it was called. In
general, this should be invoked in the frame function of an
application. In the context of VR Juggler, this would be
either preFrame() or
postFrame() in the application
object interface. An example of using this function is shown
in Example 6.14, “Sonix Frame Update”.
Example 6.14. Sonix Frame Update
void MyApp::preFrame()
{
// Update application state for the current frame
// ...
float time_delta = getTimeChangeInSeconds(); // use a system call or other API
// to get the time delta
snx::sonix::instance()->step(time_delta);
}Example 6.15. Complete Sonix Program Using OpenAL
1 #include <iostream>
#include <string>
#include <snx/sonix.h>
5 int main(int argc, char* argv[])
{
std::string filename(:808kick.wav"), api("OpenAL");
if ( ! snx::FileIO::fileExists(filename.c_str()) )
10 {
std::cout << "File not found: " << filename << "\n" << std::flush;
return 1;
}
15 // start sonix using OpenAL
snx::sonix::instance()->changeAPI(api);
// fill out a description for the sound we want to play
snx::SoundInfo sound_info;
20 sound_info.filename = filename;
sound_info.datasource = snx::SoundInfo::FILESYSTEM;
// create the sound object
snx::SoundHandle sound_handle;
25 sound_handle.init("my simple sound");
sound_handle.configure(sound_info);
// trigger the sound
sound_handle.trigger();
30 sleep(1);
// trigger the sound from a different position in 3D space
sound_handle.setPosition(10.0f, 0.0f, 0.0f);
sound_handle.trigger();
35 sleep(1);
// this simulates a running application
while ( 1 )
{
40 snx::sonix::instance()->step(time_delta);
}
return 0;
}Sonix is reconfigurable, allowing audio APIs to be swapped out at run time safely without the dependent systems noticing. Applications using Sonix expect to be completely portable. Changing sound APIs at run time can be useful so that the user can experiment with quality and latency differences of different hardware and sound APIs. If no audio API is available on a given platform, calls to Sonix simply are ignored. This means that applications do not have to have special code to enable or disable sounds based on the availability of the backend sound software. These details are handled transparently by Sonix.
Everything in Sonix can be reconfigured behind the
scenes during application execution. See Example 6.16, “Reconfiguring Sonix at Run Time” for a trivial example
of how to reconfigure Sonix in C++. In a more complex
context, this feature of Sonix has powerful implications.
Sounds can be changed on the fly, as well as the API being
used to render the sound. Such details are hidden behind
sound handles and the snx::sonix
singleton.
Example 6.16. Reconfiguring Sonix at Run Time
1 // start sonix using OpenAL
snx::sonix::instance()->changeAPI("OpenAL");
// fill out a description for the sound we want to play
5 snx::SoundInfo sound_info;
sound_info.filename = "808kick.wav";
sound_info.datasource = snx::SoundInfo::FILESYSTEM;
// create the sound object
10 snx::SoundHandle sound_handle;
sound_handle.init("my sound for testing");
sound_handle.configure(sound_info);
// trigger the sound
15 sound_handle.trigger();
sleep(1);
// trigger the sound using a different audio system
snx::sonix::instance()->changeAPI("AudioWorks");
20 sound_handle.trigger();
sleep(1);
// trigger our sound object using different source data
sound_info.filename = "303riff.wav";
25 sound_handle.configure(sound_info);
sound_handle.trigger();
sleep(1);
Before we conclude this section on using Sonix directly in applications, we examine the design details of Sonix. These details are beyond the scope of what needs to be understood in order for Sonix to be utilized by VR Juggler applications, so readers not interested in these details can skip ahead to the section called “Using the VR Juggler Sound Manager”.
Sonix was designed using modern software design principles including design patterns and object oriented design. The complete design of Sonix is depicted in UML in Figure 6.2, “The Sonix Design”. As evidenced by the diagram, the design of Sonix is quite simple, though the classes tend to have relatively large interfaces. Refer to the Sonix Programmer Reference, found on the Sonix documentaiton web page, for complete documentation of these interfaces.
Design patterns describe simple and elegant solutions to specific problems in object oriented software design. When designing Sonix, we used many design patterns appropriate for a simple audio system.
Adapter
(snx::SoundImplementation).
This adapter provides a common interface to the
underlying sound API.
Prototype
(snx::SoundImplementation).
Making
snx::SoundImplementation a
Prototype allows a new cloned object to be created
from it that has duplicate state.
Store/plugin-method
(snx::SoundFactory). Each
sound implementation is registered with a Store
called snx::SoundFactory.
This Store allows users to select items from its
inventory. Another name for Store is “Abstract
Factory.”
Abstract Factory
(snx::SoundFactory). The
Store can create new instances of the requested
sound implementation. The Abstract Factory consults
its Store of registered objects, and if found, makes
a clone of that object (Prototype pattern). The
Abstract Factory is used in Sonix to configure the
Bridge.
Bridge (snx::sonix
interface class and
snx::SoundImplementation).
The snx::sonix class is the
audio system abstraction which is decoupled from its
implementation
snx::SoundImplementation.
This way the two can vary independently. Bridge also
facilitates run-time configuration of the sound
API.
Proxy (std::string and
snx::SoundHandle).
snx::SoundHandle is how users
manipulate their sound object.
snx::SoundHandle is actually
a proxy to a std::string
proxy. The std::string Proxy
is what allows Sonix reconfiguration of resources.
Rather than using pointers which can easily be left
to dangle, the std::string
serves as a lookup for a protected sound resource
located internally to the Sonix run-time memory
space. The snx::SoundHandle
wraps this std::string to
provide a simple and familiar C++ object to use as
the sound handle. The Sonix class acts as Mediator
between every Proxy method and the actual audio
system Adapter.
Sonix supports the selection of several audio subsystems by the application through implementation plug-ins (see Figure 6.3, “Use of Plug-ins in Sonix”). Each plug-in implements an adapter to an underlying audio subsystem. The adapter supports a common interface that Sonix knows how talk to. Each adapter is then registered with a factory object, which may ask that adapter to clone itself for use by whomever called the factory.
Now that we understand how Sonix works and how to use its simple API, we can go one step further and simplify the usage even more. Similar to the graphics API-specific VR Juggler Draw Managers, there exists the concept of a Sound Manager. There is a Sonix-specific Sound Manager implementation that handles many of the details of Sonix usage.
The Sonix Sound Manager is configured using a config
element of type sound_manager_sonix. The
config element sets the sound API to use, defines the llistener
position, and sets up all the sound objects that will be used
by the application. All of this would normally have to be done
by the application programmer using the
snx::sonix singleton, as presented above
in the section called “Basic Sonix Usage”. In Example 6.17, “Example Sonix Sound Manager Conifguration”, we show a
small example of configuring the Sonix Sound Manager. Note that
some parts of the config file are removed for brevity. Aspects
of this will be referenced in the code examples shown
below.
Example 6.17. Example Sonix Sound Manager Conifguration
<sound_manager_sonix version="1">
<api>OpenAL</api>
<listener_position>0.0</listener_position>
<listener_position>0.0</listener_position>
<listener_position>0.0</listener_position>
<sound>
<sound name="bump" version="1">
<filename>${VJ_BASE_DIR}/share/vrjuggler/data/sounds/bump.wav</filename>
<ambient>false</ambient>
<retriggerable>false</retriggerable>
<loop>1</loop>
<pitch_bend>1.0</pitch_bend>
<cutoff>1.0</cutoff>
<volume>1.0</volume>
<position>0.0</position>
<position>0.0</position>
<position>0.0</position>
</sound>
</sound>
<sound>
<sound name="step" version="1">
<filename>${VJ_BASE_DIR}/share/vrjuggler/data/sounds/footstep.wav</filename>
<ambient>false</ambient>
<retriggerable>false</retriggerable>
<loop>1</loop>
<pitch_bend>1.0</pitch_bend>
<cutoff>1.0</cutoff>
<volume>1.0</volume>
<position>0.0</position>
<position>0.0</position>
<position>0.0</position>
</sound>
</sound>
<!-- Other sound objects ... -->
</sound_manager_sonix>All sound files named in the config element are
automatically loaded by the Sonix Sound Manager. User
applications simply declare sound handles that refer to the
loaded sound objects. The VR Juggler application class will
then include one snx::SoundHandle
instance for each sound object declared in the Sonix Sound
Manager config element, as shown in Example 6.18, “Declaring Sound Handles in Application Object
Class”. This is very
similar to the use of Gadgeteer device interfaces that provide
access to device input.
Example 6.18. Declaring Sound Handles in Application Object Class
class MySoundApp : public vrj::GlApp
{
public:
void init();
void preFrame();
// Other application object interface methods ...
private:
// Determines if the "bump" sound should be triggered.
bool shouldTriggerBump();
// Determines if the "step" sound should be triggered.
bool shouldTriggerStep();
snx::SoundHandle mBumpSound;
snx::SoundHandle mStepSound;
// And so on ...
};To get access to the sounds through the
snx::SoundHandle objects, the sound
handles must be initialized. This is done in the same way as
described earlier in the explanation of direct use of the Sonix
API. The sound handle initialization should be performed in the
init() method of the VR Juggler
application object, as shown in Example 6.19, “Initializing Sound Handles in an Application
Object”. The string value
passed to snx::SoundHandle::init() is
the name given in the Sonix Sound Manager config element.
Again, we see similarity with the usage of Gadgeteer device
interfaces.
Example 6.19. Initializing Sound Handles in an Application Object
void MySoundApp::init()
{
mBumpSound.init("bump");
mStepSound.init("step");
// And so on ...
}Finally, the sounds can be triggered in the application
object preFrame() method, shown below
in Example 6.20, “Triggering Sounds in an Application Object”. They should
not be triggered in the draw() method
because, as we have seen in earlier sections,
draw() can be invoked multiple times
per frame. There is no need to call the function
snx::sonix::step() in
preFrame() or in
postFrame(). This is done
automatically within the Sonix Sound Manager each time through
the kernel control loop.
Example 6.20. Triggering Sounds in an Application Object
void MySoundApp::preFrame()
{
if ( shouldTriggerBump() )
{
mBumpSound.trigger();
}
if ( shouldTriggerStep() )
{
mStepSound.trigger();
}
}Other methods in the
snx::SoundHandle class interface can be
invoked on the sound handle objects. We have focused on simple
rendering of sounds here. Refer to the Sonix
Programmer Reference, found on the Sonix
documentation web page, for complete documentation on
the interface of
snx::SoundHandle.