Trinnov Audio - Research

[ Acoustic Field | Digital Acoustics | Solutions | Publications | Links ]

Research

Trinnov Audio allocates extensive research efforts to contribute to the improvement of audio quality. We focus on one of the greatest scientific challenges: spatial audio. Our scientific papers are considered as world-class contributions by many renowned experts and have been relayed in the prestigious JAES (Journal of the Audio Engineer Society).

The result of this continuous R&D effort is a leading position in the emerging scientific area of Digital Acoustics - the processing of acoustic fields in 3D. Acoustic field processing is to spatial audio what signal processing is to discrete signals. It provides understanding and control of spatial audio to bring the performances to the next level.

The Acoustic Field Concept

The future of audio lies in the control of the spatial dimension of sound.

Any sound event presents two aspects:

A time dimension, which allows us to recognize the different sounds (a voice, a music instrument, a plane...) and also their tones.
A spatial dimension, which allows us to localize each sound (the voice is ahead, the plane is high...) and also recognize the place where they are located (a church, a street...).

The continuous progress in terms of computing power and storage capacity offers new perspectives in the field of audio.
However, any further improvement of the temporal aspect is almost useless for the major part of the listeners. For example, the new high-resolution formats (DVD-Audio and Super Audio CD) have today reached the limits of the human's hearing capacities. On the contrary, the spatial performances of today technologies remain limited.

Unfortunately, the developments dedicated to spatial audio aspect are limited by the deficiency in the theoretical foundation of solids, which is the direct opposite to the temporal aspect which is based on the signal theory (signal processing). Consequently, the need of new research to control sound as a whole is emerging.

Any sound event creates wave phenomenon expending in time and in the three space dimensions called acoustic field.
For a better understanding of the power of this model, let's suppose that one could perfectly capture an acoustic field, for example the one produced by an orchestra in a concert hall. In addition, let's suppose that one could reproduce the identical acoustical field in a listening room.
In this case, the audience in the listening room should hear exactly the same reality than the audience in the concert hall, from both a temporal point of view (as the today's techniques make it possible) and a spatial point of view.

While High-Fidelity focused on the accurate reproduction of audio signals, the accurate reproduction of the whole acoustic field opens the way to "High Spatial Fidelity" or "High Spatial Resolution". In order to fully take advantage of this more exact representation, Trinnov Audio engaged an extensive research program to provide cutting-edge solutions to these new challenges.

Fundamental acoustics offers a powerful theoretical tool which allows us to describe the acoustic fields: the Fourier-Bessel transform.
Very specialized and poorly documented, it has remained until now unexploited in audio. Trinnov Audio based his research work on this theoretical tool as well as other sciences, including mathematics and signal processing. As a result, Trinnov Audio has developed a new theory for acoustic fields processing.

The Fourier-Bessel transform decomposes any acoustic field as a superposition of elementary acoustic fields: the Fourier-Bessel functions. Historically, any actual audio process (synthesizers, filters, effects...) have been based on the signal processing theory which use the properties of the Fourier transform. Now the Fourier-Bessel functions are to the acoustic fields what Fourier functions (complex exponentionals) are to the audio signals. Utilizing Fourier-Bessel functions, Trinnov Audio developed a very powerful theory of acoustic field processing allowing infinite possibilities of manipulation.

More information

Three-dimensional sound can be described, from a physical point of view, by an acoustic field, which is defined for each point (x,y,z) in space and for each instant t using the pressure field p(x,y,z,t). Nevertheless, manipulating an acoustic field using its primary representation p(x,y,z,t) is not easy because it would be necessary to know it for each value of (x,y,z,t). Therefore, an acoustic field is decomposed, in spherical coordinates, into its Fourier-Bessel expansion, offering a much convenient and compact representation. From the four dimensional continuous function p(r,,,t), the Fourier-Bessel decomposition gives a set of signals called Fourier-Bessel coefficients of the acoustic field, denoted pl,m(t), where l and m are integers that satisfy l = 0 and -l = m = l. In the Fourier-Bessel formalism, l is called the order. In the frequency domain, P(r,,,f) and Pl,m(f) are the Fourier transforms of P(r,,,f) and pl,m(t) respectively. This decomposition is given by the following expression:

where k = 2_f=c and c is the speed of sound, approximately 340 m/s. The Fourier-Bessel expansion is generally truncated at some order L. This order determines the resolution of the acoustic field representation. The higher the order, the higher the acoustic field representation fidelity will be, but the more computation power and signals will be required.

Fourier-Bessel's functions (space)		Fourier fonctions (time)

They are composed of two parts:

spherical harmonics

and
spherical Bessel functions

Spherical Bessel functions give the radial behavior of Fourier-Bessel functions whereas spherical harmonics give their angular behavior.
The variable l is called order.
The Fourier-Bessel functions are defined for each l > 0 and for each m verifying -l < m < l.

The pressure field corresponding to each elementary Fourier-Bessel function is represented on the following figure, where each row corresponds to a value of l (from 0 to 2) and each column corresponds to a value of m (from -2 to 2).
The indication tells in which plane the field is represented (for example, X,Y indicates the horizontal plane).

Spherical harmonics are already used by Ambisonics, but most of the time only at orders 0 and 1.
Using higher order Ambisonics is indeed difficult because of the lack of good high order directivity microphones.
Spherical harmonics are defined for each l > 0 and for each m verifying -l < m < l.
They are bi-dimensional objects that allow to model the direction of arrival of sound.
Their usual representation is obtained by Spherical Fourier Transform of the Fourier-Bessel coefficients.

The first spherical harmonics are represented on the figure below, where each row corresponds to a value of l
(from 0 to 3) and each column corresponds to a value of m (from -3 to 3).

You can also play with spherical harmonics on this page.

However, considering spherical harmonics alone is not sufficient to manage acoustic fields (record, manipulate or reproduce).
Our research works are addressing the complete Fourier-Bessel representation, including its radial part, and we developed a unique knowledge to manipulate these complex mathematical objects.

The Fourier-Bessel transform of an acoustic field is similar to the Fourier transform of a signal. More precisely, the Fourier transform describe perfectly a signal as a superposition of sinusoids at different frequencies (spectral representation of the signal). In the same way, the Fourier-Bessel transform describes perfectly an acoustic field as a superposition of elementary acoustics fields having different spatial variations (spectral representation of an acoustic field).

Using this theory, it is possible to represent an acoustic field under three equivalent points of view:

A pressure field in space The acoustic pressure value is defined for each point in space ant each instant in time. This representation directly illustrates waves propagation of the acoustic field, creating a "drops-in-water"-like representation

A Fourier-Bessel spectrum The spectrum gives the contributions (or weights) of each Fourier-Bessel function in the construction the acoustic field. This representation is extremely powerful as it represents a continuous acoustic field as a set of coefficient changing with time. In other words it is a digital representation of the acoustic field! Once digitalized, an acoustic field can be controlled by digital processors. Therefore this is mathematical representation is the fundamental tool for pioneers in acoustic field processing.

A directivity function A directivity function can be associated to the Fourier-Bessel spectrum (spherical Fourier transform). This is very meaningful representation as it gives the apparent direction of the sound. The color corresponds to the phase of the directivity, connected to the distance of the sources.

In the standard Fourier theory the "quickness" of variation of a signal is described by the concept of frequency. The highest the frequency, the fastest the signal varies with time. Similarly, the Fourier-Bessel's theory describes the "quickness" of variation of an acoustic field by the "spatial frequency", usually called "order".

The following simulations provide an intuitive insight on how the concepts of frequency, order and spatial precision are connected:

Variation of the frequency	Variation of the order	Moving sound source
At constant order (or representation precision), the spatial zone of perfect representation decreases with increasing frequency while the directivity only varies in phase not in shape.	At constant frequency, both the spatial zone of perfect representation and the accuracy of the directivity increase with the order.	At constant frequency and order, the spatial zone of perfect representation is constant and the directivity pattern represents the position of the source: the main lobe points to the source while the phase represents the distance.

Click here to play video	Clic here to play video	Click here to play video

...top

Digital Acoustics

The power of today's signal processing comes from the manipulation of the Fourier functions (exponential or sine/cosine). Similarly, the power of Digital Acoustics - acoustic field processing - is based on the manipulation of Fourier-Bessel functions. This new theory has led us to develop new technologies allowing to record, manipulate and reproduce acoustic fields.

1) Spatial Pick-up: recording acoustic fields

Actual technologies of "sound capture" consider separately the signals provided by each microphone used. Thanks to a radically innovating processing of these signals, the Spatial Pick-up technology makes it possible to optimally exploit all the available microphones in order to retranscribe all the collected information concerning a sound event. It is sufficient to know the spatial characteristics and position of each microphone in the sound space.

Thus, Spatial Pick-up technology brings an optimal answer to every application in function of the specifics constraints (cost, performances, size, simplicity...). Implementation of this technology led us to three prototypes of high spatial resolution microphone using, respectively, 5, 8 and 24 classic microphones. Fourier-Bessel functions allow one to record acoustic fields using any number of capsule of any type with any array organization (any position, any orientation). More detailed information is available in our scientific paper presented at the 114th AES convention in Amsterdam, A New Comprehensive Approach of Surround Sound Recording, preprint 5717.

More information

2) Spatial Replay : acoustic fields restitution

Today's "multichannel" technologies consists in providing signals each dedicated to feed a single speaker, in order that the listener placed in the middle hear the desire effect. Spatial Replay technology gathers information describing an acoustic field and applies proprietary processing in order to determine the signals required for each speaker to optimally reproduce the acoustic field.

Thus, Spatial Replay technology gives an optimal answer to every application as a function of the specific constraints such as cost, performances, size, simplicity... The application of this technology has permitted Trinnov Audio to develop a prototype using from 2 to 16 speakers with random arrangement. As there is no restriction to optimally control tens or hundreds of loudspeakers, it gives strong basis for the future of audio technologies

More information

3) Spatial Morphing : acoustic fields manipulation

The possibility to restitute accurately the acoustic field is not enough. Since music production is a creative process, there is a strong demand for specific tools to manipulate sound environment. Trinnov Audio developed a technology that directly process the acoustic field: Spatial Morphing. This technology opens to spatial audio the same transformation possibilities than the most advanced photo-editing software offers to pictures. As an example, in a full orchestra, a particular musician may be selected and attenuated or amplified, moved around, stretched or diffused in space.

Thus, Spatial Morphing technology brings an innovative solution for a variety of sound design needs. Trinnov Audio has developed prototype software to rotate the sound image, distort or emphasize the sound stage in a particular direction.

More information

According to today?s signal processing technologies, it is often more efficient to manipulate the frequencial representation of a signal rather than to manipulate its temporal wave form. As an example filters are traditionally defined by their frequency response. The same concept applies to acoustic fields where it is more efficient to manipulate the Fourier-Bessel coefficients representation rather than it spatio-temporal wave form.
Any transformation of a linear acoustic field can be expressed as a specific recombination of its Fourier-Bessel coefficients. As a result, the Fourier-Bessel coefficients of the processed acoustic field are obtained by applying a transformation matrix on the Fourier-Bessel coefficients of the unprocessed acoustic field. This generic formulation of acoustic field processing opens infinite possibilities of manipulations. Different transformation categories can be defined, as an example:

Spatial rotations: Rotate the entire acoustic field according to the standard 3 freedom degree of 3D rotations.
Spatial distortions: Applies spatial transformation in the acoustic field, for instance the sound sources are moved according to an angular distortion law.
Spatial convolution: Adjust the level of details in the acoustic field. As an example is can blur a focused recording or, in a certain extend, improve focusness a fussy recording.
Spatial gating: makes it possible to select in space part of the entire acoustic field.

The concept of transformation matrix is the core concept of acoustic field processing. In general, the matrix coefficients for high resolution acoustic fields (order > 1) are obtained with extremely complex mathematical relations. It has been one considerable challenges for Trinnov Audio to establish and validate these relations. Now, Very complex acoustic field processing can be built by cumulating different type of elementary transformations.

...top

Solutions based on Digital Acoustics

Digital Acoustics makes possible new solutions to the real world challenges of sound engineers, acousticians and sound systems designers.

High Spatial Resolution can be used make high quality 5.1 recordings. Our SRP (Surround Recording Platform) product is a implementation of HSR.
Spatial Remapping helps to optimize loudspeaker placement in a room. It is used in our flagship product, the Optimizer.

1) High Spatial Resolution

High Spatial Resolution (HSR) is the spatial equivalent for high temporal resolution, which is also called High Fidelity (HiFi). High Fidelity aims at recording sound precisely, including the least audible details, i.e. within a wide frequency band.

Similarly, HSR aims at accurately recording the spatial characteristics of a sound environment.

Spatial resolution is directly connected to the angular selectivity of directivity patterns. Whereas temporal resolution is characterized by the frequency, spatial resolution is characterized by the "spatial frequency", which is also called the order.

The higher the order is, the higher spatial resolution is:

Order 0 corresponds to an omnidirectional directivity and does not select any direction in space.
Order 1 corresponds to a figure-of-8, combinations of order 0 and order 1 give all directivities ranging from an omni to a bidirectional pattern, including cardioids and hypercardioids.
All these directivities are directly provided by standard capsules.
Higher orders give more selective patterns.

Trinnov Audio has developed the HSR technology to overcome some limitations of current spatial audio systems.
Thanks to our unique research about both acoustic field capture and restitution, it is possible to extract more efficiently the least spatial information that any microphone array can get and to compute more efficient signals to feed any loudspeaker array.

Trinnov Audio's Surround Recording Platform delivers directivities up to order 5. Both figures below represent the most selective directivity achievable using 1st and 5th order.


1st Order		5th Order

High Spatial Resolution allows:

a wide listening area,
accurate tones,
perfect down-mix,
optimal use of the centre channel (and optionally LFE),
harmonious integration of panned sources,
true envelopment (phantom images over 360?),
pin-point phantom images,
efficient control of the reverberation,
robustness to post-processing.

For more information, please read our White Paper about HSR.

2) Spatial Remapping

Spatial Remapping is a technology to adapt multichannel sound on any loudspeaker layout. An optimal reproduction of multichannel sound is obtained only if the loudspeakers are arranged in the listening room according the ITU recommendation, where the center speaker is at 0?, the left and right speakers are at +/-30? and the surround speakers are at +/-110?. Unfortunately, this recommendation is incompatible with many listening situations such as homes or location recordings.

The ITU recommendation has been developer to overcome a limitation of mulichannel sound, where the spatial environment is described by a mean to reproduce it: loudspeaker at predefined positions and channels to feed them. If the loudspeakers are not at their correct places, the reproduction is incorrect. The Spatial Remapping technology overcomes this limitation of multichannel by providing correct imaging on any reasonable speaker arrangement.

The remapping principle is illustrated below:

During a radiation step, a unique acoustic field is associated to the discrete multichannel signals. The radiation is a linear process recreating the acoustic field produced by ideal loudspeaker perfectly respecting a predefined loudspeaker layout such as ITU recommendation. The radiation process provides the Fourier-Bessel coefficients of the acoustic field resulting from the mutual contribution and interaction of all the channels. This is a very powerful step as the acoustic field representation is independent of the original multichannel format and the loudspeaker layout.
During the decoding step, optimal loudspeaker feeds are derived from the acoustic field according to the Spatial Replay method.

In the Optimizer, the remapping technology is fully automated thanks to a 3D acoustic probe measuring the actual 3D positions of the loudspeakers. The distance is evaluated within 1cm from the propagation time for the wave front emitted by the loudspeaker to reach the 3D acoustic probe. The angles (azimuth and elevation) are measured with less than 2? error from the analysis of the orientation of the wave front crossing the 3D acoustic probe.

...top

Research Publications

		Convention Paper 6375 Presented at the 118th Convention 2005 May 28-31, Barcelona, Spain Reproducing Multichannel Sound on Any Speaker Layout
		Convention Paper 6231 Presented at the 117th Convention 2004 October 28-31, San Fransisco, USA Designing High Spatial Resolution Microphones
		Convention Paper 6116 Presented at the 116th Convention 2004 May 08-11, Berlin, Germany High Spatial Resolution Multichannel Recording
		Convention Paper 5717 Presented at the 114th Convention 2003 March 22-25, Amsterdam, The Netherlands A New Comprehensive Approach of Surround Sound Recording
		Conference Paper 4-1 Presented at 28th International Conference 2006 June, Piera, Sweden Use of a High Spatial Resolution Microphone to Characterize the Early Reflections Generated by a WFS Loudspeaker Array
		Informatique Musicale "Du signal au signe musical" Chapter 3: Techniques de spatialisation du son infoMusicale.pdf (1.3 MB)

...top

Links

...top