(51) Int.Cl.6

Description

The invention relates to an electronic recognition system for audio signals, as disclosed in the preamble of claim 1. In this case the audio signals include, in particular, but not exclusively, warning signals from other vehicles and unusual noises from one's own vehicle. In addition, the invention relates to a vehicle monitoring method, as disclosed in the preamble of claim 8.

Almost all modern vehicles are equipped with devices, like parking sensors and distance sensors. Their mode of operation is based on emitting signals and receiving the back-scattered signals. To this end, the device comprises not only a transmitter, but also a receiver for visual or audio signals. The signals, which are received by the receiver, are admitted for further processing by an evaluation unit of the device, only if they correlated in some way to the emitted signal. That is, they have the same frequency or the same pulse shape.

Moreover, there exist systems, in which random events, like honking or other warning signals, are also acquired; and in this way communications between multiple vehicles is established. Such a system is disclosed, for example, in the German utility model DE 94 10 485, as a device for direction-dependent detection of weak audio signals. This device comprises a direction-constant transmitter, which works at an angle of up to 110 deg. and with which an emergency vehicle is equipped. The respective motor vehicle of the other road users is equipped with one or more receivers, which pick up the signals of the transmitter and each of which is coupled to an electric converter, which in turn is connected to loudspeaker lines, which transmit a buzzing sound to the loudspeaker.

Another system of the aforementioned type is disclosed in the DE 30 46 862. The loudness level control, which is described in said system and which is intended for devices for an electroacoustic sound reproduction in vehicles, exhibits a detector, which emits - in the event of audio warning signals from authorized emergency vehicles - an output signal to an actuator, by means of which the loudness level of the electroacoustic sound reproducing device is decreased or set completely to zero.

However, the prior art systems consider only a sub-aspect of the acoustic information in road traffic. They serve only to assist the driver in detecting warning signals by influencing the setting of a reproduction system in the vehicle. However, audio signals in road traffic can be used in a variety of ways and for a comprehensive and more complete evaluation of the momentary traffic situation than is possible, for example, with just optical means.

The invention is based on the problem of providing a recognition system, which is intended for audio signals and warning signals and with which the road situation and the vehicle state can be determined completely and can be considered in the control of the vehicle, as well as a method for acoustic monitoring of a vehicle and for conditioning the audio signals.

This problem is solved with a recognition system, which is intended for audio signals and exhibits the features disclosed in clam 1, and with a method, which is intended for acoustic monitoring of a vehicle as well as for conditioning the audio signals and which exhibits the features disclosed in claim 8. The dependent claims relate to the respective advantageous embodiments of the invention.

The inventive recognition system, which is intended for audio signals and which comprises at least one receiver for acquiring audio signals outside the vehicle and in the interior of the vehicle as well as comprises at least one output unit for outputting information and control signals as a function of the audio signals, is characterized by an analog / digital converter for converting audio signals into digital data; by an evaluation unit for collecting the classifying criteria from the data; and by a computer having at least one memory unit for storing the comparison signals and having at least one comparator for comparing the acquired classifying criteria with the previously stored classifying criteria and for generating information and control signals as a function of the comparison results.

The inventive method for monitoring a vehicle with the steps - acquiring audio signals in and outside the vehicle and outputting the information and control signals as a function of the acquired audio signals - is characterized by the steps: converting the acquired audio signals into digital data, acquiring the classifying criteria in the data, comparing the acquired classifying criteria with the previously stored classifying criteria and generating information and control signals as a function of the comparison results.

The invention is based on the recognition that optimal acoustic monitoring of the vehicle is possible with microphones that are mounted with a preamplifier on the vehicle and that their output signals are converted into digital data with an analog / digital converter (ADC). To this end, the data are transferred from the ADC to an evaluation unit or rather a processing unit for the purpose of obtaining classifying criteria in the data. Upon completion of the classification, the results are compared with the predefined classifying criteria in a computer and then either shown to the driver, for example, on an LED or LCD display, or in the case of an autonomously operating vehicle are used via an interface, for example, a CAN bus interface, in order to control the vehicle. In addition to their visualization and their use for control elements and actuators, the data can also be outputted over a computer interface to an external (vehicle) computer for further processing.

The distinct advantage of the invention over the prior art lies in the detection of the audio signals in a vehicle and in their classification (especially with respect to warning signals) and in a reaction (synchronized with the respectively classified signals, taking into consideration the existing boundary conditions (vehicle with or without a vehicle driver)).

In a preferred embodiment it is possible to distinguish upon detection of a classified signal by way of a coarse directional detection whether it is a signal coming from the opposite or the same direction of travel or coming from a lateral direction. To this end, this embodiment provides that a plurality of microphones or sound receivers are mounted on the vehicle.

In an additional preferred embodiment of the invention, the frequency spectrum of the acquired audio signal is analyzed, and/or a voice input of control commands by the user is allowed.

Possible output signals are signals for visual or acoustic reproduction for the driver, or signals for direct intervention in the control of the vehicle. The visualization of the signals in the first case can be an indication of the direction with a Martin horn (two tone horn) or indication of information, like a localization display in the vehicle in the event that a component fails.

For greater comprehension, the invention is explained in detail below by means of embodiments that disclose additional features as well as additional advantages. These embodiments are depicted in the drawings.

Figure 1 is a block diagram of a first embodiment of the recognition system of the invention.

Figure 2 depicts one part of the block diagram in Figure 1 with additional details.

Figure 1 depicts one embodiment of the inventive electronic recognition system, which is intended for audio warnings and signals. In order to receive audio signals, this embodiment comprises microphones 1, each of which is connected directly to a preamplifier 2. The preamplifier 2 raises the signal, received by the microphone 1, above a noise level and renders it accessible for further processing.

Figure 1 depicts only two microphones 1 with preamplifiers 2 for reasons relating to a better overview. However, the recognition system exhibits preferably four microphones 1, which are mounted on the outside of the vehicle, or at least in such a manner that they can pick up the audio signals outside the vehicle. The four microphones 1 cover, starting from the corners of the vehicle, the surrounding area of the vehicle and in this way allow an exact position finding of the sound sources in the environment. In the case of two microphones, instead of four microphones, the system can distinguish only sound sources in front of and behind the vehicle, but cannot perform a lateral position finding. In addition, the interior may also be provided with microphones 1, either for monitoring the passenger compartment or for monitoring the engine chamber or any other section of the vehicle, like radar and the like. Instead of exterior microphones 1, it is also possible to use, under some circumstances, already existing microphones, if they are adequately sensitive to the respective frequency range.

The signals, which are received by the microphones 1 and are conditioned by the preamplifiers 2, are fed to an A/D converter 3, which converts the analog signals to digital data. This feature has the advantage that, starting from the A/D converter 3, the transmission quality is significantly improved (noise sources are no longer a significant factor), and the processing of the data is drastically simplified, and/or it is possible to process the data in consideration of a plurality of aspects. The operating principle of A/D converters is common knowledge and, hence, is not explained in detail at this point.

The embodiment of the recognition system that is depicted in Figure 1 provides an A/D converter 3 for all microphones 1. In other words, the A/D converter 3 is a multi-channel A/D converter, which reads in succession the individual microphone signals into the input over a multiplexer. Thereafter it converts the signals one by one. The digital data are further processed in the order of their A/D conversion from the audio signal. The configuration with a (multi-channel) A/D converter and a multiplexer has the advantage that only one A/D converter is required.

However, it is also possible to equip each microphone 1 with its own A/D converter - a single channel A/D converter. Then there is no need for a multiplexer; and the individual microphone output signals are converted directly. Their further processing is staggered with respect to time on the digital side.

In any case the system clock is defined preferably by the A/D converter, so that in the next phase of the processing the runtime difference between the signal, received by the first microphone 1, and the signal received by the second microphone 1, can be determined. That is, the digital data are outputted from the A/D converter with information that gives the time, at which it was generated in relation to the system clock. The position finding of the sound source from the runtime difference of the individual data is explained in detail below in conjunction with Figure 2.

Starting from the A/D converter 3, the lines in Figure 1 and Figure 2 are depicted with small slanted lines, which are supposed to show that the lines are for digital data. These lines represent the data buses with, for example, 8 bit width. In this case, in particular, it is possible to use the buses that already exist in the vehicle, like a CAN bus, as the data lines.

The data, formed from the audio signals by way of the A/D converter, are analyzed in an evaluation unit 4; and classifying criteria of the data are determined. The function of the evaluation unit 4 and the essential classifying criteria are explained in detail below in conjunction with Figure 2.

The classifying criteria of the data, which are correlated with the acquired audio signals and which are determined by the evaluation unit 4, are fed into a computer 5. The computer 5 specifies the classifying criteria and compares said criteria with previously stored patterns or with previously buffered data. In this case the essential classifying criteria are the frequency response of the signal and any other conspicuous features of the signals, like their repetition rate. These properties are found preferably by way of a Fourier transformation of the signals in the evaluation unit 4. In addition, the amplitude response of the signal can be tracked over a defined period of time. This feature is especially important for the decision of whether the signal transmitter is coming closer or going farther away. Additional features of the functions of the computer 5 are explained below in conjunction with Figure 2.

In addition, the embodiment of the invention that is depicted in Figure 1 and Figure 2 comprises a voice input unit 14. It is possible with the voice input unit 14 to specify in the broadest sense patterns, with which the classifying criteria of the digitized signal are compared. In particular, it is possible with the voice input unit 14 to store command words or key words, which - when repeated, for example, by the driver - allow the recognition system to perform a predefined function. However, specific noises can also be stored in this manner. These noises can be assigned, for example, to a malfunction of a component of the vehicle.

The comparison results of the computer 5 are outputted over an interface 9 as the information and control signal to an output. In the illustrated embodiment an audio output unit 15 for a voice reproduction and a visual display unit 16 for visualization of the evaluation results are a part of the recognition system. In this case the output serves to inform the driver and/or to intervene directly in the control of the vehicle, in particular in the case of autonomously operating vehicles.

In this case the audio output unit 15 comprises a voice generator (not illustrated) for assembling the voice sequences as a function of the evaluation results of the computer 5. In addition, the output unit 15 comprises a loudspeaker, which can be a part of an already existing reproduction system in the vehicle.

The display unit 16 is preferably an LCD or an LED display.

Figure 2 shows the details of the evaluation unit 4 and the computer 5, both of which are depicted as dashed lines. For reasons relating to a better overview, Figure 2 also shows once more the A/D converter 3, the voice input unit 14 and the interface 9 with the display and output unit 15 and 16.

In the evaluation unit 4 a spectral analysis of the audio signal (more precisely the corresponding data) is performed in a spectrum analyzer 6. In essence, this analysis can be done by a bandpass circuit or switching circuit for a Fourier transformation.

The classifying criteria, extracted in the spectrum analyzer 6, are compared with the classifying criteria of a specified spectrum in a spectrum comparing circuit 10. In this way, for example, a Martin horn can be clearly identified by the audio frequency and its variation over time. To this end, the audio frequency with its variation over time for a Martin horn is filed in a spectrum memory unit 13.

In a preferred embodiment of the invention the memory unit 13 for the classifying criteria of a spectrum is connected to the voice input unit 14. It is possible with this configuration to give individual specifications with respect to the stored classifying criteria. In particular, key words and/or the spectrums of spoken commands can be stored and later compared with the audio signals, acquired by the microphones. Then in the event of a command the system reacts in accordance with the request of the driver. In this case the voice input unit 14 can be coupled, in particular, to the spectrum analyzer 6 and the spectrum comparing circuit 10 in a neuronal network, so that a learning algorithm can be executed, in order to guarantee a reliable recognition of spoken commands.

In order to determine the direction, from which an audio signal is received, a reference time in each signal is determined in a marking circuit 7, which is downstream of the spectrum analyzer and which is intended for measuring the runtime difference between the audio signals. It is possible with this marking in the data, corresponding to an audio signal, to determine the signal response difference between two microphones 1 by means of a runtime measuring circuit 11. In addition, it is also possible to determine the rate of repetition of the detected sound - that is, from the low frequency sound changes. Therefore, both the position finding and the classifying criteria, which were filtered out with the spectrum analyzer 6 and the marking circuit 7, are used in combination, for example, to identify a signal of a Martin horn, which in a first interval emits an audio signal having a lower frequency and in a second interval emits an audio signal having a higher (by a fifth) frequency.

The position finding of a sound source can be improved by comparing the amplitude of two received signals, in addition to their runtime difference. The amplitude of the microphone 1, which is somewhat farther away from the sound source, is somewhat lower than the amplitude of the microphone 1, which is closer to the sound source. A corresponding comparison is made by way of an amplitude analyzer 8 and an amplitude comparator 12. The amplitude analyzer 8 determines the amplitude of the signal at a time that is fixed by the marking circuit 7. This amplitude of the signal is compared with the amplitude of a signal from the other microphone 1 at a suitable time. Depending on whether said amplitude is larger or smaller, the direction of the sound source can be determined with certainty.

In particular, the computer 5 also considers the temporal changes of the classifying criteria over wider ranges of time. That is, for example, the first time derivative is formed so that information about the movements of the sound source in relation to the vehicle is received.

If the recognition system determines on the basis of the evaluation of the audio signals in the evaluation unit 4 and on the basis of the comparison with the predefined patterns that it is necessary for the driver to intervene or that in the case of an autonomously operating vehicle it is necessary to change the driving variables, then the corresponding information and control signals are outputted by audio and visual means and/or an intervention from the interfaces occurs directly in the vehicle control.

The inventive method comprises the following steps that are necessary for controlling the aforementioned components of the recognition system.

In a first step the signals, which were conditioned by the preamplifiers, are converted into digital data. In the next step of the method, the data are analyzed for classifying criteria. This analysis occurs in the evaluation unit 4. Following completion of the classification, the results are compared with the predefined classifying criteria in the computer 5.

Then the results are shown either to the driver, for example, on an LED or an LCD display 16; or in the case of an autonomously operated vehicle the results are used via an interface (not illustrated) - for example, a CAN bus interface -, in order to control the vehicle. In addition to the visualization and their use for control elements and actuators, the data can also be outputted over a computer interface (not illustrated) to an external computer for further processing. In addition, voice reproduction with an audio output unit 15 is also possible.

The method for one embodiment of the inventive recognition system with a voice input unit 14 comprises additionally a learning algorithm for receiving and recognizing user-specific audio stimuli for the system.

Listed below are some of the reactions that are possible with the recognition system, according to the invention. In this case a distinction is made according to the boundary conditions - that is, whether the vehicle is an autonomously operating vehicle (1) (electronic drive system) or a manually operated vehicle (2).

When the inventive recognition system detects, for example, a Martin horn, then it classifies it as a sound warning device for authorized emergency vehicles:

(1) When an emergency vehicle is approaching, the speed is decreased, and the system is put into a state of higher alertness. In situations, in which the emergency vehicle is travelling in the same direction, the system looks for a possibility to allow the emergency vehicle to pass on the left. If it is not possible to detect the direction, the system is put into a state of higher alertness; and the optical sensors and the audio recognition system endeavor to detect the direction of the emergency vehicle.

(2) Following the recognition of the warning signal (irrespective of which direction), the driver is informed by acoustic and/or optical signal units (LED and/or LCD) that a vehicle having an authorized right-of-way was detected. In addition, the control elements and/or the actuators reduce the loudness level of the audio system in the vehicle to a certain level, so that the attention of the vehicle driver is shifted to the emergency vehicle. Furthermore, the intention is to provide a visual display or audio announcement of the direction of the emergency vehicle.

If, in contrast, the system detects a horn and/or any other device for emitting sound signals in road vehicles, like bicycle bells, sound warning signals of rail-bound vehicles, general sound warning signals (construction sites, factories and the like), the following reactions are triggered:

(1) Independently of the direction of the cause, the system is put into a state of higher alertness and endeavors by means of some (in particular, optical) sensors to detect the cause. In addition, depending on the speed of one's own vehicle, the speed is reduced. Furthermore, the driver and/or the passengers can be informed by means of the audio and/or visual units 15, 16 that there is a situation, in which the hazard risk is higher. If necessary, the automatic control system can be switched off in the case of a semi-automatically operated vehicle - for example, in the case of a set cruise control system, so that manual intervention is possible.

(2) Following the recognition of the warning signal (irrespective of the direction from which the audio signal is coming), the driver is informed by acoustic and/or optical signal units (15, 16) that a hazard signal was detected. Moreover, in an additional embodiment of the invention, the control elements and/or the actuators reduce the loudness level of the audio system in the vehicle to a certain level, so that the attention of the vehicle driver is shifted to the emergency vehicle. Furthermore, the audio and visual output unit 15 and 16 can give a visual display or an audio announcement of the direction of the place of origin of the hazard signal.

In another preferred embodiment of the invention, the recognition system can also be used for voice control. That is, human command words (a cry for help, stop, code words) can be programmed into the system and serve to trigger a command: (1) and (2) execute the command (as, for example, "STOP"), but also react to defined code words, which may or may not be personal (for example, for opening doors, deactivating the anti-theft device and/or the immobilizer).

Furthermore, the received audio signal can also be used for detecting anomalous driving noises (squeaking tires, noise caused by a flat tire, engine and/or exhaust defects) and can, therefore, be used for prompt detection of malfunctions of the vehicle and early warning of a repair.

(1)

(1) The system reacts as a function of the cause. For example, in the event of a noise caused by a flat tire, the vehicle is stopped, and a corresponding message is sent to the central service office. In contrast, in the event of squeaky tires, the speed is decreased. Moreover, the system can be put into a state of higher alertness.

(2) The driver is given a cause message by way of the visual and/or audio output units 15 and 16.

The recognition system can also be used to detect weather conditions (rain, snow fall):

(1) The vehicle speed is adjusted (that is, decreased) to the corresponding weather condition; and an appropriate system state is assumed.

(2) The driver is informed about the respective weather condition by way of the visual and/or audio output units 15 and 16.

Reference Numerals

1 microphone

2 preamplifier

3 A/D converter

4 evaluation unit

5 computer unit

6 spectrum analyzer

7 marking circuit

8 amplitude analyzer

9 interface for display, further processing and vehicle control

10 spectrum comparing circuit

11 runtime measuring circuit

12 amplitude comparator

13 spectrum memory unit

14 voice input unit

15 audio output unit

16 visual display unit

Patent Claims

1. Recognition system, which is intended for audio signals and which comprises

at least one receiver (1, 2) for acquiring audio signals outside the vehicle and in the interior of the vehicle,

at least one output unit (9) for outputting information and/or control signals as a function of the audio signals,

characterized by

an analog / digital converter (3) for converting audio signals into digital data;

an evaluation unit (4) for collecting the classifying criteria from the data;

a computer (5) having at least one memory unit (13) for storing the comparison signals and at least one comparator (10) for comparing the acquired classifying criteria with the previously stored classifying criteria and for generating information and/or control signals as a function of the comparison results.

2. Recognition system for audio signals, as claimed in claim 1, characterized in that the evaluation unit comprises filter units (6) for detecting the frequency distribution of the audio signal.

3. Recognition system for audio signals, as claimed in claim 1 or 2, characterized in that the evaluation unit comprises position finding units (7, 8, 11, 12) for determining the distance and/or the relative speed of the signal source of the audio signal.

4. Recognition system for audio signals, as claimed in any one of the preceding claims, characterized by a voice input unit (14) for receiving and storing useful commands.

5. Recognition system for audio signals, as claimed in claim 1 or 2, characterized by a display unit (16) for visualization and/or a sound reproducing unit (15) for audio reproduction of the evaluation steps.

6. Recognition system for audio signals, as claimed in any one of the preceding claims, characterized by actuators for automatic control of the vehicle as a function of an output signal of the computer.

7. Recognition system for audio signals, as claimed in any one of the preceding claims, characterized by a computer interface on the computer.

8. Method, which is intended for acoustic monitoring of a vehicle and for conditioning of the audio signals and which comprises the steps:

acquiring audio signals in and outside the vehicle, outputting the information and control signals as a function of the acquired audio signals,

characterized by the steps:

converting the acquired audio signals into digital data,

acquiring the classifying criteria in the data,

comparing the acquired classifying criteria with the previously stored classifying criteria and

generating information and control signals as a function of the comparison results.

9. Method, as claimed in claim 8, characterized in that the acquisition of classifying criteria comprises the filtering of audio signals for generating a frequency spectrum of the signal.

10. Method, as claimed in claim 8, characterized by a learning algorithm for storing selected classifying criteria.

--------------------

2 sheets of drawings

--------------------

DRAWINGS PAGE 1 Number: DE 197 49 372 A1

Int. Cl.⁶: G 10 K 11/18

Disclosure date: May 12, 1999

[see figure]

key to figure:

3 A/D converter 5 computer

4 evaluation unit 9 interfaces

14 voice input unit