Research Paper Analysis
On the Design of a Bioacoustic Sensor for the Early Detection of the Red Palm Weevil
link

Abstract

During the last two decades Red Palm Weevil (RPW, Rynchophorus Ferrugineus) has become one of the most dangerous threats to palm trees in many parts of the World. Its early detection is difficult, since palm trees do not show visual evidence of infection until it is too late for them to recover. For this reason the development of efficient early detection mechanisms is a critical element of RPW pest management systems. One of the early detection mechanisms proposed in the literature is based on acoustic monitoring, as the activity of RPW larvae inside the palm trunk is audible for human operators under acceptable environmental noise levels (rural areas, night periods, etc.). In this work we propose the design of an autonomous bioacoustic sensor that can be installed in every palm tree under study and is able to analyze the captured audio signal during large periods of time. The results of the audio analysis would be reported wirelessly to a control station, to be subsequently processed and conveniently stored. That control station is to be accessible via the Internet. It is programmed to send warning messages when predefined alarm thresholds are reached, thereby allowing supervisors to check on-line the status and evolution of the palm tree orchards. We have developed a bioacoustic sensor prototype and performed an extensive set of experiments to measure its detection capability, achieving average detection rates over 90%.

Introduction

In this paper, we design and develop a bioacoustic sensor prototype which efficiently detects the
sounds produced by RPW larvae after the first infestation stages. The proposed bioacoustic sensor is
able to:

• (1) effectively detect the RPW presence with high detection rates (over 90%);
• (2) perform monitoring tests at user programmable frequencies in order to achieve a fast detection response;
• (3) work autonomously during large periods of time (at least one year);
• (4) work without maintenance requirements after installation;
• (5) form a wireless sensor network to cover from little orchards to
  large plantation extensions;
• (6) allow continuous monitoring activity, since data may be checked
  on-line in real-time through an Internet connection with the control
  station;
• (7) trigger an alarm system defined at the control station to warn supervisors about the desired events by means of e-mail, Short Message System (SMS), Instantaneous Message System (IMS), etc.

Microcontroller

• Jennic (now NXP) model number JN5148-001
• Low power operation modes, from 1.5 uA in sleep mode, and close to 200 mA in full power mode with all peripherals working on.
• Thirty two-bits RISC pipelined processor.
• One memory of 128 KB ROM and one memory of 128 KB RAM, large enough to store program and data used to perform the bioacoustic detection of the RPW.
• Wireless 2.4 GHz, IEEE 802.15.4 compliant transceiver, with ZigBee network support.
• Up to 21 general purpose digital inputs/outputs.
• Twelve bit ADC with a maximum sampling frequency of 100 kHz.
• Four wire interface for digital audio.

Power Consumption

• Regarding the power supply, the bioacoustic sensor measured consumption rises up to 200 mA when performing detection process.
• When sensor wakes up from sleep mode, it enters running mode, where all the required systems are turned on, and the RPW detection software starts to work during a limited period of time (between 5 and 10 minutes, although it may be programmable).
• This working behaviour is periodically repeated during the life of the sensor node,
  performing between 3 and 12 audio analysis a day (again this may be modified). In the best case, a battery of 2,000 mAh works for no more than one month.

Power Unit

• Supercapacitor at 500 mA of one hour of continuous operation.
• 8 miniature solar cells wired in parallel generating around 350 mA.
• Every cell works at 3.0 volts and is able to give 45 mA, with dimensions: 54 mm × 43 mm × 3.0 mm (L × W × D).
• The current produced by solar cells is supplied to a Low Dropout (LDO) Linear Voltage Regulator (LT3085 from Linear Technology), to adjust the solar cells voltage to the input range demanded by supercapacitor, which operates at 2.3 volts. The output of the supercapacitor powers the sensor board using a Step-Up Voltage Regulator (L6920 from STMicroelectronics) to properly adjust capacitor’s voltage to that required by the sensor board.

Radio Interface

• The radio interface is up to create a reliable point-to-point outdoor communication over distances of 30 meters. Nominal values from manufacturer are larger, but they can be achieved only in optimal conditions of no EMI interferences, higher transmission power (up to 20 dBm), and particular antenna arrangement and orientation.

Audio Probe

• To decide the appropriate audio probe design, we have analyzed the features of RPW sound: (a) studying previous works like [11–13], (b) performing a preliminary spectral and temporal analysis of available RPW audio recordings, and (c) taking into account several aspects, as probe insertion into palm trees, environmental noise levels, signal adjust conditioning parameters (i.e., signal amplification and filtering) among others, by means of laboratory and field tests.

Microphone

• We used also KINGSTATE KECG2740PBJ, a 6 mm external diameter and
  length of 2.7 mm, with a sensitivity of –40 dB at 1 kHz.

Signal Conditioning: Filtering and Amplification Stage

Once we have defined the audio sensor and the probe, we need to perform some signal conditioning to deliver the microphone output signal to the A/D converter with the highest possible quality. The main issue here is the amplification stage, which is a well-known problem with lots of solutions. In this case, not all of these solutions are convenient, because the bioacoustic sensor has a set of requirements that limits the design: (1) it has to be powered by batteries, so the use of high and bipolar voltages should be avoided, and (2) sensor signal output has to be largely amplified, in the whole range of frequencies, up to 10 kHz.

To fit the requirements, we have used a Texas Instruments TLV2785 operational amplifier. This chip includes four operational amplifiers, with single supply since 1.8 V up to 3.6 V, and a supply current of less than 1 mA per channel, very convenient for battery operated devices. In addition, amplifiers may be shutdown to save battery when the device is not capturing sound. Its bandwidth is 8 MHz, allowing gains of more than 50 dB for a cut off frequency of 10 kHz.

A PCB board has been designed, where two of the operational amplifiers are used to amplify the microphone output. Two additional amplifiers are used to create an active low pass filter at 10 kHz, to avoid aliases in the digital part of acquisition chain. This board is inserted in a plastic box, with an RCA connector for the microphone, a potentiometer to adjust the desired gain, and two jack outputs, one for the headphones and the other for the connection with an analog input of a microcontroller (a prototype is shown at Figure 5). The goal of this design is the modularity and flexibility of the device: different microphones may be connected to the amplifier, it can be used by a human operator listening through headphones, or it can be connected to the microcontroller for the autonomous detection.

• Texas Instruments TLV2785 operational amplifier

Bioacoustic Sensor: Software Description

In running mode, the first operation performed by the sensor node is to configure and initialize the audio system with the parameters established at the initialization stage (sample bits, sampling rate, window size, etc.). Then, a Signal-to-Noise Ratio (SNR) measurement test is executed to determine the actual noise level before the audio analysis procedure;

In our studies we have chosen 4096-sample fixed size windows with the minimal overlapping that prevents potential losses of RPW signals located at window edges.

Once we have captured a window of audio samples, we proceed to apply our RPW analysis algorithm. If this window contains one or more RPW audio signals, the analysis results are time stamped and stored as part of the running mode analysis report.

RPW Sound Model

We have classified three different kinds of sounds coming from RPW larvae: “eating”, “squealing” and “moving”.

For our study, we have chosen the first one, “eating”, because (1) it is the loudest sound, (2) its frequency is clearly superior to the other identified sounds in all available recordings, and (3) it is representative of the RPW larvae feeding actions in the first stages of their evolution.

Figure 7 shows a good example of “eating” sound that was extracted from the recordings above mentioned. We can see the “eating” sound in both temporal, Figure 7(a), and frequency domains, Figure 7(b). The later was obtained from a wavelet packet transform with five decomposition levels that equally divides the input audio signal bandwidth in 32 subbands (horizontal axis). For each wavelet subband, we have computed the normalized energy of its coefficients in order to visually determine the energy distribution. As it can be seen, in Figure 7(b), there are three subbands that accumulate most of the signal energy. Thus, energy distribution across wavelet subbands may be considered a spectral fingerprint of RPW feeding activity, being an important feature to be included in our RPW “eating” sound model.

We also analyze the “eating” sample in the temporal domain, determining its main duration in terms of audio samples. We need to determine the beginning and ending positions of “eating” sound inside captured audio window using three parameters: audio signal level, signal variance, and SNR level, as shown in Figure 7(a). With the SNR level we determine the noise domain present in the captured audio, which is the actual noise level plus a 6 dB margin. The beginning of a new sound will be determined by α consecutive samples which value is above noise domain; meanwhile, the ending of the new sound will be determined just before β consecutive samples, which value falls inside noise domain. Both parameters are empirically established to properly identify potential RPW sounds. It can be seen in Figure 7(a) that the RPW “eating” temporal extension is of 492 samples (around 11 milliseconds).

The Fourier Transform, explained in one sentence
source

Prototype for apidictor/frequency analysis done

I'm happy to present our newest sensor: a microphone with an attached amplifier circuit and low pass filter with 1,6kHz shut-off frequency. In short: usable as a frequency spectrum analyser

That's the circuit, actually similar to this breakout from sparkfun but with different capacitors to low pass the audio signal at about 1600 Hz which turns in a less noisy signal. More about that on the github repository.

The Cortex M3 is quite strong enough to perform a middle class quality fast fourier transformation of about 400 samples. I used the kiss fft library for that. Depending on the sampling frequency we're able to receive frequency bins of about 2 Hz. Higher sampling frequency means we're able to detect higher frequencies, but also means we're loosing accuracy. Here are some data I received from the spark playing my guitar with a sampling frequency of about 1000Hz (means we're able to detect frequencies up to about 1000/2,2Hz)

Still that's a try and there's alot to go still.
Important points we'll have to discuss about are:

• which frequencies we'll focus on?
• how exact the fourier transformation needs to be?
• the guitar example is a quite easy job compared to listening to a bee hive and sourrounding noises. How do better the outcome of the fft? -> best would be to keep the data as small but also raw as possible to perform data analysis later on

btw: thanks to phec's work on that
@serdmanczyk thanks for the post on the fft! - any further ideas now?
@ghellard do you have any information on which frequencies to focus on?

Nice work @ron!

We now how over 400GB of high definition audio thats been recorded from our collaborators in Australia - would you like to take some samples from that and check them out for a test?

How is development going in general @jonathan and @ron? Are we looking on track for our release time?

Nice work guys.

I got some data from Aaron.
would be great to have access to more data ( in an uncompressed or high quality compressed format - mp3 is quite bad for analysis) to run the analysis over a lot of samples to get to know possible pitch-frequencies.

thanks!

Hey @Jagungal - we're wanting to test the new audio sensor - could you send some of the new high quality audio to @ron? His email is ron.martin.jacob@googlemail.com

@ron perhaps you'd like to describe exactly what you'd like him to send, since theres a lot of audio!

No problem - email sent to you @ron. I have already sent about 100GB of samples to Aaron, and these are high definition .wav files, so they should fit the bill nicely. The downside is that they are huge - the smallest one is 400Mb, and most are 2.1GB, so moving them via email is not really an option.

Not sure how physically close you are to each other - a disc with just under 400GB of data is on its way to Aaron now, and should arrive sometime next week.

Our queen died last week, and they should be replacing her now - we have had the recorder going continually to see if we can catch the piping from the new queens before they hatch, which would be fun.

Finally managed to get the large file transfer working - @ron you should have an email in your inbox now, just click on the link and the high definition audio file will download.

This is in mono only, and is basically exactly as the Tascam unit records it.

Down the track if you send us the schematics or a completed sample of the apha kit we can run it in paralell with the current recorder to set a baseline and test if a bit more. Our office apiary is on a balcony and has access to wi-fi and power - you can see the set-up here:
https://photosbykris.exposure.co/aurecons-urban-bee-colony

Sounds like good data!
I think we'll finish the alpha board technically with temp, humidity, audio, ingoing/outgoing and weight sensors, built a proper version of it end than make it ready to share so we can have a look how to connect and impove it inside the hive.

Bee's only seem to recognize frequencies up to 500Hz - quite good point for our sensor:
see:

http://www.apidologie.org/articles/apido/pdf/1993/03/Apidologie_0044-8435_1993_24_3_ART0009.pdf

for more information

Thats great data @ron!

which frequencies we'll focus on?

Hi Ron, have you scanned this thread: http://www.imkerforum.de/showthread.php?t=32478

I think there are some different sound sources you can inspect and monitor

• toot and quack of the emerging and "regnant" queens, see Seeley, Honeybee Democracy for a description of time pattern imkerforum.de/showthread.php?t=29251&p=347733#post347733

• you can find ongoing links about the frequencies under mkerforum.de/showthread.php?t=29251&p=357496#post357496

• second would bee worker piping right before swarming, also described by Seeley
  imkerforum.de/showthread.php?t=32478&p=349281#post349281

• third "warble" and "hiss" described by Eddie Woods, I have found this only by Eddie Woods so it seems nobody has replicate this, but it could be worth to investigate this in deep.

Other non-swarm related acoustic indicators can be found here:
colonymonitoring.com/cmwp/five-components/sensing/colony-acoustics/

Btw. I don't know if we need really a high sampling rate. Monitor every one or five second could perhaps be sufficient.

And it could be interesting to move the calculation of the frequency bins out of software to a dedicated hardware device. Cortex M3 is nice, but power hungry and I think bee monitoring makes only sense with a battery powered or solar device.

third "warble" and "hiss" described by Eddie Woods, I have found this only by Eddie Woods so it seems nobody has replicate this, but it could be worth to investigate this in deep.

Must correct myself: http://www.apimondia.com/congresses/2013/Economy/Plenary-Session/Remote%20Hive%20Monitoring%20-%20Huw%20Evans%20Beng.pdf