Research update: developing new bat monitoring tools with Bat Detective data

Hello from the Bat Detective team! It’s been a busy summer at Bat Detective HQ after an amazing spring with British Science Week and the World Tour. So we’ve been a bit quiet on the blog in the last couple of months while we’ve been working on updating our automated tools and road-testing them on new data. We’re now coming close to having our results ready for scientific publication, and we’ve also had the chance to put our new software tools into practice on analysing some brand new bat survey data. So over the next two blog posts, we’ll be updating you on our progress, explaining where the Bat Detective project is at right now, and showing how we’ve been using all the data you’ve helped us to label.

In this first post we’ll discuss how we’ve used Bat Detective data to improve our automated bat call detection tools, and highlight some of the challenges we’ve encountered along the way. In the next post, we’ll show some examples of where we’ve been testing out our software tools on bat survey data from the UK and Madeira – keep an eye on the blog for that one very soon.

The team have also been up to a few other things in the last few months. Bat Detective’s Rory Gibb (me) gave a project update talk at the Zooniverse’s first ever ecology workshop, where there was some fascinating discussion about how citizen scientists can become increasingly involved in some of the major challenges facing ecology and conservation in future. We’ve also just had a big article on Bat Detective and its sister project iBats published in the latest citizen science-themed issue of Environmental SCIENTIST – the article will be available to read online in the near future, so we’ll share it here when that happens.

Training machines to recognise bat calls: why and how?

In our last research update post a year ago, we explained how advances in machine learning technology have enabled us to train algorithms to automatically recognise bat calls in ultrasonic survey recordings. This is important because newer bat detectors can be deployed in the field for weeks or months, collecting so much audio data that it’s almost impossible to analyse them manually. By making it possible for bat researchers to quickly and reliably find where bat echolocation calls are in these recordings, automated tools are creating exciting new opportunities to study bat ecology, behaviour and conservation at much larger scales than ever before.

Machine learning involves training computer algorithms to automatically recognise bat echolocation calls in recordings, by showing the computer thousands of examples of what they look and sound like. In our last research update we showed how training the algorithms on increasingly large amounts of data from Bat Detective improves their performance. For that reason, and also to include a greater diversity of bat sounds from around the globe, we’ve asked for your help in labelling our World Tour data over the last year. And thanks also to the efforts put in by volunteers during British Science Week, we’ve now got thousands of new bat call annotations to incorporate into our detector tools – so one of our current challenges is exploring the best ways to use all of these new data.

We’ve now got the detector algorithms up and running, and we’re currently testing them out to assess how well they perform. The figure below shows an example of the detector in action on a snippet of audio data from the iBats global bat monitoring programme. The recording is displayed as a spectrogram underneath, with sounds showing up as bright markings. The graph above shows the computer predicting where it thinks the bat calls are – each vertical red line shows where the computer predicts there is a bat call. The height of the red lines tell us how certain the computer is about its predictions, where higher indicates more confident. The green bars show where a human expert has confirmed that bat calls are present – so in this example, the computer has successfully recognised all the bat calls.

Are you sure that’s a bat? The problem of false positives

However, there are still some errors where the computer thinks there is a bat call, when there actually isn’t one (a ‘false positive’). This is a problem for monitoring bat populations, because too many false positives could result in researchers overestimating the true number of bats in an area, which could for example have an impact on conservation efforts. You can see a clear example of these errors in the next figure below, where the computer falsely predicts that the mechanical noises at the bottom of the spectrogram are lots of bat calls.

So to improve this, we’ve also been including non-bat sounds from Bat Detective – those insect calls and mechanical noises you’ve also helped us to find. By training the algorithms to also recognise what bat calls don’t look like, we can significantly improve their accuracy. The image below shows the difference: it’s the same audio clip, but there are now far fewer false positives (red lines).

This is a great example of the importance of testing out these tools on new data from a variety of times, places and detector types. This helps us get a better idea of where they’re under-performing, and how they can be improved before we release them as open-source tools for other researchers to use. So with that in mind, keep an eye on the Bat Detective blog next week for our next research update: we’ll be showing some examples of where we’ve road-tested them on new bat survey data recorded during this summer. We’ll also be uploading a new set of data from Russia – one of our last few World Tour stops – so stay tuned for that.

And a huge thanks again for all your efforts with labeling the data on Bat Detective, both during this year and throughout the whole project – we wouldn’t have been able to get to this stage without your input, and it’s really exciting to see the work of our community of volunteers starting to produce results.

Advertisement

Bat Detective arrives in Japan, our next World Tour stop

Welcome to the latest stop on our World Tour! We’re now in Japan, after spending the last two months uploading data from Australia and New Zealand to the Bat Detective site. Firstly, a massive thanks from the Bat Detective team for all your efforts in listening to and classifying our data so far this year – thanks to the amazing efforts of citizen scientists during the World Tour as well as British Science Week back in March, we’ve got a much larger dataset of labelled bat calls to train our automated algorithms with, and the results are improving.

In the coming months we have a few more World Tour stops before we reach the end of our global bat search. This month we’re in Japan, with a new set of data uploaded to the Bat Detective site that was recorded on car-driven transects during 2010 and 2011 in locations throughout Japan: you can see where the surveys were carried out on the map shown below. We hope you’ll enjoy searching for bats in Japan, and if you have any queries just let us know via the Talk section of the Bat Detective website.

For this month’s blog post, to accompany our Japan data, we’re publishing a short piece written by iBats and Bat Detective’s founder Kate Jones during her 2010 trip to Japan, during which she collected iBats audio data and hosted training workshops for the iBats monitoring program. Scroll down below the map to start reading…

Locations throughout Japan where the data currently hosted on Bat Detective were recorded, during 2010 and 2011.

“I stare slightly queasily down at the tiny but perfectly formed green-tea plantations and rice paddies in the valley far below, as the car winds down the narrow mountain roads of the Mount Fuji highlands in Japan. We stop to try to manoeuvre around an impossibly large truck loaded with locals, and I am struck by the beauty of the mountains surrounding us, lush green forested slopes and azure blue lakes matching the skies overhead.

A crazy expansion of the iBats monitoring program over the past few months has me visiting places and people that I have only imagined. ‘What are you doing in Japan?’ asked the Japanese air attendant politely as I waited for the bathroom on my 11 hr flight to Tokyo. ‘I am hoping to develop a program with local people and scientists to monitor bat populations’, I replied carefully. ‘You see, you can use changes in bat populations like a heart monitor to check the health of nature and the impact of people on the environment’. ‘Bats?’ she squealed, ‘I LOVE bats,’ and proceeded to draw me a map of Japan marked with large crosses where I should visit to see bats. Equally unexpected was finding out at dinner on our first night given by our host, Dai Fukui, that grilled eels are actually very tasty. Especially as tempura with sesame dressing. Yum.

Stuart Parsons, sitting next to me in the car, is looking even more pale than I. Obviously the sake of the previous evening is not going well with Dai’s mountain driving. David Hill on the other hand is made of sterner stuff, alternating between calmly explaining Japanese culture to us in the back and chatting easily to Dai in Japanese in the front. Yesterday was spent exploring caves in the mountains with members of the Japanese bat group (Komori no koui) and listening to horseshoe bats bubbling and warbling over our heads.

In the evening we were introduced to a whole new concept in fieldwork, a ‘mist netting barbecue’. We left the hard work to the local experts and sat around chatting to the group while some of the students brought us mist netted bats for us to look at. The endemic tube-nosed bat (Murina ussuriensis) was especially cute — David’s favorite. Dai explains that he has found this bat hibernating in little tubes it has made in the snow in winter. We drink cold sake and ponder how this bat copes with subzero temperatures. I explain the importance of monitoring to the group and how our acoustic equipment works. Stuart displays the calls in real time from the bats flittering over our heads on his brand new iPad. Whilst outwardly dismissing this gimmick, I am secretly marveling at how Stuart is among the select few in the world that can out-geek me with Apple products.

Although bats are protected in Japan, there is no formalised monitoring of their populations and little general public awareness of the important role bats play in ecosystems. This is a fact that the bat group is trying to change with their awareness-raising activities around Japan every year, culminating in a bat festival in August. The car lurches down the mountain and I see a Bat Conservation Trust sticker on the car in front in our little convoy.

‘Are you a member?’ I ask curiously of the owner, Keiko Osawa, earlier that day, during our lunch overlooking Nagashima Dam. ‘Yes,’ motioning to her husband Yushi, ‘we like getting Bat News’. Yushi is a photographer and they seem to spend most of their time travelling the world photographing fruit bats.

‘Kate San,’ asked the secretary of the bat group Akeiko Mekosa politely. ‘How many members does Bat Conservation Trust have?’. She exclaimed in surprise when I told her over 5000, and said she struggles to get their membership up to 500. The group wants to develop an iBats project here over the next year, and hopes to raise the profile of bats and start its first national monitoring program. Stuart and I are here to help this get started, and to run a workshop on iBats monitoring, volunteer management and acoustic analysis for them.

Ussuri tube-nosed bat, Murina ussuriensis (image reproduced from http://zmmu.msu.ru/bats/rusbats/gallery/pmuss.html)

Beside me in the car, Stuart is beginning to look more normal and is checking his photos on his new iPad just to annoy me. We chat about his plans for his iBats project in New Zealand next summer (our winter). In contrast to Japan’s forty species of bats, New Zealand only has two. Stuart bristles at my dismissive tone and says that what they lack in numbers they make up in distinctiveness.

I have to agree with him for once — New Zealand is home to the short-tailed bat (Mystacina tuberculata) which spends most of its time in the moist fern-filled forests scampering on the ground hunting for fruit and insects. Although the iBats car-based acoustic monitoring would not be useful to monitor Mystacina as they are confined to deep forest, Stuart sees the potential for using iBats to monitor long-tailed bats (Chalinolobus tuberculatus). He has agreed to trial our new iBats application for the iPhone – you just attach your iPhone to an ultrasound detector and send the recording and GPS information straight to the iBats website. We did a test run in the Fijordland of New Zealand’s South Island in February and apart from us being bitten to death alternatively by sand flies and mosquitoes, it worked perfectly. Long-tailed bats happily flew over the car as we made our way along the transect through Lord Of The Rings country. Stuart is excited about using the technique to better understand the distribution of this threatened endemic species.

We head back up into the clouds with the help of the nice Japanese lady satnav to where we are staying tonight and holding the workshop. The workshop venue is a lodge in the highlands with traditional Japanese style rooms, where the bed is made every night from bedding beautifully folded and organised in wooden cupboards with ornate sliding doors. I’m especially excited about the tales of the Japanese bath houses, with their piping hot plunge pools fed from the surrounding hot springs.

The expansion and interest in the iBats project has been rather overwhelming over the last few months and has seen the team giving workshops in Hungary, Ukraine and most recently Russia, where the vodka flowed a little too easily but the welcome and enthusiasm for the project was amazing. I am overwhelmed by the generosity of the people I have met around the world and their commitment to conserve their bats in the face of conditions much more problematic than those we face. As we stop to investigate the first bat house built in Japan, one of the group asks me where next for the iBats project. Hmmmmm, what about Australia?”

The Bat Detective World Tour – join us soon!

Bat Detective has now been running for over three years, and all the input from our community of citizen scientists has been invaluable in helping us to develop machine learning algorithms for detecting bat calls in audio recordings – so thank you! As we explained in our recent post about our current research, adding more annotated data – and from a wider variety of recorded sound environments – will further improve the accuracy and reliability of our bat detector software. This will bring us closer to our goal of creating smart automated tools for monitoring global bat populations, which we hope will in turn help us to learn more about how human activities are affecting the earth’s ecosystems.

So we’re about to take Bat Detective on a World Tour, and we’re asking for your help in searching for bat calls in recordings from across the globe.

Since 2005 the amazing groups of volunteers and researchers on the iBats monitoring programme have been recording audio bat surveys in places ranging from the UK to Japan, North America to sub-Saharan Africa — each with their own distinct environmental soundscapes and unique selection of bat species. So far, however, the audio snapshots we’ve uploaded to Bat Detective have only been those from Eastern Europe. This means we still have lots of new data from all over the world in need of exploring and annotating, all of which will build into improving our automated bat detectors.

All countries where iBats surveys have been carried out to date (data: http://www.ibats.org.uk)

So throughout the World Tour we’ll be travelling from country to country, regularly uploading new sets of audio data from a selection of places where iBats volunteers have surveyed. We’ll begin in the UK, where the Bat Detective team are based, before jetting across the globe to search for bats in countries in Africa, North America, Australia and Asia. And as we go we’ll be adding posts to this blog, reporting on where and when the surveys were recorded, and highlighting some of the local bat species (and other curious sonic inhabitants) you can expect to encounter in each location.

Keep an eye on the Bat Detective blog for dates, news and updates as we progress through the tour. And until our travels start in a few weeks’ time, you can still help us track down bats in our current Eastern European data – visit the Bat Detective site to get searching. Thank you for your contributions over the last three years, and we hope you’ll enjoy helping us to search for bats worldwide!