Research update: developing new bat monitoring tools with Bat Detective data

Hello from the Bat Detective team! It’s been a busy summer at Bat Detective HQ after an amazing spring with British Science Week and the World Tour. So we’ve been a bit quiet on the blog in the last couple of months while we’ve been working on updating our automated tools and road-testing them on new data. We’re now coming close to having our results ready for scientific publication, and we’ve also had the chance to put our new software tools into practice on analysing some brand new bat survey data. So over the next two blog posts, we’ll be updating you on our progress, explaining where the Bat Detective project is at right now, and showing how we’ve been using all the data you’ve helped us to label.

In this first post we’ll discuss how we’ve used Bat Detective data to improve our automated bat call detection tools, and highlight some of the challenges we’ve encountered along the way. In the next post, we’ll show some examples of where we’ve been testing out our software tools on bat survey data from the UK and Madeira – keep an eye on the blog for that one very soon.

The team have also been up to a few other things in the last few months. Bat Detective’s Rory Gibb (me) gave a project update talk at the Zooniverse’s first ever ecology workshop, where there was some fascinating discussion about how citizen scientists can become increasingly involved in some of the major challenges facing ecology and conservation in future. We’ve also just had a big article on Bat Detective and its sister project iBats published in the latest citizen science-themed issue of Environmental SCIENTIST – the article will be available to read online in the near future, so we’ll share it here when that happens.

Training machines to recognise bat calls: why and how?

In our last research update post a year ago, we explained how advances in machine learning technology have enabled us to train algorithms to automatically recognise bat calls in ultrasonic survey recordings. This is important because newer bat detectors can be deployed in the field for weeks or months, collecting so much audio data that it’s almost impossible to analyse them manually. By making it possible for bat researchers to quickly and reliably find where bat echolocation calls are in these recordings, automated tools are creating exciting new opportunities to study bat ecology, behaviour and conservation at much larger scales than ever before.

Machine learning involves training computer algorithms to automatically recognise bat echolocation calls in recordings, by showing the computer thousands of examples of what they look and sound like. In our last research update we showed how training the algorithms on increasingly large amounts of data from Bat Detective improves their performance. For that reason, and also to include a greater diversity of bat sounds from around the globe, we’ve asked for your help in labelling our World Tour data over the last year. And thanks also to the efforts put in by volunteers during British Science Week, we’ve now got thousands of new bat call annotations to incorporate into our detector tools – so one of our current challenges is exploring the best ways to use all of these new data.

We’ve now got the detector algorithms up and running, and we’re currently testing them out to assess how well they perform. The figure below shows an example of the detector in action on a snippet of audio data from the iBats global bat monitoring programme. The recording is displayed as a spectrogram underneath, with sounds showing up as bright markings. The graph above shows the computer predicting where it thinks the bat calls are – each vertical red line shows where the computer predicts there is a bat call. The height of the red lines tell us how certain the computer is about its predictions, where higher indicates more confident. The green bars show where a human expert has confirmed that bat calls are present – so in this example, the computer has successfully recognised all the bat calls.

Are you sure that’s a bat? The problem of false positives

However, there are still some errors where the computer thinks there is a bat call, when there actually isn’t one (a ‘false positive’). This is a problem for monitoring bat populations, because too many false positives could result in researchers overestimating the true number of bats in an area, which could for example have an impact on conservation efforts. You can see a clear example of these errors in the next figure below, where the computer falsely predicts that the mechanical noises at the bottom of the spectrogram are lots of bat calls.

So to improve this, we’ve also been including non-bat sounds from Bat Detective – those insect calls and mechanical noises you’ve also helped us to find. By training the algorithms to also recognise what bat calls don’t look like, we can significantly improve their accuracy. The image below shows the difference: it’s the same audio clip, but there are now far fewer false positives (red lines).

This is a great example of the importance of testing out these tools on new data from a variety of times, places and detector types. This helps us get a better idea of where they’re under-performing, and how they can be improved before we release them as open-source tools for other researchers to use. So with that in mind, keep an eye on the Bat Detective blog next week for our next research update: we’ll be showing some examples of where we’ve road-tested them on new bat survey data recorded during this summer. We’ll also be uploading a new set of data from Russia – one of our last few World Tour stops – so stay tuned for that.

And a huge thanks again for all your efforts with labeling the data on Bat Detective, both during this year and throughout the whole project – we wouldn’t have been able to get to this stage without your input, and it’s really exciting to see the work of our community of volunteers starting to produce results.

Advertisement

Bat Detective New Zealand: our latest World Tour location

Welcome to New Zealand, the latest stop on the Bat Detective World Tour! As of today we’ve just uploaded a new set of audio data to Bat Detective, recorded along survey transects on New Zealand’s South Island. You can see the locations of the surveys on the map below, and visit the Bat Detective site now to get searching for bats.

Prior to this we’ve spent the last month hosting audio data from iBats Mexico, which was neatly timed to coincide with the publication of the latest automated bat call classifier from members of our research group – a classifier for Mexican bat species. As with our results from the algorithms we’re training with Bat Detective data, it’s another example of how advances in machine learning technology are increasingly enabling the development of tools and systems for effective acoustic monitoring of bats (as well as biodiversity more broadly). You can find out more about the Mexican classification tool and how it will assist in bat population monitoring via some great coverage in the media, including in Science and an interview with our group’s Dr. Veronica Zamora-Gutierrez and Prof. Kate Jones on the BBC.

Locations on South Island, New Zealand, where iBats data were recorded

Bats occupy a unique space in the ecology of New Zealand, since they are the country’s only endemic terrestrial mammals – before humans settled the islands, the only mammals native to New Zealand were three bat species (the greater short-tailed bat, lesser short-tailed bat and long-tailed bat) and several species of marine mammal. Since human settlement this has changed, with invasive mammalian predators (such as rats and cats) driving massive declines in the populations of endemic birds and bats. Indeed, the last sighting of the greater short-tailed bat was in 1967, and it is now believed to be extinct, while New Zealand’s other two bat species, the lesser short-tailed (pictured below) and long-tailed bat, have both experienced major declines and are priorities for conservation.

The acoustic data on Bat Detective New Zealand, recorded on South Island in 2010, are much noisier than lots of the recordings you’ll have previously heard on Bat Detective. Many clips have a great deal of background noise and static, in addition to distinctive bats and unique rattling insect calls. Although this can make it challenging to determine what sounds you’re hearing, it’s very useful to include data like these while training algorithms to automatically find bat calls – this will help improve the algorithms’ ability to detect bat echolocation calls in even the most noisy of real-world acoustic recordings. This will make them more useful for surveying bats in naturally noisy and complex acoustic environments, such as urban areas where there is lots of human-generated sound, or highly biodiverse (and therefore very loud) rainforests.

We hope you’ll enjoy helping us search for bats in our New Zealand data, and as ever if you’re struggling to figure out whether a sound is a bat, an insect, or something else, you can use the Talk page to flag it up and discuss it with us and other users.

Lesser short-tailed bat (photo via New Zealand Department of Conservation)

 

The Bat Detective World Tour – join us soon!

Bat Detective has now been running for over three years, and all the input from our community of citizen scientists has been invaluable in helping us to develop machine learning algorithms for detecting bat calls in audio recordings – so thank you! As we explained in our recent post about our current research, adding more annotated data – and from a wider variety of recorded sound environments – will further improve the accuracy and reliability of our bat detector software. This will bring us closer to our goal of creating smart automated tools for monitoring global bat populations, which we hope will in turn help us to learn more about how human activities are affecting the earth’s ecosystems.

So we’re about to take Bat Detective on a World Tour, and we’re asking for your help in searching for bat calls in recordings from across the globe.

Since 2005 the amazing groups of volunteers and researchers on the iBats monitoring programme have been recording audio bat surveys in places ranging from the UK to Japan, North America to sub-Saharan Africa — each with their own distinct environmental soundscapes and unique selection of bat species. So far, however, the audio snapshots we’ve uploaded to Bat Detective have only been those from Eastern Europe. This means we still have lots of new data from all over the world in need of exploring and annotating, all of which will build into improving our automated bat detectors.

All countries where iBats surveys have been carried out to date (data: http://www.ibats.org.uk)

So throughout the World Tour we’ll be travelling from country to country, regularly uploading new sets of audio data from a selection of places where iBats volunteers have surveyed. We’ll begin in the UK, where the Bat Detective team are based, before jetting across the globe to search for bats in countries in Africa, North America, Australia and Asia. And as we go we’ll be adding posts to this blog, reporting on where and when the surveys were recorded, and highlighting some of the local bat species (and other curious sonic inhabitants) you can expect to encounter in each location.

Keep an eye on the Bat Detective blog for dates, news and updates as we progress through the tour. And until our travels start in a few weeks’ time, you can still help us track down bats in our current Eastern European data – visit the Bat Detective site to get searching. Thank you for your contributions over the last three years, and we hope you’ll enjoy helping us to search for bats worldwide!

Bat Detective research update – and stay tuned!

This week it’s our birthday! It’s been exactly three years since we first launched the Bat Detective project on 1st October 2012. Since then we’ve had an amazing response from our community of citizen scientist bat detectives, with over 94,000 unique audio snapshots explored by nearly 4,000 volunteers, and more than 11,000 bat calls discovered.

All the hard work you’ve put in so far has been invaluable. Using the data from Bat Detective, we’ve been developing computer algorithms that can automatically search for and detect bat calls in audio recordings with a very good success rate. To do this we’ve taken advantage of recent rapid improvements in machine learning technology for recognising complex patterns within data — such as the distinctive shapes of bat calls.

We’ve had great results so far, thanks to all the audio data the bat detective community has searched through, and all the calls you’ve identified. The majority of those have been searching calls (over 7,000), but you’ve also labelled over 2,000 each of the more rarely recorded social and feeding calls. We’ve used this annotated data to train our machine learning algorithms, by showing them thousands of examples of what bat calls look and sound like. This enables them to better tell apart the sounds we’re interested in from other background sound, such as insect calls and mechanical noise.

We’re now at the stage where we can use these algorithms to detect bat calls throughout the millions of recordings collected through the iBats monitoring project. What this means is that we’re a key step closer to developing automated software for accurately detecting and species-identifying bat calls from recorded audio — a vital move towards a global monitoring programme for bat populations. To read more in-depth summaries of the work our team members have been doing towards that goal, see our recent blog post for Methods In Ecology & Evolution.

This graph shows how well our algorithms are currently performing at finding known bat calls within a large set of audio data that we’ve already annotated. The closer the curve reaches to the top right of the graph, the better the results we’re getting — this means we’re maximising the proportion of the bat calls detected within the audio (increasing the recall) while minimising the number of non-bat sounds that are incorrectly classified as bat calls (improving the precision). When we use four times as much data from Bat Detective to train the algorithms (shown as a green line), we get a large improvement in performance compared to when we use much smaller amounts of data (shown as the blue and purple lines).

So the more data we can use to train our algorithms, the more accurate and reliable they will be. This will allow them to more successfully detect even calls recorded in challenging acoustic conditions, when there’s lots of background noise or the bats are far away from the detector — those trickier cases where they’re failing now. That’s why the ongoing help from the bat detective community is so valuable for our research. So later this month we’ll be announcing some new developments in the Bat Detective project, where you can help us search for bat calls in recordings from all around the globe — stay tuned for more information very soon!