Research update: road-testing our bat call detection software
Last week we published our latest Bat Detective research update, explaining how we’ve used the data you’ve labelled to improve our algorithms for detecting bat calls in audio data. The next step is to assess how well they perform in real-world bat survey situations – and the best way to do this is to put them into practice on new data. So following last week’s update, this post will focus on examples of where we’ve road-tested our new tools this year.
The first was a two-week garden bat survey we carried out over two weeks in July, in a row of suburban gardens in southeast England. We were testing our algorithms on data collected by two different bat detector types, both of which were deployed outside to collect data autonomously over multiple nights. Alongside some specialised full-spectrum ultrasonic detectors, which are commonly used in acoustic wildlife monitoring research and citizen science projects including the Norfolk Bat Survey, we were also also using some brand new, low-cost acoustic sensors developed by our collaborators in Oxford for a variety of biodiversity monitoring applications (including citizen science). This offered a great chance to test our call detection tools on data from two different types of acoustic sensor.
The videos below show examples of how the algorithms are used on the UK garden survey data. The first shows the distinctive calls of a common pipistrelle passing by the detector, and the second are calls from a noctule, the UK’s biggest bat. Each red line on the spectrogram shows where the algorithms predict a bat call – so you can see in these cases they’re performing well, successfully detecting every echolocation call in these recordings. For each predicted call, the algorithms also calculate a probability of detection – a measure of how likely the sound is to be a bat call, with reference to our training data from Bat Detective. This enables us to set a threshold for call detection – by setting a high probability threshold, this then makes it possible to only detect sounds that are very likely to be a bat call. As we discussed in the last post, this can help to reduce the risk of false positive detections (i.e. falsely thinking there’s a bat call, when actually there isn’t one).
Once the calls have been detected, the next step in the analysis is to classify those calls to species – what bat is that? Doing this requires a second set of algorithms, trained to distinguish between the calls of different species. Our group are currently working on developing these, and incorporating them into our tools, in order to more fully automate the analysis process (and therefore make it faster and more reliable for researchers monitoring bats).
As well as the UK data we collected in July, we’re also now using our call detection tools to analyse recordings collected during a huge bat survey on the Atlantic island of Madeira. There are thought to be three bat species on Madeira, including the endemic Madeira pipistrelle (Pipistrellus maderensis), which is listed as vulnerable on the IUCN Red List. However, little is known about the distribution and abundance of bats on the island, and its relatively small size makes it an ideal study system for an island-wide bat survey.
So a member of our research group (in collaboration with M-ITI in Madeira), has been busy out in the field throughout August and September, deploying full-spectrum bat detectors in locations across the island, which were selected to provide a randomised sample of its full range of habitats and altitudes. A map of the sample sites is shown below, with each blue marker showing where bat detectors have been placed. Now the data have all been collected, we’re starting to use our automated tools to detect bat calls in all the recordings. From there we can start to ask questions about the distribution of bats on the island, and to assess what habitats and locations might be particularly important for conservation.
This is a great example of how the tools we’ve developed using the Bat Detective data can now be applied to understand bat ecology and assist in conservation efforts. Without tools like these, the sheer quantity of audio data collected during a summer-long survey at this level of detail – which clocks up to hundreds of hours of survey-time in total – would be almost impossible to analyse by hand. Keep an eye on the Bat Detective blog in the coming months, as we’ll keep you informed on the last few steps in developing our bat call detector tools for open-source release, as well as letting you know about this and other test-case projects.
Bat Detective New Zealand: our latest World Tour location
Welcome to New Zealand, the latest stop on the Bat Detective World Tour! As of today we’ve just uploaded a new set of audio data to Bat Detective, recorded along survey transects on New Zealand’s South Island. You can see the locations of the surveys on the map below, and visit the Bat Detective site now to get searching for bats.
Prior to this we’ve spent the last month hosting audio data from iBats Mexico, which was neatly timed to coincide with the publication of the latest automated bat call classifier from members of our research group – a classifier for Mexican bat species. As with our results from the algorithms we’re training with Bat Detective data, it’s another example of how advances in machine learning technology are increasingly enabling the development of tools and systems for effective acoustic monitoring of bats (as well as biodiversity more broadly). You can find out more about the Mexican classification tool and how it will assist in bat population monitoring via some great coverage in the media, including in Science and an interview with our group’s Dr. Veronica Zamora-Gutierrez and Prof. Kate Jones on the BBC.
Locations on South Island, New Zealand, where iBats data were recorded
Bats occupy a unique space in the ecology of New Zealand, since they are the country’s only endemic terrestrial mammals – before humans settled the islands, the only mammals native to New Zealand were three bat species (the greater short-tailed bat, lesser short-tailed bat and long-tailed bat) and several species of marine mammal. Since human settlement this has changed, with invasive mammalian predators (such as rats and cats) driving massive declines in the populations of endemic birds and bats. Indeed, the last sighting of the greater short-tailed bat was in 1967, and it is now believed to be extinct, while New Zealand’s other two bat species, the lesser short-tailed (pictured below) and long-tailed bat, have both experienced major declines and are priorities for conservation.
The acoustic data on Bat Detective New Zealand, recorded on South Island in 2010, are much noisier than lots of the recordings you’ll have previously heard on Bat Detective. Many clips have a great deal of background noise and static, in addition to distinctive bats and unique rattling insect calls. Although this can make it challenging to determine what sounds you’re hearing, it’s very useful to include data like these while training algorithms to automatically find bat calls – this will help improve the algorithms’ ability to detect bat echolocation calls in even the most noisy of real-world acoustic recordings. This will make them more useful for surveying bats in naturally noisy and complex acoustic environments, such as urban areas where there is lots of human-generated sound, or highly biodiverse (and therefore very loud) rainforests.
We hope you’ll enjoy helping us search for bats in our New Zealand data, and as ever if you’re struggling to figure out whether a sound is a bat, an insect, or something else, you can use the Talk page to flag it up and discuss it with us and other users.
Lesser short-tailed bat (photo via New Zealand Department of Conservation)
Bat Detective World Tour begins next week!
As we announced recently, Bat Detective is about to go on a World Tour, and we’re inviting you to join us, starting this coming Monday 2nd November. The iBats monitoring programme, which provides us with our audio data, has now been running for a decade, with our volunteers collecting recordings of bat surveys in locations worldwide. However, to date Bat Detective has made only some of that recorded audio available for our citizen scientists to explore, and that has come mainly from Eastern Europe.
So starting on 2nd November, over the course of the World Tour we’ll be regularly uploading new sets of data to Bat Detective from different countries across the globe. Each country – ranging from Europe to places in Africa, the Americas and Asia – has its own selection of bat species alongside other acoustic inhabitants, so you can expect to encounter a variety of different soundscapes while searching for bat calls worldwide. Your help with classifying bat calls, insect noises and other sounds in these places will be of valuable assistance in our work towards creating automated bat detectors – read more about our research here.
On Monday we’ll begin our trip in the United Kingdom, home of the Bat Detective team and the Bat Conservation Trust. Keep an eye on the Bat Detective blog for a more in-depth post about British bats, citizen science, and what you might hear while exploring our UK data. We hope you’ll enjoy joining us in searching for bats across the globe, so stay tuned and see you next week…
photo via Wikimedia
The Bat Detective World Tour – join us soon!
Bat Detective has now been running for over three years, and all the input from our community of citizen scientists has been invaluable in helping us to develop machine learning algorithms for detecting bat calls in audio recordings – so thank you! As we explained in our recent post about our current research, adding more annotated data – and from a wider variety of recorded sound environments – will further improve the accuracy and reliability of our bat detector software. This will bring us closer to our goal of creating smart automated tools for monitoring global bat populations, which we hope will in turn help us to learn more about how human activities are affecting the earth’s ecosystems.
So we’re about to take Bat Detective on a World Tour, and we’re asking for your help in searching for bat calls in recordings from across the globe.
Since 2005 the amazing groups of volunteers and researchers on the iBats monitoring programme have been recording audio bat surveys in places ranging from the UK to Japan, North America to sub-Saharan Africa — each with their own distinct environmental soundscapes and unique selection of bat species. So far, however, the audio snapshots we’ve uploaded to Bat Detective have only been those from Eastern Europe. This means we still have lots of new data from all over the world in need of exploring and annotating, all of which will build into improving our automated bat detectors.
All countries where iBats surveys have been carried out to date (data: http://www.ibats.org.uk)
So throughout the World Tour we’ll be travelling from country to country, regularly uploading new sets of audio data from a selection of places where iBats volunteers have surveyed. We’ll begin in the UK, where the Bat Detective team are based, before jetting across the globe to search for bats in countries in Africa, North America, Australia and Asia. And as we go we’ll be adding posts to this blog, reporting on where and when the surveys were recorded, and highlighting some of the local bat species (and other curious sonic inhabitants) you can expect to encounter in each location.
Keep an eye on the Bat Detective blog for dates, news and updates as we progress through the tour. And until our travels start in a few weeks’ time, you can still help us track down bats in our current Eastern European data – visit the Bat Detective site to get searching. Thank you for your contributions over the last three years, and we hope you’ll enjoy helping us to search for bats worldwide!
Bat Detective research update – and stay tuned!
This week it’s our birthday! It’s been exactly three years since we first launched the Bat Detective project on 1st October 2012. Since then we’ve had an amazing response from our community of citizen scientist bat detectives, with over 94,000 unique audio snapshots explored by nearly 4,000 volunteers, and more than 11,000 bat calls discovered.
All the hard work you’ve put in so far has been invaluable. Using the data from Bat Detective, we’ve been developing computer algorithms that can automatically search for and detect bat calls in audio recordings with a very good success rate. To do this we’ve taken advantage of recent rapid improvements in machine learning technology for recognising complex patterns within data — such as the distinctive shapes of bat calls.
We’ve had great results so far, thanks to all the audio data the bat detective community has searched through, and all the calls you’ve identified. The majority of those have been searching calls (over 7,000), but you’ve also labelled over 2,000 each of the more rarely recorded social and feeding calls. We’ve used this annotated data to train our machine learning algorithms, by showing them thousands of examples of what bat calls look and sound like. This enables them to better tell apart the sounds we’re interested in from other background sound, such as insect calls and mechanical noise.
We’re now at the stage where we can use these algorithms to detect bat calls throughout the millions of recordings collected through the iBats monitoring project. What this means is that we’re a key step closer to developing automated software for accurately detecting and species-identifying bat calls from recorded audio — a vital move towards a global monitoring programme for bat populations. To read more in-depth summaries of the work our team members have been doing towards that goal, see our recent blog post for Methods In Ecology & Evolution.
This graph shows how well our algorithms are currently performing at finding known bat calls within a large set of audio data that we’ve already annotated. The closer the curve reaches to the top right of the graph, the better the results we’re getting — this means we’re maximising the proportion of the bat calls detected within the audio (increasing the recall) while minimising the number of non-bat sounds that are incorrectly classified as bat calls (improving the precision). When we use four times as much data from Bat Detective to train the algorithms (shown as a green line), we get a large improvement in performance compared to when we use much smaller amounts of data (shown as the blue and purple lines).
So the more data we can use to train our algorithms, the more accurate and reliable they will be. This will allow them to more successfully detect even calls recorded in challenging acoustic conditions, when there’s lots of background noise or the bats are far away from the detector — those trickier cases where they’re failing now. That’s why the ongoing help from the bat detective community is so valuable for our research. So later this month we’ll be announcing some new developments in the Bat Detective project, where you can help us search for bat calls in recordings from all around the globe — stay tuned for more information very soon!
FAQs & Feedback
By Kate E. Jones and Kim Mroz
The Bat Detective project has passed its one-month anniversary! Since the launch, everyone who’s gotten involved has been fantastic. We’ve also had quite a lot of press coverage including that on the BBC News website; interviews on BBC Radio 4’s Material World Programme, Irish Radio, BBC World News; and a podcast for The Guardian.
Since our launch (up until Halloween), we’ve had 671 of you register with the site to explore the data our iBats volunteers recorded from Bulgaria. In total you helped us look at 18,729 snapshots and you provided 73,421 classifications – amazing! You found 12,653 bat calls in our Bulgarian data and also 22,275 insect calls and 23,003 machine sounds. We expected that a lot of the bat calls found would those ‘searching’ calls made when a bat is navigating around, and that is indeed the case (8,821 calls or 70% of the total). However the rest of them were classified as feeding calls (1,591) and a really impressive 2,239 calls were social. This is really exciting as comparatively little is known about social calls and this is first time that they have been recorded over such a large area.
We have been really amazed by how much support you have given us and many of you have done thousands of classifications each, for example one person provided over 9,000 classifications! On Halloween we released the Romanian data which of course includes those recordings made in Transylvania (home of Bram Stoker’s Dracula)! Despite what you may have heard about the legend of Dracula, you will not hear any vampire bats (bats that feed on blood) in the recordings. In fact there are only 3 species of ‘vampire’ bats out of over 1200 bat species in total with only one species known to associated with humans and they all live in South America. However, some exciting species that you might hear in the recordings include the Schreiber’s bent-wing bat (Miniopterus schreibersii), which is one of most threatened species in Europe and the European free-tailed bat (Tadarida teniotis) which might possibly be one of the smelliest!
A huge thank you for those of you that agreed to become moderators on the Bat Detective talk and helped field all the questions! There is a constant demand for more information, and you have given us lots of useful feedback about the site. We’ve noticed particular questions about the recordings (and the bats within them) keep cropping up and we have answered some of the most commonly asked questions below, which we would be happy to hear your thoughts about! That’s it for now, watch out for the Transylvanian report coming in a few weeks!
FAQs:
- How can we hear the high frequencies?
Rob covered in his blog how the volunteers used ultrasound microphones to record the sounds beyond our hearing. These are then played back at a much slower speed to be recorded to the hard-drive, which is what we’re classifying here on BatDetective.org… So 0.3 seconds of sound beyond our feeble hearing becomes 3 seconds of audible calls!
. - Why can I hear frogs, birds and tigers, but no button to tag them?
The recordings were all made across Europe shortly after sunset, and have been time-expanded (see FAQ #1). This means that it’s very unlikely you’re actually hearing tigers or birds, and if you were then they wouldn’t sound like you’d expect! Instead, we have chirpy bats and roaring insects, whose calls have been lowered in pitch and lengthened in time. So if you hear a noise that sounds familiar, you have to think about whether it would sound as familiar if it was 10 times higher and quicker!
. - More examples? I’m not seeing many bats!
The examples we’ve included in the field guide are particularly loud, clean sounds to help with your classifications. However, there isn’t enough room to include all variations of bat, insect and machine sounds! It could be that you’re over-looking the different, slightly obscured or quiet calls … you might like to check out the collections that users have made for more (possibly noisier) examples of interesting sounds!
. - Is this a bat or an insect?
Fig. 1: Bat or Insect? … Insect!
There are certain insects sounds (see Fig. 1) that masquerade as bat feeding calls (scroll down in the link) in the spectrograms, and there’s a general rule to help with these. Does the sound have a clicky or a chirpy quality? If it’s a clicky sound, then this is almost certainly an insect!
- If you’re only interested in bats, should I bother with labelling insects and machine sounds?
Yes, please! We’re hoping to branch out into insects too; it would be a shame to waste all of this fantastic data. On the other hand, flagging machine noise can help tremendously with data quality control.
. - What happens if I realise I’ve made mistakes? No back button?!
It’s okay! Don’t worry! We obviously appreciate the care you’re taking with the tagging, but if you happen to miss something or you later disagree with your own tag then in the long-run it shouldn’t matter. Hopefully other users will spot these things too and everyone’s opinion will balance out to good classifications!
. - What should I do with harmonics?
Most naturally-made sound will contain more than one harmonic. Bat calls and insect sounds are no different!
Fig. 2: Harmonics contain information too!
Sometimes these harmonics are very quiet, and may not even reach the microphone at all (see Tim’s blog) but if there is a nice distinct harmonic (e.g., Fig. 2) then there’s no harm in marking it. Feel free to make a separate frequency range to let us know about harmonics.
. - What are these odd un-cut recordings?
Rob’s blog discusses how the processing of an evening’s recordings is meant to work, but sometimes our automated methods break down. Instead of only getting lovely 3 second snippets, there are also a few crazy recordings floating around … you can see some examples here! Don’t worry about these; just ask for another sound (sorry!).
. - Audio doesn’t work … help?
The site currently relies on Flash to play the audio recordings. If you’re having problems while using a major browser (e.g., Firefox, IE, Chrome) you may need to update your browser or Flash install to use the latest version. If you’re still having problems, then please let us know on the Help boards.
. - I’m not sure about something, what should I do?
Ask us! If you have a general topic in mind, you can start a discussion on the Science, Chat or Help boards or see if someone else beat you to it. If a particular sound has you stumped then Talk about it! The science team, moderators and other users may be just as stumped as you, but (hopefully) they might just have an answer!
Feedback:
We’re very appreciative of all of the feedback you’ve provided, which mostly concerns the mechanics of the Bat Detective site. We’re now at the stage of having a think on what changes we can make to allow you all to help us more easily.
- Improve the tutorial and guide terminology … and the tutorial itself!
The tutorial is at the top of our list of improvements. Hopefully a dedicated page containing a tutorial video, call guide and site-guide will make it easier for new and old users alike to find answers to their questions, and feel more confident while helping Bat Detective. Watch this space!
. - Add frequency scales on talk images.
This is also something that we’re very keen to add ourselves. Discussing recordings in Talk will certainly be a lot easier with some reference to frequencies!
. - Confirmation button after selecting ‘Next Sound’.
Several users have noted how other Zooniverse projects (e.g., MoonZoo) have some form of confirmation request when the ‘Next Sound’ button is hit while classifying. This would prevent slightly over-enthusiastic clicking leading to missed recordings, and is definitely something that’s possible.
. - ‘Did you spot this?’ sounds.
So you can check how well you’re spotting the bats and insects, it has been suggested that every now and then we could give you an already classified sound to look at, and then point out the calls to see if we agree. Another great idea that we can filch (thanks Planet Hunters)!
Welcome to Bat Detective!
Bat Detective is an online citizen science project where the public can help to monitor bats across Europe and track changes in the environment by listening to the weirdly wonderful ultrasonic tweets of bats.
Bat Detective project allows visitors to take part in conservation by listening out for bat tweets in recordings collected over 80,000 km of roads across Europe by thousands of volunteers from the iBats program, including bat recordings from the heart of Transylvania.
By sorting the sounds in the recordings into insect and bat calls, bat detectives will help the Bat Detective team learn how to reliably distinguish bat tweets to develop new automatic identification tools.
Bats use lots of different types of sounds, from singing to each other to find a mate, to using the echoes from their tweets to find their way around. Usually bat sounds are inaudible to humans as they are too high for us to hear, but special ‘time expansion’ ultrasonic detectors convert these sounds to a lower frequency, and visitors to Bat Detective can listen to these unique recordings and help the Bat Detective team distinguish different sounds.
One out of every five species of bats is threatened with extinction and better automatic identification tools are desperately needed to quickly process vast amounts of sound data collected by volunteers from bat monitoring programs who survey bat populations each year.
Bats are found all over the world from local parks to pristine rainforests and monitoring their population trends provides an important indicator of healthy ecosystems. Developing new tools that allow the Bat Detective team to interpret population trends from sound will allow bats’ tweets to act as a way to track environmental change.
Bat Detective has been developed by the science team at University College London and Bat Conservation Trust with the development team at Citizen Science Alliance, which runs Zooniverse.org with funding from the Alfred P. Sloan foundation.
Coming Soon…
Holy Citizen Science! Watch this space.
Bat Detective 
