Edge Based Infant Crying Reason Classifier Hunger

Bottle prepping machines make life just a little bit less hectic for overwhelmed parents, but they do still need to be started and that is easier said than done when you’ve got a wailing baby in your arms. That’s why Manivannan created this edge AI-powered system that automatically starts a bottle prepping machine when it hears a baby crying.

Manivannan designed this project to work with a Tommee Tippee bottle prepping machine, but you should be able to apply the same principles to any machine that starts with a single button or switch. An Arduino Portenta H7 board starts the machine through a DFRobot Gravity relay module. The only hardware modification required is connecting that relay in parallel with the machine’s start button.

The crying baby detection magic happens with a machine learning model running directly on the Arduino — no internet connection or cloud service required. Manivannan trained that model using Edge Impulse by recording both normal room noise and the baby crying. Manivannan’s tutorial explains how to perform the training so the model can recognize the difference between the two.

After training, the Arduino will constantly listen for the sound of the baby crying. When it hears that, it will automatically activate the bottle prepping machine, so formula is ready in a couple of minutes and you can concentrate on soothing the baby.

BABL leverages tinyML to distinguish a baby's cry from other noise, preventing false alarms, and alerting parents only when needed.

With Edge Impulse's recent announcement of support for the popular Arduino Nano 33 BLE Sense , the exciting world of Machine Learning is now easily accessible to even novice Arduino developers . To demonstrate how quick and easy it is to get started and create results with Edge Impulse Studio , I wanted to target a conventional application that could be dramatically improved with machine learning . Baby monitors work by transmitting audio wirelessly from a transmitter in the baby's room to a receiver which the parent can monitor. The problem is that any sound - whether it's a baby crying out with hunger - or a baby playfully babbling as it falls back asleep - produces the same results for the overtired parent: disruption of their precious few moments of sleep . By leveraging machine learning, the system can recognize crying from other noises , and behave accordingly. In this first project, we will see just how quickly such advanced functionality can be achieved with tinyML and Edge Impulse .

Edge Impulse has fantastic documentation , which I highly recommend you go back and review in detail as time allows, but for now, our goal is to get up and running and monitoring our babies as quickly as possible, so here is my CliffsNotes version; I'm using Ubuntu 20.04 but if you run into problems on a different version or platform, let me know in the comments and I'll be more than happy to help you out.

First, you must install the Arduino CLI - if you are not familiar, this glorious piece of software brings the functionality of the Arduino IDE that we all know and love to the command line - meaning you can automate things, build and deploy from other IDEs and editors, and so on. And the tools from Edge Impulse require it in order to work properly, so follow the installation guide , or (on Linux):

If, after this, you are presented with a list of available commands, then you are all set. If not, feel free to drop your error into the comments below and I'll gladly help you get it sorted.

OK, onto the Edge Impulse CLI ! These tools are distributed as an npm package , and require Node.js 10 or higher. If you're not familiar with Node, npm packages, and Node version management, I highly recommend you check out nvm , which lets you easily switch between the Node version required for your project. Here's how I used nvm to install the EI CLI with the latest stable version of Node:

If this results in usage instructions, you are all set! If you run into problems, check out the troubleshooting section in the docs, or drop a comment below and I'll be more than happy to help figure things out with you!

With all of the CLI tools sorted, it's time to get the Arduino itself set up! On Linux, connect your Nano via micro-USB, press reset twice to enter bootloader mode , and grab and install the latest firmware as follows:

I ran into a problem with flash_linux.sh and had to hand-edit it for it to work successfully, but that was a beta version of the software, and I'm told it's been fixed since. If all went well, you should receive a message saying the firmware was flashed and it's time to start the daemon. Press reset one last time , but don't actually start the daemon yet. One thing that the docs never seem to mention anywhere is the (admittedly obvious, but worthy of mention) requirement to create and Edge Impulse account and project . If you haven't already done this, then the config steps of the daemon won't actually make sense. Once you've created an account and project , go ahead and start the daemon:

go back to your project in the Studio , click Devices , and you should see your device listed and connected. At this point, setup is complete, and it's time to move onto the fun stuff! 🚀

With your device connected, you can use Edge Impulse Studio's Data acquisition functionality to interactively gather audio data in real time. In the Record new data section of this page, ensure your device is selected, enter noise for Label , set Sample length to 16000 ms (16 seconds - the maximum possible with the Arduino Nano 33 BLE Sense's available memory), select Built-in microphone for Sensor , and set Frequency to 16000 Hz. We need around 5 minutes of background noise data, so take your Nano and laptop to your baby's room, press Start sampling , and repeat around 20 times until you have ~5 minutes of data for your noise class. There's a handy Data collected tally at the top of the page to help you keep track, and you can click on an individual sample to see its waveform or play back its audio. By definition, our background noise samples aren't going to be very interesting:

Now repeat this process for your crying class - just update the Label , and keep everything else the same. My baby was asleep while I was working on this project, so I found some baby crying sound effects . In order to ensure I had some unique data to test with, I made sure to not use all of the available samples when building my dataset. Once you're up to 10 minutes total, (5 for each class) it's time to create what EI call an "Impulse".

On the left nav, select Impulse design , and you will be presented with the "blocks" interface, the first of which is pre-filled for us with Time series data . The audio samples will be analyzed in Windows , and we want to determine if our baby is crying within one second of data, so we set Window size to 1000 ms. Instead of only looking at each one second sample once, we will set the Window increase to 300 ms in order to extract several overlapping blocks from each second of audio. For our Processing block , click and add Audio ( MFCC ) , and then Neural Network (Keras) for our Learning block , and click Save Impulse .

The Impulse design section on the left nav will now display some new options, with grey bullets next to them, indicating that they are still to do. Click on MFCC and configure your parameters as follows (should be the default):

Next, from the Generate features tab, click the Generate features button. When the process is complete, you'll get a nice graph on the Feature explorer which allows you to visualize your dataset; there's actually quite a lot of overlap in ours - perhaps because the audio between actual cries is extremely similar to our noise audio.

With our features generated, you'll see NN Classifier is the last grey bullet in our process, so click on that in the right nav, and set:

and leave the Neural network architecture as-is. Click Start training and wait for the job to complete.

86.3% accuracy sounds good, but let's just give it a try for ourselves with the Live classification tool in the left nav. With your device still connected, select it from the Device dropdown, set Sample length to 5000 ms, and select Built-in microphone for Sensor , and set Frequency to 16000 Hz as before. Click Start sampling , and play one of the unused baby crying samples. Review the results to confirm that the classification is accurate. You can also use the Classify existing test sample functionality to work with existing data. Our results are looking pretty good here: