A construction and development consultancy company was exploring a potential solution to track worker locations, tools in use, and the stage of the building project on construction sites in order to enhance project efficiency and safety amongst its workforce.
They reached out to The Curve to find ways in which this could be done with minimal costs whilst ensuring safety on site was unaffected.
The Problem
The project’s premise was to research and design ways in which cost-effective IoT devices (single-board computers) could be used on a construction site to identify the types of tools and machinery used during a specific time. Being able to identify the tools and machinery in use would inform project managers of the stage of the building project they were currently at, for instance, if the devices recognised that there was a large number of individuals using angle grinders on metal in certain rooms and locations, then the assumption could be made that they were working on fire suppression piping.
One of the key challenges to this project was training a Machine Learning model using enough diverse sound samples to allow it to identify the type of equipment being used throughout a relatively loud and harsh environment. This was a particularly challenging aspect of the project as it wasn’t feasible to be on an active construction site regularly to record the audio sources of the equipment which would then be used to train the model to better identify the tools in use.
How We Helped
The Approach
To gather and train the machine learning model with accurate audio sources of the different tools that are in use, various sources for the audio were used, including open source data sets, google sound libraries as well as platforms such as YouTube. And whilst some of these open source datasets had issues due to their quality – when supplemented with high quality sources they performed much better and made the ML model much more effective.
When the audio of tools or equipment is identified and processed by the device, this information is transmitted wirelessly to an access point within the building. This is then saved to a database that can process the audio information further which helps remove the need for the devices to be able to process high levels of data which would not allow the devices to run on a low-power setup.
Utilising machine learning correctly within this project was important to ensure that the model and devices could be trained to correctly identify the tools and equipment in use. The machine learning system used for this project was the Edge Impulse platform due to its ability to build datasets, train models and optimise libraries of data that could be run directly on the devices. Using Edge Impulse as the machine learning platform also allowed pre-written code to be used alongside custom code so that it could be shaped and utilised in a way that best suited this particular project.
The Outcome
Whilst this project mainly focused on the concept in which IoT and Machine Learning devices could be used to detect audio, identify the source and tools, and then report into a dashboard, it did highlight some potential future use cases. For instance, where this was originally intended to act as a progress and schedule reporting tool, it soon became clear that with further development and refining, the product could also be utilised as a health and safety tool or system by identifying when and where heavy and loud machinery or tools are being used within a site.
There is also an opportunity for this type of audio detection system to be used within other industries, more specifically the healthcare industry. These devices and systems could be used to detect sounds such as hospital equipment on wards and alert staff to the ongoing issues and with the IoT devices being relatively low cost to produce it provide a feasible alternative to in-person staff monitoring.
Get started on your next IoT project with The Curve!