New B2B commerce study
Download now
Optimized user experience with AI: intelligent volume detection through computer vision
Automatic beverage selection via AI
A common scenario in image processing with AI is the detection and volume determination of objects or containers. Especially in confidential production environments, but also for devices without or with a slow data connection, processing must take place locally and offline.
The challenge
Object detection and volume measurement in real-time
We have set out to continuously improve the interaction between man and machine - on a large and small scale. To achieve this, we also think beyond direct customer inquiries. We asked ourselves how we could further improve the user experience when selecting a product from a fully automatic coffee machine.
Our approach: greater user convenience through intelligent recognition of coffee cups.
The solution
Combination of AI models to meet requirements
We have developed a practical application of such a scenario based on computer vision with AI support for precise detection and volume determination of different vessels. The combination of AI models and embedded hardware creates an efficient and reliable solution that can be used offline because AI models on embedded hardware enable local real-time processing. Architectures and optimization techniques are required to maximize the performance of the models on limited resources.
Our solution recognizes different cup types and automatically selects the appropriate beverage on this basis. Cameras recognize the container types based on AI using embedded computer vision.
To make this possible, we have developed a customized Convolutional Neural Network (CNN) based on the TensorFlow framework, which can reliably classify different cup types. This technology was integrated directly into the coffee machines via an API and enables efficient and error-free cup recognition, whereupon the machine automatically selects the appropriate coffee beverage.
The solution
Robust AI models for segmentation and regression
The solution was developed using a camera and models. Image processing networks recognize structures in the images of the cups. We did the training of the models ourselves, including calculations and optimizations by the TensorFlow framework. The original untrained model was not initialized, and we developed our own training methodology to achieve high accuracy in cup recognition.
Another focus is on improving the robustness and resilience of the AI models by training with generated image data. Our approach to automated training data generation simulates different environmental conditions to increase the reliability of the system under real-world conditions. This makes it much easier to make changes that affect the optical conditions: replacing or changing the position of the cameras, changing the lighting conditions, or altering objects can thus be realized with considerably less effort.
In this project, we use a total of three AIs: one to identify the type of cup and two others to recognize the volume of the vessel. The volume AIs are specifically divided into two parts: segmentation and regression.
Segmentation AI: This AI takes the camera image and removes everything that is not a vessel. This makes it easier to determine the exact area that contains the volume of the vessel.
Regression AI: The image processed by the segmentation AI is then passed to the regression AI, which derives the volume of the vessel from this image. This is done with the help of filters that can recognize various properties such as size.
Technical background
Agile development for optimal performance
The process began with a detailed requirements analysis, followed by the development of the AI models and integration into the existing systems. We always rely on agile development methods and continuous testing to adapt the solution optimally to the requirements, which is possible for every API and recipe-enabled machine.
The captured images are adapted to the recipes for various drinks such as latte macchiato, espresso, and Americano. The training of the AI was optimized by combining real and generated data, which enabled significant time savings through the use of generated images and further increased the performance of the system.
Phytec
Phytec has been developing and manufacturing embedded components for reliable electronic series products in Germany since 1985. Processor modules such as System on Modules and Single Board Computers as well as OEM products based on them are just as much a part of the core business as customer-specific embedded systems including software, housing design, and assembly. The family-owned company employs over 400 people in 5 branches worldwide.