Diffuse Your Data Blues: Augmenting Low-Resource Datasets via User-Assisted Diffusion

Подробнее о видео

Object detection in the wild is a well-known problem in the domain of computer vision (CV). From a computer vision problem, it turns into an intelligent user interface (IUI) problem for augmented and mixed reality interfaces. The International Standardization Organization defined (ISO/IEC 5927 : 2024) mixed reality system as a system that uses a mixture of representations of physical world data and virtual world data as its presentation medium [20]. The accuracy and latency of detecting and registering real life objects dictates the accuracy, reliability and finally usability of mixed reality applications. This paper addresses an important problem of developing mixed reality applications for domains where the object detection problem is too complex for classical image processing technique as well as hard enough for standard deep learning-based computer vision models due to scarcity of datasets. In particular, we highlighted two applications in the context of Industry 4.0 and aircraft maintenance, where standard labelled data were not available and exact deployment scenario could not be assumed apriori requiring a robust object detection model. While there
is plethora of works in computer vision domain on developing in the wild object detection models for autonomous vehicles, human body parts, we rather took a different approach on training standard computer vision model with synthetic data, and testing MR applications on diverse real life situations. The novelty of the work lies on:
(1) Developing a diffusion model-based image mixing pipeline for generating customized dataset
(2) Training a standard computer vision model with synthetic data and reporting object detection accuracy through MR applications
The proposed technique uses segmentation methods to automatically annotate relevant areas of interests of a set of images, mixing the images using the proposed pipeline to generate realistic synthetic data. Results indicate that the use of proposed model led to a substantial enhancement in detection performance. Specifically, incorporating diffusion techniques into the dataset of components of pneumatic cylinder elevated the 𝐹 1 score from 69.77 to 84.21, while the 𝑚𝐴𝑃@50 increased from 76.48 to 88.77. This represents a 11% improvement in object detection accuracy compared to conventional methods.