Developed applications for human detection and visualization with robotic operating system using time-of-flight (3D) camera

Client Overview

A global leader in the field of automation, covering a broad spectrum ranging from industrial automation, electronic components to social systems including automated ticket gates and solar power conditioners, healthcare devices, etc. At present, the client provides products and services in around 120 countries and regions.

Business Challenge

The client has recently developed a prototype of Time-of-Flight (ToF) camera IR sensor, which captures the images and provides the same in point cloud format over USB interface. To demonstrate the efficiency of this ToF sensor in the market, the client wanted to develop an application for their business team who can use it for product showcases to win business. For the same, the client was looking for a technology partner who can develop demo applications for the product that can be used by the client’s sales team.

VOLANSYS Contribution

VOLANSYS created two demo applications for the client’s new camera sensor. Firstly, a human detection demo application in C++11 on Linux (Ubuntu 18.04) platform using Point Cloud Library and secondly, a visualization application in Robotic Operating System viewer using Linux desktop that showcases captured images as a 3D stream.

Human detection

  • Fetched the point Cloud 3D data from the ToF camera over USB interface
  • Displayed the incoming data in a 3D viewer on the screen
  • Allowed users to define a detection zone. Here, people available within detection zone would be detected by the application and considered in the counting
  • Store the detection zone, to be used as pre-defined zone for upcoming sessions
  • Detected the human presence within detection zone, using pre-trained SVM (Support Vector Machines) with HOG (Histogram of Oriented Gradients) classifier
  • Added feature to draw a translucent cube around the detected humans and run-time show the total number of humans moving in/out of the detection zone, at any given point of time

Visualization

  • Developed new package for ToF sensor code in Robotic Operating System (ROS2) package format
  • Fetched the point cloud 3D data from the ToF camera sensor over USB interface
  • Created a publisher-subscriber mechanism between interface code and Robotic Operating  System viewer that would render the point cloud 3D data in the Robotic Operating System viewer – Linux desktop, whenever a frame is being captured and published on a defined topic.
  • Added support for configurable resolution
Technologies | Engineering Expertise

Image processing | Support Vector Machines | Histogram of Oriented Gradients | Point Cloud Library | Robotic Operating  System – ROS2 | C++11 | CMake | Machine Learning | Artificial Intelligence

Solution Architecture

Benefits Delivered
  • The demo applications developed enabled the client to target new markets in the field of object detection
  • Integration of client’s ToF sensors with Robotic Operating Systems helps client to convince their end customer to use ToF sensor for robotic applications with reduced time to market
Similar Success Stories