Safe Person Recognition at Industrial Robots with an Active NIR Camera System

Status:

completed 03/2016

Aims:

Research and prototypical implementation of a method for safe optical person recognition and positioning in real-time within the operating range of industrial robots. Approach: a dedicated camera system working within the near-infrared (NIR) spectral range which allows the reliable detection of visible skin areas independent from skin types.

Activities/Methods:

Analysis of requirements and design of system specification; detailed analysis of relevant material samples (skin, typical workpieces, etc.); identification of optimal spectral signatures; development of an optimized camera system; development of suited calibration methods, as well as algorithms for skin detection with face and limb recognition; evaluation of the system in the lab and a field test.

Results:

This project covered the research about feasibility and benefits of a Vision Based Protective Device (VBPD) based on an active near-infrared camera system. This kind of system allows a robust pixel-by-pixel detection of skin using its spectral fingerprint in the near-infrared region. This technique can be used for an automatic distinction between persons and other objects within the image data. Especially in the working area of industrial robots, the system offers an additional benefit for the safety of employees and the availability of the robot.

Summary of the results:

• Robust skin detection using NIR camera system
  Pixels of human skin can be distinguished from other pixels with a sensitivity of 0.999974 according to the selected generic footage for the intended application. The data was acquired using a functional prototype, which also shows the feasibility of this kind of a system.
• Person detection
  98.1% of all picked pixels of persons in a sample of images are recognized by the system automatically as person-pixels. Distinction between persons and other objects allows a "smart" way of Muting, in which materials and the robot are ignored, but persons in the same area are detected safely.
• Real-time capability
  Image acquisition can be performed with a speed of 25 multispectral images per second in a sufficient amount of time using currently available near-infrared cameras. The processing time scales with hardware-effort.
• Depth information
  A stereoscopic setup with two near-infrared cameras allows the acquisition of three-dimensional images with state-of-the-art methods. The combination of a NIR-camera and a camera for the visible spectrum is not suitable for this purpose, regarding our investigation.
• Compensation of motion artifacts
  A large amount of motion artifacts, caused by the necessary single image acquisition of each near-infrared channel, can be compensated with state-of-the-art algorithms. Nevertheless, fast movements and the combination of other artifacts show the performance limits of those algorithms.

Summarizing, the feasibility and suitability of this kind of protective device for the observation of dangerous areas can be confirmed. The size of the observable area is more restricted compared to common camera systems working in the visible range. This is due to the relatively young development of near-infrared images sensors compared to silicon-based image sensors, as well as the performance limits concerning the active light source. These and other problems, in particular an image acquisition method more robust against motion artifacts, are addressed in the follow-up research project "beyondSPAI". In this context, the application of a NIR-camera system as part of a multimodal sensor system including different kinds of sensors to safeguard collaborative robots will be investigated.

Last Update: