Embedded Systems for Real Time Video Processing

Embedded video processing in real time is needed as pre-processing step for AI inference algorithms. We support AMD Zynq (32 bit processor architecture, 28 nm technology) and AMD Zynq Ultrascale+ systems (64 bit processor architecture, 16 nm technology) with programmable logic on single device.

The AMD device is positioned on HW module with DDR4 memory and runs Linux OS. We integrate HW accelerators into the programmable logic part to reach high performance with reasonable power consumption.

This is example of Single Instruction Multiple Data (SIMD) HW accelerator operating in floating point 32 bit arithmetic. HW accelerated algorithm with floating point matrix operations. The concrete algorithm can be changed in the runtime by change of the firmware for the state machine sequencer used by the HW accelerator.  In parallel, the HW accelerated „Lucas Kanade Dense Optical Flow” (LK-DOF) algorithm computes relative movement direction and relative movement speed for each pixel of video and displays it by colour and colour intensity for each pixel.

This is example of embedded acceleration for matrix multiplication reached on eight SIMD HW accelerators in comparison with a SW computation on a single core of an average Intel PC and in comparison to SW computation on four core arm processor on the same AMD device.

The HW accelerated LK-DOF algorithm on AMD ZU15 device is computed in SW with rate 0.1 frames per second while the HW accelerated version is 600 times faster.