Making cities smarter and safer with innovation from the Edge to the Cloud

Smart cities bring together technology and infrastructure to improve the lives of citizens through greater safety and efficiency. However, rising populations in large urban areas intensifies the challenges that cities face in terms of managing infrastructures and actively protecting citizens. Combine this with increased mobility of urbanising populations, with more people travelling in greater distances than ever before, and it’s clear that additional technology solutions will need to be utilised and existing ones improved. Highways agencies and police forces already use video extensively to monitor traffic congestion, respond to accidents, and spot threatening behaviour in town centres. But could these systems be optimised to make roads safer?

With greater camera resolution, they could. Enhancing the ability to recognise vehicle number plates could help tighten up enforcement of traffic rules, ultimately reducing contraventions – particularly disobeying traffic lights or junction restrictions – that cause congestion as drivers come to understand they are highly likely to be prosecuted. Enhanced traffic-flow cameras and speed cameras can also help reduce congestion and encourage safer driving. Other opportunities to improve the quality of information provided by cameras include extending the typical field of view and improving night-vision performance, both of which could be achieved by combining multiple image sensors.

In addition, combined with greater intelligence in decoding human body language, high-resolution smart cameras can become better at pre-empting disturbances so that help can be dispatched.

To achieve this, improvements are needed at both the front-end and back-end of the system. Simply upgrading to high-resolution image sensors, such as 4K sensors, greatly increases the data associated with each frame that must be captured and conditioned by edge devices and transmitted to the cloud to be stored and analysed.

Field Programmable Gate Arrays (FPGAs) are known for their ability to perform massively parallel signal processing on multiple streams of high-speed data in real-time. Xilinx’s Zynq UltraScale+ MPSoC (Multi-Processor System-on-Chip) devices, which are fabricated using today’s leading 16nm process technology, enable a single-chip solution containing a 4K video encoder implemented as hard IP that eliminates the typical signal latencies associated with chip-to-chip communications. Moreover, this year Xilinx has delivered new MIPI IP with 2.5Gbps/lane, creating the fastest peripheral interface in the market today.

Xilinx is also involved in a number of projects to develop systems such as night-vision cameras, stereo traffic snap cameras, panoramic cameras, and AI boxes for smart-city use cases.

In addition, AI technology is becoming deeply integrated into front-end and back-end video equipment, enhancing the accuracy and efficiency of the whole system. A typical back-end system that connects with smart cameras comprises a smart network video recorder, smart server and video management software (VMS). AI-powered intelligent video analysis technology is the essence of these products and smart NVR/server offers real-time video analysis with metadata. In addition, smart NVR supports intelligent index by event, achieves precision recording and saves storage space.

To improve performance here, Xilinx innovations include the Alveo™ U50 adaptable accelerator card that integrates 16nm UltraScale+™ FPGAs and high-bandwidth memory (HBM2) chips with 460GB/s bandwidth for cloud acceleration. Moreover, the latest Versal™ Adaptive Compute Acceleration Platform (ACAP) – with its multi-terabit-per-second Network on Chip (NoC) interconnect and advanced AI Engine that contains hundreds of tightly integrated VLIW SIMD processors – now moves computing capacity beyond 100 tera operations per second (TOPS @ INT8). This could dramatically improve the AI capability of intelligent video systems and significantly accelerate AI applications at the edge and in the cloud.

Xilinx is also building the innovative ecosystem developers need to apply these advanced devices in their projects, including tools such as Vitis™ for application development and Vitis AI™ for optimising and deploying accelerated machine-learning inference.

Vitis Unified Software Platform Overview

AI technology inspires cities as they pursue their journeys towards smart safety, smart infrastructure, and smart transportation. Various advanced technologies, including innovative FPGA, MPSoC and ACAP devices, as well as AI accelerator cards that integrate these devices to accelerate their deployment in edge and cloud applications, to enable autonomous AI platforms that are capable of handling, in real-time, the enormous quantities of data that these smart cities will generate.

Future generations of AI could move our understanding of the smart city forwards from a basically reactive concept to instead embrace predictive resource management, introducing data from potentially thousands or millions of pervasively distributed sensors such as traffic sensors, parking sensors, air-quality sensors, weather sensors, ambient-light sensors, and maybe many other types, too. The smart city of today is an embryonic version of what is to come .

Trevor Weng

Industrial, Vision, Healthcare & Sciences Marketing Manager at Xilinx