Hybrid evaluation for HW-NAS on embedded heterogeneous SoCs: A case study in semantic segmentation on Nvidia Jetson Orin

Adapting together Deep Neural Networks (DNNs) and hardware accelerators can help to better address SWaP constraints such as latency or energy. However, such approaches have many design challenges, especially for vision applications on heterogeneous SoCs. To efficiently handle the complex design spaces, we present a new comprehensive methodology to speed up a Hardware-aware Neural Architecture Search (HW-NAS) flow while guaranteeing precise solutions. This new HW-NAS implementation relies on a complete and convenient methodology for faster exploration duration compatible with Nvidia's deployment tools. With less than 1% exploration loss, our HW-NAS flow improves the default Nvidia deployment strategy for multiple power modes with 33% less time duration and still precise solutions. The experimental results show that our mapping approach can propose better mIoU-latency-power Pareto fronts, i.e., 50% less power-intensive for equivalent accuracy or a +6% better mIoU with 28% less power consumption.