SimART

A Unified and Open Real-world Multimodal Simulation Platform for 6G Integrated Sensing and Communication

Code Paper

Research Brief

From "Tool Tinkering" to a Unified Simulation Pipeline

6G ISAC demands visual, LiDAR, IMU, GPS, and wireless data in one pipeline — but existing simulators each cover only half the problem.

SimART bridges the gap with ROS: a shared clock and coordinate frame connect AirSim, Sionna RT/SYS, and CKM generation. A full session is recorded as one rosbag file.

Platform Contribution

Connects physical sensing, ray tracing, PHY/MAC wireless evaluation, and CKM construction into a modular workflow.

Data Contribution

All sensor streams, channels, KPIs, and beam labels share a common clock and coordinate frame — ideal for multimodal learning.

Scenario Contribution

Supports real OpenStreetMap maps and user-defined layouts, generating both visual and electromagnetic assets.

Problem Context

Key Gaps in ISAC Dataset Generation

Robotic Simulators

CARLA, AirSim, Gazebo, and Isaac Sim handle 3D physics, cameras, LiDAR, and platform motion, but wireless propagation and link evaluation are limited.

Wireless Simulation Tools

Sionna RT/SYS, DeepMIMO, and Wireless InSite excel at channel and link analysis, but lack onboard sensor and closed-loop robot motion capabilities.

SimART

Combines mature tools through ROS contracts so the same backend can serve aerial, ground, indoor, and maritime ISAC scenarios.

Architecture

ROS as the Spatio-Temporal Backbone

Four modules — physics & sensing, ray tracing, link evaluation, and CKM — exchange data over standard ROS topics and custom SimART messages.

SimART platform architecture with physics, ray tracing, link system, CKM generator and rosbag dataset outputs — Architecture of SimART and the resulting multimodal ISAC dataset.

Physics and Sensing

Publishes platform pose, RGB/Depth/Semantic images, LiDAR point cloud, IMU, GPS, and ground truth.

Ray Tracing

Computes propagation paths, CIR, delay, angle, and Doppler using spatially aligned simplified meshes.

Link and System

Evaluates SINR, BLER, achievable rate, and optimal beam index via Sionna SYS.

CKM Generator

Scans channel and link metrics across grid positions to produce multi-layer channel knowledge maps.

Scene Construction

One Scene, Two Asset Sets

Visual and electromagnetic assets are built separately but share a single coordinate frame, keeping observations aligned with propagation paths.

Real regions: extract roads, buildings, and geographic elements from OpenStreetMap.
Custom layouts: construct controlled scenes via RoadRunner, Unreal Engine, or custom meshes.
EM conversion: apply geometry simplification, edge preservation, and material assignment with Blender scripts.

AirSim rendered urban campus scene — High-fidelity visual scene

Simplified ray-tracing mesh for electromagnetic simulation — Simplified RT mesh

Multimodal Dataset

One Simulation, One Complete Recording

Modules publish under a shared clock; rosbag captures everything in one session, fully compatible with ROS tools — rviz, rqt bag, SLAM, and perception pipelines.

RGB image captured from UAV flight — RGB image of UAV motion

Depth map captured from UAV flight — Depth map of UAV motion

LiDAR point cloud captured in the urban environment — LiDAR point cloud stream

Semantic segmentation captured from UAV flight — Semantic segmentation of UAV motion

Ray tracing paths between base station and receivers — Ray-traced propagation paths

Channel Knowledge Map

Turning Channel Statistics into Spatial Priors

The CKM generator grids the region, computing per-cell channel statistics and link metrics as spatial priors for ISAC learning.

Dense CKM received power map — Received power

Dense CKM best base station rate map — Achievable rate

Dense CKM effective SINR map — Effective SINR

Research Value

Cross-Scenario ISAC Algorithm Validation

Aerial ISAC

Sensing-assisted communication for UAVs, low-altitude economy, and urban air mobility.

Connected Driving

Joint sensing, localization, and beam management for autonomous driving and V2X.

Indoor Robotics

Coupled connectivity and sensing for factory and warehouse robots in rich multipath.

Maritime ISAC

Swap the front-end simulator while reusing the back-end wireless and CKM generation.

Project Credits

Team

Kang Yan

University of Electronic Science and Technology of China

Yuqi Cao

University of Electronic Science and Technology of China

Jiaqi Li

Nanjing University

Luping Xiang

Nanjing University

Kun Yang

Nanjing University