3D modellingAISimulation2026

Virtual environments for simulators

How to generate roads, towns and trails automatically for simulators: real enough to be believed, light enough for dynamic, immersive simulation.

Role: Design Researcher
My part: Research, 3D environments and AI workflow
With: Francesco Bottino
Timeline: May 2026 – now
Tools: Blender, Unity, Unreal Engine, Claude

The need

Simulation needs new virtual environments, fast. To generate them automatically, we first need the techniques that keep a scene light enough to run smoothly and ready to use with physics and in Unity.

The outcome

An AI tool that turns a description into a virtual environment that is realistic, light and ready for simulation. It picks the right technique for terrain, roads, buildings and vegetation, and checks the result against the simulator’s requirements.

The goal

Realistic environments that stay light enough for the simulator.

Every simulation requires a virtual environment, whether a road, an urban street or a mountain trail. Visual realism is necessary but not sufficient: the scene must sustain a stable frame rate in real time, provide reliable surfaces for vehicle and pedestrian physics, and integrate directly into Unity. The following requirements apply to every environment.

Performance

A defined triangle budget and levels of detail, so that only nearby geometry is rendered in full.

Realism

Real terrain and road layouts, buildings true to the local architecture, and markings and signage that comply with traffic regulations.

Immersion

Vegetation as crossed billboards with alpha clipping, stereo-compatible shaders and consistent colour.

Unity integration

FBX tiles and a Unity package that rebuilds the scene and its materials automatically.

Manual workflow

Open map data as a base, every layer finished by hand.

We started with a manual pipeline in Blender: OpenStreetMap and Blosm for roads and footprints, Buildify for the buildings, everything else by hand, then tiles for Unity. This phase defined what works best in the simulator, which texture sources to use and an efficient workflow from an empty scene to simulation.

A town

Top view of the town: road network with a roundabout, Buildify buildings and green lots

Driver’s view of a straight street between brick and stone buildings lined with trees

Driver’s view of a street with a dashed centre line, ochre buildings with shutters and trees

Rubble stone wall material on a sphere — Stone

Concrete panel material on a sphere — Concrete

Stone paving slab material on a sphere — Paving

A motorway

Driver’s view of the motorway under a variable message sign announcing roadworks and a compulsory exit

Three-lane carriageway on a viaduct between guardrails and mesh safety barriers

Inside a motorway tunnel with lane markings and guardrails, the exit light in the distance

Motorway with a service area sign at 500 m and a noise barrier with bird stickers

Green motorway direction sign material on a sphere — Road sign

Cast concrete wall material on a sphere — Concrete

Roadside grass material on a sphere — Grass

Rusted steel material on a sphere — Rusted steel

Metal grating material on a sphere, see-through — Grating

AI-assisted workflow

An AI agent builds the place in Blender, following the rules of the simulation.

The manual workflow is now automated. Claude works inside Blender and writes the Python that repeats the same steps, from data to Unity, following the rules defined by hand. Every layer is reviewed against the requirements and sent back until it is right.

Oltrepò: a real road in the vineyards

A real road in the Oltrepò hills, built on real data: the Regione Lombardia terrain model for every climb and descent, aerial photos for the vine rows, and OpenStreetMap for roads, buildings, woods, hedges and walls. The vegetation is made of cut-out images of local species on crossed planes.

The Oltrepò road along a ridge, with trees, a farmhouse and hills in the distance

A vine row seen from its end, generated procedurally

The same road in Unity, from the driver’s height between vineyards

Aerial photo of vineyards used as ground texture, on a sphere — Aerial photo

Weathered plaster wall material on a sphere — Plaster

Vine leaf billboard material on a sphere — Vine leaves

Worn asphalt material on a sphere — Asphalt

A mountain trail

Here the terrain already existed: the work was to texture it and add the vegetation, keeping the trail clear and walkable.

Aerial view of the trail network winding through meadows and conifer forests

Two trails crossing in a clearing, seen from above between conifers

Eye-level view of the gravel trail through a meadow towards the forest and mountains

Rock face material with patches of moss and snow, on a sphere — Rock

Colour map of the mountain terrain with trails, meadows and forest, on a sphere — Terrain map

Mossy rock material on a sphere — Mossy rock

Conifer billboard atlas on a sphere, see-through — Conifers

A Milan city street

Top view of the Milan neighbourhood: red roofs, courtyards, crossings and parked cars

Aerial view of the neighbourhood in late afternoon light: blocks of flats, red roofs and tree-lined streets

Street level at a crossing: parked cars, lane markings and tall blocks of flats

A street running past an underpass, with street lamps, zebra crossings and parked cars

What keeps it light

Detail only where people drive, walk and look.

The two workflows turned into a set of techniques that cut the weight of a scene without losing what matters for the simulation.

Exact where you move

The terrain is simplified everywhere except on roads and trails, where the geometry stays exact.

Only what can be seen

Visibility is computed from eye height along the route; hidden trees and stones are removed.

Trees as billboards

Two planes crossed at right angles: four triangles per plant.

Next steps

Towards software that builds a simulation-ready map from a single prompt.

The next step is a tool, built on skills and agents, that already knows the simulator’s requirements and the techniques defined in this project. Choose a place, and it is reconstructed as faithfully as possible, from terrain and roads to buildings, vegetation and signage, ready for simulation.

What still needs work

Every step still needs a precise brief and a human review.
Procedural objects can look repetitive at close range, buildings in particular.
Roundabouts, underpasses, overpasses and complex junctions are still modelled by hand.