How Do We Teach a Computer to See a Car’s Direction?

We started with a simple visual idea for LocalParts: put a car on an OpenStreetMap-based hero section and let it drive around Beenleigh. It sounded like a front-end animation task. Put a map on the page, load a car model, move it along a route, and rotate it as it travels.

The interesting part was not making the car move. The interesting part was asking what ‘move in the right direction’ actually means.

When a human looks at the screen, the answer seems obvious. The car should point along the road. When a computer looks at the same scene, there is no single thing called ‘direction’. There are geographic coordinates, projected screen coordinates, model axes, wheel axes, camera rotation, map pitch, and several different meanings of ‘forward’.

That gap between what the human sees and what the computer can calculate became the real project.

A 3D car driving over OpenStreetMap tiles on a curved Earth-scale surface
The finished Three.js scene streams visible map tiles onto a curved Earth-scale surface while the car follows the precomputed route.

First, give the computer a world to move through

The first layer was the map. A browser can render a two-dimensional map easily, but the visual goal was closer to an isometric or pitched map: streets receding into the scene, with the car sitting in the same visual plane as the map.

That introduced our first useful distinction: latitude and longitude describe a place on Earth; a map projection turns that place into display coordinates; the browser turns display coordinates into pixels; and a 3D renderer turns model coordinates into pixels through a camera.

Those are related systems, but they are not interchangeable. A heading measured between two latitude/longitude points is a geographic bearing. It is not automatically the angle the car should have on the screen, especially once the map is pitched or rotated.

The finished prototype uses a precomputed OSRM driving route through Beenleigh. The route is stored locally as GeoJSON, so the browser can replay it without depending on a live routing request every time the hero loads. That gives the animation convincing road-following geometry while keeping the visual demo deterministic.

A Three.js car grounded on the curved Earth-scale map surface
The same latitude/longitude position is projected onto the curved world that carries the route and vehicle.

The sprite that looked right in code but wrong in the design

The first car was a sprite. It was quick to add, but it did not belong in the scene. A flat image could be moved around, yet it did not share the map’s perspective or lighting. It looked like a sticker placed on top of the interface rather than a vehicle travelling through the map.

This was a good reminder that technical correctness is only one kind of correctness. The sprite had a position and a rotation value. The human eye still rejected it.

We changed to a 3D model loaded with Three.js. Because a remote model can fail to load, be the wrong size, or arrive with an unexpected origin, the prototype also includes a procedural low-poly fallback car. The fallback is not a final asset; it is a diagnostic instrument. It guarantees that we can test movement, camera framing, visibility, and orientation even when the production model is unavailable.

A 3D car driving over OpenStreetMap tiles on a curved Earth-scale surface
The procedural fallback car gives the scene a readable, perspective-aware vehicle while the production asset is being refined.

What does ‘forward’ mean on a 3D model?

This was the question that unlocked the rest of the debugging: if the car is pointing the wrong way, which part of the model tells us what forward is?

The answer is not necessarily the model’s positive X axis, positive Z axis, or its visible bonnet. Different modelling tools and asset pipelines use different conventions. A model can arrive rotated, mirrored, offset from its origin, or with a parent object that adds another rotation.

So we looked for something physical in the model: a wheel. A wheel gives us a useful normal or axle direction. The wheel’s axis tells us its left-to-right orientation. It does not directly tell us which way the car drives, but it gives us a stable lateral reference.

With an approximate up vector, we can derive two possible forward directions by taking a cross product. The same wheel axle is compatible with a car facing either end of the vehicle. The camera and the route provide the missing information: project each candidate into the scene and choose the one that visually agrees with the desired travel direction.

That is a much better question than ‘what rotation value looks right?’ It turns a visual mystery into a measurable relationship between a model feature, a coordinate frame, and a camera.

A Three.js car scene used to diagnose model orientation and route direction
The development view exposes the car’s orientation and makes the model-forward-axis problem visible.

The camera is part of the answer

At first, we compared the car’s geographic heading to the route heading and assumed the numbers should match the model rotation. They did not. The map was pitched, the camera had its own bearing, and the 3D model lived in a different coordinate frame.

The fix was to ask a more precise question: if I rotate the model by this amount, where does its forward vector land on the camera’s screen?

The harness now projects the candidate forward vector through the Three.js camera. It compares the resulting screen direction with the direction the car needs to travel. This lets the prototype calibrate the model against the actual view rather than against an abstract angle.

The on-screen harness reports the useful evidence: latitude, longitude, geographic heading, route number, screen vector, map pitch, map bearing, model rotation, calibrated rotation, wheel axle, screen position, and whether the car is inside the camera frustum.

That last check is important. A car can be perfectly positioned in world coordinates and still be invisible because it is outside the camera’s view, behind another layer, too small, clipped by the near plane, or hidden by a failed asset load.

A Three.js car scene used to diagnose model orientation and route direction
The harness connected route heading, model rotation, camera framing, and the human-visible result.

Why the rotation had to lerp

Once the target rotation was correct, the car still looked artificial when it turned. The heading changed instantly from one route segment to the next, so the vehicle snapped around.

The answer was interpolation, but even that has a detail worth asking about: what happens when the shortest turn crosses the -180°/180° boundary? If the current angle is 179° and the target is -179°, a naive subtraction suggests a 358° turn. The visually correct turn is only 2° in the opposite direction.

The prototype normalises the angular difference, then smoothly interpolates toward the target on every animation frame. In plain terms, the route provides a target. The car eases toward it. The result is a vehicle that appears to steer into the next segment instead of teleporting its nose to a new heading.

This is another example of the computer needing a more granular instruction than the human does. A person says, ‘make it turn smoothly.’ The implementation needs to define shortest-angle arithmetic, frame timing, smoothing strength, and what happens when the target changes again before the car arrives there.

What the computer can see, and what the human sees

The phrase ‘get AI to see the car’ can sound like a computer-vision problem, but this prototype used AI in a more grounded way. AI helped us inspect screenshots, reason about code, propose instrumentation, and translate visual feedback into testable hypotheses. The browser and Three.js supplied the scene. The human supplied the judgment that the car looked wrong.

Computer-side evidenceHuman-side judgment
The model has non-zero coordinates.I can actually see the car.
The heading calculation returns a plausible angle.The nose points along the road.
The mesh is inside the camera frustum.The car is not hidden behind the text.
The wheel axis is mathematically stable.The vehicle does not look sideways.
The rotation value changes smoothly.The turn feels natural rather than sluggish.
The asset loaded without an exception.The model belongs in this visual style.

Neither side is enough on its own. Human vision is excellent at noticing ‘that feels wrong’, but it is not always good at naming the cause. Code is excellent at measuring causes, but it can report success while the result still looks wrong.

The productive loop is to look at the screen and describe the mismatch, reduce it to a smaller question, add a measurement or visual harness, change one relationship at a time, and look at the screen again.

This is the style of questioning I enjoy most: not ‘which magic rotation fixes it?’, but ‘which normal, axis, projection, or layer explains what I am seeing?’

A Three.js car grounded on the curved Earth-scale map surface
The visual scene is the evidence: the car must remain visible, grounded, and aligned with the map surface.

The pitfalls we hit

Several failures were useful because each exposed a hidden assumption.

A flat sprite did not share the map’s perspective

It moved, but it did not look embedded in the world. The lesson was to choose an asset representation that matches the scene, not just one that is easy to animate.

The 3D car was initially invisible

Model scale, camera distance, origin placement, and loading state all affect visibility. A bounding-box fit and a guaranteed fallback geometry made the scene testable before the final asset was trustworthy.

The loader API was not where the first code expected it

The initial GLTF loader approach assumed a global that was not present in the chosen module setup. Importing the loader explicitly fixed the error. This is a classic front-end trap: an example from one Three.js distribution style is copied into another.

The car and the text competed for the same space

The car stage had to sit behind the hero content, while the map and vehicle remained visible. Explicit stacking order made the intended relationship clear: map and car at the visual base, content above them, controls still reachable.

Geographic heading was not screen heading

Map pitch and bearing changed how a direction appeared to a viewer. Comparing raw geographic angles directly with model rotations produced a car that was mathematically plausible and visually wrong.

A wheel normal is not automatically the nose direction

The wheel gave us an axle, not a forward arrow. We needed the up vector, a cross product, both possible signs, and a camera-based comparison to choose the correct candidate.

Instant rotation looked mechanical

The route was correct, but the animation was not. Shortest-angle interpolation and smoothing closed the gap between a route calculation and a believable turn.

A prototype route is not real navigation

The route is now real OSRM road geometry, but it is still a visual hero animation rather than turn-by-turn navigation. A full navigation product would need live routing, speed rules, traffic handling, user-selected destinations, and a clear explanation of what the car represents. Visual confidence should never be mistaken for data accuracy.

Remote assets create production questions

External models and map tiles bring questions about licensing, availability, caching, performance, privacy, and attribution. The production hero now keeps the rendering in Three.js, streams only the tiles inside the camera footprint, uses a CORS-friendly tile source, and preserves visible OpenStreetMap and CARTO attribution.

What this way of thinking gives us

The result is more than a car that rotates correctly. It is a repeatable way to investigate interactive graphics. Start with the visible mismatch. Then ask what the computer would need to know for the human judgment to become measurable: which object is the reference, which axis or normal is stable, which coordinate system are we in, what does the camera do to the measurement, is the object visible, is the layer order correct, and does the animation preserve continuity over time?

Those questions are useful far beyond this one hero section. They apply to robotics, CAD viewers, games, mapping, browser-based product configurators, and any interface where a 3D object has to agree with a human interpretation of space.

What comes next

The next engineering steps are to replace the procedural fallback with a licensed vehicle model that fits LocalParts, add a reduced-motion or static fallback for users who do not want animation, and continue tuning tile level-of-detail and caching. The diagnostic harness remains useful during development but stays out of the public production experience.

The larger lesson is simple: computers do not need us to stop asking visual questions. They need us to keep asking them until the visual idea has been translated into coordinates, vectors, normals, projections, and carefully measured relationships.

That is how a vague complaint — ‘the car is pointing the wrong way’ — turns into a useful technical investigation.

A 3D car driving over OpenStreetMap tiles on a curved Earth-scale surface
The polished result: a car, route, map tiles, curved world surface, and shadow rendered together by Three.js.

Leave a Reply

Your email address will not be published. Required fields are marked *