A learned autopilot,
watched by a safer one.
This project teaches a computer to fly a simulated small plane by copying a traditional autopilot that already works well — then adds a safety system that takes the controls back the instant the AI starts flying badly.
imore technical detail
In v1 a neural network is trained via behavior cloning to imitate a PID controller flying a JSBSim Cessna 172 simulation; in v2 it is replaced by a policy trained with reinforcement learning (PPO), behind the same monitor and fallback. A separate runtime safety monitor checks the aircraft's true state every control step and switches to the PID whenever bank, pitch, altitude error, or airspeed leaves a validated envelope.
The dashed line shows what happens if the AI flies completely on its own for the full two minutes — it never recovers, drifting further off course the whole time. The solid line starts exactly the same way, but this time the safety monitor is watching: it steps in at 31 seconds and hands control to the traditional autopilot, which brings the plane back under control. Move your cursor over the chart to scrub through the flight.
imore technical detail
Monitor trips on: |bank| > 30°, |pitch| > 12°, |altitude error| > 35ft, or airspeed outside 85–120kts. In this recorded run, altitude error crossed the 35ft threshold at t = 31.1s. The fallback is latching — once engaged, it flies for the rest of the episode.
how it flies
One controller you can trust, one you're testing.
The safety monitor checks on the plane 20 times every second. As long as things look normal, the AI does the flying. The moment something looks wrong, control switches to the traditional autopilot for the rest of that flight — once it steps in, it doesn't hand control back.
imore technical detail
Checked every 50ms (20Hz): bank angle, pitch angle, altitude error, and airspeed, each against a threshold set with margin above the PID's own worst-case nominal behavior. The fallback latches by design — re-entering the envelope does not hand control back mid-episode.
what went wrong
Seven real bugs, found by testing, not guessing.
Every threshold and design choice on this page traces back to one of these. Click any one for the root cause, how it was caught, and the fix.
Heading-wrap discontinuity
Crossing 360°/0° produced a 358° error instead of 2°, commanding a hard turn the wrong way.
Reading the very first hands-off trim plot, before writing any controller.
Wrapped heading error to [-180°, 180°) inside the environment itself.
Deterministic turbulence
JSBSim's gust model defaults to a fixed internal seed — every training episode got the identical disturbance.
Tracing raw state across different episode seeds and finding identical values.
Explicitly randomized atmosphere/randomseed once per episode.
The imitation model loses integral memory
The PID's altitude loop integrates error over time; the cloned model only sees instantaneous state, so small regression errors compound over the aircraft's ~30s phugoid mode.
Extending the evaluation horizon to 120s, after the model looked fine at 60s.
Not fixed — this is the reason the safety monitor exists, and why v2 moves to RL.
PID mis-trims when engaged mid-flight
The PID captured whatever pitch it saw on its first call as “level.” Correct at a fresh reset, wrong when the monitor engages it mid-oscillation.
Integration-testing the full monitor+fallback system — invisible to a PID-only unit test.
The environment now computes true trim pitch and the PID takes it explicitly.
The fallback has no airspeed protection
The PID only flies elevator and aileron. A bank plus a headwind gust can bleed airspeed past the stall floor with nothing to stop it.
A systematic stress sweep across turbulence severity, independent of the model or monitor.
Documented scope limit: reliable to severity 2, roughly 50–75% crash rate above it, for every controller tested (the v2 AI included). Flagged for a throttle loop.
The reward taught the AI to crash on purpose
The first scoring rule docked more points the further the plane drifted, plus a one-time penalty for crashing. Once a flight started going badly, ending it early scored better than carrying on.
Reading the whole training curve, not just seeing that training finished: average flight length collapsed from ~400 steps to ~25. My pre-training check had covered a perfect flight and an instant crash, but not a slow drift.
Capped the per-step penalty and added a small reward for every step survived. Flights then ran the full 1,200 steps.
Results depend on the simulator version
On JSBSim 1.3.3 the backup autopilot never crashed in heavy turbulence; on 1.2.4 it crashes 50–65% of the time.
Re-running the stress test in a fresh environment with a newer version: it failed to reproduce my own earlier numbers.
Pinned jsbsim==1.2.4 for every reported result. The v2 policies were trained on 1.3.3 and evaluated on 1.2.4.
does the monitor actually help?
Stress-tested against turbulence, sensor noise, and bad starts.
I ran the same test scenario 20 times per condition, each a minute and a half long, comparing the AI flying completely alone against the AI with the safety monitor watching over it.
The baseline everything else rests on. Before teaching a computer anything, I made sure the traditional autopilot could handle a real test: turning and climbing at the same time. It handled both smoothly — exactly why it's trustworthy enough to be the safety net.
imore technical detail
Combined 30° heading change and 80ft altitude change, commanded simultaneously. Max bank angle 20.6°. Heading settled to within 2° by 15.2s; altitude settled to within 5ft by 42.5s.
How far off-target the plane ended up, on average, flying alone vs. with the safety monitor watching — shorter bars are better. The monitor made a real difference in every scenario tested, from calm air to turbulence, sensor noise, and bad starting positions.
imore technical detail
20 episodes per condition, 90s each. Mean final |altitude error|, model alone → monitored: baseline 7.2→5.8ft; moderate gusts 14.7→8.9ft; large off-nominal start 18.7→6.2ft; sensor noise 11.8→7.2ft; combined stress 17.1→7.5ft.
This one has nothing to do with the AI or the monitor. It's the traditional autopilot flying completely on its own in increasingly rough air. It handles light-to-moderate turbulence fine, but this simple version can't adjust engine power mid-flight, so strong turbulence plus a turn can slow the plane down dangerously. I found this limit through deliberate testing, and I'm documenting it rather than hiding it.
imore technical detail
Turbulence severity follows JSBSim's MIL-F-8785C-style scale (roughly 1=light to 6=severe). PID-alone crash rate: severity 0–2 → 0%; severity 3 → 60%; severity 4 → 75% (measured on JSBSim 1.2.4; a newer 1.3.3 build did not show the cliff). Root cause: no throttle/autothrottle loop, so a bank plus a headwind gust can bleed airspeed through the 50kt floor.
Full technical plots (all five build steps, matplotlib output): see the repository.
v2: learning by trial and error
Replacing the copycat with a pilot that learned by flying.
In v1 the AI copied an autopilot. In v2 it learns the way a person would: it flies thousands of simulated flights, gets a score for staying on course, and adjusts. The safety monitor and the backup autopilot are exactly the same as before.
imore technical detail
PPO (Stable-Baselines3), 500k steps, two random seeds, trained on JSBSim 1.3.3 with turbulence severity 0–2 randomized per episode. Reward per step: +1 alive bonus, minus capped altitude/heading/airspeed tracking penalties and a control-effort penalty; −20 when a hard limit trips. All results below: JSBSim 1.2.4, 20 identical flights per cell, v1's conditions.
Shorter bars are better. The pilot that learned by trial and error (blue) beats the copycat (gold) in every scenario, and beats the traditional autopilot (grey) whenever the air is rough or the start is bad. In calm air the traditional autopilot is still the most precise.
imore technical detail
Mean final |altitude error| in ft (PID / BC / RL seed 0 / RL seed 1): calm 1.4 / 6.6 / 3.8 / 2.8; gusts 8.1 / 18.6 / 5.7 / 4.7; large start 8.3 / 26.1 / 5.9 / 5.4; sensor noise 9.4 / 12.9 / 6.0 / 4.0; combined 10.5 / 25.6 / 6.1 / 9.1. Alone, the RL policies crashed in 2 of 200 flights (combined stress, seed 1); with the monitor, none.
This is the test the monitor exists for. I deliberately broke the AI's controls (a stuck elevator offset, or a roll command that never lets go) while leaving the backup autopilot untouched. Flying alone, every AI crashed in the stuck-roll test. With the monitor watching, none of the 180 flights crashed: it noticed within about a second and a half and took over.
imore technical detail
Faults added to the learned controller's output only: elevator +0.4, aileron fixed at +0.3, and elevator +0.4 in severity-2 turbulence. Unmonitored crash rates: BC 100% in all three; RL 0% / 5% for the elevator faults and 100% for stuck aileron. Fault types and sizes were chosen by me; they are a stress test, not a model of a real failure.
Strong turbulence plus a turn can slow the plane to a dangerous speed, and none of these pilots can adjust engine power. Every controller, the AI and the backup alike, struggles above severity 2. Because the backup shares the limit, the monitor can't rescue a flight there. Adding a throttle loop is the obvious next step.
Two training runs and 20 flights per scenario is a small sample, so differences of a few feet in calm air are within noise. The v2 policy banks harder in turns than the traditional autopilot, so it trips the monitor in about half of its calm-air flights; the monitor's limits were set in v1 for the copycat model and are conservative for this one. Everything here is simulation.