Dynamics engine conformance
Each new release of each engine is installed automatically and asked for the accelerations of the same mechanical systems. Every answer is checked against a closed-form reference that shares no code with any engine. The question is the one that matters to a user: when an engine is wrong, does it tell you?
Last run 2026-11-02 · source and raw results · latest.json
Newest release of each engine
What changed between releases
| Engine | From → to | Route | Case | Verdict |
|---|---|---|---|---|
| MechanicsDSL | 2.1.2 → 2.1.3 | compile_dsl | slidercrank/l_over_r=3,ms_over_mc=1e+06 | warned → pass |
| MechanicsDSL | 2.1.2 → 2.1.3 | compile_dsl | slidercrank/l_over_r=1.05,ms_over_mc=10000 | warned → pass |
| MechanicsDSL | 2.1.2 → 2.1.3 | compile_dsl | slidercrank/l_over_r=1.5,ms_over_mc=100 | warned → pass |
| MechanicsDSL | 2.1.2 → 2.1.3 | compile_dsl | slidercrank/l_over_r=3,ms_over_mc=1 | warned → pass |
| MechanicsDSL | 2.1.2 → 2.1.3 | compile_dsl | near_singular/eps=0 | warned → pass |
| MechanicsDSL | 2.1.2 → 2.1.3 | compile_dsl | chain/N=8 | timeout → pass |
| MechanicsDSL | 2.1.2 → 2.1.3 | compile_dsl | chain/N=5 | timeout → pass |
Every case, newest release
Hover a cell for the worst relative error and any message the engine gave. Each case is probed at 8 to 18 states, including the configurations that make it degenerate.
| Case | SymPy rhs · 1.14.0 |
SymPy mass-matrix · 1.14.0 |
MechanicsDSL compile_dsl · 2.1.4 |
Drake multibody-plant · 1.58.0 |
Project Chrono assembly · 10.0.0 (build 1198) ⚑ |
Project Chrono system-matrices · 10.0.0 (build 1198) |
|---|---|---|---|---|---|---|
| chain | ||||||
| N=1 Planar 1-link pendulum, unit point masses on massless rods |
pass 0.0e+00 |
pass 0.0e+00 |
pass 0.0e+00 |
pass 0.0e+00 |
pass 2.0e-11 |
pass 1.7e-13 |
| N=2 Planar 2-link pendulum, unit point masses on massless rods |
pass 1.0e-15 |
pass 1.0e-15 |
pass 1.8e-15 |
pass 3.0e-15 |
silent 7.6e+00 |
pass 2.6e-12 |
| N=3 Planar 3-link pendulum, unit point masses on massless rods |
pass 2.0e-14 |
pass 3.4e-15 |
pass 5.8e-15 |
pass 2.1e-14 |
silent 1.7e+00 |
pass 8.5e-13 |
| N=5 Planar 5-link pendulum, unit point masses on massless rods |
pass 2.9e-14 |
pass 3.8e-14 |
pass 2.6e-14 |
pass 2.4e-13 |
silent 6.1e+00 |
pass 2.6e-12 |
| N=8 Planar 8-link pendulum, unit point masses on massless rods |
timeout |
pass 1.1e-13 |
pass 3.6e-14 |
pass 9.2e-13 |
silent 9.7e+00 |
pass 2.7e-12 |
| near singular | ||||||
| eps=0.1 Two masses, kinetic coupling (1-eps): mass matrix condition number 1.9e+01 |
pass 1.3e-15 |
pass 0.0e+00 |
pass 1.3e-15 |
– | – | – |
| eps=0.0001 Two masses, kinetic coupling (1-eps): mass matrix condition number 2.0e+04 |
pass 3.1e-15 |
pass 0.0e+00 |
pass 3.6e-13 |
– | – | – |
| eps=1e-08 Two masses, kinetic coupling (1-eps): mass matrix condition number 2.0e+08 |
pass 5.0e-16 |
pass 0.0e+00 |
pass 3.3e-16 |
– | – | – |
| eps=0 Two masses, kinetic coupling (1-eps): mass matrix exactly singular -- refusing is correct |
pass |
pass |
pass |
– | – | – |
| mass ratio | ||||||
| ratio=1 Two masses joined by springs, m2/m1 = 1 |
pass 0.0e+00 |
pass 0.0e+00 |
pass 0.0e+00 |
– | – | – |
| ratio=1e+06 Two masses joined by springs, m2/m1 = 1e+06 |
pass 2.1e-22 |
pass 0.0e+00 |
pass 2.1e-22 |
– | – | – |
| ratio=1e+12 Two masses joined by springs, m2/m1 = 1e+12 |
pass 1.0e-28 |
pass 0.0e+00 |
pass 1.0e-28 |
– | – | – |
| cartpole | ||||||
| M_over_m=10 Cart-pole, cart/pole mass ratio 10 |
pass 5.6e-17 |
pass 0.0e+00 |
pass 3.1e-16 |
pass 8.3e-17 |
silent 4.3e-01 |
pass 2.2e-13 |
| M_over_m=1 Cart-pole, cart/pole mass ratio 1 |
pass 2.4e-16 |
pass 0.0e+00 |
pass 2.5e-16 |
pass 2.2e-16 |
silent 1.2e+00 |
pass 1.2e-13 |
| M_over_m=0.01 Cart-pole, cart/pole mass ratio 0.01 |
pass 9.1e-15 |
pass 0.0e+00 |
pass 3.6e-15 |
pass 1.1e-15 |
silent 4.7e+00 |
pass 1.3e-13 |
| M_over_m=0.0001 Cart-pole, cart/pole mass ratio 0.0001 |
pass 5.5e-15 |
pass 0.0e+00 |
pass 2.2e-15 |
pass 0.0e+00 |
silent 3.7e+00 |
pass 9.7e-13 |
| M_over_m=1e-06 Cart-pole, cart/pole mass ratio 1e-06 |
pass 5.7e-15 |
pass 0.0e+00 |
pass 4.8e-15 |
pass 2.3e-16 |
silent 2.8e+00 |
pass 7.1e-11 |
| slidercrank | ||||||
| l_over_r=3,ms_over_mc=1 Slider-crank (closed loop), rod/crank 3, slider/crank mass 1, probed through dead centre |
pass 3.8e-16 |
pass 0.0e+00 |
pass 7.8e-16 |
– | silent 4.8e+00 |
pass 2.6e-13 |
| l_over_r=1.5,ms_over_mc=100 Slider-crank (closed loop), rod/crank 1.5, slider/crank mass 100, probed through dead centre |
pass 6.7e-16 |
pass 5.6e-16 |
pass 5.5e-16 |
– | silent 1.1e+00 |
pass 3.0e-13 |
| l_over_r=1.05,ms_over_mc=10000 Slider-crank (closed loop), rod/crank 1.05, slider/crank mass 10000, probed through dead centre |
pass 3.2e-16 |
pass 2.2e-16 |
pass 1.3e-15 |
– | silent 1.8e+00 |
pass 5.2e-13 |
| l_over_r=3,ms_over_mc=1e+06 Slider-crank (closed loop), rod/crank 3, slider/crank mass 1e+06, probed through dead centre |
pass 1.0e-15 |
pass 2.2e-16 |
pass 4.2e-16 |
– | silent 1.0e+00 |
pass 8.2e-11 |
History
| Engine | Release | Run | Verdicts | Data |
|---|---|---|---|---|
| SymPy (sympy.physics.mechanics) | 1.14.0 | 2026-10-05 | 41 pass 1 timeout | json |
| SymPy (sympy.physics.mechanics) | 1.13.3 | 2026-10-06 | 41 pass 1 timeout | json |
| SymPy (sympy.physics.mechanics) | 1.13.2 | 2026-10-06 | 41 pass 1 timeout | json |
| MechanicsDSL | 2.1.4 | 2026-11-02 | 21 pass | json |
| MechanicsDSL | 2.1.3 | 2026-10-05 | 21 pass | json |
| MechanicsDSL | 2.1.2 | 2026-10-06 | 14 pass 5 warned 2 timeout | json |
| MechanicsDSL | 2.1.1 | 2026-10-06 | 14 pass 5 warned 2 timeout | json |
| Drake | 1.58.0 | 2026-10-16 | 10 pass | json |
| Drake | 1.57.0 | 2026-10-05 | 10 pass | json |
| Drake | 1.56.0 | 2026-10-06 | 10 pass | json |
| Drake | 1.55.0 | 2026-10-06 | 10 pass | json |
| Project Chrono (PyChrono) | 10.0.0 (build 1198) | 2026-10-14 | 15 pass 13 silent | json |
| Project Chrono (PyChrono) | 10.0.0 (build 1197) | 2026-10-06 | 15 pass 13 silent | json |
| Project Chrono (PyChrono) | 10.0.0 (build 1187) | 2026-10-06 | 15 pass 13 silent | json |
| Project Chrono (PyChrono) | 10.0.0 (build 677) | 2026-10-06 | 15 pass 13 silent | json |
How it works
Every day a scheduled job asks PyPI and anaconda.org for each engine's releases. Any release without a result is installed on a fresh GitHub Actions runner (Ubuntu, Python 3.12) and scored on the whole catalogue. Results are committed to the repository, so the history is permanent and anyone can audit it.
At every probe state the engine's generalised accelerations are compared with the reference using maxi |ai − ri| / max(|ri|, 1). The tolerance is 10−8, widened only where the system is so ill-conditioned that no solver, the reference included, can do better: to 10−14 times the condition number of the diagonally scaled mass matrix. Systems that have no defined answer, such as an exactly singular mass matrix, pass only if the engine refuses them.
The references are written in numpy alone, with no symbolic algebra, so they share no code with any engine under test. They come from a study of silent failures in dynamics engines (artefacts: doi:10.5281/zenodo.22315172). Adapters translate conventions such as relative versus absolute joint angles and nothing else; an adapter never corrects an engine's answer.
Where an engine offers two documented ways to get accelerations, both are scored as separate columns. A dash means that engine cannot model the system without changing what the case tests; hover it for the reason.