When faced with a problem, always start with the simplest solution.

Or

When you have two competing theories which predict the same outcomes, the simpler one is the better.

See also

Ockam’s Razor