Approaching the world of statistics from the perspective of a musical empiricist, I was actually a bit surprised to learn that the core curriculum of OSU's Statistics PhD program was so mathematical. After all, it would seem that study of uncertainty itself -- and statistical methods for quantifying it -- ought to be motivated by either philosophical or practical concerns. It therefore seemed a non sequitur that Statistical Theory I (620) would begin with mathematical first principles and proceed deductively.
I was appropriately humbled, then, when I became aware of how imprecise natural language truly is compared to mathematical ones. Statistical theory can't be defined with sufficient precision using the language of a philosopher or practician (hat tips on this topic to both Bertrand Russel and Richard Feynman). Regardless, as a non-mathematician, more suggestions for how to think like one would have been very, very useful.
In honesty, I still can't claim to have the same mature intuition for mathematical language and relationships that I have for musical ones, notwithstanding the considerable overlap (and arguable essential equivalence) of the fields. Nonetheless, I have begun to enjoy a certain amused elan when I stumble upon unexpected fluency with certain mathematical ideas, despite their basicness.
It seems to me that most universally applied aspects of statistics -- namely mathematical models to help estimate unobservables -- cannot be intuitively understood without geometric imagination. They probably can't be effectively taught unless distance is introduced as a core concept. The typical human mind (and most of ours are typical) has capabilities reflecting adaptive solutions to evolutionary imperatives, and, at least for me, I can think of no better crutch than visual and spatial reasoning with which to approach statistical modeling. This idea is obviously not a new one.
What I've only now begun to perceive, however, is that mathematicians must be thinking and reasoning spatially, even when they're actually representing this reasoning by manipulating equations, functions, and other symbols of notation. Even if the last several centuries of mathematical work have been dedicated to formalizing ideas into algebra for more precise (or even automated) manipulation, I'd be willing to bet that geometrical thinking still drives our intuition.
Perhaps my more mathematically-minded friends would be able to confirm or deny!