In this world, think of magic as a great lake. When someone casts a spell, it’s as if the person dipped their hand in and took a scoop of water out, with the accompanying ripples radiating out from the point of contact. The more powerful the spell cast, the greater the scoop, thus, the greater the ripples caused and the farther they reach. These “magic ripples” manifest themselves in a variety of ways from visual, to auditory, to physical even. The higher level the spell and the more powerful the caster, the greater the effect the ripple has on the surrounding area. Thus attempting to be subtle when casting spells is very difficult, especially the higher the power you command.
In ages past, there were great wars between mages of unbelievable power and the effects unleashed by their struggles were sometimes cataclysmic in scope. The sheer magnitude of the energy they were manipulating caused unintended devastation to the poor folks that happened to be nearby. It is largely due to the backlash of these titanic battles that have caused such fear and hatred against those who use magic. Now that the great mages are gone, people have taken drastic measures to ensure that they never return.