Overloaded functions are not your friend
Haskell's overloading (using type classes) is elegant, neat, etc., etc., but it is death to performance if left to linger in an inner loop. Each type class constraint on an overloaded function corresponds to an extra argument passed at runtime (called a dictionary), and a reference to an overloaded function is implemented by extracting a field from the dictionary. The presence of overloading can prevent many other optimisations, such as Strictness, from kicking in.
How can you squash overloading?
Give explicit type signatures: Signatures are the basic trick; putting them on exported, top-level functions is good software-engineering practice, anyway. (GHC Tip: using -fwarn-missing-signatures can help enforce good signature-practice).
The automatic specialisation of overloaded functions (with -O) should take care of overloaded local and/or unexported functions.