[WARNING: This page is hasn't been updated since 2015.]
Lazy evaluation is a method to evaluate a Haskell program. It means that expressions are not evaluated when they are bound to variables, but their evaluation is deferred until their results are needed by other computations. In consequence, arguments are not evaluated before they are passed to a function, but only when their values are actually used.
Lazy evaluation is part of operational semantics, i.e. how a Haskell program is evaluated. The counterpart in denotational semantics, i.e. what a Haskell program computes, is called Non-strict semantics. This semantics allows one to bypass undefined values (e.g. results of infinite loops) and in this way it also allows one to process formally infinite data.
While lazy evaluation has many advantages, its main drawback is that memory usage becomes hard to predict. The thing is that while two expressions, like
2+2 :: Int and
4 :: Int, may denote the same value 4, they may have very different sizes and hence use different amounts of memory.
An extreme example would be the infinite list
1 : 1 : 1 … and the expression
let x = 1:x in x. The latter is represented as a cyclic graph, and takes only finite memory, but its denotation is the former infinite list.