C# 2 and yet to be released VB 9 allow developers to reference local variables in anonymous functions. When an anonymous function 'closes over' a variable, the local variable is promoted to an instance variable and stored in an object called a closure. This allows the variable to exist long after the method call that created it has been completed.
The closure referenced by anonymous functions is normally eligible for garbage collection when any delegates pointing to the anonymous functions go out of scope. Notice that functions is plural. All anonymous functions in the same scope share the same closure. Eric Lippert thinks this point is not being given enough attention.
However, a related issue which I haven't seen anyone talk much about is what the consequences of having one closure per scope are. Though it makes the closure semantics conceptually easier to think about (and implement!), it can lead to an unfortunate problem with garbage collection.
In the example he gives on his blog, two anonymous functions are created. The short-lived function is released immediately while the long-lived function is returned from the method and may exist for an indeterminate amount of time.
The closure for the short-lived delegate owns the expensive resource. But since there is one closure per scope, both the short-lived and the long-lived delegates own a single closure. The closure cannot be collected until every delegate that owns it is dead. Therefore the expensive resource is not released until the long-lived delegate is released, even though the long-lived delegate does not reference the expensive resource!