The Dark Side of Decorators

Recently a bug report was filed on the Flask-Classy issue tracker at Github which caught me by surprise. This was a bug so glaring that the fact I hadn’t seen it myself was a shock, but even more shocking was that nobody else had reported it either.

The bug was simple to describe:

If you used any decorator (but didn’t use the @route decorator), Flask-Classy would not auto generate the correct route.

To be honest, when I realized the bug was related to decorators I wasn’t that surprised. I’d always known that there was something funky with them and I even hinted about that in the docs. As it’s turned out though I haven’t used decorators with FlaskViews that much and when I did I always had a @route decorator in the mix.

Fortunately the issue submitter – @shuhaowu – was awesome enough to not only submit some failing tests, but also took the time to do some research into the problem It turns out that decorators obfuscate the signature of the method or functions they are applied to.

This comes as absolutely no surprise, since decorators are essentially syntactic sugar for wrapping a function with another function. Somehow though the implications of this escaped me when I was writing earlier versions of Flask-Classy.

To illustrate the effect this has on decorated methods in a class take a look at this code snippet:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
from inspect import getmembers, getargspec
from functools import wraps

# Some super basic decorators
def std_decorator(f):
    def std_wrapper(*args, **kwargs):
        return f(*args, **kwargs)
    return std_wrapper

def wraps_decorator(f):
    @wraps(f)
    def wraps_wrapper(*args, **kwargs):
        return f(*args, **kwargs)
    return wraps_wrapper

# A simple class with example decorators used
class SomeClass(object):

    def method_one(self, x, y):
        pass

    @std_decorator
    def method_two(self, x, y):
        pass

    @wraps_decorator
    def method_three(self, x, y):
        pass

obj = SomeClass()
for name, func in getmembers(obj, predicate=inspect.ismethod):
    print
    print "Bound Name: %s" % name
    print "Func Name: %s" % func.func_name
    print "Args: %s" % getargspec(func)[0]

Which outputs the following:

Bound Name: method_one
Func Name: method_one
Args: ['self', 'x', 'y']

Bound Name: method_two
Func Name: std_wrapper
Args: []

Bound Name: method_three
Func Name: method_three
Args: []

As we can see, the first method is totally transparent. The second method though has been completely obfuscated. We’re only able to see the original method name because inspect.getmembers was kind enough to share the bound name. But the original arguments? Completely gone. The functools.wraps method at least keeps the original method name intact, but without the arguments we don’t have enough information to construct a meaningful route that will map back to this method at runtime.

So what’s left? How can we get the argspec of the base method that’s been decorated? Fortunately for us python’s reflection capabilities are more than enough to get us there. Here’s a function that can get the base method’s argspec from most decorated methods. I’m certain there must be a case where this will not work, but for all the cases I’ve tried it works fine. (Remember, this only works for methods of a class, not plain functions.)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def get_true_argspec(method):
    """Drills through layers of decorators attempting to locate 
       the actual argspec for the method.
    """

    argspec = inspect.getargspec(method)
    args = argspec[0]
    if args and args[0] == 'self':
        return argspec
    if hasattr(method, '__func__'):
        method = method.__func__
    if not hasattr(method, 'func_closure') or method.func_closure is None:
        raise Exception("No closure for method.")

    method = method.func_closure[0].cell_contents
    return get_true_argspec(method)

Now, I haven’t run any speed tests on this, but I presume by it’s very nature that it’s going to perform much slower than a typical inspect.getargspec (if for no other reason than the fact that it makes that exact call one or more times). Fortunately in the case of Flask-Classy, the price is only paid during application loading and won’t have any impact on an actively running application.

Comments