If Python Had Lambdas…

…the world might be a bit of a better place, but not really.

“But Quad, Python does have lambdas. Look!”

>>> double = lambda x: x+x
>>> double(5)
10
>>> quadruple = lambda x: double(double(x))
>>> quadruple(5)
20

I tell you, they’re lies.

SICP taught us message passing. Here’s a variation of an example they used.

>>> def complex_number(a, b):
...     return (lambda msg:
...                 if msg == "real part":
  File "<stdin>", line 3
    if msg == "real part":
     ^
SyntaxError: invalid syntax

BZZT! No can do. Lambdas can’t contain keywords/control flow (well, ternary if). Lambdas contain expressions. Instead, we have to give it a name.

def complex_number(a, b):
    def dispatch(msg):
        if msg == "real":
            return a
        elif msg == "imag":
            return b
        else:
            raise Exception("Invalid message.")
    return dispatch

[Edit 4/4/12: To make this less trivial, instead of returning just a, have it print it additionally.]

And with that, we can do the following.

>>> z = complex_number(1, 2)
>>> z("real")
1
>>> z("imag")
2
>>> z("complex")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 8, in dispatch
Exception: Invalid message.

In Scheme or Lisp, we don’t need this extra naming baggage. We can make lambdas do arbitrary things. The equivalent of the Python definition using lambdas would be:

(define (complex-number a b)
  (lambda (msg)
    (cond
      ((eq? msg 'real) a)
      ((eq? msg 'imag) b)
      (else (error "Invalid message.")))))

And with this, we have:

> (define z (complex-number 1 2))
> (z 'real)
1
> (z 'imag)
2
> (z 'magic)
error: Invalid message.

In the Scheme code, we could literally return a lambda function itself. In Python, if we want the full power of functions, we need to give them a name.

This reminds me of the C era of programming. In order to provide a callback to, say, some GUI action, we needed to make a function (with a name). But of course, functions in Python have an advantage: they can be created in many places (but not everywhere! keep reading.) and they can grab data from the environment in which they’re created (closures).

So, if we can def stuff in most places in Python code, then we can replace lambdas somewhat easily. A “true” lambda with unrestricted usage,

lambda NAME(ARGS):
    BODY

could be translated into

def NAME(ARGS):
    BODY
NAME

If only we could do that…

In Common Lisp, we have something like def that works for internal definitions (definitions inside another function definition). The mechanism is called labels. The translation of the complex_number function in Python to Lisp is this.

(defun complex-number (a b)
  (labels ((dispatch (msg)
             (cond
               ((string-equal msg "real") a)
               ((string-equal msg "imag") b)
               (t (error "Invalid message.")))))  ; end labels defs
    #'dispatch))

This is essentially the same.

Now, suppose Lisp had lambdas that were as bad as Python’s, but labels worked just as well as Python’s def.

Behold, the power of macros!

(defmacro %lambda ((&rest ARGS) &body BODY)
  (let ((NAME (gensym)))
    `(labels ((,NAME (,@ARGS)
                ,@BODY))
       #',NAME)))

Look ma, a lambda maker!

The way this works is this. We generate a symbol using gensym and give it to NAME (Scheme doesn’t need to do this, since macros are hygienic). Then we fill in the “structure” which is prepended with a backquote. The places we fill things in are prefixed with a comma (or a comma-at, which splices in longer bits of code, like the body of a function). In a sense, we are just filling a template out.

Let me prove to you it works!

First define the function-returning-function:

(defun complex-number (a b)
  (%lambda (msg)
    (cond
     ((string-equal msg "real") a)
     ((string-equal msg "imag") b)
     (t (error "Invalid message.")))))

And then try it.

CL-USER> (defvar z (complex-number 1 2))
Z
CL-USER> (funcall z "real")
1
CL-USER> (funcall z "imag")
2

(Note: we need that extra funcall due to the nature of how Common Lisp functions work.)

We can check further stuff, like being able to list lambda functions.

CL-USER> (list (%lambda (x) (+ 2 x))
               (%lambda (p) (* 2 p)))
(#<function 293DAB6A> #<function 293D22A2>)

Unfortunately, even if Python had macros, this would fail (ergo, Python defs aren’t really lambdas either, in that they cannot be created anywhere, and aren’t first class in any sense). For example, you cannot make a list of these defs or return one. Python chose not to have the equivalent of begin or progn which starts a new “block” that returns its last value. And as a result, we can’t just define a new function anywhere a value could be used. Instead, we have to create the function where a function can be defined, and then put the function value elsewhere. This has the unfortunate effect of leaky, oozing “encapsulation” (kind of like Python’s way of doing classes!). Concretely, to make functions available in a list, one must do [Edit 4/4/12: This was a poor example. The lambdas need to be less trivial. Consider adding a print to f and g]:

def funlist():
    def f(x):
        return 2+x
    def g(p):
        return 2*p
    return [f, g]

So, despite people telling me how great Python is, and how you can work around the fact that lambdas aren’t first class, I still wonder, have any of you guys ever read SICP?

18 comments to If Python Had Lambdas…

  • Nice post. Python lambdas are painfully restrictive. But they are pretty darn useful. I guess the philosophy is that if you need something more complex than an expression go write a method. Agreee or disagree, that appears to be the fact by design (mostly because GVR said so)

    Just one thing, ternary operator was introduced in python sometime ~v2.5

    Example:
    >>> a=1
    >>> b=2

    >>> ternary=lambda msg: a if msg==”test” else b
    >>> ternary(“test”)
    1

    #Some people also like to use boolean as ternary (ugly i know, python returns the value in case #expression evals to true)

    >>> ternary=lambda msg: msg==”test” and a or b
    >>> ternary(“test”)
    1
    >>> ternary(“blah”)
    2

  • anish

    Use following syntax

    def complex_number(a, b):
        return (lambda msg: (msg =="real" and a) or ( msg=='imag' and b ))
    

  • I don’t know much about Lisp, and I haven’t read SICP (though it is now on my reading list). That being said, I too have been frustrated with Python’s lambdas. There are still a few hacks to get around the lack of flow control through. The first would be:

    def raise_exception(err):
        raise Exception(err)
    
    def complex_number(a,b):
        return (lambda msg:a if msg=='real' else (b if msg=='imag' else raise_exception('fail')))
    
    z = complex_number(1,2)
    print z('real')
    print z('imag')
    print z('cheeseburger')
    

    and the second would be:

    def complex_number(a,b):
        return lambda msg:[None,a,b][(msg=='real')+ 2*(msg=='imag')]
    z = complex_number(1,2)
    print z('real')
    print z('imag')
    print z('cheeseburger')
    

    Neither of these are good and both are quite ugly, but they work in a pinch.

  • Well, to your first example, you can use a lambda, you just have to use the expression version of if/then/else instead of the statement version.

    def complex_number(a, b):
         return lambda msg: a if msg == "real" else b if msg == 'imag' else None
    

    Of course, this doesn’t actually raise an exception. If you want it to raise an exception, you can use a dictionary (which is basically what you are doing right? A dynamic string lookup)

    def complex_number(a, b):
        return lambda msg: dict(real = a, imag = b)[msg]
    

    That’s a lot cleaner too, and in python readability is a large focus. But in reality, using a string argument to the lambda is a bit silly. Yes, it’s do-able, but part of learning to use Python really effectively is balancing when a functional concept is more useful, and when an imperative or object-oriented approach is more useful. In Python we already have a facility for looking up keys in a dictionary and throwing an exception when the key isn’t found. It’s called a class:

    class complex_number(object):
        def __init__(self, a,b):
            self.real = a
            self.imag = b
    >>> complex_number(3,4).real
    3
    >>> complex_number(4,5).imag
    5
    

    You’re probably thinking that’s a lot of overhead for one little thing that is basically a tuple, and you’d be right. There is a nice convenience function for created these “tuples with field names” called namedtuple. This is how I would write complex_number if I had to do it in real Python code:

    from collections import namedtuple
    
    complex_number = namedtuple('complex_number', 'real imag')
    

    Again, I definitely agree that some strategies are more verbose in Python, but that’s just because python is very opinionated about making things clear to read and having a correct way to do things. Once you code in it for a while, you start to see convoluted code as a warning sign that there is a better way to do it.

  • Also, real quick, the final example is easy enough to use lambdas for:

    def funlist():
        return [lambda x: 2 + x,
                lambda p: 2 * p]
    

    That’s a great example of how lambdas are first class

  • No, the CL code:

    (list (%lambda (x) (+ 2 x))
          (%lambda (p) (* 2 p)))
    

    …is this in Python:
    [lambda x: x+2,lambda p: p*2]
    

    …which is more concise, as I find is usually the case.

    • Note, I was testing that these lambdas could be listed.

      I could enrich this by having state or non-trivial control flow in the CL lambdas, and they would still be listable.

      I have added edits (but have not changed for posterity) to the post to reflect the point I was trying to convey.

    • SciK

      In Perl 6:

      my @functions = * + 2, * * 2;
      # or:
      my @functions = {$_ + 2}, {$_ * 2};
      # or:
      my @functions = {$^x + 2}, {$^p * 2};
      # or:
      my @functions = -> $x {$x + 2}, -> $p {$p * 2};
      # or:
      my @functions = sub ($x) {$x + 2}, sub ($p) {$p * 2};
      
      # now, if I want to apply them both to 5:
      say @functions».(5);
      
      # → 7 10
      

  • You already know that Python hasn’t got a “real” lambda, so why don’t you use a class, which I’m sure you already know is Python’s “real” replacement for lambda?!

  • Stu

    I’m curious what a python syntax for lambdas might look like, care to make an example ?

  • As for your edit about printing in the lambda, the print function is one of my favorite things about Python 3. I often find myself import from __future__ when I’m using python 2 just so I can print in a lambda or map.

  • Well I 2nd your point that Python doesn’t have “real” lambda, and I know that lambda is the ultimate so I guess we are out of luck in Python land for now lol.

  • Roel

    g = lambda helper: (lambda b: {True: “ok”, False: “nok”}[helper(b)])
    f = g(bool)

    f(15) # –> “ok”
    f(True) # –> “ok”
    f(None) # –> “nok”
    f(False) # –> “nok”

  • Now, now, if you could put anything in a lambda, you might just try to create a lambda that returns a class. Now, we can’t have that, could we?

    That’s a made up problem, by the way: I’ve never hand the desire to create a class inside a lambda. Of course, I could *almost* imagine why that would be useful…and if I ever stumbled onto a problem where it would be *very* useful to do such a thing–indeed, if it would simplify my problem greatly–I’m up a creek without a paddle, because of Python’s arbitrary restrictions.

    Sigh.

Leave a Reply

  

  

  

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Before you post, please prove you are sentient.

what is 4 plus 7?