Things I Want in Common Lisp

[Updated 04 September 2014]

This post contains some things I want in Common Lisp, in no particular order. I’ll try to keep things short and to the point.

Shared Libraries

Right now, the concept of a “library” in idiomatic Lisp is a .asd file along with some Lisp source code. I don’t always want to distribute the source code. (Though most of the time I do.)

It would be nice if we had both statically and dynamically linkable/loadable libraries like those from C. Of course Lisp’s model of how things work doesn’t quite match up to this, but I think it could be done if implementations standardized on some loading (and unloading?) strategy.

As far as I know, the closest thing we can do to this is something like concatenating FASL files. But I would like something more robust and semi-standard across implementations.

Standard Calling Interface

Lisp isn’t too friendly with the outside world. It is difficult to call Lisp code from another application, unless you’re using a special implementation. Right now, message passing seems to be the only way to go.

It would be nice if there was some standard interface to call into Lisp with. Maybe even with just other Lisp applications. Something like a calling convention.

A Graphics/GUI Library

I honestly can’t recommend Lisp to anyone personally who wants to do graphical programming. There’s the commercial LispWorks CAPI which works relatively well, but it comes at a cost.

Right now, all of the existing open source “solutions” are junk. Qt is junk. SDL is marginally better but still junk. Garnet is old and junk. McCLIM is broken and junk (though it’s interesting!). Tcl/Tk is antiquated and junk.

This has been the largest pain point for me for writing open source scientific simulation code.

Better Type Abstractions

I want to be able to have, for example, a hash table that can specialize efficiently on different types. In Lisp, you essentially either support every type (perhaps with a few restrictions) inefficiently, or you support a single type efficiently.

I don’t think there exists a good way to have generic and efficient code reuse right now. All of the polymorphic stuff built in to Lisp is efficient only because the compiler knows about it.

I ranted about this a little bit here.

TYPECASE is not a solution. Inlining is not a solution.

Polymorphic/algebraic/recursive types that the compiler knows about is probably the most difficult thing on this list, since it gets into language semantics.

An Interface/API Mechanism

[Updated 27 July 2014]

It would be nice to programmatically specify interfaces/APIs. C has header files. C++ has class declarations. SML has signatures/structures/functors. I would love to in my source code to specify programmatically what is provided.

Packages do not solve this problem. Packages just let us group symbols together. Packages cannot tell the difference between a function, type, class, or variable. Many people still use packages as the way to write an “interface” though. I do, usually with an annotation of what each symbol is. But that isn’t very good.

I’ve tried my hand at some sort of DEFINE-API and DEFINE-INTERFACE mechanism. See below for work on the latter. FarĂ© also made a very extensive library with how he thinks interfaces might work. He also wrote a paper about it for ILC2012. Some people (e.g., drewc) have said LIL has made it into their daily routine.

As an extension, although it doesn’t quite fit with the philosophy of generic functions, it would be nice to specify which methods a subclass should implement to really be a subclass of that class. And it would be nice if this was checked statically. I started working on something at least dynamic many years ago but never finished it. You can find it here.

Interfaces/Implementations

I implemented some form of interfaces and implementations. It allows one to write things like this:

(define-interface stack ()
  (make-stack (&rest r))
  (push-stack (s x))
  (peek-stack (s))
  (pop-stack (s)))

(define-implementation list-stack (stack)
  :make-stack
  (lambda (&rest r)
    r)

  :push-stack
  (lambda (s x)
    (cons x s))

  :peek-stack
  (lambda (s)
    (car s))
  
  :pop-stack
  (lambda (s)
    (cdr s)))

(define-implementation vector-stack (stack)
  :make-stack
  (lambda (&rest r)
    (let ((length (length r)))
      (make-array length
                  :adjustable t
                  :fill-pointer length
                  :initial-contents r)))
  
  :push-stack
  (lambda (s x)
    (vector-push-extend x s)
    s)
  
  :peek-stack
  (lambda (s)
    (aref s (1- (length s))))
  
  :pop-stack
  (lambda (s)
    (vector-pop s)
    s))

It can be used like this:

CL-USER> (pop-stack vector-stack
                    (push-stack vector-stack
                                (make-stack vector-stack 1 2 3)
                                5))
#(1 2 3)
CL-USER> (pop-stack list-stack
                    (push-stack list-stack
                                (make-stack list-stack 1 2 3)
                                5))
(1 2 3)

This will complain if all implementation functions aren’t there.

I think it’s a start, but I’ve yet to see performance and how it actually works day-to-day.

Better Static Analyzers

It would be nice if we could have better static analysis tools for Lisp that aren’t baked into a single Lisp compiler. Lisp should be amenable to this, even if the analyzer has to make some bold assumptions about the code.

Better Executable Support

Whole program optimization, tree shaking, all that would be nice. There’s LispWorks’ deliver mechanism, which has some sophisticated features, but that’s about it. Everyone else just dumps out an image with a little bit of start up code, and maybe some compression.

Fast start-up times would also be very useful for small, oft-called utilities.

Seamless Arbitrary Precision Floats

Arbitrary precision floats (including complex floats) would be nice if they were seamlessly integrated into the language compiler/runtime without any performance penalty on other types.

CLISP has this, but that’s it. An MPFR interface is in development. (SB-MPFR, Fateman’s MPFR interface.) Something portable would be nice. I myself started working on an MPFR interface with CFFI but have little to show. And of course it’s not integrated with the rest of Lisp like all of the other numeric functions are.

A standard way of controlling precision would need to be developed. I think this is a non-trivial problem; simply setting some global *long-float-precision* might not be good enough.

Numeric Tower Extension

It would be nice if there was an efficient, portable way to extend the numeric tower.

If I make my own numeric type that allows coercion, arithmetic, and so on, integrating it into Lisp’s existing library of numeric functions would make it a lot more usable.

Damage Control Mechanisms

Lisp has some built-in things to ensure that standard symbols and definitions are not screwed around with. It would be nice if implementations had some options to protect the current state of the Lisp image from being destructively modified in certain ways. It would also be nice if some operations were reversible, such as unloading a library or cleanly removing the existence of a package (and ensuring it gets cleaned up from memory).

Efficient Finalizers on Classes

It would be nice if there existed efficient class-wide finalizers, so there could be smoother integration with C stuff. If a Lisp object gets deallocated, then the corresponding C stuff gets free’d as well. (Of course, this should be programmed by the programmer, not some special construct.)

Right now, trivial-garbage allows one to set finalizers on individual Lisp objects, and of course we can set a finalizer for every object allocated by a class easily.

Predictable Tail Call Elimination

I don’t know if it would be right to bake TCE into the standard itself, but it would be nice if I could turn it on and off via some DECLARE or DECLAIM syntax.

Some implementations of course do some TCE with the right compiler policy, but more granularity and less “you get TCE and the kitchen sink of optimizations” would be nice.

Purity Annotations

It would be nice if I could declare a function as pure, even if it doesn’t appear to be pure locally. That way the compiler could easily elide/inline/precompute it. Something like this, ignoring the fact this isn’t a good way to compute factorial:

(declaim (pure fact))
(defun fact (n)
  (let ((result 1))
    (loop :while (plusp n) :do
      (setf result (* result n))
      (decf n))
    result))

Coroutines

[Updated 04 September 2014]

As in the generators/yield/whatever you want to call them. They can be implemented with call/cc. They exist (and are a huge boon) in Python. I want them in Common Lisp.

CL-CONT is good enough to make basic ones, as I’ve done here, but I want them efficient and general to support almost all of CL without any artificial limitations. They’re useful for functional programming (especially stream-based programming) and a million other things.

I made a half-baked library for using generators called poor-mans-generators, but they’re really “completely broke man’s generators”.

4 comments to Things I Want in Common Lisp

Leave a Reply

  

  

  

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Before you post, please prove you are sentient.

what is 2 + 6?