Let's look at various ways languages support extensiblity to understand one of macro's benefits.

Our definition of a language is based on the idea that a programming language consists of a core and a standard library built on top of that core.

One benefit of this perspective is that it is easy to understand and implement. But the real advantage is that it makes the language highly extensible—what we consider part of the language core may actually be implemented by the standard library.

In Common Lisp, while supporting the standard library, macros provide another way to extend the language.

Macros allow you to define your own syntax, determining how s-expressions are transformed into Lisp forms. By incorporating macros into the core of the language, we can create new syntax to build the standard library without having to embed that new syntax into the core itself.

For example, Common Lisp provides the special operator IF, but IF only accepts a single Lisp form. If you need to execute more than one form, you have to wrap them with PROGN.

  (if (spam-p current-message)
      (progn
        (file-in-spam-folder current-message)
        (update-spam-database current-message)))

If you frequently find yourself writing IF together with PROGN, it indicates a need for a new feature (or new syntax) to meet your requirements. This new syntax can be implemented using macros, without needing to incorporate it into the language core.

  (defmacro when (condition &rest body)
    `(if ,condition (progn ,@body)))

  (defmacro unless (condition &rest body)
    `(if (not ,condition) (progn ,@body)))

Another example: AND and OR are implemented as macros in Common Lisp, whereas in other programming languages (C, Java, Python), they are often part of the language core.

Yet another example: Loop control structures in Common Lisp are built using macros (based on TAGBODY and GO), while in other programming languages, such control structures are typically part of the language core.

Defining Macros

The key to understanding macros is to grasp the distinction between the code that generates code (the macro) and the code that is generated. When we define a macro, we write code that the compiler uses to generate the resulting code, which is then compiled and executed. Execution can only occur after all macros have been expanded into code and that code has been compiled.

The time when a macro executes is called macro expansion time, while the time when normal program execution happens is called runtime.

It's crucial to always keep this distinction in mind, because the environment in which code executes during macro expansion time is different from the environment during runtime. In other words, data that exists only at runtime cannot be accessed during macro expansion time.

The basic syntax for defining a macro is:

  (defmacro name (parameters)
    "Optional documentation string."
    body-form*)

What a macro does is translate a macro form (a Lisp form whose first element is the macro name) into code that performs specific operations.

Steps for defining a macro:

1. Clearly determine what code you want to generate and how the macro will be called.
2. Based on the macro's parameters, write the macro that can generate the desired code.
3. Ensure the macro is free from leaks.

Rules of thumb for avoiding leaks:

1. Unless there is a specific reason to alter the evaluation order, ensure that subforms in the expansion are evaluated in the same order as they appear in the macro call.
2. Unless there is a specific reason to evaluate something multiple times, create variables in the expansion to hold the evaluation results of argument forms, and use these variables wherever the value is needed subsequently, ensuring each subform is evaluated only once.
3. During macro expansion, use GENSYM to generate variable names needed in the expanded code.

Backquotes

Backquotes (`) placed before an expression can prevent evaluation, similar to quotes ('). However, within a backquoted expression, any subexpression preceded by a comma (,) will be evaluated.

For example:

CL-USER> `(and ,(list 1 2 3))
(AND (1 2 3))
CL-USER> `(and ,@(list 1 2 3))
(AND 1 2 3)  

Backquote-at(,@) is similar to quote('), but it expands the list, using the elements of the list directly.

Sometimes we see double backquotes (``), and also double commas (,,).

How should we understand the operation of backquotes and commas?

• One backquote delays evaluation once.
• One comma cancels the evaluation delay once.

The Evaluation of Macro Forms

The evaluation of macro forms is different from function call forms.

The evaluation of a macro form proceeds in two stages:

1. The elements of the macro form (unevaluated) are passed to the macro function;
2. The form returned by the macro function (called its expansion) is evaluated according to the normal evaluation rules.

Each macro defines its own local syntax by specifying how a symbolic expression in a macro form is to be transformed into its expansion.