> a big difference that Common Lisp brings to the table is that it has separate namespaces for variables and functions. This may not sound like much but it makes it much easier to write macros (code that writes code) and facilitates compile-time computing.
A little more detail here: it has a lot more namespaces than that. Indeed, you can attach arbitrary different namespaces of things to symbols if you want to.
It turns out that when you're writing macros in Lisp, the majority of what you use from outside the macro are functions and the majority of symbols that you bind and use within the macro are variables, so if you put them in different namespaces, you get rid of the majority of cases where symbols don't do what you were expecting in a macro.
Now, it's not quite all of the cases, and when it does happen, it's a pain to track down. Which is why the Scheme folks spent the effort and complexity on hygienic macros (where the system rewrites all symbols internally to make variable capture impossible). At the time of Common Lisp's creation, multiple namespaces and unhygienic macros were the obvious choice for production code, since there weren't hygienic systems available. Today you can go either way. In terms of research, hygienic is the route to go, as the work over the last couple of years on strongly typed, hygienic systems has shown.
> A little more detail here: it has a lot more namespaces than that.
And I don't think the function/value split is even the most important
namespacing it does with regards to macro hygiene ("functions are less likely
to be shadowed" always struck me as a kind of weak argument). Symbols being
namespaced under packages is a much more robust solution:
CL:CAR isn't accessible in P, so (car x) by itself would signal
UNDEFINED-FUNCTION, but the macro works correctly. Defining a P::CAR function
wouldn't change anything.
That isn't just for functions, either, eg if you do:
(where you want to leak IT), and then import AWHEN but not IT, you'll get an
error that YOUR-PACKAGE::IT isn't bound to anything when you try to use it; it
doesn't matter that AWHEN-PACKAGE:IT is bound.
I don't disagree with anything you said, I just think it made CL macros look
flimsier than they are.
The problem is that it is quite common for cl:car to be accessible in packages, because of the practice of use-ing the cl package everywhere.
Even if I'm in my own package, I can't do this:
(flet ((car (x) ...)))
(m arg))
because in order to have a hassle-free Lisp coding experience in my package, I brought in all the public symbols from the common-lisp package. So my flet is in fact shadowing cl:car.
Even if I just import the specific common-lisp things I need, and car is not one of them, that could change. Today, that car above is mypkg:car. Tomorrow, someone edits the defpackage to import car (because they needed it in a function they added), and now when that file is re-read, car is cl:car.
Packages have a theoretical solution to the hygiene problem that will not be air-tight in practice due to use/importation.
Another problem is that programmers aren't going to define a large number of packages to protects parts of their program from each other. A common practice is just to make one package for an entire project.
You need fine-grained package use to achieve near perfect hygiene. Ideally, each module that provides macros should have its own package, and use only symbols from that package in the macro expansion.
If module A uses a macro from package B, and that macro generates a local function F, that will be B::F, not interfering with the A::F function. If the modules are in the same master project package, the clash is not averted, obviously.
A little more detail here: it has a lot more namespaces than that. Indeed, you can attach arbitrary different namespaces of things to symbols if you want to.
It turns out that when you're writing macros in Lisp, the majority of what you use from outside the macro are functions and the majority of symbols that you bind and use within the macro are variables, so if you put them in different namespaces, you get rid of the majority of cases where symbols don't do what you were expecting in a macro.
Now, it's not quite all of the cases, and when it does happen, it's a pain to track down. Which is why the Scheme folks spent the effort and complexity on hygienic macros (where the system rewrites all symbols internally to make variable capture impossible). At the time of Common Lisp's creation, multiple namespaces and unhygienic macros were the obvious choice for production code, since there weren't hygienic systems available. Today you can go either way. In terms of research, hygienic is the route to go, as the work over the last couple of years on strongly typed, hygienic systems has shown.