Load Common Lisp Module Programmatically

Question

I just started learning Common Lisp a few weeks ago, so sorry if this is an obvious question. How can I load modules programmatically? I have a directory of "tasks", each a Lisp program, and would like to import each and run a specific function that they all contain.

I found out a way to iterate over a directory (via UIOP:DIRECTORY-FILES). But I'm stuck trying to figure out a way to load a module "as" a specific name, as in Python. That would allow me to load "module-1.lisp" as mod and then load "module-2.lisp" as mod in a loop.

Pseudocode:

for path in directory
    (load path as mod)
    (mod:function)

If there is a better way to achieve what I am trying to do, feel free to say so! Thanks in advance for any help!

question from:https://stackoverflow.com/questions/65847843/load-common-lisp-module-programmatically

Answer 1

Quick summary of packages and systems

"Loading as" is not something meaningful in Common Lisp, because loading is not the same as defining a new system or package. In Python, a file is a module etc. but not in Lisp.

Lisp has a concept of packages, which are namespaces. They are used to organize symbols (symbols are first-class values). In other words, they use, export or maybe shadow symbols from other packages and that's it. When you evaluate (in-package pack), you can write all accessible symbols from pack directly, like my-fun, otherwise you need to fully qualify symbols, as some-package:their-func. There are extensions that allows you to import a package with a different name, but this is not standard (see https://github.com/phoe/trivial-package-local-nicknames)

Loading a Lisp script is not necessarily the same as defining a package, it depends on what belongs in the file (if it as defpackage forms).

When you want to organize your source code in Lisp, you define a system, using ASDF. A system list all its dependencies, its components (files) and the dependencies between their components. That's how you describe in which order files should be loaded, compiled, tested, etc.

Packages and systems are independant but in small systems you often have the the same name for a system and the unique package it defines. For larger systems there might be multiple packages.

See 21. Programming in the Large: Packages and Symbols.

Your question

Each of your file needs to define a package, to avoid polluting a single namespace, for example:

(defpackage :common-name.01
   (:use :cl :utils)
   (:export #:run-me))

(in-package :common-name.01)

;; some code

But once they are all loaded, either using your approach or by defining a proper ASDF system, you want to be able to access all the RUN-ME functions, in all the packages.

You can write some introspective code that lists packages, etc. but I think a better approach would be to have a way for each of your file to declare that their RUN-ME functions should be registered in your framework, like test frameworks do.

For example, at the end of your files, you could write:

(provide-function 'run-me)

This assumes that e.g. your utils package defines and exports a function named provide-function that stores values in a central registry.

For example, in utils.lisp:

(defpackage :utils
   (:use :cl) 
   (:export #:provide-function
            #:run-all))

(in-package :utils)

(defvar *all-interesting-functions* nil)

(defun provide-function (f)
  (pushnew f *all-interesting-functions* :test #'eql))

And when you want to run all the functions, you can iterate over this list:

(defun run-all ()
  (mapcar #'funcall *all-interesting-functions*))