%eclent; ]> Internals &ECL; is an implementation of the Common-Lisp language that is based on a kernel written in C plus a set of libraries written in Common-Lisp. The kernel includes a bytecodes compiler, an interpreter, and enough functions to create and manipulate all lisp objects. The lisp libraries provide higher level constructs such as macro definitions, LOOPs, an implementation of CLOS, and a translator from Lisp to C. As a result of this design, which dates back to the Kyoto CL and was later improved in Giuseppe Attardi's ECoLisp, &ECL; can be used as As a standalone implementation of the Common-Lisp language As an embedded interpreter subject to the control of a larger C program. As a Common-Lisp environment with C/C++ extensions. This manual describes the facility of &ECL; to interface the C language and &ECL;. With this facility, the user can arrange his or her C-language programs so that they can be invoked from &ECL;. In addition, the user can write Lisp function definitions in the C language to increase runtime efficiency. Building programs In this section we describe how you can use &ECL; to build programs and loadable extensions that you can later on distribute to other people.
What can &ECL; do? Some day for some reasons you will be in the need to distribute code that has been developed using &ECL;. In the following sections we will describe the means that &ECL; offers you to do so. Basically, these are the alternatives Source code You distribute your programs in source code form. This is the easiest and most portable way, but not the fastest one. Standalone programs You translate all your lisp code to C using the &ECL; compiler. The final object files can be linked against other C/C++ libraries to obtain a standalone executable. You can build statically and dynamically linked libraries. You translate all your lisp code to C and combine the resulting object files into a single library with .a extension. You can distribute this library to other people and the final users can utilize these libraries to build standalone programs. You can build dynamically loadable files. This is the most flexible way. You translate all lisp code to C and link it against possibly other C/C++ libraries to obtain a dynamically loadable library (file type .so under unix). This library can be loaded a startup time to add new functionality to the &ECL; environment. In several of these options, we have mentioned the possibility to include C/C++ code. Even if this is possible, you cannot use ordinary C/C++ compilers and makefiles to build &ECL; extensions, let it be programs or libraries. Briefly, you have to organize your code as follows Organize the C code as a library, let it be static or dynamic. Build a function, say mymain(), in which the initialization phase for your library is performed. Group the code that interfaces to Lisp in separate C files, all of which should include #include <ecl/ecl.h> at the beginning. Compile your lisp source files. Let &ECL; build the final executable or library. In the final step there are ways to instruct &ECL; to call your initialization function (mymain() in the example above). These means are explained in the following sections.
Compiling files &ECL; supports two types of compilation. One is bytecodes compilation. This process is performed on-the-fly, as you load source files with lisp code. This leads to a series of bytes for each instruction, the so called "bytecodes". These bytecodes are interpreted in a virtual machine, which is written in C and which is reasonably fast. The other type of compilation is the so-called "native" compilation. This process consists on translating the lisp source file to C language. The intermediate file is later compiled using a C compiler. The result is an object file which may have different purposes. Dynamically loadable files or FASL (FASt Loadable) files These are produced in a &ECL; built with support for dynamically loadable libraries (Feature :DLOPEN is in *features*), when no extra arguments are passed to compile-file. These object files typically have the .fas extension, and can be loaded with load. They cannot be used to build libraries nor standalone executable programs. linkable object files These are produced when invoking compile-file with the keyword argument :system-p set to true. The object file typically has the .o extension. It cannot be loaded with load, but it can be used to build libraries, standalone executable programs, or larger FASL files.
Building standalone executables To build an executable you need a working &ECL; image with the compiler. The function to build customized images is c::build-program. The description of this function is as follows. Care should be taken that image-name differs from any filename in lisp-files.
c:build-program Function: c:build-program {image-name &key lisp-files ld-flags prologue-code epilogue-code} This function builds a lisp image up from the core lisp library, plus all components listed in lisp-files. Each component is either: A symbol: Names a statically linked library built from lisp code. A string: Denotes an object file built from lisp code. ld-flags is a list of strings with additional parameters to be passed to the linker. You can include here your favorite C/C++ libraries. prologue-code and epilogue-code are used to customize the initialization process of the lisp image. In order to build the executable, c:build-program first writes down a piece of C code which initializes the lisp environment. You can customize the initialization process by suppling code to be executed before (prologue-code) or after (epilogue-code) setting up the lisp environment. Typically prologue-code defaults to an empty string, while epilogue-code invokes the classical lisp top-level. Additionally, as a convenience, epilogue-code can be either a string with C code or also a list with a lisp form, which will be interpreted at run time.
Building libraries To build a library you proceed more or less the same way as with standalone executables. There are two different functions depending on whether you need to build static or shared libraries.
c:build-static-library Function: c:build-static-library {library-name &key lisp-files prologue-code epilogue-code init-name} c:build-shared-library Function: c:build-shared-library {library-name &key lisp-files prologue-code epilogue-code ld-flags init-name} This function builds a library file up from the object files listed in lisp-files. Each of the arguments to lisp-file must name a single object file produced with compile-file. library-name is the physical pathname corresponding to the library. The value of library-name must follow some system-specific conventions. To make your program portable, library-name should be built using the output of compile-file-pathname. prologue-code and epilogue-code are strings with C code to be executed before and after initializing the library, respectively. For dynamically linked libraries you can also provide a list of strings in ld-flags. These strings are additional parameters for the linker and their purpose is to link C/C++ extensions into the library. init-name gives the initialization function of the library a user-specified name. Thus a the generated library may be used and/or linked to a C application. The recommended way to invoke init-name is following: cl_object the_block = read_VV(OBJNULL, init_FOO /* function name specified by init-name */); Be sure to call cl_boot before the invocation of init-name. In order to avoid that the returned object is garbage collected, you should keep the result of read_VV in a local variable.
File names
compile-file-pathname Function: compile-file-pathname {filename-base &key output-file type} When compiling lisp files, creating libraries, etc, a number of files are produced which are of interest for the user or programmer. However, the name of these files will change from system to system. The purpose of the function compile-file-pathname is to query the compiler about the name of the different files that it can produce. Possible values of the type argument include: :fas (default) Standard compiled files that can be loaded with load. :c, :data, :h Intermediate files produced by the Lisp-to-C translator. :o Linkable object files. :lib, :static-library A normal library produced with c:build-static-library. :dll, :shared-library A dynamically linked library produced with c:build-shared-library. :program An executable produced with c:build-program. The output of this function is system specific. For example, under FreeBSD > (compile-file-pathname "/this/path/mylib" :type :lib) #P"/this/path/libmylib.a" > (compile-file-pathname "/this/path/mylib" :type :dll) #P"/this/path/libmylib.so" > (compile-file-pathname "/this/path/mycode") #P"/this/path/mycode.fas"
Compiler examples
The <filename>hello.lisp</filename> file In the following examples we will use the same lisp program. You have to create a file called hello.lisp which contains the following lines (princ "Hello world!") (terpri) (quit) If you start &ECL; and load this file in the Common-Lisp environment you will see the "Hello world!" message and the interpreter will be closed. ECL (Embeddable Common-Lisp) 0.9d Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya Copyright (C) 1993 Giuseppe Attardi Copyright (C) 2000 Juan J. Garcia-Ripoll ECL is free software, and you are welcome to redistribute it under certain conditions; see file 'Copyright' for details. Type :h for Help. Top level. > (load "hello.lisp") ;;; Loading "hello.lisp" Hello World!
Example of loadable object file You can only perform the example in this section if your &ECL; image supports dynamically loading of object files. This is true if you find the keyword :dlopen in the *features* variable. This is true, for instance, in a typical FreeBSD or Linux box, Type :h for Help. Top level. > *features* (:IEEE-FLOATING-POINT :IBM-PC :I386 :BSD :UNIX :DLOPEN :ANSI-CL :CLOS :BOEHM-GC :ECL :COMMON) In this example we build a loadable extension which prints the "Hello world!" message. First you need to create a the hello.lisp file. Next you have to enter the &ECL; environment and type (compile-file "hello.lisp"). This produces a loadable object file. Type :h for Help. Top level. > (compile-file "hello.lisp") ;;; Loading #P"/usr/lib/ecl/cmp.fas" ;;; Loading #P"/usr/lib/ecl/sysfun.lsp" ;;; Compiling hello.lisp. ;;; End of Pass 1. ;;; Calling the C compiler... ;;; Invoking external command: gcc -O2 -march=i686 -pipe -fomit-frame-pointer -fPIC -fstrict-aliasing -Dlinux -O "-I/usr/lib/ecl/" -w -c "hello.c" -o "hello.o" ;;; Invoking external command: gcc -o "hello.fas" -L"/usr/lib/ecl/" "hello.o" -Wl,–rpath,/usr/lib/ecl/ -shared -lecl -lgmp -lgc -ldl -lm ;;; OPTIMIZE levels: Safety=2, Space=0, Speed=3 ;;; Finished compiling hello.lisp. #P"hello.fas" Top level. > (load "hello") ;;; Loading #P"hello.fas" Hello World!
Example of standalone program In this example we build a standalone program which prints the "Hello world!" message and does nothing else. First you must create the hello.lisp file shown above. Next you have to enter the &ECL; environment and type (compile-file "hello.lisp" :system-p t). This produces an object file that can be linked against the &ECL; core image. Type :h for Help. Top level. > (compile-file "hello.lisp" :system-p t) ;;; Loading #P"/usr/lib/ecl/cmp.fas" ;;; Loading #P"/usr/lib/ecl/sysfun.lsp" ;;; Compiling hello.lisp. ;;; End of Pass 1. ;;; Calling the C compiler... ;;; Invoking external command: gcc -O2 -march=i686 -pipe -fomit-frame-pointer -fPIC -fstrict-aliasing -Dlinux -O "-I/usr/lib/ecl/" -w -c "hello.c" -o "hello.o" ;;; OPTIMIZE levels: Safety=2, Space=0, Speed=3 ;;; Finished compiling hello.lisp. #P"hello.o" The final step is to build the executable using the c:build-program instruction. > (c:build-program "myecl" :lisp-files '("hello.o")) ;;; Invoking external command: gcc -O2 -march=i686 -pipe -fomit-frame-pointer -fPIC -fstrict-aliasing -Dlinux -O "-I/usr/lib/ecl/" -w -c "myecl.c" -o "myecl.o" ;;; Invoking external command: gcc -o "myecl" -L"/usr/lib/ecl/" "myecl.o" "hello.o" -Wl,–rpath,/usr/lib/ecl/ -lecl -lgmp -lgc -ldl -lm #P"myecl" Top level. Now you can execute this program from your favorite shell. % ./myecl Hello world!
Combining files into a larger FASL You can only perform the example in this section if your &ECL; image supports dynamically loading of object files. In this example we build a loadable library which prints the "Hello world!" message and does nothing else. First you must create the hello.lisp file shown above. Next you have to enter the &ECL; environment and type (compile-file "hello.lisp" :system-p t). This produces an object file that can be linked to form a loadable library. Type :h for Help. Top level. > (compile-file "hello.lisp" :system-p t) ;;; Loading #P"/usr/lib/ecl/cmp.fas" ;;; Loading #P"/usr/lib/ecl/sysfun.lsp" ;;; Compiling hello.lisp. ;;; End of Pass 1. ;;; Calling the C compiler... ;;; Invoking external command: gcc -O2 -march=i686 -pipe -fomit-frame-pointer -fPIC -fstrict-aliasing -Dlinux -O "-I/usr/lib/ecl/" -w -c "hello.c" -o "hello.o" ;;; OPTIMIZE levels: Safety=2, Space=0, Speed=3 ;;; Finished compiling hello.lisp. #P"hello.o" The final step is to build the library using the c:build-fasl instruction. > (c:build-fasl "myecl" :lisp-files '("hello.o")) ;;; Invoking external command: gcc -O2 -march=i686 -pipe -fomit-frame-pointer -fPIC -fstrict-aliasing -Dlinux -O "-I/usr/lib/ecl/" -w -c "myecl.c" -o "myecl.o" ;;; Invoking external command: gcc -o "libmyecl.so" -L"/usr/lib/ecl/" "myecl.o" "hello.o" -Wl,–rpath,/usr/lib/ecl/ -shared -lecl -lgmp -lgc -ldl -lm #P"libmyecl.so" Now you can load this extension from any &ECL; image, even those you produce with c:build-program. <<<<<<<< THIS EXAMPLE IS WRONG?! >>>>>>>>> > (load "myecl") ;;; Loading myecl.fas Hello world! Bye.
Manipulating Lisp objects If you want to extend, fix or simply customize &ECL; for your own needs, you should understand how the implementation works.
Objects representation In &ECL; a lisp object is represented by a type called cl_object. This type is a word which is long enough to host both an integer and a pointer. The least significant bits of this word, also called the tag bits, determine whether it is a pointer to a C structure representing a complex object, or whether it is an immediate data, such as a fixnum or a character. | word-1 | word-2 | ... | word-n | |----------------------| |--------|--------|-----|--------| | ...................00| | actual data of the object | |----------------------| |--------------------------------| ]]> The fixnums and characters are called immediate datatypes, because they require no more than the cl_object datatype to store all information. All other &ECL; objects are non-immediate and they are represented by a pointer to a cell that is allocated on the heap. Each cell consists of several words of memory and contains all the information related to that object. By storing data in multiples of a word size, we make sure that the least significant bits of a pointer are zero, which distinguishes pointers from immediate data. In an immediate datatype, the tag bits determine the type of the object. In non-immediate datatypes, the first byte in the cell contains the secondary type indicator, and distinguishes between different types of non immediate data. The use of the remaining bytes differs for each type of object. For instance, a cons cell consists of three words: There is one important function which tells the type of an object, plus several macros which group several tests.
cl_object C type: cl_object This is the type of a lisp object. For your C/C++ program, a cl_object can be either a fixnum, a character, or a pointer to a union of structures (See the header object.h). The actual interpretation of that object can be guessed with the macro ecl_t_of. For example, if x is of type cl_object, and it is of type fixnum, we may retrieve its value if (ecl_t_of(x) == t_fixnum) printf("Integer value: %d\n", fix(x)); If x is of type cl_object and it does not contain an immediate datatype, you may inspect the cell associated to the lisp object using x as a pointer. For example, if (ecl_t_of(x) == t_cons) printf("CAR = %x, CDR = %x\n", x->cons.car, x->cons.cdr); else if (ecl_t_of(x) == t_string) printf("String: %s\n", x->string.self); You should see the following sections and the header object.h to learn how to use the different fields of a cl_object pointer.
cl_type C type: cl_type Enumeration type which distinguishes the different types of lisp objects. The most important values are t_cons, t_fixnum, t_character, t_bignum, t_ratio, t_singlefloat, t_doublefloat, t_complex, t_symbol, t_package, t_hashtable, t_array, t_vector, t_string, t_bitvector, t_stream, t_random, t_readtable, t_pathname, t_bytecodes, t_cfun, t_cclosure, t_gfun, t_instance, t_foreign and t_thread.
ecl_t_of Function: cl_type ecl_t_of (cl_object O) If O is a valid lisp object, ecl_t_of(O) returns an integer denoting the type that lisp object. That integer is one of the values of the enumeration type cl_type.
FIXNUMP Function: bool FIXNUMP (cl_object o) CHARACTERP Function: bool CHARACTERP (cl_object o) CONSP Function: bool CONSP (cl_object o) LISTP Function: bool LISTP (cl_object o) ATOM Function: bool ATOM (cl_object o) ARRAYP Function: bool ARRAYP (cl_object o) VECTORP Function: bool VECTORP (cl_object o) STRINGP Function: bool STRINGP (cl_object o) Different macros that check whether o belongs to the specified type. These checks have been optimized, and are preferred over several calls to ecl_t_of.
IMMEDIATE Function: bool IMMEDIATE (cl_object o) Tells whether o is an immediate datatype.
Constructing objects On each of the following sections we will document the standard interface for building objects of different types. For some objects, though, it is too difficult to make a C interface that resembles all of the functionality in the lisp environment. In those cases you need to build the objects from their textual representation, or use the evaluator to build these objects. The first way makes use of a C or Lisp string to construct an object. The two functions you need to know are the following ones.
c_string_to_object Function: cl_object c_string_to_object (const char *s) string_to_object Function: cl_object string_to_object (cl_object o) c_string_to_object builds a lisp object from a C string which contains a suitable representation of a lisp object. string_to_object performs the same task, but uses a lisp string, and therefore it is less useful. Two examples of their use /* Using a C string */ cl_object array1 = c_string_to_object("#(1 2 3 4)"); /* Using a Lisp string */ cl_object string = make_simple_string("#(1 2 3 4)"); cl_object array2 = string_to_object(string);
Integers Common-Lisp distinguishes two types of integer types: bignums and fixnums. A fixnum is a small integer, which ideally occupies only a word of memory and which is between the values MOST-NEGATIVE-FIXNUM and MOST-POSITIVE-FIXNUM. A bignum is any integer which is not a fixnum and it is only constrained by the amount of memory available to represent it. In &ECL; a fixnum is an integer that, together with the tag bits, fits in a word of memory. The size of a word, and thus the size of a fixnum, varies from one architecture to another, and you should refer to the types and constants in the ecl.h header to make sure that your C extensions are portable. All other integers are stored as bignums, they are not immediate objects, they take up a variable amount of memory and the GNU Multiprecision Library is required to create, manipulate and calculate with them.
cl_fixnum C type: cl_fixnum This is a C signed integer type capable of holding a whole fixnum without any loss of precision. The opposite is not true, and you may create a cl_fixnum which exceeds the limits of a fixnum and should be stored as a bignum.
cl_index C type: cl_index This is a C unsigned integer type capable of holding a nonnegative fixnum without loss of precision. Typically, a cl_index is used as an index into an array, or into a proper list, etc.
MOST_NEGATIVE_FIXNUM Constant: MOST_NEGATIVE_FIXNUM MOST_POSITIVE_FIXNUM Constant: MOST_POSITIVE_FIXNUM These constants mark the limits of a fixnum.
FIXNUM_MINUSP Function: bool FIXNUM_MINUSP (cl_object o) FIXNUM_PLUSP Function: bool FIXNUM_PLUSP (cl_object o) These functions perform the checks (o < 0) and (0 <= o), respectively.
MAKE_FIXNUM Function: cl_object MAKE_FIXNUM (cl_fixnum n) fix Function: cl_fixnum fix (cl_object o) MAKE_FIXNUM and fix convert from an integer to a lisp object of fixnum type and vice versa. These functions no not check their arguments.
fixint Function: cl_fixnum fixint (cl_object o) Converts a lisp fixnum to a C integer of the appropriate size. Signals an error if o is not of fixnum type.
fixnnint Function: cl_index fixnnint (cl_object o) Similar to fixint but also ensures that o is not negative.
Characters &ECL; has only one type of characters, which fits in the C type char. The following constants and functions operate on characters.
CHAR_CODE_LIMIT Constant: CHAR_CODE_LIMIT Each character is assigned an integer code which ranges from 0 to (CHAR_CODE_LIMIT-1).
CHAR_CODE Function: cl_fixnum CHAR_CODE (cl_object o) char_code Function: cl_fixnum char_code (cl_object o) Returns the integer code associated to a lisp character. Only char_code checks its arguments.
CODE_CHAR Function: cl_object CODE_CHAR (cl_fixnum o) Returns the lisp character associated to an integer code. It does not check its arguments.
coerce_to_character Function: cl_object coerce_to_character (cl_object o) Coerces a lisp object to type character. Valid arguments are a character, or a string designator of length 1. In all other cases an error is signaled.
char_eq Function: bool char_eq (cl_object x, cl_object y) char_equal Function: bool char_equal (cl_object x, cl_object y) Compare two characters for equality. char_eq take case into account and char_equal ignores it.
char_cmp Function: int char_cmp (cl_object x, cl_object y) char_compare Function: int char_compare (cl_object x, cl_object y) Compare the relative order of two characters. char_cmp takes care of case and char_compare converts all characters to uppercase before comparing them.
Arrays An array is an aggregate of data of a common type, which can be accessed with one or more nonnegative indices. &ECL; stores arrays as a C structure with a pointer to the region of memory which contains the actual data. The cell of an array datatype varies depending on whether it is a vector, a bytevector, a multidimensional array or a string. If x contains a vector, you can access the following fields: x->vector.elttype The type of the elements of the vector. x->vector.dim The maximum number of elements. x->vector.fillp Actual number of elements in the vector or "fill pointer". x->vector.self Union of pointers of different types. You should choose the right pointer depending on x->vector.elltype x->vector.hasfillp Whether x->vector.fillp can be smaller than x->vector.dim. If x contains a multidimensional array, the cell elements become x->array.elttype The type of the elements of the array. x->array.dim Number of elements in the array. x->array.rank Number of dimensions of the array. x->array.dims[] Array with the dimensions of the array. The elements range from x->array.dim[0] to x->array.dim[x->array.rank-1]. x->array.self Union of pointers to the actual data. You should choose the right pointer depending on x->array.elltype. x->array.rank Whether x->vector.fillp can be smaller than x->vector.dim. Bitvectors and strings are treated separately. Each array is of an specialized type which is the type of the elements of the array. &ECL; has arrays only a few following specialized types, and for each of these types there is a C integer which is the corresponding value of x->array.elttype or x->vector.elttype. We list those types together with the C constant that denotes that type: T aet_object CHARACTER aet_ch FIXNUM aet_fix BIT aet_bit SINGLE-FLOAT aet_sf DOUBLE-FLOAT aet_df
array_elttype Function: cl_elttype array_elttype (cl_object o) Returns the element type of the array o, which can be a string, a bitvector, vector, or a multidimensional array. For example, the code array_elttype(c_string_to_object("\"AAA\"")) returns aet_ch, while the array_elttype(c_string_to_object("#(A B C)")) returns aet_object.
aref Function: cl_object aref (cl_object array, cl_index index) aset Function: cl_object aset (cl_object array, cl_index index, cl_object value) These functions are used to retrieve and set the elements of an array. The elements are accessed with one index, index, as in the lisp function ROW-MAJOR-AREF. For example cl_object array = c_string_to_object("#2A((1 2) (3 4))"); cl_object x = aref(array, 3); cl_print(1, x); /* Outputs 4 */ aset(array, 3, MAKE_FIXNUM(5)); cl_print(1, array); /* Outputs #2A((1 2) (3 5)) */
aref1 Function: cl_object aref1 (cl_object vector, cl_index index) aset1 Function: cl_object aset1 (cl_object vector, cl_index index, cl_object value) These functions are similar to aref and aset, but they operate on vectors. cl_object array = c_string_to_object("#(1 2 3 4)"); cl_object x = aref1(array, 3); cl_print(1, x); /* Outputs 4 */ aset1(array, 3, MAKE_FIXNUM(5)); cl_print(1, array); /* Outputs #(1 2 3 5) */
Strings A string, both in Common-Lisp and in &ECL; is nothing but a vector of characters. Therefore, almost everything mentioned in the section of arrays remains valid here. The only important difference is that &ECL; stores strings as a lisp object with a pointer to a zero terminated C string. Thus, if a string has n characters, &ECL; will reserve n+1 bytes for the string. This allows us to pass the string self pointer to any C routine. If x is a lisp object of type string, we can access the following fields: x->string.dim Maximum number of characters that it can contain. x->string.fillp Actual number of characters in the string. x->string.self Pointer to the characters. x->string.hasfillp True if x->string.fillp can be smaller than x->string.dim.
make_simple_string Function: cl_object make_simple_string (char *s) make_string_copy Function: cl_object make_string_copy (char *s) Both routines build a lisp string from a C string. make_string_copy allocates new space and copies the content of the string to it. make_simple_string simply uses the memory pointed by s, which should not be deallocated. Both routines use strlen to calculate the length of the string.
Bitvectors
Streams
Structures
Instances
Bytecodes A bytecodes object is a lisp object with a piece of code that can be interpreted. The objects of type t_bytecode are implicitly constructed by a call to eval, but can also be explicitly constructed with the make_lambda function.
cl_safe_eval Function: cl_object cl_safe_eval (cl_object form, cl_object env, cl_object err_value cl_eval Function: cl_object cl_eval (cl_object form) cl_safe_eval evaluates form in the lexical environment env, which can be nil. Before evaluating it, the expression form must be bytecompiled. cl_eval is the equivalent of cl_safe_eval but without environment and with err_value set to nil. It exists only for compatibility with previous versions. cl_object form = c_string_to_object("(print 1)"); cl_safe_eval(form,Cnil); cl_safe_eval(form, Cnil);
si_make_lambda Function: cl_object si_make_lambda (cl_object name, cl_object def) Builds an interpreted lisp function with name given by the symbol name and body given by def. For instance, we would achieve the equivalent of (funcall #'(lambda (x y) (block foo (+ x y))) 1 2) with the following code cl_object def = c_string_to_object("((x y) (+ x y))"); cl_object name = _intern("foo") cl_object fun = si_make_lambda(name, def); return funcall(fun, MAKE_FIXNUM(1), MAKE_FIXNUM(2)); Notice that si_safe_lambda performs a bytecodes compilation of the definition and thus it may signal some errors. Such errors are not handled by the routine itself you might consider using cl_safe_eval or cl_eval instead: cl_object def = c_string_to_object("#'(lambda-block foo (x y) (+ x y))"); cl_object fun = cl_eval(def); return funcall(fun, MAKE_FIXNUM(1), MAKE_FIXNUM(2));
The interpreter
&ECL; stacks &ECL; uses the following stacks: Frame Stack consisting of catch, block, tagbody frames Bind Stack for shallow binding of dynamic variables Interpreter Stack acts as a Forth data stack, keeping intermediate arguments to interpreted functions, plus a history of called functions. C Control Stack used for arguments/values passing, typed lexical variables, temporary values, and function invocation.
Procedure Call Conventions &ECL; employs standard C calling conventions to achieve efficiency and interoperability with other languages. Each Lisp function is implemented as a C function which takes as many argument as the Lisp original plus one additional integer argument which holds the number of actual arguments. The function sets NValues to the number of Lisp values produced, it returns the first one and the remaining ones are kept in a global (per thread) array (VALUES). To show the argument/value passing mechanism, here we list the actual code for the Common-Lisp function cons. cl_cons(int narg, object car, object cdr) { object x; check_arg(2); x = alloc_object(t_cons); CAR(x) = car; CDR(x) = cdr; NValues = 1; return x; } &ECL; adopts the convention that the name of a function that implements a Common-Lisp function begins with a short package name (cl for COMMON-LISP, si for SYSTEM, etc), followed by L, and followed by the name of the Common-Lisp function. (Strictly speaking, `-' and `*' in the Common-Lisp function name are replaced by `_' and `A', respectively, to obey the syntax of C.) check_arg(2) in the code of cl_cons checks that exactly two arguments are supplied to cons. That is, it checks that narg is 2, and otherwise, it causes an error. allocate_object(t_cons) allocates a cons cell in the heap and returns the pointer to the cell. After the CAR and the CDR fields of the cell are set, the cell pointer is returned directly. The number assigned to NValues set by the function (1 in this case) represents the number of values of the function. In general, if one is to play with the C kernel of &ECL; there is no need to know about all these conventions. There is a preprocessor that takes care of the details, by using a lisp representation of the statements that output values, and of the function definitions. For instance, the actual source code for cl_cons in src/c/lists.d @(defun cons (car cdr) @ @(return CONS(car, cdr)) @)
The lexical environment The &ECL; interpreter uses two A-lists (Association lists) to represent lexical environments. One for variable bindings One for local function/macro/tag/block bindings When a function closure is created, the current two A-lists are saved in the closure along with the lambda expression. Later, when the closure is invoked, the saved A-lists are used to recover the lexical environment.
The interpreter stack The bytecodes interpreter uses a stack of its own to save and restore values from intermediate calculations. This Forth-like data stack is also used in other parts of the C kernel for various purposes, such as saving compiled code, keeping arguments to FORMAT, etc. However, one of the most important roles of the Interpreter Stack is to keep a log of the functions which are called during the execution of bytecodes. For each function invoked, the interpreter keeps three lisp objects on the stack: The first item is the object which is funcalled. It can be a bytecodes object, a compiled function or a generic function. In the last two cases the lexical environment is just NIL. In the first case, the second item on the stack is the lexical environment on which the code is executed. Each of these records are popped out of the stack after function invocation. Let us see how these invocation records are used for debugging. >(defun fact (x) ;;; Wrong definition of the (if (= x 0) ;;; factorial function. one ;;; one should be 1. (* x (fact (1- x))))) FACT >(fact 3) ;;; Tries 3! Error: The variable ONE is unbound. Error signalled by IF. Broken at IF. >>:b ;;; Backtrace. Backtrace: eval > fact > if > fact > if > fact > if > fact > IF ;;; Currently at the last IF. >>:h ;;; Help. Break commands: :q(uit) Return to some previous break level. :pop Pop to previous break level. :c(ontinue) Continue execution. :b(acktrace) Print backtrace. :f(unction) Show current function. :p(revious) Go to previous function. :n(ext) Go to next function. :g(o) Go to next function. :fs Search forward for function. :bs Search backward for function. :v(ariables) Show local variables, functions, blocks, and tags. :l(ocal) Return the nth local value on the stack. :hide Hide function. :unhide Unhide function. :hp Hide package. :unhp Unhide package. :unhide-all Unhide all variables and packages. :bds Show binding stack. :m(essage) Show error message. :hs Help stack. Top level commands: :cf Compile file. :exit or ^D Exit Lisp. :ld Load file. :step Single step form. :tr(ace) Trace function. :untr(ace) Untrace function. Help commands: :apropos Apropos. :doc(ument) Document. :h(elp) or ? Help. Type ":help help" for more information. >>:p ;;; Move to the last call of FACT. Broken at IF. >>:b Backtrace: eval > fact > if > fact > if > fact > if > FACT > if ;;; Now at the last FACT. >>:v ;;; The environment at the last call Local variables: ;;; to FACT is recovered. X: 0 ;;; X is the only bound variable. Block names: FACT. ;;; The block FACT is established. >>x 0 ;;; The value of x is 0. >>(return-from fact 1) ;;; Return from the last call of 6 ;;; FACT with the value of 0. ;;; The execution is resumed and > ;;; the value 6 is returned. ;;; Again at the top-level loop.
The compiler
The compiler translates to C The &ECL; compiler is essentially a translator from Common-Lisp to C. Given a Lisp source file, the compiler first generates three intermediate files: a C-file which consists of the C version of the Lisp program an H-file which consists of declarations referenced in the C-file a Data-file which consists of Lisp data to be used at load time The &ECL; compiler then invokes the C compiler to compile the C-file into an object file. Finally, the contents of the Data-file is appended to the object file to make a Fasl-file. The generated Fasl-file can be loaded into the &ECL; system by the Common-Lisp function load. By default, the three intermediate files are deleted after the compilation, but, if asked, the compiler leaves them. The merits of the use of C as the intermediate language are: The &ECL; compiler is highly portable. Cross compilation is possible, because the contents of the intermediate files are common to all versions of &ECL;. For example, one can compile his or her Lisp program by the &ECL; compiler on a Sun, bring the intermediate files to DOS, compile the C-file with the gcc compiler under DOS, and then append the Data-file to the object file. This procedure generates the Fasl-file for the &ECL; system on DOS. This kind of cross compilation makes it easier to port &ECL;. Hardware-dependent optimizations such as register allocations are done by the C compiler. The demerits are: At those sites where no C compiler is available, the users cannot compile their Lisp programs. The compilation time is long. 70% to 80% of the compilation time is used by the C compiler. The &ECL; compiler is therefore slower than compiler generating machine code directly.
The compiler mimics human C programmer The format of the intermediate C code generated by the &ECL; compiler is the same as the hand-coded C code of the &ECL; source programs. For example, supposing that the Lisp source file contains the following function definition: (defvar *delta* 2) (defun add1 (x) (+ *delta* x)) The compiler generates the following intermediate C code. /* function definition for ADD1 */ static cl_object L1(cl_object V1) { VT2 VLEX2 CLSR2 cl_object value0; value0=number_plus(symbol_value(VV[0]),V1); NVALUES=1; return value0; } /* initialization of this module */ void init_CODE(cl_object flag) { VT1 CLSR1 cl_object value0; if (!FIXNUMP(flag)){ Cblock=flag; #ifndef ECL_DYNAMIC_VV flag->cblock.data = VV; #endif flag->cblock.self_destruct=0; flag->cblock.data_size = VM; flag->cblock.data_text = compiler_data_text; flag->cblock.data_text_size = compiler_data_text_size; return;} #ifdef ECL_DYNAMIC_VV VV = Cblock->cblock.data; #endif T0= MAKE_FIXNUM(2); si_Xmake_special(VV[0]) if(SYM_VAL(T0)!=OBJNULL) cl_setq(VV[0],T0); cl_def_c_function(VV[1],(void*)L1,1); } The C function L1 implements the Lisp function add1. This relation is established by cl_def_c_function in the initialization function init_CODE, which is invoked at load time. There, the vector VV consists of Lisp objects; VV[0] and VV[1] in this example hold the Lisp symbols *delta* and add1. VM in the definition of L1 is a C macro declared in the corresponding H-file. The actual value of VM is the number of value stack locations used by this module, i.e., 2 in this example. Thus the following macro definition is found in the H-file. #define VM 2
Implementation of Compiled Closures The &ECL; compiler takes two passes before it invokes the C compiler. The major role of the first pass is to detect function closures and to detect, for each function closure, those lexical objects (i.e., lexical variable, local function definitions, tags, and block-names) to be enclosed within the closure. This check must be done before the C code generation in the second pass, because lexical objects to be enclosed in function closures are treated in a different way from those not enclosed. Ordinarily, lexical variables in a compiled function f are allocated on the C stack. However, if a lexical variable is to be enclosed in function closures, it is allocated on a list, called the "environment list", which is local to f. In addition, a local variable is created which points to the lexical variable's location (within the environment list), so that the variable may be accessed through an indirection rather than by list traversal. The environment list is a pushdown list: It is empty when f is called. An element is pushed on the environment list when a variable to be enclosed in closures is bound, and is popped when the binding is no more in effect. That is, at any moment during execution of f, the environment list contains those lexical variables whose binding is still in effect and which should be enclosed in closures. When a compiled closure is created during execution of f, the compiled code for the closure is coupled with the environment list at that moment to form the compiled closure. Later, when the compiled closure is invoked, a pointer is set up to each lexical variable in the environment list, so that each object may be referenced through a memory indirection. Let us see an example. Suppose the following function has been compiled. (defun foo (x) (let ((a #'(lambda () (incf x))) (y x)) (values a #'(lambda () (incf x y))))) foo returns two compiled closures. The first closure increments x by one, whereas the second closure increments x by the initial value of x. Both closures return the incremented value of x. >(multiple-value-setq (f g) (foo 10)) #<compiled-closure nil> >(funcall f) 11 >(funcall g) 21 > After this, the two compiled closures look like: | 10 | --|------>| 21 | nil | |-------|------| |-------|------| |------|------| ^ first closure | |-------|------| | | * | --|----------| |-------|------| * : address of the compiled code for #'(lambda () (incf x)) ** : address of the compiled code for #'(lambda () (incf x y)) ]]>
Use of Declarations to Improve Efficiency Declarations, especially type and function declarations, increase the efficiency of the compiled code. For example, for the following Lisp source file, with two Common-Lisp declarations added, (eval-when (compile) (proclaim '(function tak (fixnum fixnum fixnum) fixnum)) (defun tak (x y z) (declare (fixnum x y z)) (if (not (< y x)) z (tak (tak (1- x) y z) (tak (1- y) z x) (tak (1- z) x y)))) The compiler generates the following C code: /* local entry for function TAK */ static int LI1(register int V1,register int V2,register int V3) { VT3 VLEX3 CLSR3 TTL: if (V2 < V1) { goto L2;} return(V3); L2: { int V5; V5 = LI1((V1)-1,V2,V3); { int V6; V6 = LI1((V2)-1,V3,V1); V3 = LI1((V3)-1,V1,V2); V2 = V6; V1 = V5;}} goto TTL; ;;; Note: Tail-recursive call of TAK was replaced by iteration. }
Inspecting generated C code Common-Lisp defines a function disassemble, which is supposed to disassemble a compiled function and to display the assembler code. According to Common-Lisp: The Language, This is primary useful for debugging the compiler, ..\\ This is, however, useless in our case, because we are not concerned with assembly language. Rather, we are interested in the C code generated by the &ECL; compiler. Thus the disassemble function in &ECL; accepts not-yet-compiled functions only and displays the translated C code. > (defun add1 (x) (1+ x)) ADD1 > (disassemble *) ;;; Compiling (DEFUN ADD1 ...). ;;; Emitting code for ADD1. /* function definition for ADD1 */ static L1(int narg, object V1) { VT3 VLEX3 CLSR3 TTL: VALUES(0) = one_plus((V1)); RETURN(1); }
Porting &ECL; To port &ECL; to a new architecture, the following steps are required: Ensure that the GNU Multiprecision library supports this machine. Ensure that the Boehm-Weiser garbage collector is supported by that architecture. Alternatively, port ECL's own garbage collector src/c/alloc.d and src/c/gbc.d to that platform. Fix src/aclocal.in, src/h/config.h.in and src/h/ecl.h so that they supply flags for the new host machine. Fix the machine dependent code in src/c/. The most critical parts are in the unix*.d and thread*.d files. Compile as in any other platform. Run the tests and compare to the results of other platforms.