%eclent; ]>

A Foreign Function Interface, or &FFI; for short, is a means for a programming language to interface with libraries written in other programming languages, the foreign code. You will see this concept most often being used in interpreted environments, such as Python, Ruby or Lisp, where one wants to reuse the big number of libraries written in C and C++ for dealing with graphical interfaces, networking, filesystems, etc. A FFI is made of at least three components: Foreign objects management This is the data that the foreign code will use. A &FFI; needs to provide means to build and manipulate foreign data, with automatic conversions to and from lisp data types whenever possible, and it also has to deal with issues like garbage collection and finalization. Foreign code loader To actually use a foreign routine, the code must reside in memory. The process of loading this code and finding out the addresses of the routines we want to use is normally done by an independent component. Foreign function invocation This is the part of the &FFI; that deals with actually calling the foreign routines we want to use. For that one typically has to tell the &FFI; what are the arguments that these routines expect, what are the calling conventions and where are these routines to be found. On top of these components sits a higher level interface written entirely in lisp, with which you will actually declare and use foreign variables, functions and libraries. In the following sections we describe both the details of the low-level components (, ), and of the higher level interface (). It is highly recommended that you read all sections.

&ECL; allows for two different appraoches when building a &FFI;. Both approaches have a different implementation philosophy and affect the places where you can use the &FFI; and how. Static &FFI; For every foreign function and variable you might need to use, a wrapper is automatically written in C with the help of . These wrappers are compiled using an ordinary C compiler and linked against both the foreign libraries you want to use and against the &ECL; library. The result is a &FASL; file that can be loaded from &ECL; and where the wrappers appear as ordinary lisp functions and variables that the user may directly invoked. Dynamic &FFI; First of all, the foreign libraries are loaded in memory using the facilities of the operating system. Similar routines are used to find out and register the memory location of all the functions and variables we want to use. Finally, when actually accessing these functions, a little piece of assembly code does the job of translating the lisp data into foreign objects, storing the arguments in the stack and in CPU registers, calling the function and converting back the output of the function to lisp. &ECL; for this purpose utilizes the library "A Portable Foreign Function Interface Library" commonly known as libffi.

As you see, the first approach uses rather portable technices based on a programming language (C, C++) which is strongly supported by the operating system. The conversion of data is performed calling routines in the &ECL; library and we need not care about the precise details (organizing the stack, CPU registers, etc) when calling a function: the compiler does this for us. On the other hand, the dynamic approach allows us to choose the libraries we load at any time, look for the functions and invoke them even from the toplevel, but it relies on unportable techniques and requires the developers to know very well both the assembly code of the machine the code runs on and the calling conventions of that particular operating system. For these reasons &ECL; doesn't maintain it's own implementation of the DFFI but rather relies on the third party library. &ECL; currently supports the static method on all platforms, and the dynamical one a wide range of the most popular ones, shown in libffi. You can test if your copy of &ECL; was built with DFFI by inspecting whether the symbol :DFFI is present in the list from variable *FEATURES*.

While the foreign function invocation protocols differ strongly between platforms and implementations, foreign objects are pretty easy to handle portably. For &ECL;, a foreign object is just a bunch of bytes stored in memory. The lisp object for a foreign object encapsulates several bits of information: A list or a symbol specifying the C type of the object. The pointer to the region of memory where data is stored. A flag determining whether &ECL; can automatically manage that piece of memory and deallocated when no longer in use. A foreign object may contain many different kinds of data: integers, floating point numbers, C structures, unions, etc. The actual type of the object is stored in a list or a symbol which is understood by the higher level interface (). The most important component of the object is the memory region where data is stored. By default &ECL; assumes that the user will perform automatic management of this memory, deleting the object when it is no longer needed. The first reason is that this block may have been allocated by a foreign routine using malloc(), or mmap(), or statically, by referring to a C constant. The second reason is that foreign functions may store references to this memory which &ECL; is not aware of and, in order to keep these references valid, &ECL; should not attempt to automatically destroy the object. In many cases, however, it is desirable to automatically destroy foreign objects once they have been used. The higher level interfaces &UFFI; and &CFFI; provide tools for doing this. For instance, in the following example adapted from the &UFFI; documentation, the string NAME is automatically deallocated (def-function "gethostname" ((name (* :unsigned-char)) (len :int)) :returning :int) (if (zerop (c-gethostname (ffi:char-array-to-pointer name) 256)) (format t "Hostname: ~S" (ffi:convert-from-foreign-string name)) (error "gethostname() failed."))

Up to now we have only discussed vague ideas about how a &FFI; works, but you are probably more interested on how to actually code all these things in lisp. You have here three possibilities: &ECL; supplies a high level interface which is compatible with &UFFI; up to version 1.8 (api for >=v2.0 is provided by cffi-uffi-compat system shipped with &CFFI;). Code designed for &UFFI; library should run mostly unchanged with &ECL;. Note, that api resides in ffi package, not uffi, to prevent conflicts with cffi-uffi-compat. The &CFFI; library features a mostly complete backend for &ECL;. This is however a work in progress, as the fact that &CFFI; allows for calling arbitrary functions without declaring them causes some troubles with &ECL;. &ECL;'s own low level interface. Only to be used if &ECL; is your deployment platform. It features some powerful constructs that allow you to mix arbitrary C code with lisp ( and ). In the following two subsections we will discuss two practical examples of using the native &UFFI; and the &CFFI; library.

The example below shows how to use &UFFI; in an application. There are several important ingredients: You need to specify the libraries you use and do it at the toplevel, so that the compiler may include them at link time. Every function you will use has to be declared using ffi:def-function. #| Build and load this module with (compile-file "uffi.lsp" :load t) |# ;; ;; This toplevel statement notifies the compiler that we will ;; need this shared library at runtime. We do not need this ;; statement in windows. ;; #-(or ming32 windows) (ffi:load-foreign-library #+darwin "/usr/lib/libm.dylib" #-darwin "/usr/lib/libm.so") ;; ;; With this other statement, we import the C function sin(), ;; which operates on IEEE doubles. ;; (ffi:def-function ("sin" c-sin) ((arg :double)) :returning :double) ;; ;; We now use this function and compare with the lisp version. ;; (format t "~%Lisp sin:~t~d~%C sin:~t~d~%Difference:~t~d" (sin 1.0d0) (c-sin 1.0d0) (- (sin 1.0d0) (c-sin 1.0d0)))

The &CFFI; library is an independent project and it is not shipped with &ECL;. If you wish to use it you can go to their homepage, download the code and build it using &ASDF;. &CFFI; differs slightly from &UFFI; in that functions may be used even without being declared beforehand. This poses a few problems to the &ECL; backend, but hopefully these should have been solved in the latest releases. #| Build and load this module with (compile-file "cffi.lsp" :load t) |# ;; ;; This toplevel statement notifies the compiler that we will ;; need this shared library at runtime. We do not need this ;; statement in windows. ;; #-(or ming32 windows) (cffi:load-foreign-library #+darwin "/usr/lib/libm.dylib" #-darwin "/usr/lib/libm.so") ;; ;; With this other statement, we import the C function sin(), ;; which operates on IEEE doubles. ;; (cffi:defcfun ("sin" c-sin) :double '(:double)) ;; ;; We now use this function and compare with the lisp version. ;; (format t "~%Lisp sin:~t~d~%C sin:~t~d~%Difference:~t~d" (sin 1.0d0) (c-sin 1.0d0) (- (sin 1.0d0) (c-sin 1.0d0))) ;; ;; The following also works: no declaration! ;; (let ((c-cos (cffi:foreign-funcall "cos" :double 1.0d0 :double))) (format t "~%Lisp cos:~t~d~%C cos:~t~d~%Difference:~t~d" (cos 1.0d0) c-cos (- (cos 1.0d0) c-cos)))

To compare with the previous pieces of code, we show how the previous programs would be written using and #| Build and load this module with (compile-file "ecl.lsp" :load t) |# ;; ;; With this other statement, we import the C function sin(), which ;; operates on IEEE doubles. Notice that we include the C header to ;; get the full declaration. ;; (defun c-sin (x) (ffi:clines "#include <math.h>") (ffi:c-inline (x) (:double) :double "sin(#0)" :one-liner t)) ;; ;; We now use this function and compare with the lisp version. ;; (format t "~%Lisp sin:~t~d~%C sin:~t~d~%Difference:~t~d" (sin 1.0d0) (c-sin 1.0d0) (- (sin 1.0d0) (c-sin 1.0d0)))

ffi:clines Insert C declarations and definitions ffi:clines c-code* c-code One or more strings with C definitions. Not evaluated. returns No value. This special form inserts C code directly in the file that results from compiling lisp sources. Contrary to , this function may have no executable statements, accepts no input value and returnsn no value. The main use of FFI:CLINES is to declare or define C variables and functions that are going to be used later in other &FFI; statements. FFI:CLINES is a special form that can only be used in lisp compiled files as a toplevel form. Other uses will lead to an error being signaled, either at compilation time or when loading the file. In this example the FFI:CLINES statement is required to get access to the C function cos() (ffi:clines "#include <math.h>") (defun cos (x) (ffi:c-inline (x) (:double) :double "cos(#0)" :on-liner t)) ffi:c-inline Inline C code in a lisp form. ffi:c-inline (lisp-value*) (c-type*) return-type C-code &key; one-liner side-effects lisp-value A lisp expression, evaluated. c-type A valid FFI type. return-type A valid FFI type or (VALUES). C-code A string with valid C code plus some valid escape forms. one-liner A boolean, defaults to NIL. side-effects A boolean, defaults to T. returns One or more lisp values. This is an special form which can be only used in compiled code and whose purpose is to execute some C code getting and returning values from and to the lisp environment. The first argument to ffi:c-inline is a list of lisp forms. These forms are going to be evaluated and their lisp values will be transformed to the corresponding C types denoted by c-type. The input values are used to create a valid C expression using the template in C-code. This is a string of arbitrary size which mixes C expressions with two kind of escape forms. The first kind of escape form are made of a hash and a letter or a number, as in: #0, #1, ..., until #z. These codes are replaced by the corresponding input values. The second kind of escape form has the format @(return n), it can be used as lvalue in a C expression and it is used to set the n-th output value of the ffi:c-inline form. When the parameter one-liner is true, then the C template must be a simple C statement that outputs a value. In this case the use of @(return) is not allowed. When the parameter one-liner is false, then the C template may be a more complicated block form, with braces, conditionals, loops and spanning multiple lines. In this case the output of the form can only be set using @(return). Note that the conversion between lisp arguments and FFI types is automatic. Note also that ffi:c-inline cannot be used in interpreted or bytecompiled code! The following example implements the transcendental function SIN using the C equivalent (ffi:c-lines "#include <math.h>") (defun mysin (x) (ffi:c-inline (x) (:double) :double "sin(#0)" :one-liner t :side-effects nil)) This function can also be implemented using the @(return) form as follows: (defun mysin (x) (ffi:c-inline (x) (:double) :double "@(return)=sin(#0);" :side-effects nil)) The following example is slightly more complicated as it involves loops and two output values: (defun sample (x) (ffi:c-inline (n1 n2) (:int :int) (values :int :int) "{ int n1 = #0, n2 = #1, out1 = 0, out2 = 1; while (n1 <= n2) { out1 += n1; out2 *= n1; n1++; } @(return 0)= out1; @(return 1)= out2; }" :side-effects nil))