ciel/README.org

* CIEL Is an Extended Lisp                                         :noexport:

STATUS: +highly+ WIP, the API WILL change, but it is usable.

I am dogfooding it in public and private projects since several months.


* What is this ?                                                   :noexport:

  CIEL is a collection of useful libraries.

  It's Common Lisp, batteries included.

  Questions, doubts? See the [[file:docs/FAQ.md][FAQ]].

** Rationale

One of our goals is to make Common Lisp useful out of the box for
mundane tasks -by today standards. As such, we ship libraries to
handle JSON and CSV, as well as others to ease string manipulation, to
do pattern matching, to bring regular expressions, for threads and
jobs scheduling, for HTTP and URI handling, to create simple GUIs with
Ltk, and so on. You can of course do all this without CIEL, but
then you have to install the library manager first and load these libraries
into your Lisp image every time you start it. Now, you have them at
your fingertips whenever you start CIEL.

We also aim to soften the irritating parts of standard Common Lisp.
A famous one, puzzling for beginners and non-optimal for seasoned
lispers, is the creation of hash-tables. We include the =dict= function
from the Serapeum library (which we enhanced further with a pull request):

#+begin_src
CIEL-USER> (dict :a 1 :b 2 :c 3)
#+end_src

which prints:

#+begin_src txt
(dict
 :A 1
 :B 2
 :C 3
)
#+end_src

In standard Common Lisp, the equivalent is more convoluted:

#+BEGIN_SRC lisp
  (let ((ht (make-hash-table :test 'equal)))
    (setf (gethash :a ht) 1)
    (setf (gethash :b ht) 2)
    (setf (gethash :c ht) 3)
    ht)
;; #<HASH-TABLE :TEST EQUAL :COUNT 3 {1006CE5613}>
;; (and we don't get a readable representation, so our example is not even equivalent)
#+end_src

Moreover, we want to bring a **full featured REPL on the terminal**
(see more below).

See [[docs/README.md][the documentation]].

* TODOs                                                            :noexport:

- [-] settle on libraries that help newcomers
- [-] automate the documentation
- distribute (Quicklisp, Qlot, Quicklisp distribution, [[https://ultralisp.org/projects/ciel-lang/CIEL][Ultralisp]],
  Ultralisp distribution (upcoming)…)
- [-] ship a binary +and a core image+.
- optional: create a tool that, given a CIEL code base, explains what
  packages to import in order to switch to "plain CL".

How to procede ?

This is an experiment. I'd be happy to give push rights to more
maintainers. We will send pull requests, discuss, and in case we don't
find a consensus for what should be on by default, we can create other
packages.

Rules

- don't install libraries that need a Slime helper to work in the REPL (cl-annot).
- reader syntax changes may not be enabled by default.

* Table of contents :TOC:
- [[#install][Install]]
  - [[#with-quicklisp][With Quicklisp]]
  - [[#with-a-core-image][With a core image]]
  - [[#with-a-binary-use-ciels-custom-repl][With a binary. Use CIEL's custom REPL.]]
- [[#usage][Usage]]
  - [[#use-ciel-at-startup][Use CIEL at startup]]
- [[#libraries][Libraries]]
- [[#language-extensions][Language extensions]]
- [[#final-words][Final words]]
- [[#how-to-generate-the-documentation][How to generate the documentation]]

* Install

  You will probably need the following system dependencies (names for
  a Debian system):

: libmagic-dev libc6-dev gcc  # from magicffi

** With Quicklisp

You need a Lisp implementation and Quicklisp installed.

CIEL is not yet on Quicklisp, but it is on [[https://ultralisp.org][Ultralisp]].

So, either clone this repository:

: git clone https://github.com/ciel-lang/CIEL ~/quicklisp/local-projects/CIEL

either install the Ultralisp distribution and pull the library from there:

: (ql-dist:install-dist "http://dist.ultralisp.org/" :prompt nil)

Then, load the .asd file (with =asdf:load-asd= or =C-c C-k= in Slime), quickload "ciel":

#+BEGIN_SRC lisp
(ql:quickload "ciel")
#+end_src

and enter the =ciel-user= package:

#+BEGIN_SRC lisp
(in-package :ciel-user)
#+end_src

** With a core image

You need a Lisp implementation, but you don't need Quicklisp.

Build a /core image/ for your lisp with all CIEL's dependencies:

: sbcl --load build-image.lisp

and use it:

: sbcl --core ciel --eval '(in-package :ciel-user)'

Then you can configure Slime to have the choice of the Lisp image to
start. See below in [[*Use CIEL at startup]]

We +will distribute ready-to-use core images+ can not distribute core
images, you must build it yourself.

** With a binary. Use CIEL's custom REPL.

   You don't need anything, just download the CIEL executable and run
   its REPL. You need to build the core image yourself though.

   - we provide an experimental binary for Debian systems: go to
     https://gitlab.com/vindarel/ciel/-/pipelines, download the latest
     artifacts, unzip the =ciel-v0.zip= archive and run
     =ciel-v0/ciel-repl=.

   TODO: build it for different platforms.

   To build it, clone this repository and run =make build=.

   Start it with =./ciel-repl=.

   You are dropped into a custom Lisp REPL, freely based on [[https://github.com/hellerve/sbcli][sbcli]].

   This REPL is more user friendly than the default SBCL one:

- it has readline capabilities, meaning that the arrow keys work by
  default (wouhou!) and there is a persistent history, like in any shell.
- it has *multiline input*.
- it has *TAB completion*.
- it handles errors gracefully: you are not dropped into the debugger
  and its sub-REPL, you simply see the error message.
- it has optional *syntax highlighting*.
- it has an optional *lisp critic* that scans the code you enter at
  the REPL for instances of bad practices.
- it has a *shell pass-through*: try =!ls=.

- it has *documentation lookup* shorthands: use =:doc symbol= or =?=
  after a symbol to get its documentation: =ciel-user> (dict ?=.

- it has *developer friendly* macros: use =(printv code)= for an
  annotated trace output.

- and it defines more helper commands:

#+begin_src txt
  %help => Prints this general help message
  %doc => Prints the available documentation for this symbol
  %? => Inspect a symbol: %? str
  %w => Writes the current session to a file <filename>
  %d => Dumps the disassembly of a symbol <sym>
  %t => Prints the type of a expression <expr>
  %lisp-critic => Toggles the lisp-critic
  %q => Ends the session.
#+end_src

See more in [[docs/README.md][the documentation]].

* Usage

Create a package and "use" =ciel= in addition of =cl=:

#+BEGIN_SRC lisp
  (defpackage yourpackage
    (:use :cl :ciel))
#+end_src

You can also use =generic-ciel=, based on [[https://github.com/alex-gutev/generic-cl/][generic-cl]]:

#+begin_src
  (defpackage yourpackage
    (:use :cl :generic-ciel))
#+end_src

generic-cl allows us to define our =+= or =equalp= methods for our
own objects (and more).

** Use CIEL at startup

You can enter the =CIEL-USER= package when you start your Lisp image
from your editor.

A working, but naive and slow-ish approach is to add this in your =~/.sbclrc=:

#+BEGIN_SRC lisp
(ql:quickload "ciel")
(in-package :ciel-user)
(ciel-user-help)
#+end_src

A faster way is to use CIEL's core and to use SLIME's or your editor's
feature to [[https://common-lisp.net/project/slime/doc/html/Multiple-Lisps.html#Multiple-Lisps][configure multiple Lisps]].

You need to:

- build CIEL's core for your machine (=make image=),
- add this to your Emacs init file:

#+BEGIN_SRC lisp
  (setq slime-lisp-implementations
        `((sbcl    ("sbcl" "--dynamic-space-size" "2000"))  ;; default. Adapt if needed.
          (ciel-sbcl  ("sbcl" "--core" "/path/to/ciel/ciel" "--eval" "(in-package :ciel-user)"))))
  (setq slime-default-lisp 'ciel-sbcl)
#+end_src

- and start a new Lisp process.
- optional: if you didn't set it as default with =slime-default-lisp=,
  then start a new Lisp process with =M-- M-x slime= (alt-minus prefix),
  and choose ciel-sbcl. You can start more than one Lisp process from SLIME.

The Lisp process should start as fast as the default SBCL.

* Libraries

  We import, use and document libraries to fill various use cases:
  generic access to data structures, functional data structures,
  string manipulation, JSON, database access, web, URI handling, GUI,
  iteration helpers, type checking helpers, syntax extensions,
  developer utilities, etc.

  See [[docs/README.md][the documentation]].

  To see the full list of dependencies, see the =ciel.asd= project
  definition or this [[file:docs/dependencies.md][dependencies list]].

* Language extensions

  We provide arrow macros, easy type declaratons in the function
  lambda list, macros for exhaustiveness type checking, pattern
  matching, etc.

  See [[https://ciel-lang.github.io/CIEL/#/language-extensions][the documentation]].

* Final words

That was your life in CL:

#+html: <p align="center"><img src="docs/before.jpeg" /></p>

and now:

#+html: <p align="center"><img src="docs/after-plus.jpeg" /></p>

* How to generate the documentation

See =src/ciel.lisp= and run =(generate-dependencies-page-reference)=.