• What is this ?
• TODOs
• Install
  • With Quicklisp
  • With a core image
  • With a binary. Use CIEL's custom REPL.
• Libraries
  • Data structures
    • Generic and nested access to datastructures (access)
    • Hash-table utilities (Serapeum)
    • Sequences utilities (Serapeum)
    • String manipulation (str)
  • Data formats
    • CSV
    • JSON
  • Databases
  • Pattern matching
  • Numbers
    • parse-float
    • parse-number
    • Parsing numbers, floats, decimals
      • cl-decimals: parse and format decimal numbers
  • Regular expressions
  • Threads, monitoring, scheduling
  • HTTP and URI handling
  • Web
  • Conditions
  • Types, type checking, exhaustiveness type checking
    • Example with enums
    • Example with union types
  • Syntax extensions
    • Arrow macros
    • Pythonic triple quotes docstring
    • Lambda shortcuts
  • Development
    • Testing (Fiveam)
    • Logging (log4cl)
    • Discoverability of documentation (repl-utilities' readme, summary,…)
    • printv
    • Getting a function's arguments list (trivial-arguments)
  • generic-cl
• Final words

DISCLAIMER: this is an EARLY DRAFT, but it is usable.

CIEL Is an Extended Lisp

What is this ?

CIEL is a collection of useful Quicklisp libraries.

It's Common Lisp, batteries included.

Questions, doubts? See the FAQ.

TODOs

• settle on libraries that help newcomers
• automate the documentation
• distribute (Quicklisp, Qlot, Quicklisp distribution, Ultralisp, Ultralisp distribution (upcoming)…)
• ship a core image and a binary
• optionnal: create a tool that, given a CIEL code base, explains what packages to import in order to switch to "plain CL".

How to procede ?

This is an experiment. I'd be happy to give push rights to more maintainers. We will send pull requests, discuss, and in case we don't find a consensus for what should be on by default, we can create other packages.

Rules

• don't install libraries that need a Slime helper to work in the REPL (cl-annot).
• reader syntax changes may not be enabled by default.

Install

With Quicklisp

You need a Lisp implementation and Quicklisp installed.

CIEL is not yet on Quicklisp (but it is on Ultralisp), so clone this repository and load the .asd (with load or C-c C-k in Slime).

git clone https://github.com/ciel-lang/CIEL ~/quicklisp/local-projects/CIEL

Then, quickload it:

(ql:quickload "ciel")

and enter the ciel-user package, instead of the default common-lisp-user (or cl-user):

(in-package :ciel-user)

With a core image

You need a Lisp implementation, but you don't need Quicklisp.

Build a core image for your lisp with all CIEL's dependencies:

sbcl --load build-image.lisp

and use it:

sbcl --core ciel --eval '(in-package :ciel-user)'

TODO: we will distribute ready-to-use core images.

With a binary. Use CIEL's custom REPL.

You don't need anything, just download the CIEL executable and run its REPL.

TODO: build it on CI for different platforms.

To build it, clone this repository and run make build.

Start it with ./ciel-repl.

You are dropped into a custom Lisp REPL, freely based on sbcli.

This REPL is more user friendly than the default SBCL one:

• it has readline capabilities, meaning that the arrow keys work (!) and there is a persistent history, like in any shell.
• it has multiline input and reset.
• it has TAB completion.
• it handles errors gracefully: you are not dropped into the debugger and its sub-REPL, you simply see the error message. It also defines short helper commands:

  :help => Prints this general help message
  :doc => Prints the available documentation for this symbol
  :? => Gets help on a symbol <sym>: :? str
  :w => Writes the current session to a file <filename>
  :d => Dumps the disassembly of a symbol <sym>
  :t => Prints the type of a expression <expr>
  :q => Ends the session.

Note: the documentation is also available by appending a "?" after a function name:

ciel-user> (dict ?

Libraries

Data structures

Generic and nested access to datastructures (access)

From Access, we import access and accesses (plural).

It's always

(access my-structure :elt)

for an alist, a hash-table, a struct, an object… Use accesses for nested access (specially useful with JSON).

Hash-table utilities (Serapeum)

We import functions from Serapeum. https://github.com/ruricolist/serapeum/blob/master/REFERENCE.md#hash-tables

:dict
:do-hash-table ;; see also trivial-do
:dict*
:dictq  ;; quoted
:href  ;; for nested lookup.
:href-default
:pophash
:swaphash
:hash-fold
:maphash-return
:merge-tables
:flip-hash-table
:set-hash-table
:hash-table-set
:hash-table-predicate
:hash-table-function
:make-hash-table-function
:delete-from-hash-table
:pairhash

;; create a hash-table:
(dict :a 1 :b 2 :c 3)

Sequences utilities (Serapeum)

See https://github.com/ruricolist/serapeum/blob/master/REFERENCE.md#sequences

:assort
:batches
:iota
:runs
:partition
:partitions
:split-sequence

String manipulation (str)

Available with the str prefix.

https://github.com/vindarel/cl-str/

Data formats

CSV

You have cl-csv, under its cl-csv package name and the csv local nickname.

  ;; read a file into a list of lists
  (cl-csv:read-csv #P"file.csv")
  => (("1" "2" "3") ("4" "5" "6"))

  ;; read csv from a string (streams also supported)
  (cl-csv:read-csv "1,2,3
  4,5,6")
  => (("1" "2" "3") ("4" "5" "6"))

  ;; read a file that's tab delimited
  (cl-csv:read-csv #P"file.tab" :separator #\Tab)

  ;; loop over a CSV for effect
  (let ((sum 0))
    (cl-csv:do-csv (row #P"file.csv")
      (incf sum (parse-integer (nth 0 row))))
    sum)

See also:

• auto-text, automatic detection for text files (encoding, end of line, column width, csv delimiter etc). inquisitor for detection of asian and far eastern languages.
• CLAWK, an AWK implementation embedded into Common Lisp, to parse files line-by-line.

JSON

We use cl-json (GitHub). It has a json nickname.

To encode an object to a string, use encode-json-to-string:

(json:encode-json-to-string (list (dict :a 1)))
;; "[{\"A\":1}]"

To decode from a string: decode-json-from-string.

To encode or decode objects from a stream, use:

• encode-json object &optional stream
• decode-json &optional stream

as in:

(with-output-to-string (s)
   (json:encode-json (dict :foo (list 1 2 3)) s))
;; "{\"FOO\":[1,2,3]}"

(with-input-from-string (s "{\"foo\": [1, 2, 3], \"bar\": true, \"baz\": \"!\"}")
  (json:decode-json s))
;; ((:|foo| 1 2 3) (:|bar| . T) (:|baz| . "!"))

cl-json can encode and decode from objects. Given a simple class:

(defclass person ()
  ((name :initarg :name)
   (lisper :initform t)))

We can encode an instance of it:

(json:encode-json-to-string (make-instance 'person :name "you"))
;; "{\"NAME\":\"you\",\"LISPER\":true}"

By default, cl-json wants to convert our lisp symbols to camelCase, and the JSON ones to lisp-case. We disable that in the ciel-user package.

You can set this behaviour back with:

(setf json:*json-identifier-name-to-lisp* #'json:camel-case-to-lisp)
(setf json:*lisp-identifier-name-to-json* #'json:lisp-to-camel-case)

Databases

Mito and SxQL are available.

https://lispcookbook.github.io/cl-cookbook/databases.html

Pattern matching

Use Trivia, also available with the match local nickname.

Numbers

parse-float

https://github.com/soemraws/parse-float

parse-number

https://github.com/sharplispers/parse-number

Parsing numbers, floats, decimals

cl-decimals: parse and format decimal numbers

https://github.com/tlikonen/cl-decimals

The main interface are the functions parse-decimal-number and format-decimal-number. The former is for parsing strings for decimal numbers and the latter for pretty-printing them as strings.

Reading:

DECIMALS> (parse-decimal-number "0.24")
6/25


DECIMALS> (parse-decimal-number "−12,345"
                                :decimal-separator #\,
                                :negative-sign #\−)
-2469/200

Parsing:

DECIMALS> (format-decimal-number -100/6 :round-magnitude -3)
"-16.667"
("-" "16" "." "667")

DECIMALS> (loop for e from -5 upto 5
                do (print (format-decimal-number
                           (expt 10 e) :round-magnitude -5
                           :decimal-separator ","
                           :integer-minimum-width 7
                           :integer-group-separator " "
                           :fractional-minimum-width 7
                           :fractional-group-separator " ")))

"      0,000 01"
"      0,000 1 "
"      0,001   "
"      0,01    "
"      0,1     "
"      1       "
"     10       "
"    100       "
"  1 000       "
" 10 000       "
"100 000       "
NIL

Regular expressions

Use ppcre.

Threads, monitoring, scheduling

Bordeaux-Threads (bt prefix)

Lparallel

Moira (monitor and restart background threads)

trivial-monitored-thread

Trivial Monitored Thread offers a very simple (aka trivial) way of spawning threads and being informed when one any of them crash and die.

cl-cron (with our fork here)

For example, run a function every minute:

  (defun say-hi () (print "Hi!"))
  (cl-cron:make-cron-job #'say-hi)
  (cl-cron:start-cron)

Wait a minute to see some output.

Stop all jobs with stop-cron.

make-cron's keyword arguments are:

(minute :every) (step-min 1) (hour :every) (step-hour 1) (day-of-month :every)
(step-dom 1) (month :every) (step-month 1) (day-of-week :every)
(step-dow 1)
(boot-only nil) (hash-key nil))

HTTP and URI handling

See:

• Dexador (dex nickname)
• Quri
• Lquery

Web

Imported:

• Hunchentoot
• Easy-routes
• Djula
• Spinneret

https://lispcookbook.github.io/cl-cookbook/web.html

Conditions

From Serapeum, we import ignoring.

An improved version of ignore-errors. The behavior is the same: if an error occurs in the body, the form returns two values, nil and the condition itself.

ignoring forces you to specify the kind of error you want to ignore:

  (ignoring parse-error
            ...)

Types, type checking, exhaustiveness type checking

From Serapeum, we import:

  :etypecase-of
  :ctypecase-of
  :typecase-of
  :case-of
  :ccase-of

etypecase-of allows to do compile-time exhaustiveness type checking.

Example with enums

We may call a type defined using member an enumeration. Take an enumeration like this:

(deftype switch-state ()
  '(member :on :off :stuck :broken))

Now we can use ecase-of to take all the states of the switch into account.

(defun flick (switch)
  (ecase-of switch-state (state switch)
    (:on (switch-off switch))
    (:off (switch-on switch))))
=> Warning

(defun flick (switch)
  (ecase-of switch-state (state switch)
    (:on (switch-off switch))
    (:off (switch-on switch))
    ((:stuck :broken) (error "Sorry, can't flick ~a" switch))))
=> No warning

Example with union types

(defun negative-integer? (n)
(etypecase-of t n
((not integer) nil)
((integer * -1) t)
((integer 1 *) nil)))
=> Warning

(defun negative-integer? (n)
(etypecase-of t n
((not integer) nil)
((integer * -1) t)
((integer 1 *) nil)
((integer 0) nil)))
=> No warning

See Serapeum's reference.

Syntax extensions

Arrow macros

We provide the Clojure-like arrow macros and "diamond wands" from the arrow-macros library.

  ;; -> inserts the previous value as its first argument:
  (-> "  hello macros   "
    str:upcase
    str:words) ; => ("HELLO" "MACROS")

  ;; ->> inserts it as its second argument:
  (->> "  hello macros   "
    str:upcase
    str:words
    (mapcar #'length)) ; => (5 6)


  ;; use as-> to be flexible on the position of the argument:
  (as-> 4 x
    (1+ x)
    (+ x x)) ; => 10

And there is more. All the available macros are:

:->
:->>
:some->
:some->>
:as->
:cond->
:cond->>
:-<>
:-<>>
:some-<>
:some-<>>

Pythonic triple quotes docstring

https://github.com/smithzvk/pythonic-string-reader

We can use triple quotes for docstrings, and double quotes within them.

(defun foo ()
  """foo "bar"."""
  t)

Lambda shortcuts

You have to enable cl-punch's syntax yourself.

https://github.com/windymelt/cl-punch/ - Scala-like anonymous lambda literal.

(cl-punch:enable-punch-syntax)

;; ^() is converted into (lambda ...) .
;; Each underscore is converted into a lambda argument.

(mapcar ^(* 2 _) '(1 2 3 4 5))
;; => '(2 4 6 8 10)

;; One underscore corresponds one argument.

(^(* _ _) 2 3)
;; => 6

;; <_ reuses last argument.

(mapcar ^(if (oddp _) (* 2 <_) <_) '(1 2 3 4 5))
;; => '(2 2 6 4 10)

;; _! corresponds one argument but it is brought to top of the argument list.
;; It can be useful when you want to change argument order.

(^(cons _ _!) :a :b)
;; => (:b . :a)

(^(list _! _! _!) 1 2 3)
;; => '(3 2 1)

Development

Testing (Fiveam)

The FiveAM test framework is available for use.

Below we create a package to contain our tests and we define the most simple one:

(defpackage ciel-5am
(:use :cl :5am))

(in-package :ciel-5am)

(test test-one
(is (= 1 1)))

Run the test with:

(run! 'test-one)

Running test TEST-ONE .
 Did 1 check.
    Pass: 1 (100%)
    Skip: 0 ( 0%)
    Fail: 0 ( 0%)

T
NIL
NIL

If the test fails you will see explanations:

> (run! 'test-one)

Running test TEST-ONE .f
 Did 2 checks.
    Pass: 1 (50%)
    Skip: 0 ( 0%)
    Fail: 1 (50%)

 Failure Details:
 --------------------------------
 TEST-ONE []:

1

 evaluated to

1

 which is not

=

 to

2


 --------------------------------

NIL
(#<IT.BESE.FIVEAM::TEST-FAILURE {1007307ED3}>)
NIL

Use run to not print explanations.

You can use (!) to re-run the last run test.

You can ask 5am to open the interactive debugger on an error:

(setf *debug-on-error* t)

Logging (log4cl)

https://github.com/sharplispers/log4cl/

(log:info …)

Discoverability of documentation (repl-utilities' readme, summary,…)

We use readme and summary from repl-utilities.

Learn more with:

(readme repl-utilities)

printv

printv

(:printv
(defvar *y*)
(defparameter *x* 2)
(setf *y* (sqrt *x*))
(setf *y* (/ 1 *y*)))

;; This produces the following text to PRINTV's output stream, and still results in the same returned value: 0.70710677.

;;;   (DEFVAR *Y*) => *Y*
;;;   (DEFPARAMETER *X* 2) => *X*
;;;   (SETF *Y* (SQRT *X*)) => 1.4142135
;;;   (SETF *Y* (/ 1 *Y*)) => 0.70710677

Getting a function's arguments list (trivial-arguments)

https://github.com/Shinmera/trivial-arguments

(defun foo (a b c &optional d) nil)
(arglist #'foo)
;; (a b c &optional d)

generic-cl

https://github.com/alex-gutev/generic-cl/

todo:

generic-ciel

Example:

;; with a struct or class "point":
(defmethod equalp ((p1 point) (p2 point))
   (…))

Final words

That was your life in CL:

and now: