ciel/docs/scripting.md

# Scripting

CIEL provides a fast-starting scripting solution for Common Lisp.

It is based on a standalone binary (created with the fast SBCL
implementation) and it ships useful built-in utilities, for real-world
needs: HTTP, JSON, CSV handling, plotting, and more. You just have to
get the binary and run your script. Use a shebang line if you wish.

It's a fast and easy solution to write Lisp code for your day-to-day tasks.

> Note: this is brand new!  Expect limitations and changes.

Get the `ciel` binary (it's under 30MB) and call it with your .lisp script:

```
$ ciel script.lisp
```

(or just `./script.lisp` with a shebang line, see below)

Call built-in scripts:

```
$ ciel -s simpleHTTPserver 9000
```

> Note: script names are case insensitive.

### Example script

```lisp
#!/usr/bin/env ciel
;; optional shebang line, only for the short ./script call)

;; Start your script with this to access all CIEL goodies:
(in-package :ciel-user)

(defun hello (name)
  "Say hello."
  ;; format! prints on standard output and flushes the streams.
  (format! t "Hello ~a!~&" name))

;; Access CLI args:
(hello (second *script-args*))

;; We have access to the DICT notation for hash-tables:
(print "testing dict:")
(print (dict :a 1 :b 2))

;; We can run shell commands:
(cmd:cmd "ls")

;; Access environment variables:
(hello (os:getenv "USER"))  ;; os is a nickname for uiop/os

(format! t "Let's define an alias to run shell commands with '!'. This gives: ")
(defalias ! #'cmd:cmd)
(! "pwd")

;; In cas of an error, we can ask for a CIEL toplevel REPL:
(handler-case
    (error "oh no")
  (error (c)
    (format! t "An error occured: ~a" c)
    (format! t "Here's a CIEL top level REPL: ")
    (sbcli::repl :noinform t)))
```

Output:


```
$ ciel script.lisp you
=>

Hello you!
"testing dict:"

 (dict
  :A 1
  :B 2
 )
cmd? ABOUT.org	    ciel		     ciel-core
   bin  		    docs		     src
 […]
Hello vindarel!
Let's define an alias to run shell commands with '!'. This gives:
/home/vindarel/projets/ciel
ciel-user>
```

## Command line arguments

Access them with `ciel-user:*script-args*`. It is a list of strings that
contains your script name as first argument.

This list of arguments is modified by us (especially if you call
scripts with the `-s` option). You can always check the full original
list with `(uiop:command-line-arguments)`.

You can use CL built-ins to look what's into this list, such as `(member "-h" *script-args* :test #'equal)`.

<!-- todo: show example. -->
You can use a proper command-line options parser, which is shipped with CIEL: [Clingon](https://github.com/dnaeon/clingon). This top-notch library supports:

- Short and long option names support
- Automatic generation of help/usage information for commands and sub-commands
- Support for various kinds of options like *string*, *integer*, *boolean*, *switches*, *enums*, *list*, *counter*, *filepath*, etc.
- Out of the box support for `--version` and `--help` flags
- Subcommands
- Support for pre-hook and post-hook actions for commands, which allows invoking functions before and after the respective handler of the command is executed
- Support for Bash and Zsh shell completions
- and more.

All *unknown* free arguments coming after your script name are passed along to the script in the `*script-args*` variable. This works:

    $ ./simpleHTTPserver.lisp -b 4242

However, in the following case the "-v" option would be intercepted by the ciel binary, because it is one of its known options:

    $ ./simpleHTTPserver.lisp -v -b 4242

To give "-v" to your script, use a double slash, as in:

    $ ./simpleHTTPserver.lisp -- -v -b 4242

Pull requests are accepted to make this better.


## Executable file and shebang line

We can also make a CIEL file executable and run it directly, like this:

```
$ chmod +x script.lisp
$ ./script.lisp
```

Add the following shebang at the beginning:

```sh
#!/usr/bin/env ciel

(in-package :ciel-user)
;; lisp code follows.
```

You also need to add the `ciel` binary in your path. A possibility:

    $ ln -s /home/path/to/ciel/bin/ciel ~/.local/bin/ciel

It magically works because before LOAD-ing this Lisp file, we remove the shebang line, and load the remaining Lisp code.

<!-- daaaamn no need of a complex shebang like Roswell like at did at the beginning. See previous commits. Do we want a complex shebang to pass options to the CIEL binary ?

#!/bin/sh
#|-*- mode:lisp -*-|#
#|
exec /path/to/ciel `basename $0` "$@"

How it works:

- it starts as a /bin/sh script
  - all lines starting by `#` are shell comments
- the exec calls the `ciel` binary with this file name as first argument,
  the rest of the file (lisp code) is not read by the shell.
  - before LOAD-ing this Lisp file, we remove the #!/bin/sh shebang line.
  - Lisp ignores comments between `#|` and `|#` and runs the following lisp code.

-->

## Main function and interactive development

TLDR: use the `#+ciel` feature flag as in:

~~~lisp
(in-package :ciel-user)

(defun main ()
  …)

#+ciel
(main)
~~~

Writing scripts is nice, but it is even better when doing so in a
fully interactive Lisp environment, such as in Emacs and Slime (which
is not the only good one anymore ;) ). We then must have a way to have
a piece of code executed when we run the script (the call to the
"main" function doing the side effects), but *not* executed when we
`load` the file or when we compile and load the whole buffer during
development (`C-c C-k`) (note that we can always compile functions
individually with `C-c C-c`).

In Python, the pattern is `__name__ == "__main__"`. In CIEL, we use
Common Lisp's feature flags: the variable `*features*` (inside the
`ciel-user` package) is a list containing symbols that represent
features currently enabled in the Lisp image. For example, here's an
extract:

~~~lisp
CIEL-USER> *features*
(…
 :CL-PPCRE-UNICODE :THREAD-SUPPORT :SWANK :QUICKLISP :ASDF3.3
 :ASDF :OS-UNIX :ASDF-UNICODE :X86-64 :64-BIT
 :COMMON-LISP :ELF :IEEE-FLOATING-POINT :LINUX :LITTLE-ENDIAN
 :PACKAGE-LOCAL-NICKNAMES :SB-LDB :SB-PACKAGE-LOCKS :SB-THREAD :SB-UNICODE
 :SBCL :UNIX)
~~~

Before running your script, we add the `:CIEL` symbol to this
list. The `#+foo` reader macro is the way to check if the feature
"foo" is enabled. You can also use `#-foo` to check its absence. To
always disable a piece of code, the pattern is `#+(or)`, that always
evaluates to nil.

Make sure you are "in" the `ciel-user` package when writing this `#+ciel` check.


## Eval and one-liners

Use `--eval` or `-e` to eval some lisp code.

Example:

```sh
$ ciel -e "(uiop:file-exists-p \"README.org\")"
/home/vindarel/projets/ciel/README.org

$ ciel -e "(-> \"README.org\" (uiop:file-exists-p))"
/home/vindarel/projets/ciel/README.org

$ ciel -e "(-> (http:get \"https://fakestoreapi.com/products/1\") (json:read-json))"

 (dict
  "id" 1
  "title" "Fjallraven - Foldsack No. 1 Backpack, Fits 15 Laptops"
  "price" 109.95
  "description" "Your perfect pack for everyday use and walks in the forest. Stash your laptop (up to 15 inches) in the padded sleeve, your everyday"
  "category" "men's clothing"
  "image" "https://fakestoreapi.com/img/81fPKd-2AYL._AC_SL1500_.jpg"
  "rating"
  (dict
   "rate" 3.9
   "count" 120
  )
 )
```

## Built-in scripts

Call built-in scripts with `--script <scriptname>` or `-s`.

Call `ciel --scripts` to list the available scripts.

Those are for demo purposes and are subject to evolve. Ideas and contributions welcome.

### Simple HTTP server

```
$ ciel -s simpleHTTPserver 9000
```

open `http://localhost:9000` and see the list of files.

See `src/scripts/simpleHTTPserver.lisp` in the CIEL repository.

You can preview HTML files and have static assets under a `static/` directory.

Given you have an `index.html` file:

```html
<html>
  <head>
    <title>Hello!</title>
  </head>
  <body>
    <h1>Hello CIEL!</h1>
    <p>
    We just served our own files.
    </p>
  </body>
</html>
```

The script will serve static assets under a `static/` directory.

Now load a .js file as usual in your template:

        <script src="/static/ciel.js"></script>

which can be:

~~~javascript
// ciel.js
alert("hello CIEL!");
~~~

Example output:

```
$ ciel -s simpleHTTPserver 4242
Serving files on port 4242…

  ⤷ http://127.0.0.1:4242

[click on the index.html file]

127.0.0.1 - [2022-12-14 12:06:00] "GET / HTTP/1.1" 200 200 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:103.0) Gecko/20100101 Firefox/103.0"
```

### Quicksearch

Search for Lisp libraries on Quicklisp, Cliki and Github.

see `src/scripts/quicksearch.lisp`.

```lisp
$ ciel -s quicksearch color

SEARCH-RESULTS: "color"
=======================

 Quicklisp
 ---------
  cl-colors
      /home/vince/quicklisp/dists/quicklisp/software/cl-colors-20180328-git/
      http://beta.quicklisp.org/archive/cl-colors/2018-03-28/cl-colors-20180328-git.tgz
      http://quickdocs.org/cl-colors/
[…]
 Cliki
 -----
  colorize
      http://www.cliki.net/colorize
      Colorize is an Application for colorizing chunks of Common Lisp, Scheme,
      Elisp, C, C++, or Java code
[…]
 GitHub
 ------
  colorize
      https://github.com/kingcons/colorize
      A Syntax Highlighting library
  cl-colors
      https://github.com/tpapp/cl-colors
      Simple color library for Common Lisp
[…]
```

### API Pointer

Call a JSON API and access nested data with a JSON pointer:

    ciel -s apipointer URL "/json/pointer"

Example:

    $ ciel -s apipointer https://fakestoreapi.com/products\?limit\=3 "/0/rating/rate"
    3.9

We welcome more capable, expanded scripts!

---

Now, let us iron out the details ;)