Use a dedicated type to represent interpreted-function values

Change `function` so that when evaluating #'(lambda ...)
we return an object of type `interpreted-function` rather than
a list starting with one of `lambda` or `closure`.
The new type reuses the existing PVEC_CLOSURE (nee PVEC_COMPILED)
tag and tries to align the corresponding elements:

- the arglist, the docstring, and the interactive-form go in the
  same slots as for byte-code functions.
- the body of the function goes in the slot used for the bytecode string.
- the lexical context goes in the slot used for the constants of
  bytecoded functions.

The first point above means that `help-function-arglist`,
`documentation`, and `interactive-form`s don't need to
distinguish interpreted and bytecode functions any more.

Main benefits of the change:

- We can now reliably distinguish a list from a function value.
- `cl-defmethod` can dispatch on `interactive-function` and `closure`.
  Dispatch on `function` also works now for interpreted functions but still
  won't work for functions represented as lists or as symbols, of course.
- Function values are now self-evaluating.  That was alrready the case
  when byte-compiled, but not when interpreted since
  (eval '(closure ...)) signals a void-function error.
  That also avoids false-positive warnings about "don't quote your lambdas"
  when doing things like `(mapcar ',func ...)`.

* src/eval.c (Fmake_interpreted_closure): New function.
(Ffunction): Use it and change calling convention of
`Vinternal_make_interpreted_closure_function`.
(FUNCTIONP, Fcommandp, eval_sub, funcall_general, funcall_lambda)
(Ffunc_arity, lambda_arity): Simplify.
(funcall_lambda): Adjust to new representation.
(syms_of_eval): `defsubr` the new function.  Remove definition of `Qclosure`.

* lisp/emacs-lisp/cconv.el (cconv-make-interpreted-closure):
Change calling convention and use `make-interpreted-closure`.

* src/data.c (Fcl_type_of): Distinguish `byte-code-function`s from
`interpreted-function`s.
(Fclosurep, finterpreted_function_p): New functions.
(Fbyte_code_function_p): Don't be confused by `interpreted-function`s.
(Finteractive_form, Fcommand_modes): Simplify.
(syms_of_data): Define new type symbols and `defsubr` the two
new functions.

* lisp/emacs-lisp/cl-print.el (cl-print-object) <interpreted-function>:
New method.

* lisp/emacs-lisp/oclosure.el (oclosure): Refine the parent
to be `closure`.
(oclosure--fix-type, oclosure-type): Simplify.
(oclosure--copy, oclosure--get, oclosure--set): Adjust to
new representation.

* src/callint.c (Fcall_interactively): Adjust to new representation.

* src/lread.c (bytecode_from_rev_list):

* lisp/simple.el (function-documentation):
* lisp/help.el (help-function-arglist): Remove the old `closure` case
and adjust the byte-code case so it handles `interpreted-function`s.

* lisp/emacs-lisp/cl-preloaded.el (closure): New type.
(byte-code-function): Add it as a parent.
(interpreted-function): Adjust parent (the type itself was already
added earlier by accident).

* lisp/emacs-lisp/bytecomp.el (byte-compile--reify-function): Adjust to
new representation.
(byte-compile): Use `interpreted-function-p`.

* lisp/emacs-lisp/byte-opt.el (byte-compile-inline-expand): Adjust to
new representation.
(side-effect-free-fns): Add `interpreted-function-p` and `closurep`.

* src/profiler.c (trace_hash, ffunction_equal): Simplify.
* lisp/profiler.el (profiler-function-equal): Simplify.

* lisp/emacs-lisp/nadvice.el (advice--interactive-form-1):
Use `interpreted-function-p`; adjust to new representation; and take
advantage of the fact that function values are now self-evaluating.

* lisp/emacs-lisp/lisp-mode.el (closure):
Remove `lisp-indent-function` property.

* lisp/emacs-lisp/disass.el (disassemble-internal): Adjust to
new representation.
* lisp/emacs-lisp/edebug.el (edebug--strip-instrumentation):
Use `interpreted-function-p`.
* lisp/emacs-lisp/comp-common.el (comp-known-type-specifiers):
Add `closurep` and `interpreted-function-p`.

* test/lisp/help-fns-tests.el (help-fns-test-lisp-defun): Adjust to
more precise type info in `describe-function`.
* test/lisp/erc/resources/erc-d/erc-d-tests.el (erc-d--render-entries):
Use `interpreted-function-p`.
* test/lisp/emacs-lisp/macroexp-resources/vk.el (vk-f4, vk-f5):
Don't hardcode function values.

* doc/lispref/functions.texi (Anonymous Functions): Don't suggest that
function values are lists.  Reword "self-quoting" to reflect the
fact that #' doesn't return the exact same object.  Update examples
with the new shape of the return value.

* doc/lispref/variables.texi (Lexical Binding):
* doc/lispref/lists.texi (Rearrangement):
* doc/lispref/control.texi (Handling Errors): Update examples to reflect
new representation of function values.

lisp/emacs-lisp/byte-opt.el

@@ -164,7 +164,7 @@ Earlier variables shadow later ones with the same name.")
        ;; The byte-code will be really inlined in byte-compile-unfold-bcf.
        (byte-compile--check-arity-bytecode form fn)
        `(,fn ,@(cdr form)))
-      ((or `(lambda . ,_) `(closure . ,_))
+      ((pred interpreted-function-p)
        ;; While byte-compile-unfold-bcf can inline dynbind byte-code into
        ;; letbind byte-code (or any other combination for that matter), we
        ;; can only inline dynbind source into dynbind source or lexbind
@@ -1870,6 +1870,7 @@ See Info node `(elisp) Integer Basics'."
          charsetp
          ;; data.c
          arrayp atom bare-symbol-p bool-vector-p bufferp byte-code-function-p
+         interpreted-function-p closurep
          byteorder car-safe cdr-safe char-or-string-p char-table-p
          condition-variable-p consp eq floatp indirect-function
          integer-or-marker-p integerp keywordp listp markerp

lisp/emacs-lisp/bytecomp.el

@@ -2915,9 +2915,14 @@ otherwise, print without quoting."
 (defun byte-compile--reify-function (fun)
   "Return an expression which will evaluate to a function value FUN.
 FUN should be an interpreted closure."
-  (pcase-let* ((`(closure ,env ,args . ,body) fun)
-               (`(,preamble . ,body) (macroexp-parse-body body))
-               (renv ()))
+  (let* ((args (aref fun 0))
+         (body (aref fun 1))
+         (env (aref fun 2))
+         (docstring (function-documentation fun))
+         (iform (interactive-form fun))
+         (preamble `(,@(if docstring (list docstring))
+                     ,@(if iform (list iform))))
+         (renv ()))
     ;; Turn the function's closed vars (if any) into local let bindings.
     (dolist (binding env)
       (cond
@@ -2954,11 +2959,11 @@ If FORM is a lambda or a macro, byte-compile it as a function."
                  (if (symbolp form) form "provided"))
         fun)
        (t
-        (when (or (symbolp form) (eq (car-safe fun) 'closure))
+        (when (or (symbolp form) (interpreted-function-p fun))
           ;; `fun' is a function *value*, so try to recover its
           ;; corresponding source code.
-          (when (setq lexical-binding (eq (car-safe fun) 'closure))
-            (setq fun (byte-compile--reify-function fun)))
+          (setq lexical-binding (not (null (aref fun 2))))
+          (setq fun (byte-compile--reify-function fun))
           (setq need-a-value t))
         ;; Expand macros.
         (setq fun (byte-compile-preprocess fun))
@@ -5148,7 +5153,6 @@ binding slots have been popped."
             ;; `arglist' is the list of arguments (or t if not recognized).
             ;; `body' is the body of `lam' (or t if not recognized).
             ((or `(lambda ,arglist . ,body)
-                 ;; `(closure ,_ ,arglist . ,body)
                  (and `(internal-make-closure ,arglist . ,_) (let body t))
                  (and (let arglist t) (let body t)))
              lam))

lisp/emacs-lisp/cconv.el

@@ -902,7 +902,7 @@ lexically and dynamically bound symbols actually used by FORM."
                                     (delete-dups cconv--dynbindings)))))
         (cons fvs dyns)))))
 
-(defun cconv-make-interpreted-closure (fun env)
+(defun cconv-make-interpreted-closure (args body env docstring iform)
   "Make a closure for the interpreter.
 This is intended to be called at runtime by the ELisp interpreter (when
 the code has not been compiled).
@@ -911,22 +911,27 @@ ENV is the runtime representation of the lexical environment,
 i.e. a list whose elements can be either plain symbols (which indicate
 that this symbol should use dynamic scoping) or pairs (SYMBOL . VALUE)
 for the lexical bindings."
-  (cl-assert (eq (car-safe fun) 'lambda))
+  (cl-assert (consp body))
+  (cl-assert (listp args))
   (let ((lexvars (delq nil (mapcar #'car-safe env))))
-    (if (or (null lexvars)
-            ;; Functions with a `:closure-dont-trim-context' marker
-            ;; should keep their whole context untrimmed (bug#59213).
-            (and (eq :closure-dont-trim-context (nth 2 fun))
-                 ;; Check the function doesn't just return the magic keyword.
-                 (nthcdr 3 fun)))
+    (if (or
+         ;; Functions with a `:closure-dont-trim-context' marker
+         ;; should keep their whole context untrimmed (bug#59213).
+         (and (eq :closure-dont-trim-context (car body))
+              ;; Check the function doesn't just return the magic keyword.
+              (cdr body)
+              ;; Drop the magic marker from the closure.
+              (setq body (cdr body)))
+         ;; There's no var to capture, so skip the analysis.
+         (null lexvars))
         ;; The lexical environment is empty, or needs to be preserved,
         ;; so there's no need to look for free variables.
-        ;; Attempting to replace ,(cdr fun) by a macroexpanded version
-        ;; causes bootstrap to fail.
-        `(closure ,env . ,(cdr fun))
+        ;; Attempting to replace body by a macroexpanded version
+        ;; caused bootstrap to fail.
+        (make-interpreted-closure args body env docstring iform)
       ;; We could try and cache the result of the macroexpansion and
       ;; `cconv-fv' analysis.  Not sure it's worth the trouble.
-      (let* ((form `#',fun)
+      (let* ((form `#'(lambda ,args ,iform . ,body))
              (expanded-form
               (let ((lexical-binding t) ;; Tell macros which dialect is in use.
 	            ;; Make the macro aware of any defvar declarations in scope.
@@ -935,10 +940,10 @@ for the lexical bindings."
                          (append env macroexp--dynvars) env)))
                 (macroexpand-all form macroexpand-all-environment)))
              ;; Since we macroexpanded the body, we may as well use that.
-             (expanded-fun-cdr
+             (expanded-fun-body
               (pcase expanded-form
-                (`#'(lambda . ,cdr) cdr)
-                (_ (cdr fun))))
+                (`#'(lambda ,_args ,_iform . ,newbody) newbody)
+                (_ body)))
 
              (dynvars (delq nil (mapcar (lambda (b) (if (symbolp b) b)) env)))
              (fvs (cconv-fv expanded-form lexvars dynvars))
@@ -946,7 +951,8 @@ for the lexical bindings."
                             (cdr fvs))))
         ;; Never return a nil env, since nil means to use the dynbind
         ;; dialect of ELisp.
-        `(closure ,(or newenv '(t)) . ,expanded-fun-cdr)))))
+        (make-interpreted-closure args expanded-fun-body (or newenv '(t))
+                                  docstring iform)))))
 
 
 (provide 'cconv)

lisp/emacs-lisp/cl-preloaded.el

@@ -444,13 +444,24 @@ For this build of Emacs it's %dbit."
   )
 (cl--define-built-in-type compiled-function (function)
   "Abstract type of functions that have been compiled.")
-(cl--define-built-in-type byte-code-function (compiled-function)
+(cl--define-built-in-type closure (function)
+  "Abstract type of functions represented by a vector-like object.
+You can access the object's internals with `aref'.
+The fields are used as follows:
+
+  0 [args]       Argument list (either a list or an integer)
+  1 [code]       Either a byte-code string or a list of Lisp forms
+  2 [constants]  Either vector of constants or a lexical environment
+  3 [stackdepth] Maximum amount of stack depth used by the byte-code
+  4 [docstring]  The documentation, or a reference to it
+  5 [iform]      The interactive form (if present)")
+(cl--define-built-in-type byte-code-function (compiled-function closure)
   "Type of functions that have been byte-compiled.")
 (cl--define-built-in-type subr (atom)
   "Abstract type of functions compiled to machine code.")
 (cl--define-built-in-type module-function (function)
   "Type of functions provided via the module API.")
-(cl--define-built-in-type interpreted-function (function)
+(cl--define-built-in-type interpreted-function (closure)
   "Type of functions that have not been compiled.")
 (cl--define-built-in-type special-form (subr)
   "Type of the core syntactic elements of the Emacs Lisp language.")

lisp/emacs-lisp/cl-print.el

@@ -237,6 +237,38 @@ into a button whose action shows the function's disassembly.")
                               'byte-code-function object)))))
     (princ ")" stream)))
 
+(cl-defmethod cl-print-object ((object interpreted-function) stream)
+  (unless stream (setq stream standard-output))
+  (princ "#f(lambda " stream)
+  (let ((args (help-function-arglist object 'preserve-names)))
+    ;; It's tempting to print the arglist from the "usage" info in the
+    ;; doc (e.g. for `&key` args), but that only makes sense if we
+    ;; *don't* print the body, since otherwise the body will tend to
+    ;; refer to args that don't appear in the arglist.
+    (if args
+        (prin1 args stream)
+      (princ "()" stream)))
+  (let ((env (aref object 2)))
+    (if (null env)
+        (princ " :dynbind" stream)
+      (princ " " stream)
+      (cl-print-object
+       (vconcat (mapcar (lambda (x) (if (consp x) (list (car x) (cdr x)) x))
+                        env))
+       stream)))
+  (let* ((doc (documentation object 'raw)))
+    (when doc
+      (princ " " stream)
+      (prin1 doc stream)))
+  (let ((inter (interactive-form object)))
+    (when inter
+      (princ " " stream)
+      (cl-print-object inter stream)))
+  (dolist (exp (aref object 1))
+    (princ " " stream)
+    (cl-print-object exp stream))
+  (princ ")" stream))
+
 ;; This belongs in oclosure.el, of course, but some load-ordering issues make it
 ;; complicated.
 (cl-defmethod cl-print-object ((object accessor) stream)

lisp/emacs-lisp/comp-common.el

@@ -118,7 +118,9 @@ Used to modify the compiler environment."
     (buffer-substring
      (function ((or integer marker) (or integer marker)) string))
     (bufferp (function (t) boolean))
+    (closurep (function (t) boolean))
     (byte-code-function-p (function (t) boolean))
+    (interpreted-function-p (function (t) boolean))
     (capitalize (function ((or integer string)) (or integer string)))
     (car (function (list) t))
     (car-less-than-car (function (list list) boolean))

lisp/emacs-lisp/disass.el

@@ -129,7 +129,7 @@ redefine OBJECT if it is a symbol."
 	   (setq args (help-function-arglist obj))	;save arg list
 	   (setq obj (cdr obj))		;throw lambda away
 	   (setq obj (cdr obj)))
-	  ((byte-code-function-p obj)
+	  ((closurep obj)
 	   (setq args (help-function-arglist obj)))
           (t (error "Compilation failed")))
     (if (zerop indent) ; not a nested function
@@ -178,7 +178,9 @@ redefine OBJECT if it is a symbol."
 	  (t
 	   (insert "Uncompiled body:  ")
 	   (let ((print-escape-newlines t))
-	     (prin1 (macroexp-progn obj)
+	     (prin1 (macroexp-progn (if (interpreted-function-p obj)
+		                        (aref obj 1)
+		                      obj))
 		    (current-buffer))))))
   (if interactive-p
       (message "")))

lisp/emacs-lisp/edebug.el

@@ -4254,7 +4254,7 @@ code location is known."
           ((pred edebug--symbol-prefixed-p) nil)
           (_
            (when (and skip-next-lambda
-                      (not (memq (car-safe fun) '(closure lambda))))
+                      (not (interpreted-function-p fun)))
              (warn "Edebug--strip-instrumentation expected an interpreted function:\n%S" fun))
 	   (unless skip-next-lambda
              (edebug--unwrap-frame new-frame)

lisp/emacs-lisp/lisp-mode.el

@@ -1347,7 +1347,6 @@ Lisp function does not specify a special indentation."
 (put 'condition-case 'lisp-indent-function 2)
 (put 'handler-case 'lisp-indent-function 1) ;CL
 (put 'unwind-protect 'lisp-indent-function 1)
-(put 'closure 'lisp-indent-function 2)
 
 (defun indent-sexp (&optional endpos)
   "Indent each line of the list starting just after point.

lisp/emacs-lisp/nadvice.el

@@ -185,7 +185,7 @@ DOC is a string where \"FUNCTION\" and \"OLDFUN\" are expected.")
 (defun advice--interactive-form-1 (function)
   "Like `interactive-form' but preserves the static context if needed."
   (let ((if (interactive-form function)))
-    (if (or (null if) (not (eq 'closure (car-safe function))))
+    (if (not (and if (interpreted-function-p function)))
         if
       (cl-assert (eq 'interactive (car if)))
       (let ((form (cadr if)))
@@ -193,14 +193,14 @@ DOC is a string where \"FUNCTION\" and \"OLDFUN\" are expected.")
             if
           ;; The interactive is expected to be run in the static context
           ;; that the function captured.
-          (let ((ctx (nth 1 function)))
+          (let ((ctx (aref function 2)))
             `(interactive
               ,(let* ((f (if (eq 'function (car-safe form)) (cadr form) form)))
                  ;; If the form jut returns a function, preserve the fact that
                  ;; it just returns a function, which is an info we use in
                  ;; `advice--make-interactive-form'.
                  (if (eq 'lambda (car-safe f))
-                     `',(eval form ctx)
+                     (eval form ctx)
                    `(eval ',form ',ctx))))))))))
 
 (defun advice--interactive-form (function)

lisp/emacs-lisp/oclosure.el

@@ -146,7 +146,7 @@
 (setf (cl--find-class 'oclosure)
       (oclosure--class-make 'oclosure
                             "The root parent of all OClosure types"
-                            nil (list (cl--find-class 'function))
+                            nil (list (cl--find-class 'closure))
                             '(oclosure)))
 (defun oclosure--p (oclosure)
   (not (not (oclosure-type oclosure))))
@@ -431,75 +431,57 @@ ARGS and BODY are the same as for `lambda'."
 
 (defun oclosure--fix-type (_ignore oclosure)
   "Helper function to implement `oclosure-lambda' via a macro.
-This has 2 uses:
-- For interpreted code, this converts the representation of type information
-  by moving it from the docstring to the environment.
-- For compiled code, this is used as a marker which cconv uses to check that
-  immutable fields are indeed not mutated."
-  (if (byte-code-function-p oclosure)
-      ;; Actually, this should never happen since `cconv.el' should have
-      ;; optimized away the call to this function.
-      oclosure
-    ;; For byte-coded functions, we store the type as a symbol in the docstring
-    ;; slot.  For interpreted functions, there's no specific docstring slot
-    ;; so `Ffunction' turns the symbol into a string.
-    ;; We thus have convert it back into a symbol (via `intern') and then
-    ;; stuff it into the environment part of the closure with a special
-    ;; marker so we can distinguish this entry from actual variables.
-    (cl-assert (eq 'closure (car-safe oclosure)))
-    (let ((typename (nth 3 oclosure))) ;; The "docstring".
-      (cl-assert (stringp typename))
-      (push (cons :type (intern typename))
-            (cadr oclosure))
-      oclosure)))
+This is used as a marker which cconv uses to check that
+immutable fields are indeed not mutated."
+  (cl-assert (closurep oclosure))
+  ;; This should happen only for interpreted closures since `cconv.el'
+  ;; should have optimized away the call to this function.
+  oclosure)
 
 (defun oclosure--copy (oclosure mutlist &rest args)
+  (cl-assert (closurep oclosure))
   (if (byte-code-function-p oclosure)
       (apply #'make-closure oclosure
              (if (null mutlist)
                  args
                (mapcar (lambda (arg) (if (pop mutlist) (list arg) arg)) args)))
-    (cl-assert (eq 'closure (car-safe oclosure))
-               nil "oclosure not closure: %S" oclosure)
-    (cl-assert (eq :type (caar (cadr oclosure))))
-    (let ((env (cadr oclosure)))
-      `(closure
-           (,(car env)
-            ,@(named-let loop ((env (cdr env)) (args args))
-                (when args
-                  (cons (cons (caar env) (car args))
-                        (loop (cdr env) (cdr args)))))
-            ,@(nthcdr (1+ (length args)) env))
-           ,@(nthcdr 2 oclosure)))))
+    (cl-assert (consp (aref oclosure 1)))
+    (cl-assert (null (aref oclosure 3)))
+    (cl-assert (symbolp (aref oclosure 4)))
+    (let ((env (aref oclosure 2)))
+      (make-interpreted-closure
+       (aref oclosure 0)
+       (aref oclosure 1)
+       (named-let loop ((env env) (args args))
+         (if (null args) env
+           (cons (cons (caar env) (car args))
+                 (loop (cdr env) (cdr args)))))
+       (aref oclosure 4)
+       (if (> (length oclosure) 5)
+           `(interactive ,(aref oclosure 5)))))))
 
 (defun oclosure--get (oclosure index mutable)
-  (if (byte-code-function-p oclosure)
-      (let* ((csts (aref oclosure 2))
-             (v (aref csts index)))
-        (if mutable (car v) v))
-    (cl-assert (eq 'closure (car-safe oclosure)))
-    (cl-assert (eq :type (caar (cadr oclosure))))
-    (cdr (nth (1+ index) (cadr oclosure)))))
+  (cl-assert (closurep oclosure))
+  (let* ((csts (aref oclosure 2)))
+    (if (vectorp csts)
+        (let ((v (aref csts index)))
+          (if mutable (car v) v))
+      (cdr (nth index csts)))))
 
 (defun oclosure--set (v oclosure index)
-  (if (byte-code-function-p oclosure)
-      (let* ((csts (aref oclosure 2))
-             (cell (aref csts index)))
-        (setcar cell v))
-    (cl-assert (eq 'closure (car-safe oclosure)))
-    (cl-assert (eq :type (caar (cadr oclosure))))
-    (setcdr (nth (1+ index) (cadr oclosure)) v)))
+  (cl-assert (closurep oclosure))
+  (let ((csts (aref oclosure 2)))
+    (if (vectorp csts)
+        (let ((cell (aref csts index)))
+          (setcar cell v))
+      (setcdr (nth index csts) v))))
 
 (defun oclosure-type (oclosure)
-  "Return the type of OCLOSURE, or nil if the arg is not a OClosure."
-  (if (byte-code-function-p oclosure)
-      (let ((type (and (> (length oclosure) 4) (aref oclosure 4))))
-        (if (symbolp type) type))
-    (and (eq 'closure (car-safe oclosure))
-         (let* ((env (car-safe (cdr oclosure)))
-                (first-var (car-safe env)))
-           (and (eq :type (car-safe first-var))
-                (cdr first-var))))))
+  "Return the type of OCLOSURE, or nil if the arg is not an OClosure."
+  (and (closurep oclosure)
+       (> (length oclosure) 4)
+       (let ((type (aref oclosure 4)))
+         (if (symbolp type) type))))
 
 (defconst oclosure--accessor-prototype
   ;; Use `oclosure--lambda' to circumvent a bootstrapping problem: