* lisp/emacs-lisp/comp.el (comp-mvar-symbol-p): As all slots into
a `comp-cstr' are in or fix this logic.
(comp-mvar-cons-p): Likewise.
* test/src/comp-tests.el (bug-44968): New testcase.
* test/src/comp-test-funcs.el (comp-test-44968-f): New test
function.
We used to store module runtime and environment pointers in the static
lists Vmodule_runtimes and Vmodule_environments. However, this is
incorrect because these objects have to be kept per-thread. With this
naive approach, interleaving module function calls in separate threads
leads to environments being removed in the wrong order, which in turn
can cause local module values to be incorrectly garbage-collected.
The fix isn't completely trivial: specbinding the lists wouldn't work
either, because then the garbage collector wouldn't find the
environments in other threads than the current ones, again leading to
objects being garbage-collected incorrectly. While introducing custom
pseudovector types would fix this, it's simpler to put the runtime and
environment pointers into the specbinding list as new specbinding
kinds. This works since we need to unwind them anyway, and we only
ever treat the lists as a stack. The thread switching machinery
ensures that the specbinding lists are thread-local, and that all
elements of the specbinding lists in all threads are marked during
garbage collection.
Module assertions now have to walk the specbinding list for the
current thread, which is more correct since they now only find
environments for the current thread. As a result, we can now remove
the faulty Vmodule_runtimes and Vmodule_environments variables
entirely.
Also add a unit test that exemplifies the problem. It interleaves two
module calls in two threads so that the first call ends while the
second one is still active. Without this change, this test triggers
an assertion failure.
* src/lisp.h (enum specbind_tag): Add new tags for module runtimes and
environments.
* src/eval.c (record_unwind_protect_module): New function to record a
module object in the specpdl list.
(do_one_unbind): Unwind module objects.
(backtrace_eval_unrewind, default_toplevel_binding, lexbound_p)
(Fbacktrace__locals): Deal with new specbinding types.
(mark_specpdl): Mark module environments as needed.
* src/alloc.c (garbage_collect): Remove call to 'mark-modules'.
Garbage collection of module values is now handled as part of marking
the specpdl of each thread.
* src/emacs-module.c (Fmodule_load, funcall_module): Use specpdl to
record module runtimes and environments.
(module_assert_runtime, module_assert_env, value_to_lisp): Walk
through specpdl list instead of list variables.
(mark_module_environment): Rename from 'mark_modules'. Don't attempt
to walk though current thread's environments only, since that would
miss other threads.
(initialize_environment, finalize_environment): Don't change
Vmodule_environments variable; environments are now in the specpdl
list.
(finalize_environment_unwind, finalize_runtime_unwind): Make 'extern'
since do_one_unbind now calls them.
(finalize_runtime_unwind): Don't change Vmodule_runtimes variable;
runtimes are now in the specpdl list.
(syms_of_module): Remove Vmodule_runtimes and Vmodule_environments.
* test/data/emacs-module/mod-test.c (Fmod_test_funcall): New test
function.
(emacs_module_init): Bind it.
* test/src/emacs-module-tests.el (emacs-module-tests--variable): New
helper type to guard access to state in a thread-safe way.
(emacs-module-tests--wait-for-variable)
(emacs-module-tests--change-variable): New helper functions.
(emacs-module-tests/interleaved-threads): New unit test.
As the constraint logic of the compiler is not trivial and largely
independent from the rest of the code move it into comp-cstr.el to
ease separation and maintainability.
This commit improve the conversion type
specifier -> constraint for generality.
Lastly this should help with bootstrap time as comp.el compilation
unit is slimmed down.
* lisp/emacs-lisp/comp-cstr.el: New file.
(comp--typeof-types, comp--all-builtin-types): Move from comp.el.
(comp-cstr, comp-cstr-f): Same + rename.
(comp-cstr-ctxt): New struct.
(comp-supertypes, comp-common-supertype-2)
(comp-common-supertype, comp-subtype-p, comp-union-typesets)
(comp-range-1+, comp-range-1-, comp-range-<, comp-range-union)
(comp-range-intersection): Move from comp.el.
(comp-cstr-union-no-range, comp-cstr-union): Move from comp.el and
rename.
(comp-cstr-union-make): New function.
(comp-type-spec-to-cstr, comp-cstr-to-type-spec): Move from
comp.el, rename it and rework it.
* lisp/emacs-lisp/comp.el (comp-known-func-cstr-h): Rework.
(comp-ctxt): Remove two fields and include `comp-cstr-ctxt'.
(comp-mvar, comp-fwprop-call): Update for `comp-cstr' being
renamed.
(comp-fwprop-insn): Use `comp-cstr-union-no-range' or
`comp-cstr-union'.
(comp-ret-type-spec): Use `comp-cstr-union' and rework.
* test/lisp/emacs-lisp/comp-cstr-tests.el: New file.
(comp-cstr-test-ts, comp-cstr-typespec-test): New functions.
(comp-cstr-typespec-tests-alist): New defconst to generate tests
on.
(comp-cstr-generate-tests): New macro.
* test/src/comp-tests.el (comp-tests-type-spec-tests): Update.
(ret-type-spec): Initialize constraint context.
* src/keymap.c (describe_vector): Say which command shadows this
binding. (Bug#9293)
* test/src/keymap-tests.el
(help--describe-vector/bug-9293-one-shadowed-in-range): Adapt
test.
* src/keymap.c (describe_vector): Do not say binding is shadowed if
the other key binding points to the same command. (Bug#9293)
* test/src/keymap-tests.el
(help--describe-vector/bug-9293-same-command-does-not-shadow): New
test.
* src/keymap.c (describe_vector): Make sure found consecutive keys
are either not shadowed or, if they are, that they are shadowed by
the same command. (Bug#9293)
* test/src/keymap-tests.el
(help--describe-vector/bug-9293-one-shadowed-in-range): New test.
Set the default value when `set` encounters a PER_BUFFER variable
which has been let-bound globally, to match the behavior seen with
`make-variable-buffer-local`.
* test/src/data-tests.el (binding-test--let-buffer-local):
Add corresponding test.
(data-tests--set-default-per-buffer): Add tentative test for the
performance problem encountered in bug#41029.
All-ASCII strings cannot have substrings with non-ASCII characters in
them; use this fact to avoid searching entirely.
* src/fns.c (Fstring_search): For multibyte non-ASCII needle and
unibyte haystack, don't check if the haystack is all-ASCII; it's a
waste of time. For multibyte non-ASCII needle and multibyte
all-ASCII haystack, fail immediately.
* test/src/fns-tests.el (string-search): Add more test cases.
Also add some tests for braces and parse-partial-sexp amongst Lisp style
comments.
* test/src/syntax-tests.el (\;-in, \;-out): Add syntax for { and }.
(top-level): Add new tests for Lisp style comments.
(\#|-in, \#|-out): New functions.
(top-level): Add new tests for nested Lisp style comments, and mixtures of
nested comments with "ordinary" comments.
* test/src/syntax-resources/syntax-comments.txt (top-level): Add new test
fragments for #|...|#, etc.
* lisp/emacs-lisp/comp.el (comp-mvar-symbol-p): Improve it.
(comp-cond-rw-func): Fix logic for multiple predecessor on target
block.
* test/src/comp-tests.el (comp-test-cond-rw-1): New test.
* test/src/comp-test-funcs.el (comp-test-cond-rw-1-1-f)
(comp-test-cond-rw-1-2-f): New functions.
* lisp/help.el (describe-map): Fix running `menu-item' :filter
functions. This fixes a mistake in the previous conversion of this
defun from the old C function describe_map. See the discussion in
Bug#39149.
* test/src/keymap-tests.el
(keymap---get-keyelt/runs-menu-item-filter)
(describe-buffer-bindings/menu-item-filter-show-binding)
(describe-buffer-bindings/menu-item-filter-hide-binding):
New tests.
(keymap-tests--test-menu-item-filter): New defun.
* lisp/emacs-lisp/comp.el (comp-constraint-to-type-spec): New
function splitting out code from comp-ret-type-spec + better
handle boolean type specifier.
(comp-ret-type-spec): Rework to leverage
`comp-constraint-to-type-spec'.
* test/src/comp-tests.el (comp-tests-type-spec-tests): Add a
testcase.
* lisp/emacs-lisp/comp.el (comp-known-type-specifiers): New const
in place of `comp-known-ret-types' and `comp-known-ret-ranges'.
(comp-constraint): New struct to separate the constraint side of
an mvar.
(comp-constraint-f): Analogous for functions.
(comp-mvar): Rework and include `comp-constraint'.
(comp-type-spec-to-constraint): New function.
(comp-known-constraints-h): New const.
(comp-func-ret-typeset, comp-func-ret-range): Rework.
(comp-fwprop-insn): Fix.
* test/src/comp-tests.el (destructure-type-spec): New testcase.
As we have derived return type specifiers as some test for them. Also
rewrite some propagation related test using return type specifiers too
as it's way more convenient.
* test/src/comp-tests.el (fw-prop-1): Nit rename.
(comp-tests-check-ret-type-spec): New function.
(comp-tests-type-spec-tests): New variable.
(comp-tests-cond-rw-0-var) Remove variable.
(cond-rw-0, cond-rw-1, cond-rw-2, cond-rw-3, cond-rw-4, cond-rw-5)
Remove tests as now covered by `comp-tests-check-ret-type-spec'.
Having this while re-debugging the boostrap would have saved few hours
of debug so let's add it.
* test/src/comp-tests.el (and-3): Add test.
* test/src/comp-test-funcs.el (comp-test-and-3-var): New var.
(comp-test-and-3-f): New function.
This commit add an initial support for a better type propagation and
integer range propagation.
Each mvar can be now characterized by a set of types, a set of values
and an integral range.
* lisp/emacs-lisp/comp.el (comp-known-ret-types): Store into
typeset and remove fixnum.
(comp-known-ret-ranges, comp-type-predicates): New variables.
(comp-ctxt): Remove supertype-memoize slot and add
union-typesets-mem.
(comp-mvar): Remove const-vld, constant, type slots. Add typeset,
valset, range slots.
(comp-mvar-value-vld-p, comp-mvar-value, comp-mvar-fixnum-p)
(comp-mvar-symbol-p, comp-mvar-cons-p)
(comp-mvar-type-hint-match-p, comp-func-ret-typeset)
(comp-func-ret-range): New functions.
(make-comp-mvar, make-comp-ssa-mvar): Update logic.
(comp--typeof-types): New variable.
(comp-supertypes, comp-common-supertype): Logic update.
(comp-subtype-p, comp-union-typesets, comp-range-1+)
(comp-range-1-, comp-range-<, comp-range-union)
(comp-range-intersection): New functions.
(comp-fwprop-prologue, comp-mvar-propagate)
(comp-function-foldable-p, comp-function-call-maybe-fold)
(comp-fwprop-insn, comp-call-optim-func, comp-finalize-relocs):
Logic update.
* src/comp.c (emit_mvar_rval, emit_call_with_type_hint)
(emit_call2_with_type_hint): Logic update.
* lisp/emacs-lisp/cl-preloaded.el (cl--typeof-types): Undo the add
of fixnum and bignum as unnecessary.
* test/src/comp-tests.el
(comp-tests-mentioned-p-1, comp-tests-cond-rw-checker-val)
(comp-tests-cond-rw-checker-type, cond-rw-1, cond-rw-2)
(cond-rw-3, cond-rw-4, cond-rw-5): Update for new type interface.
(range-simple-union, range-simple-intersection): New integer range
tests.
(union-types): New union type test.
* test/src/keymap-tests.el
(keymap-keymap-set-parent/returns-parent): Remove test for the return
value of set-keymap-parent. It is not clear that returning the value
is a very good idea.
Problem pointed out by Stefan Monnier <monnier@iro.umontreal.ca>.
The variable now only controls whether characters are printed, not
the radix. Control chars are printed in human-readable syntax
only when special escapes such as ?\n are available. Spaces,
formatting and combining chars are excluded (bug#44155).
Done in collaboration with Juri Linkov.
* src/character.c (graphic_base_p):
* src/print.c (named_escape): New functions.
(print_object): Change semantics as described above.
(syms_of_print): Rename integer-output-format. Update doc string.
* doc/lispref/streams.texi (Output Variables):
* etc/NEWS:
* test/src/print-tests.el (print-integers-as-characters):
Rename and update according to new semantics. The test now passes.
* doc/lispref/files.texi (Contents of Directories):
Fix description of directory-files, directory-empty-p and
directory-files-and-attributes.
* etc/NEWS: Fix entry for directory-files-and-attributes. Fix typos.
* lisp/dired.el (directory-empty-p): Move function from here ...
* lisp/files.el (directory-empty-p): ... to here.
* lisp/net/ange-ftp.el (ange-ftp-directory-files): Call `nreverse' later.
* lisp/net/tramp.el (tramp-handle-directory-files):
* lisp/net/tramp-adb.el
(tramp-adb-handle-directory-files-and-attributes): Do not call
`nreverse'.
* src/dired.c (Fdirectory_files)
(Fdirectory_files_and_attributes): Fix docstrings.
* test/src/dired-tests.el: Removed. Tests moved to
test/lisp/dired-tests.el.
* test/lisp/dired-tests.el (dired-test-bug27899): Tag it :unstable.
(dired-test-directory-files)
(dired-test-directory-files-and-attributes): New tests.
* lisp/help.el (describe-map-tree): Insert header into the original
buffer, not in standard-output.
* test/src/keymap-tests.el
(describe-buffer-bindings/header-in-current-buffer)
(describe-buffer-bindings/returns-nil): New tests.
Ref: https://debbugs.gnu.org/39149#31
* test/src/comp-tests.el (comp-tests-cond-rw-checked-function):
Declare var.
(comp-tests-cond-rw-checker-val): New function.
(comp-tests-cond-rw-checker-type): Declare var.
(comp-tests-cond-rw-checker-type): New function.
(comp-tests-cond-rw-0-var): Declare var.
(comp-tests-cond-rw-0, comp-tests-cond-rw-1, comp-tests-cond-rw-2)
(comp-tests-cond-rw-3, comp-tests-cond-rw-4)
(comp-tests-cond-rw-5): New testcases.
* test/src/comp-tests.el (comp-tests-map-checker): New function
returning a list holding checker results.
(comp-tests-tco-checker, comp-tests-fw-prop-checker-1)
(comp-tests-pure-checker-1, comp-tests-pure-checker-2): Make use
of `comp-tests-map-checker'.
* doc/lispref/streams.texi (Output Variables): Add integer-output-format.
* src/print.c (print_object): In case of Lisp_Int, print integers
as characters when Vinteger_output_format is Qt, and in hex format
when Vinteger_output_format is 16.
(Vinteger_output_format): New variable.
* test/src/print-tests.el (print-integer-output-format): New test.
Fix bug#44070, which causes the minibuffer display to jump upon minor edit
(redisplay_window): Obey it.
* lisp/simple.el (end-of-buffer): Obey it.
* test/src/xdisp-tests.el (xdisp-tests--in-minibuffer): New macro,
extracted from `xdisp-tests--minibuffer-resizing`.
(xdisp-tests--minibuffer-resizing): Use it.
(xdisp-tests--minibuffer-scroll): New test.
* lisp/emacs-lisp/comp.el (comp-emit-set-call-subr): Rework based
on the fact that the subr can now be redefined.
* test/src/comp-tests.el (primitive-redefine-compile-44221):
New testcase.
This is only the first step towards a full Lisp implementation, and
does not remove the old C code. On the contrary, it is partly based
on using the old C code, which is to be replaced in steps. This also
makes it easy to test that it produces the same output as the old.
* src/doc.c (Fsubstitute_command_keys_old): Rename from
Fsubstitute_command_keys.
(Fget_quoting_style): New defun to expose text_quoting_style to Lisp.
(syms_of_doc): Expose above symbols.
* lisp/help.el (substitute-command-keys): New Lisp version of
substitute-command-keys. (Bug#8951)
* src/keymap.c
(Fdescribe_map_tree): New defun to expose describe_map_tree to Lisp.
(syms_of_keymap): New defsubr for Fdescribe_map_tree.
* src/keyboard.c (help_echo_substitute_command_keys):
* src/doc.c (Fdocumentation, Fdocumentation_property):
* src/print.c (print_error_message):
* src/syntax.c (Finternal_describe_syntax_value): Fix calls to use new
Lisp implementation of substitute-command-keys.
* test/src/doc-tests.el: Remove file.
* test/lisp/help-tests.el: Add tests for substitute-command-keys
copied from above file.
* lisp/emacs-lisp/comp.el (native-compile): Return the compiled
function when the input is a symbol or a form.
* test/src/comp-tests.el (free-fun, tco, fw-prop): Update tests
for new `native-compile' interface.
