* test/src/comp-tests.el (61917-1): New test.
* src/comp.c (syms_of_comp): New variable.
* lisp/loadup.el: Store primitive arities before dumping.
* lisp/emacs-lisp/comp.el (comp--func-arity): New function.
(comp-emit-set-call-subr): Make use of `comp--func-arity'.
* test/src/fns-tests.el (fns-tests-collate-strings): Use
en_XY.UTF-8 instead of en_DE.UTF-8 as an example of an invalid
locale; en_DE.UTF-8 is actually a valid locale on Cygwin.
Note: The test will still fail on Cygwin releases < 3.5
because of a Cygwin bug. (Bug#62420)
* lisp/emacs-lisp/comp.el (comp-emit-set-call-subr): Fix compilation
of calls to redefined primtives with dedicated op-bytecode.
* test/src/comp-tests.el (61917-1): New test.
Before this change, when you use a tree-sitter navigation function to
move to the next beginning of a thing, it jumps over the immediate
next thing and lands you at the beginning of the next-next thing.
Eg, when point is at the "|", and we evaluate
(treesit--navigate-thing pos 1 'beg), we go from
| (thing) (thing)
to
(thing) |(thing)
But some might expect point to go to
|(thing) (thing)
instead, which makes sense. Also, that's how Emacs expect defun
navigation functions to work. The discrepancy in expectation causes
bug#61617.
In this change I made tree-sitter navigation functions to work as what
Emacs expects. And what I described for moving to the next beginning
of thing is similarly applicable to moving to the end of previous end
of thing.
* lisp/treesit.el (treesit-beginning-of-defun)
(treesit-end-of-defun): Handle the case where defun-skipper moves
point back to where we started, by adding a retry.
(treesit--navigate-thing): Add a single condition checking for
progress to the condition form responsible for checking whether to
skip the next defun. Namely (eq pos (funcall advance next)))).
* test/src/treesit-tests.el:
(treesit--ert-defun-navigation-nested-master)
(treesit--ert-defun-navigation-top-level-master): Change tests to
reflect the new expectation.
bug#61514 exhibited some undesirable backtracking in a case where
it's easy to avoid it by making `mutually_exclusive_p` just a bit
more careful.
* src/regex-emacs.c (mutually_exclusive_p): Handle `on_failure_jump`s.
* test/src/regex-emacs-tests.el (regexp-tests-backtrack-optimization):
Add a few tests.
It still doesn't work for an AFTER that's a key, though, since
`key-parse' produces vectors, and keymaps contain integers.
* lisp/keymap.el (keymap-set-after): Only parse AFTER as a key if it's
a string. For consistency, use `key-parse' on the definition if it's
a string, just like `keymap-set'.
* test/src/keymap-tests.el (keymap-tests--command-3): New dummy command.
(keymap-set-after-menus): New test. Check that we can insert a menu
item after a specific entry.
It's stricter than `kbd', and doesn't try to do anything with key
sequences that look like macros.
* lisp/keymap.el (keymap-lookup): Use `key-parse' instead of `kbd'.
* test/src/keymap-tests.el (keymap-set-after-menus): Test the
`keymap-set-after' API.
Fmake_indirect_buffer can be told whether to run buffer hooks since
bug#49160, but until now it ran buffer-list-update-hook irrespective
of this.
* src/buffer.c (Fmake_indirect_buffer): Don't run
buffer-list-update-hook when called with a non-nil
INHIBIT-BUFFER-HOOKS argument.
(run_buffer_list_update_hook): Don't special-case NULL argument, as
no such callers remain.
* test/src/buffer-tests.el
(buffer-tests-inhibit-buffer-hooks-indirect): Test whether indirect
buffer hooks are run regardless of whether base buffer hooks are
inhibited. Check that all three buffer hooks, not just
kill-buffer-query-functions, are inhibited.
* src/coding.c (encode_coding_utf_8): Fix encoding with
'utf-8-auto': it should produce BOM, per the documentation of
the :bom attribute. (Bug#60750)
* lisp/international/mule.el (define-coding-system): Doc fix.
* test/src/coding-tests.el (coding-tests): Use
'with-coding-priority' instead of 'prefer-coding-system', as the
latter has global persistent effect and affects further tests.
Current implementation of treesit--things-around only searches forward
for REGEXP and go up the tree until it finds a valid thing, if nothing
matches it gives up. This makes it sometimes miss defuns. The new
implementation tries multiple times (of search forward + go up) until
it exhausts all possible defun nodes.
* lisp/treesit.el (treesit--things-around): New implementation.
(treesit--navigate-defun): Refactor to use treesit-node-top-level to
simplify code, and add some guards in the predicate function.
* test/src/treesit-tests.el:
(treesit--ert-defun-navigation-elixir-program): New variable.
(treesit-defun-navigation-nested-4): New test.
I realized that using an arbitrary function as the predicate in
queries is very helpful for some queries I'm writing for python and
javascript, and presumably most other languages[1].
Granted, we can already filter out unwanted nodes by using a function
instead of a face for the capture name, and (1) determine whether the
captured node is valid and (2) fontify that node if it's valid.
However, such approach is a bit more cumbersome and more importantly
gets in the way of another potential use of the fontification queries:
context extraction.
For example, I could use the query for the 'variable' feature to get
all the variables in a certain region. In this use-case, we want the
filtering happen before returning the captured nodes.
Besides, the change is relatively small and straightforward: most code
are already there, I just need to add some boilerplate.
[1] For a code like aa.bb(cc), we want bb to be in function face,
because obviously its a function. But for aa.bb, we want bb to be in
property face, because it's a property. In the AST, bb is always a
property, the difference between the two cases is the enclosing node:
in the first case, aa.bb is in a "call_expression" node, indicating
that bb is used as a function (a method). So we want a predicate
function that checks whether bb is used as a function or a property,
and determine whether it should be in function or property face.
* doc/lispref/parsing.texi (Pattern Matching): Update manual.
* src/treesit.c (Ftreesit_pattern_expand): Handle :pred.
(treesit_predicate_capture_name_to_node): A new function extracted
from treesit_predicate_capture_name_to_text.
(treesit_predicate_capture_name_to_text): Use the newly extracted
function.
(treesit_predicate_pred): New predicate function.
(treesit_eval_predicates): Add new predicate. Also fix a bug: we want
to AND the results of each predicate.
* test/src/treesit-tests.el (treesit--ert-pred-last-sibling): New
helper function.
(treesit-query-api): Test #pred predicate.
ts_node_parent has bugs (bug#60054), using cursor API avoids that.
Tree-sitter's author might remove ts_node_parent in the future, so
might as well switch to use cursors now. We are basically
reimplementing some of the logic of ts_node_prev_sibling and
ts_node_parent in the sibling helper and cursor helper functions.
See also https://github.com/tree-sitter/tree-sitter/issues/1992
* src/treesit.c (treesit_traverse_sibling_helper)
(treesit_traverse_child_helper)
(treesit_traverse_match_predicate): Reimplemented to use the cursor API.
(treesit_search_dfs)
(treesit_search_forward): Use the new cursor helper functions.
(Ftreesit_search_subtree)
(Ftreesit_search_forward)
(Ftreesit_induce_sparse_tree): Use cursors.
* test/src/treesit-tests.el (treesit-search-subtree): New test.
(treesit--ert-search-setup): New macro.
(treesit-search-forward)
(treesit-search-forward-named-only)
(treesit-search-backward)
(treesit-search-backward-named-only)
(treesit-cursor-helper-with-missing-node): New tests.
This new set of functions (and tests) should eliminate
defun-navigation bugs and limitations we currently have. This commit
doesn't change any existing bahavior: treesit-beginning/end-of-defun
and friends are unchanged. The plan is to later switch gear and
replace the current functions with the new ones introduced in this
change.
This is a relatively big change, but I've setup a comprehensive test,
and it should fix current bugs, so I think it's ok to put it on the
release branch.
The gist of the new navigation is to use treesit--defuns-around to
find the previous sibling defun, next sibling defun, and the parent
defun, then use this information to move to previous/next
beginning/end of defun in treesit--navigate-defun.
I also added comprehensive testing that tests all four possible
operations (prev-beg, next-beg, prev-end, next-end) starting at all
possible positions (between two sibling defuns, inside a sibling
defun, etc).
* lisp/treesit.el (treesit-defun-type-regexp): Expand definition to
allow (REGEXP . FILTER). Old functions don't support this, but it
should be fine since we are soon replacing them.
(treesit-defun-tactic)
(treesit-defun-skipper): New variables.
(treesit-default-defun-skipper)
(treesit--defuns-around)
(treesit--top-level-defun)
(treesit--navigate-defun): New functions.
* test/src/treesit-tests.el (treesit--ert-insert-and-parse-marker)
(treesit--ert-collect-positions)
(treesit--ert-test-defun-navigation): New helper functions.
(treesit--ert-defun-navigation-python-program)
(treesit--ert-defun-navigation-js-program)
(treesit--ert-defun-navigation-bash-program)
(treesit--ert-defun-navigation-nested-master): New variables.
(treesit-defun-navigation-nested-1)
(treesit-defun-navigation-nested-2)
(treesit-defun-navigation-nested-3)
(treesit-defun-navigation-top-level): New tests.
Fix the problem described in bug#59693.
* src/treesit.c (treesit_record_change): Always use the base buffer.
(Ftreesit_parser_create): Always use the base buffer. Also change the
for loop into FOR_EACH_TAIL (stylistic change).
(Ftreesit_parser_list): Always use the base buffer.
* doc/lispref/parsing.texi (Using Parser): Update manual.
* test/src/treesit-tests.el (treesit-indirect-buffer): New test.
* src/treesit.c (treesit_check_position): Extract out new function.
(Ftreesit_node_first_child_for_pos)
(Ftreesit_node_descendant_for_range): Replace code with the new
function.
(Ftreesit_query_capture): Add missing check for node and parser. Add
check for range for BEG and END. Move treesit_initialize to the
beginning of the function.
* test/src/treesit-tests.el (treesit-node-api)
(treesit-query-api): Add tests for out-of-range error.
* test/src/buffer-tests.el (overlay-modification-hooks): Remove noisy
`message' calls and use `ert-info' to log context of test
failures. (bug#59028)
(overlay-tests-start-recording-modification-hooks): ditto.
* doc/lispref/parsing.texi (Retrieving Node): Update manual.
* lisp/treesit.el (treesit-node-at): Change semantic. It tries to
return the node that a user would expect in various circumstances.
* test/src/treesit-tests.el (treesit-node-at): New test.
These tests had been annoying to me when run in interactive mode.
They failed to bind `kill-buffer-delete-auto-save' as needed, so they
depended on the user's settings, and they failed to mock out the
"Buffer modified, save?" prompt.
* test/src/buffer-tests.el (test-kill-buffer-auto-save): New helper
function that mocks the two different kinds of prompts that occur when
killing modified buffers that visit a file. Tests fail if expected
prompts are not issued.
(test-kill-buffer-auto-save-default): Use it, and explicitly bind
`kill-buffer-delete-auto-save' to nil.
(test-kill-buffer-auto-save-delete): Delete it.
(test-kill-buffer-auto-save-delete-yes): New test for the "yes" half
of the old `test-kill-buffer-auto-save-delete'.
(test-kill-buffer-auto-save-delete-yes): Ditto for the "no" half.
This fixes bug#58928. The patch works by moving the
`evaporate_overlays` check closer to `itree_delete_gap`.
* src/buffer.c (adjust_overlays_for_delete_in_buffer): New function,
integrating the functionality of `evaporate_overlays`.
(adjust_overlays_for_delete): Use it.
(evaporate_overlays): Delete function.
* src/buffer.h (evaporate_overlays): Delete declaration.
* src/insdel.c (adjust_markers_for_replace): Minor optimization.
(adjust_after_replace, replace_range, replace_range_2, del_range_2):
Don't call `evaporate_overlays`.
* test/src/buffer-tests.el (buffer-tests--overlays-indirect-evaporate):
Remove the `:expected-result :failed` attribute.
* test/src/buffer-tests.el (buffer-tests--overlays-indirect-evaporate):
Test evaporation of overlays triggered by deleting text in base
and in indirect buffers. Test doesn't pass at the moment.
* test/src/buffer-tests.el (test-overlay-insert-before-markers-empty):
Move code down to the other tests related to insertion. Test all
front/rear insert combinations. To make the test more clear, at least
to me, hard code all character positions.
(test-overlay-insert-before-markers-at-start): For both front-advance
modes verify that `insert-before-markers' at and overlay's start
advances it.
(test-overlay-insert-before-markers-at-end): For both rear-advance
modes test that `insert-before-markers' at an overlay's end advances
it.
(test-overlay-insert-before-markers-non-empty): Delete, replaced by
the two tests above.
* doc/lispref/parsing.texi (Multiple Languages): Extend and update
manual.
* lisp/treesit.el (treesit-range-functions): Remove variable.
(treesit-range-settings): New variable.
(treesit-range-rules): New function.
(treesit--merge-ranges): New function.
(treesit-update-ranges): Use treesit-range-settings instead of
treesit-range-functions.
(treesit-font-lock-rules): Fix docstring.
(treesit-indent)
(treesit-indent-region): Only update ranges in a region.
* test/src/treesit-tests.el (treesit-range): New test.
The most obvious problem was the lack of support for
`insert-before-markers`, but the behavior was also different in a few
other cases.
* src/itree.h (itree_insert_gap):
* src/itree.c (itree_insert_gap): Add `before_markers` arg.
* src/lisp.h (adjust_overlays_for_insert):
* src/buffer.c (adjust_overlays_for_insert): Add `before_markers` arg.
* src/insdel.c (adjust_markers_for_replace, adjust_markers_for_insert)
(adjust_markers_for_delete): Adjust overlays directly from here.
(insert_1_both, insert_from_string_1, insert_from_gap)
(insert_from_buffer_1, adjust_after_replace, replace_range)
(replace_range_2, del_range_2): Don't adjust overlays explicitly here
any more.
* test/src/buffer-tests.el (test-overlay-insert-before-markers-empty)
(test-overlay-insert-before-markers-non-empty): New tests.
