Current implementation of treesit--things-around only searches forward
for REGEXP and go up the tree until it finds a valid thing, if nothing
matches it gives up. This makes it sometimes miss defuns. The new
implementation tries multiple times (of search forward + go up) until
it exhausts all possible defun nodes.
* lisp/treesit.el (treesit--things-around): New implementation.
(treesit--navigate-defun): Refactor to use treesit-node-top-level to
simplify code, and add some guards in the predicate function.
* test/src/treesit-tests.el:
(treesit--ert-defun-navigation-elixir-program): New variable.
(treesit-defun-navigation-nested-4): New test.
I realized that using an arbitrary function as the predicate in
queries is very helpful for some queries I'm writing for python and
javascript, and presumably most other languages[1].
Granted, we can already filter out unwanted nodes by using a function
instead of a face for the capture name, and (1) determine whether the
captured node is valid and (2) fontify that node if it's valid.
However, such approach is a bit more cumbersome and more importantly
gets in the way of another potential use of the fontification queries:
context extraction.
For example, I could use the query for the 'variable' feature to get
all the variables in a certain region. In this use-case, we want the
filtering happen before returning the captured nodes.
Besides, the change is relatively small and straightforward: most code
are already there, I just need to add some boilerplate.
[1] For a code like aa.bb(cc), we want bb to be in function face,
because obviously its a function. But for aa.bb, we want bb to be in
property face, because it's a property. In the AST, bb is always a
property, the difference between the two cases is the enclosing node:
in the first case, aa.bb is in a "call_expression" node, indicating
that bb is used as a function (a method). So we want a predicate
function that checks whether bb is used as a function or a property,
and determine whether it should be in function or property face.
* doc/lispref/parsing.texi (Pattern Matching): Update manual.
* src/treesit.c (Ftreesit_pattern_expand): Handle :pred.
(treesit_predicate_capture_name_to_node): A new function extracted
from treesit_predicate_capture_name_to_text.
(treesit_predicate_capture_name_to_text): Use the newly extracted
function.
(treesit_predicate_pred): New predicate function.
(treesit_eval_predicates): Add new predicate. Also fix a bug: we want
to AND the results of each predicate.
* test/src/treesit-tests.el (treesit--ert-pred-last-sibling): New
helper function.
(treesit-query-api): Test #pred predicate.
ts_node_parent has bugs (bug#60054), using cursor API avoids that.
Tree-sitter's author might remove ts_node_parent in the future, so
might as well switch to use cursors now. We are basically
reimplementing some of the logic of ts_node_prev_sibling and
ts_node_parent in the sibling helper and cursor helper functions.
See also https://github.com/tree-sitter/tree-sitter/issues/1992
* src/treesit.c (treesit_traverse_sibling_helper)
(treesit_traverse_child_helper)
(treesit_traverse_match_predicate): Reimplemented to use the cursor API.
(treesit_search_dfs)
(treesit_search_forward): Use the new cursor helper functions.
(Ftreesit_search_subtree)
(Ftreesit_search_forward)
(Ftreesit_induce_sparse_tree): Use cursors.
* test/src/treesit-tests.el (treesit-search-subtree): New test.
(treesit--ert-search-setup): New macro.
(treesit-search-forward)
(treesit-search-forward-named-only)
(treesit-search-backward)
(treesit-search-backward-named-only)
(treesit-cursor-helper-with-missing-node): New tests.
This new set of functions (and tests) should eliminate
defun-navigation bugs and limitations we currently have. This commit
doesn't change any existing bahavior: treesit-beginning/end-of-defun
and friends are unchanged. The plan is to later switch gear and
replace the current functions with the new ones introduced in this
change.
This is a relatively big change, but I've setup a comprehensive test,
and it should fix current bugs, so I think it's ok to put it on the
release branch.
The gist of the new navigation is to use treesit--defuns-around to
find the previous sibling defun, next sibling defun, and the parent
defun, then use this information to move to previous/next
beginning/end of defun in treesit--navigate-defun.
I also added comprehensive testing that tests all four possible
operations (prev-beg, next-beg, prev-end, next-end) starting at all
possible positions (between two sibling defuns, inside a sibling
defun, etc).
* lisp/treesit.el (treesit-defun-type-regexp): Expand definition to
allow (REGEXP . FILTER). Old functions don't support this, but it
should be fine since we are soon replacing them.
(treesit-defun-tactic)
(treesit-defun-skipper): New variables.
(treesit-default-defun-skipper)
(treesit--defuns-around)
(treesit--top-level-defun)
(treesit--navigate-defun): New functions.
* test/src/treesit-tests.el (treesit--ert-insert-and-parse-marker)
(treesit--ert-collect-positions)
(treesit--ert-test-defun-navigation): New helper functions.
(treesit--ert-defun-navigation-python-program)
(treesit--ert-defun-navigation-js-program)
(treesit--ert-defun-navigation-bash-program)
(treesit--ert-defun-navigation-nested-master): New variables.
(treesit-defun-navigation-nested-1)
(treesit-defun-navigation-nested-2)
(treesit-defun-navigation-nested-3)
(treesit-defun-navigation-top-level): New tests.
Fix the problem described in bug#59693.
* src/treesit.c (treesit_record_change): Always use the base buffer.
(Ftreesit_parser_create): Always use the base buffer. Also change the
for loop into FOR_EACH_TAIL (stylistic change).
(Ftreesit_parser_list): Always use the base buffer.
* doc/lispref/parsing.texi (Using Parser): Update manual.
* test/src/treesit-tests.el (treesit-indirect-buffer): New test.
* src/treesit.c (treesit_check_position): Extract out new function.
(Ftreesit_node_first_child_for_pos)
(Ftreesit_node_descendant_for_range): Replace code with the new
function.
(Ftreesit_query_capture): Add missing check for node and parser. Add
check for range for BEG and END. Move treesit_initialize to the
beginning of the function.
* test/src/treesit-tests.el (treesit-node-api)
(treesit-query-api): Add tests for out-of-range error.
* test/src/buffer-tests.el (overlay-modification-hooks): Remove noisy
`message' calls and use `ert-info' to log context of test
failures. (bug#59028)
(overlay-tests-start-recording-modification-hooks): ditto.
* doc/lispref/parsing.texi (Retrieving Node): Update manual.
* lisp/treesit.el (treesit-node-at): Change semantic. It tries to
return the node that a user would expect in various circumstances.
* test/src/treesit-tests.el (treesit-node-at): New test.
These tests had been annoying to me when run in interactive mode.
They failed to bind `kill-buffer-delete-auto-save' as needed, so they
depended on the user's settings, and they failed to mock out the
"Buffer modified, save?" prompt.
* test/src/buffer-tests.el (test-kill-buffer-auto-save): New helper
function that mocks the two different kinds of prompts that occur when
killing modified buffers that visit a file. Tests fail if expected
prompts are not issued.
(test-kill-buffer-auto-save-default): Use it, and explicitly bind
`kill-buffer-delete-auto-save' to nil.
(test-kill-buffer-auto-save-delete): Delete it.
(test-kill-buffer-auto-save-delete-yes): New test for the "yes" half
of the old `test-kill-buffer-auto-save-delete'.
(test-kill-buffer-auto-save-delete-yes): Ditto for the "no" half.
This fixes bug#58928. The patch works by moving the
`evaporate_overlays` check closer to `itree_delete_gap`.
* src/buffer.c (adjust_overlays_for_delete_in_buffer): New function,
integrating the functionality of `evaporate_overlays`.
(adjust_overlays_for_delete): Use it.
(evaporate_overlays): Delete function.
* src/buffer.h (evaporate_overlays): Delete declaration.
* src/insdel.c (adjust_markers_for_replace): Minor optimization.
(adjust_after_replace, replace_range, replace_range_2, del_range_2):
Don't call `evaporate_overlays`.
* test/src/buffer-tests.el (buffer-tests--overlays-indirect-evaporate):
Remove the `:expected-result :failed` attribute.
* test/src/buffer-tests.el (buffer-tests--overlays-indirect-evaporate):
Test evaporation of overlays triggered by deleting text in base
and in indirect buffers. Test doesn't pass at the moment.
* test/src/buffer-tests.el (test-overlay-insert-before-markers-empty):
Move code down to the other tests related to insertion. Test all
front/rear insert combinations. To make the test more clear, at least
to me, hard code all character positions.
(test-overlay-insert-before-markers-at-start): For both front-advance
modes verify that `insert-before-markers' at and overlay's start
advances it.
(test-overlay-insert-before-markers-at-end): For both rear-advance
modes test that `insert-before-markers' at an overlay's end advances
it.
(test-overlay-insert-before-markers-non-empty): Delete, replaced by
the two tests above.
* doc/lispref/parsing.texi (Multiple Languages): Extend and update
manual.
* lisp/treesit.el (treesit-range-functions): Remove variable.
(treesit-range-settings): New variable.
(treesit-range-rules): New function.
(treesit--merge-ranges): New function.
(treesit-update-ranges): Use treesit-range-settings instead of
treesit-range-functions.
(treesit-font-lock-rules): Fix docstring.
(treesit-indent)
(treesit-indent-region): Only update ranges in a region.
* test/src/treesit-tests.el (treesit-range): New test.
The most obvious problem was the lack of support for
`insert-before-markers`, but the behavior was also different in a few
other cases.
* src/itree.h (itree_insert_gap):
* src/itree.c (itree_insert_gap): Add `before_markers` arg.
* src/lisp.h (adjust_overlays_for_insert):
* src/buffer.c (adjust_overlays_for_insert): Add `before_markers` arg.
* src/insdel.c (adjust_markers_for_replace, adjust_markers_for_insert)
(adjust_markers_for_delete): Adjust overlays directly from here.
(insert_1_both, insert_from_string_1, insert_from_gap)
(insert_from_buffer_1, adjust_after_replace, replace_range)
(replace_range_2, del_range_2): Don't adjust overlays explicitly here
any more.
* test/src/buffer-tests.el (test-overlay-insert-before-markers-empty)
(test-overlay-insert-before-markers-non-empty): New tests.
* src/buffer.c (adjust_overlays_for_insert)
(adjust_overlays_for_delete): Repeat for all buffers sharing the same text.
* src/itree.c (itree_insert_gap, itree_delete_gap): Allow an empty tree.
* test/src/buffer-tests.el (buffer-tests--overlays-indirect-bug58928):
New test.
Because now it just calls a user-supplied function rather than doing
any work by itself.
* test/src/treesit-tests.el (treesit-parser-supplemental): Remove the
test.
This version: central variable, everything controlled by
treesit-settings. Major mode sets up tree-sitter/non-tree-sitter
in a conditional branch, based on the setting.
* lisp/treesit.el (treesit-settings): New option.
(treesit-defun-type-regexp): Change docstring.
(treesit-mode-supported)
(treesit-required-languages)
(treesit--local-variable-backup): Remove variables.
(treesit--backup-local-variable)
(treesit-mode)
(global-treesit-mode--turn-on)
(global-treesit-mode): Remove functions.
(treesit--setting-for-mode): New function.
(treesit-ready-p): New argument MODE, changed REPORT to QUIET, and
LANGUAGEs to LANGUAGE (now it can be a single symbol or a list of
them).
(treesit-major-mode-setup): New function. Mostly comes from
treesit-mode.
* test/src/treesit-tests.el (treesit-misc): New test.
* lisp/progmodes/python.el (python-mode): Move some setup code into
the conditional branch at the end.
* lisp/progmodes/js.el (js-json-mode)
(js-mode): Move some setup code into the conditional branch at the
end.
* lisp/progmodes/ts-mode.el: Move tree-sitter setup into the
conditional branch.
This bug occurs when an atomic pattern is matched at the end of
a string and the on-failure-keep-string-jump optimisation is
in effect, as in:
(string-match "\\'\\(?:ab\\)*\\'" "a")
which succeeded but clearly should not (bug#58726).
Reported by Michael Heerdegen.
* src/regex-emacs.c (PREFETCH): Add reset parameter.
(re_match_2_internal): Use it for proper atomic pattern treatment.
* test/src/regex-emacs-tests.el (regexp-atomic-failure): New test.
* doc/lispref/lists.texi (Plist Access): Improve description of
default predicate.
* lisp/emacs-lisp/cl-extra.el (cl-getf, cl--set-getf): Assume
plist-member always returns a cons.
* lisp/emacs-lisp/gv.el (plist-get): Support new optional predicate
argument (bug#47425#91).
* lisp/emacs-lisp/map.el: Bump minor version.
(map--dispatch): Remove now that bug#58563 is fixed. Break two
remaining uses out into corresponding cl-defmethods.
(map--plist-p): Add docstring.
(map--plist-has-predicate, map--plist-member-1, map--plist-member)
(map--plist-put-1, map--plist-put): New definitions for supporting
predicate argument backward compatibly.
(map-elt): Fix generalized variable getter under a
predicate (bug#58531). Use predicate when given a plist.
(map-put): Avoid gratuitous warnings when called without the hidden
predicate argument. Improve obsoletion message.
(map-put!): Use predicate when given a plist.
(map-contains-key): Ditto. Declare forgotten
advertised-calling-convention (bug#58531#19).
(map--put): Group definition in file together with that of map-put!.
* lisp/files-x.el (connection-local-normalize-criteria): Simplify
using mapcan + plist-get.
* lisp/net/eudc.el (eudc--plist-member): New convenience function.
(eudc-plist-member, eudc-plist-get, eudc-lax-plist-get): Use it
instead of open-coding plist-member.
* src/fns.c (Fplist_get, plist_get, Fplist_put, plist_put): Pass the
plist element as the first argument to the predicate, for
consistency with assoc + alist-get.
(Fplist_member, plist_member): Move from widget to plist section.
Open-code the EQ case in plist_member, and call it from
Fplist_member in that case, rather than the other way around.
* test/lisp/apropos-tests.el (apropos-tests-format-plist): Avoid
polluting obarray.
* test/lisp/emacs-lisp/cl-extra-tests.el (cl-getf): Extend test with
generalized variables, degenerate plists, and improper lists.
* test/lisp/emacs-lisp/gv-tests.el: Byte-compile file; in the
meantime bug#24402 seems to have been fixed or worked around.
(gv-setter-edebug): Inhibit printing messages.
(gv-plist-get): Avoid modifying constant literals. Also test with a
predicate argument.
* test/lisp/emacs-lisp/map-tests.el (with-maps-do): Simplify
docstring.
(test-map-elt-testfn): Rename...
(test-map-elt-testfn-alist): ...to this. Also test with a predicate
argument.
(test-map-elt-testfn-plist, test-map-elt-gv, test-map-elt-signature)
(test-map-put!-plist, test-map-put!-signature)
(test-map-contains-key-signature, test-map-plist-member)
(test-map-plist-put): New tests.
(test-map-contains-key-testfn): Also test with a predicate argument.
(test-map-setf-alist-overwrite-key, test-map-setf-plist-insert-key)
(test-map-setf-plist-overwrite-key): Avoid modifying constant
literals.
(test-hash-table-setf-insert-key)
(test-hash-table-setf-overwrite-key): Fix indentation.
(test-setf-map-with-function): Make test more precise.
* test/lisp/net/eudc-tests.el: New file.
* test/lisp/subr-tests.el (test-plistp): Extend test with circular
list.
* test/src/fns-tests.el (test-cycle-equal, test-cycle-nconc): Move
from plist section to circular list section.
(plist-put/odd-number-of-elements): Avoid modifying constant
literals.
(plist-member/improper-list): Simplify.
(test-plist): Move to plist section. Also test with a predicate
argument.
See bug#58703.
* src/buffer.c (overlays_in): Add a new TRAILING arg expressing the
behavior wanted by `overlays-at', namely to include all overlays
beginning at the POS passed to `overlays-at', even if POS is the end
of the narrowed region. Pass true and the search range is extended to
ZV+1 if END is greater than ZV, just as is done for EMPTY.
(overlays_at): Pass 'true' for the new trailing arg. At present this
is the only caller passing 'true'.
(mouse_face_overlay_overlaps): Pass 'false' for the new trailing arg.
(disable_line_numbers_overlay_at_eob): ditto.
(Foverlays_in): ditto.
* src/editfns.c (overlays_around): ditto.
* test/src/buffer-tests.el (sorted-overlays): Add a spot test for
this.
When passed EMPTY, 'overlays_in' should return empty overlays at END.
It was doing so, but it was also returning any other overlay that
happened to begin at END.
bug#58672
* src/buffer.c (overlays_in): Don't return overlays at END unless they
are empty overlays.
(disable_line_numbers_overlay_at_eob): Pass 'false' instead of 'NULL'
for the bool 'empty' arg.
* test/src/buffer-tests.el (sorted-overlays-in): New helper function.
(test-overlays-in-empty-range): New test exhaustively covering these
edge conditions.
(test-overlays-in-empty-range-bug58672): Simple test for one case.
Correct `get-char-property` which failed to ignore empty overlays.
* src/textprop.c (get_char_property_and_overlay): Make sure the
overlay covers the character.
* test/src/buffer-tests.el (buffer-tests--overlay-bug58479): New test.
* src/buffer.h (buffer_has_overlays): Avoid `interval_tree_size`.
