(treesit-sexp-type-down-list, treesit-sexp-type-up-list):
New buffer-local variables.
(treesit-down-list): Use 'treesit-sexp-type-down-list'.
(treesit-up-list): Use 'treesit-sexp-type-up-list'.
* lisp/progmodes/elixir-ts-mode.el (elixir-ts-mode):
Set 'treesit-sexp-type-down-list' and
'treesit-sexp-type-up-list' to 'list'.
* lisp/treesit.el (treesit-ready-p): More accurate wording of
message when grammar library fails to load.
* src/treesit.c (treesit_load_language): Improve message when
reporting library ABI version mismatch. Suggested by Soham
Gumaste <sohamg2@gmail.com>.
(treesit-parsers-at): Use it (bug#77906).
(treesit-forward-sexp, treesit--forward-list-with-default)
(treesit-up-list): Let-bind 'treesit--parser-overlay-offset'.
* lisp/treesit.el (treesit--simple-indent-eval): Allow EXP to be
anything, so higher-order indent presets can take anything as an
argument: t, nil, symbols, keywords, etc.
Add line-column tracking for tree-sitter parsers. Copied from
comments in treesit.c:
Technically we had to send tree-sitter the line and column
position of each edit. But in practice we just send it dummy
values, because tree-sitter doesn't use it for parsing and
mostly just carries the line and column positions around and
return it when e.g. reporting node positions[1]. This has
been working fine until we encountered grammars that actually
utilizes the line and column information for
parsing (Haskell)[2].
[1] https://github.com/tree-sitter/tree-sitter/issues/445
[2] https://github.com/tree-sitter/tree-sitter/issues/4001
So now we have to keep track of line and column positions and
pass valid values to tree-sitter. (It adds quite some
complexity, but only linearly; one can ignore all the linecol
stuff when trying to understand treesit code and then come
back to it later.) Eli convinced me to disable tracking by
default, and only enable it for languages that needs it. So
the buffer starts out not tracking linecol. And when a
parser is created, if the language is in
treesit-languages-require-line-column-tracking, we enable
tracking in the buffer, and enable tracking for the parser.
To simplify things, once a buffer starts tracking linecol, it
never disables tracking, even if parsers that need tracking
are all deleted; and for parsers, tracking is determined at
creation time, if it starts out tracking/non-tracking, it
stays that way, regardless of later changes to
treesit-languages-require-line-column-tracking.
To make calculating line/column positons fast, we store
linecol caches for begv, point, and zv in the
buffer (buf->ts_linecol_cache_xxx); and in the parser object,
we store linecol cache for visible beg/end of that parser.
In buffer editing functions, we need the linecol for
start/old_end/new_end, those can be calculated by scanning
newlines (treesit_linecol_of_pos) from the buffer point
cache, which should be always near the point. And we usually
set the calculated linecol of new_end back to the buffer
point cache.
We also need to calculate linecol for the visible_beg/end for
each parser, and linecol for the buffer's begv/zv, these
positions are usually far from point, so we have caches for
all of them (in either the parser object or the buffer).
These positions are far from point, so it's inefficient to
scan newlines from point to there to get up-to-date linecol
for them; but in the same time, because they're far and
outside the changed region, we can calculate their change in
line and column number by simply counting how much newlines
are added/removed in the changed
region (compute_new_linecol_by_change).
* doc/lispref/parsing.texi (Using Parser): Mention line-column
tracking in manual.
* etc/NEWS: Add news.
* lisp/treesit.el:
(treesit-languages-need-line-column-tracking): New variable.
* src/buffer.c: Include treesit.h (for TREESIT_EMPTY_LINECOL).
(Fget_buffer_create):
(Fmake_indirect_buffer): Initialize new buffer fields.
(Fbuffer_swap_text): Add new buffer fields.
* src/buffer.h (ts_linecol): New struct.
(buffer): New buffer fields.
(BUF_TS_LINECOL_BEGV):
(BUF_TS_LINECOL_POINT):
(BUF_TS_LINECOL_ZV):
(SET_BUF_TS_LINECOL_BEGV):
(SET_BUF_TS_LINECOL_POINT):
(SET_BUF_TS_LINECOL_ZV): New inline functions.
* src/casefiddle.c (casify_region): Record linecol info.
* src/editfns.c (Fsubst_char_in_region):
(Ftranslate_region_internal):
(Ftranspose_regions): Record linecol info.
* src/insdel.c (insert_1_both):
(insert_from_string_1):
(insert_from_gap_1):
(insert_from_buffer):
(replace_range):
(del_range_2): Record linecol info.
* src/treesit.c (TREESIT_BOB_LINECOL):
(TREESIT_EMPTY_LINECOL):
(TREESIT_TS_POINT_1_0): New constants.
(treesit_debug_print_linecol):
(treesit_buf_tracks_linecol_p):
(restore_restriction_and_selective_display):
(treesit_count_lines):
(treesit_debug_validate_linecol):
(treesit_linecol_of_pos):
(treesit_make_ts_point):
(Ftreesit_tracking_line_column_p):
(Ftreesit_parser_tracking_line_column_p): New functions.
(treesit_tree_edit_1): Accept real TSPoint and pass to
tree-sitter.
(compute_new_linecol_by_change): New function.
(treesit_record_change_1): Rename from treesit_record_change,
handle linecol if tracking is enabled.
(treesit_linecol_maybe): New function.
(treesit_record_change): New wrapper around
treesit_record_change_1 that handles some boilerplate and sets
buffer state.
(treesit_sync_visible_region): Handle linecol if tracking is
enabled.
(make_treesit_parser): Setup parser's linecol cache if tracking
is enabled.
(Ftreesit_parser_create): Enable tracking if the parser's
language requires it.
(Ftreesit__linecol_at):
(Ftreesit__linecol_cache_set):
(Ftreesit__linecol_cache): New functions for debugging and
testing.
(syms_of_treesit): New variable
Vtreesit_languages_require_line_column_tracking.
* src/treesit.h (Lisp_TS_Parser): New fields.
(TREESIT_BOB_LINECOL):
(TREESIT_EMPTY_LINECOL): New constants.
* test/src/treesit-tests.el (treesit-linecol-basic):
(treesit-linecol-search-back-across-newline):
(treesit-linecol-col-same-line):
(treesit-linecol-enable-disable): New tests.
* src/lisp.h: Declare display_count_lines.
* src/xdisp.c (display_count_lines): Remove static keyword.
* lisp/treesit-x.el (define-treesit-generic-mode): Add keyword
':copy-queries t' to the end of 'source'.
* lisp/treesit.el (treesit-language-source-alist):
Document the keyword ':copy-queries'.
(treesit--install-language-grammar-1): Add &rest args.
Process the keyword args. Call 'treesit--copy-queries'
when :copy-queries is non-nil.
(treesit--copy-queries): Add arg 'source-dir'. Copy queries
from source-dir as well. Copy only the file "highlights.scm".
* lisp/progmodes/elixir-ts-mode.el (elixir-ts--range-rules):
Rename to a shorter name from 'elixir-ts--treesit-range-rules'.
(elixir-ts--font-lock-feature-list, elixir-ts--thing-settings)
(elixir-ts--range-rules): New variables with default values
extracted from 'elixir-ts-mode'.
(elixir-ts-mode): Use 'elixir-ts--font-lock-feature-list',
'elixir-ts--thing-settings', 'elixir-ts--range-rules'
and 'heex-ts--range-rules'.
Use 'treesit-merge-font-lock-feature-list' to merge
'heex-ts--font-lock-feature-list'.
* lisp/progmodes/heex-ts-mode.el
(heex-ts--font-lock-feature-list, heex-ts--range-rules):
New variables.
(heex-ts-mode): Use 'heex-ts--font-lock-feature-list',
'heex-ts--range-rules'. Merge 'elixir-ts--font-lock-settings',
'elixir-ts--font-lock-feature-list', 'elixir-ts--thing-settings'
for embedding elixir in heex. Enable the 'sexp' navigation
by default with 'treesit-cycle-sexp-type'.
* lisp/progmodes/c-ts-mode.el (c-ts-mode): Append
'treesit-range-rules' to possibly already existing list in
'treesit-range-settings'.
* lisp/treesit.el (treesit-language-at-point-default):
Optimize to use 'when-let*'.
* lisp/treesit.el (treesit-language-at-point-default): Return
nil if there's no parser in the buffer.
(treesit-parsers-at): Add docstring for parser order.
Don't add the primary parser or the first of treesit-parser-list
when non-nil 'with-host' is provided since there is no host
for the primary parser (bug#76788).
* lisp/treesit.el (treesit-parsers-at): Add treesit-primary-parser
only when it's non-nil. When the result list is still empty,
add the first parser from 'treesit-parser-list'.
https://lists.gnu.org/archive/html/emacs-devel/2025-04/msg00627.html
* test/src/treesit-tests.el (treesit-node-supplemental)
(treesit-node-at, treesit-node-check)
(treesit-search-subtree-forward-1)
(treesit-search-subtree-backward-1): Wrap test body in 'with-temp-buffer'.
* lisp/outline.el (outline-after-change-functions): New variable.
(outline--fix-buttons-after-change):
Run hook 'outline-after-change-functions'.
* lisp/treesit.el (treesit--after-change): Improve docstring.
(treesit-major-mode-setup): Simplify to just add
'treesit--after-change' to 'outline-after-change-functions' hook.
* lisp/progmodes/rust-ts-mode.el (rust-ts-mode): Add explicit
'treesit-outline-predicate' that is like
'treesit-simple-imenu-settings', but also adds "trait_item".
* lisp/treesit.el (treesit-language-at-point-function):
Change the default value from nil to 'treesit-language-at-point-default'.
(treesit-language-at): Funcall 'treesit-language-at-point-function'
unconditionally.
(treesit-language-at-point-default): New function with body from
'treesit-language-at'.
(treesit-node-at): Simplify by replacing duplicate code
with the call to 'treesit-parsers-at'.
* lisp/treesit.el (treesit-forward-sexp): Don't use
'treesit-sexp-type-regexp' reserved for list commands
to override their default 'list' thing.
(treesit-down-list, treesit-up-list): Use
'treesit-sexp-type-regexp' instead of 'list'
when it's non-nil.
(treesit-toggle-sexp-mode): New command.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--thing-settings):
Improve 'sexp' thing settings to exclude the top-level
"translation_unit" that just moves to EOF and also "comment".
* lisp/progmodes/elixir-ts-mode.el (elixir-ts--sexp-regexp): Remove.
(elixir-ts--forward-sexp): Remove to use the default 'sexp' navigation.
(elixir-ts--with-parens-0-p, elixir-ts--with-parens-1-p):
New internal functions.
(elixir-ts-mode): Add 'treesit-thing-settings'
instead of 'forward-sexp-function' (bug#76788).
* lisp/progmodes/heex-ts-mode.el (heex-ts--sexp-regexp): Remove.
(heex-ts--forward-sexp): Remove to use the default 'sexp' navigation.
(heex-ts--thing-settings): New variable.
(heex-ts-mode): Use 'heex-ts--thing-settings'
instead of 'forward-sexp-function'.
* lisp/progmodes/java-ts-mode.el (java-ts-mode):
Improve 'sexp' thing to use settings like in c-ts-mode.
* lisp/progmodes/php-ts-mode.el (php-ts-mode):
Improve 'sexp' thing settings to exclude the top-level
"program" that just moves to EOF and also "comment".
* lisp/progmodes/ruby-ts-mode.el (ruby-ts-mode):
Improve 'sexp' thing to use settings like in c-ts-mode.
* lisp/textmodes/css-mode.el (css--treesit-thing-settings):
Add 'sexp' thing. Add "string_value" to 'text' thing.
* lisp/textmodes/html-ts-mode.el (html-ts-mode--treesit-things-settings):
Improve 'sexp' thing to use settings like in c-ts-mode.
Add "doctype" to the 'list' thing.
* lisp/treesit.el (treesit-outline-search): Remove
temporary arg 'recursive'. Use iteration to visit
more ranges until the next outline heading is found.
* lisp/progmodes/php-ts-mode.el (php-ts-mode):
Set 'treesit-outline-predicate' explicitly.
(treesit-major-mode-setup): Set 'outline-minor-mode-hook'
to add 'treesit--after-change' with 'treesit-update-ranges'
to 'after-change-functions' (bug#77256).
* doc/lispref/parsing.texi (Multiple Languages): The variable
'treesit-language-at-point-function' is now optional for
multi-language major modes. Add description of 'treesit-parsers-at'.
* lisp/treesit.el (treesit-language-at-point-function):
Change the the docstring to remove the dissuasion against
deriving the language from parser ranges.
(treesit-language-at): Use the first parser from
'treesit-parsers-at' as the default return value when
'treesit-language-at-point-function' is nil. Adapt the docstring.
(treesit-node-at): Use 'treesit-parsers-at'.
(treesit-parsers-at): New function.
(treesit-local-parsers-at): Use 'treesit-parsers-at'
with the most part of the body moved to it.
(treesit-local-parsers-on): Replace the overlay property
'treesit-parser' with 'treesit-parser-local-p' in the docstring.
(treesit-up-list, treesit-simple-imenu, treesit-outline-level):
Use 'treesit-parsers-at'.
* lisp/progmodes/c-ts-mode.el (c-ts-mode): Don't set
'treesit-language-at-point-function'.
* lisp/progmodes/elixir-ts-mode.el (elixir-ts--treesit-language-at-point):
Remove.
(elixir-ts-mode): Don't set 'treesit-language-at-point-function'.
* lisp/progmodes/js.el (js--treesit-language-at-point): Remove.
(js-ts-mode): Don't set 'treesit-language-at-point-function'.
* lisp/progmodes/php-ts-mode.el (php-ts-mode--html-language-at-point)
(php-ts-mode--language-at-point): Remove.
(php-ts-mode): Don't set 'treesit-language-at-point-function'.
* lisp/textmodes/mhtml-ts-mode.el (mhtml-ts-mode--language-at-point):
Remove.
(mhtml-ts-mode): Don't set 'treesit-language-at-point-function'.
Use 'treesit-language-at' for mode-line lighter.
* lisp/treesit.el (treesit-replace-font-lock-feature-settings):
Check the query language in addition to checking the feature.
(treesit-font-lock-fontify-region): Use treesit-font-lock-setting-*
accessors.
* lisp/treesit-x.el (treesit-generic-mode-setup): Append new
font-lock rules to an existing treesit-font-lock-settings
possibly inherited from the parent.
(treesit-generic-mode-setup): Use treesit-merge-font-lock-feature-list
to merge with an existing feature list inherited from the parent.
(liquid-generic-ts-mode): New treesit generic mode.
(alpinejs-generic-ts-setup): New treesit generic setup.
* lisp/treesit.el (treesit-forward-sentence): When no more
sentences are found, move to the end of the buffer or to
the range boundary at the 'treesit-parser' overlay's end.
* lisp/treesit.el (treesit-closest-parser-boundary):
Remove temporary function.
(treesit-outline-search): Use 'previous/next-single-char-property-change'
with 'treesit-parser' property to find the closest parser boundary.
(treesit-outline-level): Get the 'treesit-host-parser' overlay
and find the parent parser node with 'treesit-node-at'.
(Unrelated to the topic of the bug, but mentioned in the
thread.)
* lisp/treesit.el (treesit--explorer-refresh-1): New function.
(treesit--explorer-refresh): Handle 'treesit-parser-deleted'
error.
* lisp/treesit.el (treesit-local-parsers-at):
(treesit-local-parsers-on): Exclude non-local parsers.
(treesit--cleanup-local-range-overlays): Don't delete non-local
parsers (because those are managed by the major mode).
(treesit--update-ranges-non-local): Apply overlay for each ranges.
(treesit--update-ranges-local): Ignore
overlays with non-local parsers, and set
'treesit-parser-local-p' property to t.
(treesit--update-range-1): Additionally pass modified-tick to
treesit--update-ranges-non-local.
(treesit-major-mode-setup): Don't delete non-local parsers.
This fixes a bug where we call treesit-query-language on a
function, which happens before the QUERY in a range setting can
be either a function or a query, but we call it with
treesit-query-language before knowing whether it's a query or
function.
* lisp/treesit.el:
(treesit--update-range-1): Skip function range settings.
(treesit-update-ranges): Handle function range settings here.
