Some embedded parser needs to exclude child nodes from the
range, like markdown-inline. So I added this keyword that
allows users to customize the range for the embedded parser.
This can also be potentially useful for markdown comments in
rust, for example, because we want to exclude the comment
starters (//) from the embedded markdown parser.
* lisp/treesit.el (treesit-query-range):
(treesit-query-range-by-language): Add new parameter RANGE-FN.
(treesit-range-settings): Add new field RANGE-FN.
(treesit-range-rules): Add new keyword RANGE-FN.
(treesit-range-fn-exclude-children): New function.
(treesit--update-ranges-non-local):
(treesit--update-ranges-local):
(treesit--update-range-1): Support the RANGE-FN field.
Make use of the newly added embed-level parser property, we now
iterativeLy create/update embedded parsers at each embed level,
allowing arbitrary levels of nesting.
At the beginning, we start with the primary parser. We query it
with each range settings, gets some ranges and their
corresponding language. Then we create the first level of
embedded parsers using the ranges and language we got. This is
one iteration. For the next iteration/level, we start with the
embedded parsers we create/updated in the previous
iteration/level, and query each of them with each of the range
settings, creating/updating the next level of embedded parsers.
And we keep doing this until we don't get more matches from the
queries.
We now also support the :embed keyword in 'treesit-range-rules'
to be a function that returns a language. This allows major
modes like markdown and org mode to support code blocks of which
the language isn't known ahead of time.
* lisp/treesit.el (treesit-primary-parser): Move to front.
(treesit-query-range-by-language): New function.
(treesit-range-rules): Allow :embed to be a function, update docstring.
(treesit-local-parsers-on): Update docstring.
(treesit--parser-at-level):
(treesit--update-ranges-non-local): New functions.
(treesit--update-ranges-local): Use the new logic, support
:embed being a function.
(treesit--update-range-1): New function, has the meat of
original treesit-update-ranges.
(treesit-update-ranges): Implements the iteration logic using
treesit--update-range-1 to do the actual work.
* lisp/treesit.el (treesit-outline-search): Add optional arg
'recursive' to avoid infinite recursion when it gets stuck.
* lisp/textmodes/markdown-ts-mode.el (markdown-ts-mode):
Set buffer-local 'treesit-outline-predicate' to "section".
* doc/lispref/modes.texi (Outline Minor Mode):
Add 'treesit-aggregated-outline-predicate'.
* lisp/treesit.el (treesit-aggregated-outline-predicate):
New buffer-local variable.
(treesit-outline--at-point):
Use 'treesit-aggregated-outline-predicate'.
(treesit-closest-parser-boundary): New function.
(treesit-outline-search): Use 'treesit-aggregated-outline-predicate'
and 'treesit-closest-parser-boundary'.
(treesit-outline-level): Use 'treesit-aggregated-outline-predicate'.
(treesit-major-mode-setup): Add 'treesit-aggregated-outline-predicate'.
* lisp/textmodes/html-ts-mode.el (html-ts-mode--outline-predicate):
Improve.
* lisp/textmodes/mhtml-ts-mode.el (mhtml-ts-mode):
Set 'treesit-aggregated-outline-predicate'.
* lisp/treesit.el (treesit-simple-indent-modify-rules): Rename
and fix docstring. It should be matcher, not anchor. Also
reword a little bit.
* etc/NEWS: Update.
* lisp/textmodes/mhtml-ts-mode.el:
(mhtml-ts-mode--treesit-indent-rules): Update accordingly.
New major-mode alternative to mhtml-mode, based on treesitter, for
editing files containing html, javascript and css.
* etc/NEWS: Mention the new mode and new functions.
* lisp/textmodes/mhtml-ts-mode.el: New file.
* lisp/progmodes/js.el
(js--treesit-thing-settings): New variable.
(js--treesit-font-lock-feature-list); New variable.
(js--treesit-simple-imenu-settings): New variable.
(js--treesit-defun-type-regexp): New variable.
(js--treesit-jsdoc-comment-regexp): New variable.
(js-ts-mode): Use of new variables instead of direct assignment of
values.
* lisp/textmodes/css-mode.el
(css-mode--menu): New variable.
(css-mode-map): Use new variable.
(css--treesit-font-lock-feature-list): New variable.
(css--treesit-simple-imenu-settings): New variable.
(css--treesit-defun-type-regexp): New variable.
(cs-ts-mode): Use of new variables instead of direct assignment of
values.
* lisp/textmodes/html-ts-mode.el
(html-ts-mode--treesit-things-settings): New variable.
(html-ts-mode--treesit-font-lock-feature-list): New variable.
(html-ts-mode--treesit-simple-imenu-settings): New variable.
(html-ts-mode--treesit-defun-type-regexp): New variable.
(html-ts-mode): Use of new variables instead of direct assignment of
values.
* lisp/treesit.el
(treesit-merge-font-lock-feature-list): New fuction.
(treesit-replace-font-lock-feature-settings): New fuction.
(treesit-modify-indent-rules): New function.
* lisp/treesit.el:
(treesit-simple-indent-standalone-predicate): Allow it to return
an anchor instead of t.
(treesit-simple-indent-presets): Supports number.
* lisp/progmodes/c-ts-common.el:
(c-ts-common--standalone-parent):
(c-ts-common--prev-standalone-sibling): Supports number.
* lisp/treesit.el:
(treesit-simple-indent-standalone-predicate): New variable.
(treesit-simple-indent-presets): Use the predicate.
(treesit-simple-indent-presets): Update docstring.
* lisp/progmodes/c-ts-common.el:
(c-ts-common--standalone-parent):
(c-ts-common--prev-standalone-sibling): Use the predicate if
non-nil. Also, handle method chaining by default.
* doc/lispref/modes.texi (Parser-based Indentation): Add
documentation.
* lisp/treesit.el (treesit-outline-level): Use level 1 by default
since treesit-outline--at-point now always returns the current node.
* lisp/progmodes/ruby-ts-mode.el (ruby-ts-mode):
Use 'bos' instead of 'bol'. Add "singleton_class" to
'treesit-outline-predicate'. Use new condition 'named' in
'treesit-outline-predicate' (bug#74963).
At this point, thing-at-point with all standard things
is already supported by treesit, for example:
"(thing-at-point 'defun)" by treesit-beginning-of-defun,
"(thing-at-point 'sexp)" by treesit-forward-sexp,
"(thing-at-point 'list)" by treesit-up-list,
"(thing-at-point 'sentence)" by treesit-forward-sentence,
"(thing-at-point 'comment)" by treesit-forward-comment.
https://lists.gnu.org/archive/html/emacs-devel/2025-02/msg00384.html
* lisp/progmodes/json-ts-mode.el (json-ts-mode): Add 'list' thing
to 'treesit-thing-settings'.
(json-ts-mode): Disable outlines.
* lisp/textmodes/html-ts-mode.el (html-ts-mode--defun-name):
Get a grandchild 'tag_name' from 'element' that was already
defined by 'treesit-defun-type-regexp'.
(html-ts-mode--outline-predicate): New function.
(html-ts-mode): Add "comment" to the 'list' thing
in 'treesit-thing-settings'. Use "tag_name" and "attribute"
in 'sentence' to conform to sentence navigating arguments
in other ts-modes. Remove unnecessary heading
from 'treesit-simple-imenu-settings' and use "element"
supported by 'html-ts-mode--defun-name'.
Set 'treesit-outline-predicate' to 'html-ts-mode--outline-predicate'.
* lisp/textmodes/toml-ts-mode.el (toml-ts-mode):
Add 'treesit-thing-settings'.
* lisp/textmodes/yaml-ts-mode.el (yaml-ts-mode--defun-name)
(yaml-ts-mode--outline-predicate): New functions.
(yaml-ts-mode): Set 'treesit-defun-type-regexp',
'treesit-defun-name-function', 'treesit-defun-tactic'.
Add 'sentence' to 'treesit-thing-settings'.
Set 'treesit-simple-imenu-settings' and 'treesit-outline-predicate'.
Use 'kill-local-variable' for 'forward-sexp-function'
and 'show-paren-data-function' instead of resetting their value.
* lisp/treesit.el (treesit-outline-search): Check for the thing
before the end of the line to support such case when the thing
fits on the current line and ends before the end of the line
such as e.g. '<h1>...</h1>' in html-ts-mode.
(treesit-hs-find-next-block, treesit-hs-inside-comment-p):
Use anchors for "\\`comment\\'" (bug#75609).
* lisp/treesit.el (treesit-node-at): Make the logic for finding
the parser (and the root node) more explicit and
straightforward. Don't use treesit-buffer-root-node; instead,
find the parser and use treesit-parser-root-node directly.
6447634f17 ; * admin/MAINTAINERS: Add CC Mode.
486d5d524c Update cc-mode URL to point to nongnu.org
17ef46e849 ; * etc/NEWS: Note CVE-2024-53920 further up also.
05ee2b741f ; * CONTRIBUTE: Suggest to run more tests sometimes.
e74efd9a42 * CONTRIBUTE: Recommend running the unit test prior to co...
a9cde2463a Don't signal an error in treesit-node-at
5d021a711a ; Improve documentation of '.dir-locals.el'
84595cbcc7 ; (let-alist): Document double-dot escape syntax. (Bug#75...
5617b07a45 ; Prefer HTTPS to HTTP in docs
fdd23023c8 ; * admin/MAINTAINERS: Sort external packages alphabetica...
74dcfe155a ; * etc/NEWS: Remove temporary documentation markers.
a87c382cab ; * etc/NEWS: Add missing temporary documentation tags.
3c820cd265 Document insert-directory-program as a user option
2c1edf5f62 doc/lispref/modes.texi (Syntactic Font Lock): Update for ...
# Conflicts:
# etc/NEWS
* lisp/treesit.el:
(treesit-add-simple-indent-rules): New function.
* etc/NEWS: Update NEWS.
*
test/src/treesit-tests.el:
(treesit-test-add-simple-indent-rules): Add a test for the new
function.
* lisp/progmodes/hideshow.el (hs-inside-comment-p-func):
New buffer-local variable.
(hs-hide-block-at-point): Check if 'hs-block-end-regexp' is
a string or a function.
(hs-inside-comment-p): Move body to 'hs-inside-comment-p--default'.
Call 'hs-inside-comment-p-func' if it's a function.
(hs-inside-comment-p--default): New function with body from
'hs-inside-comment-p'.
(hs-hide-all): Don't use 'hs-block-start-regexp' when it's not a string.
(hs-minor-mode): Don't call 'hs-grok-mode-type' when
'hs-inside-comment-p-func' already has a buffer-local value.
* lisp/treesit.el (treesit-hs-block-end)
(treesit-hs-find-block-beginning, treesit-hs-find-next-block)
(treesit-hs-looking-at-block-start-p)
(treesit-hs-inside-comment-p): New functions.
(treesit-major-mode-setup): Set hs-minor-mode buffer-local variables.
* lisp/treesit.el (treesit-forward-comment): New function.
(treesit-major-mode-setup): Set 'forward-comment-function' to
'treesit-forward-comment' if the 'comment' thing is defined.
* src/syntax.c (forward-comment-function): New variable.
(Fforward_comment): Call the function from 'forward-comment-function'
when it's non-nil (bug#75609).
* lisp/treesit.el (treesit--imenu-merge-entries):
(Ftreesit_parser_set_included_ranges): Warn in the docstring
that the function is destructive/owns the argument.
* test/src/treesit-tests.el (treesit-range-fixup-after-edit):
(treesit-imenu): Fix tests.
Up to this point, functions in
treesit.el (font-lock/indentation/etc) creates a parser if one
doesn't exist. This doesn't work well with local parsers,
because local parsers are not visible at global level and
font-lock or indentation functions might accidentally create a
global parser for an embedded language.
The solution is to not create parsers automatically. This has
the potential of breaking things, but I don't expect any actual
breakage: all the major modes create parsers they need in the
major mode body, even though previously it isn't technically
necessary.
* lisp/treesit.el (treesit-buffer-root-node):
(treesit--update-ranges-local):
(treesit-update-ranges):
(treesit--guess-primary-parser): Use treesit-parser-list instead
of treesit-parser-create.
This should help major modes to support grammar versions.
* doc/lispref/parsing.texi (Pattern Matching): Mention new function.
* lisp/treesit.el (treesit-query-first-valid): New function.
(treesit-query-valid-p): New function.
* lisp/treesit.el (treesit-major-mode-setup): Move setting of
'transpose-sexps-function' outside of 'treesit-thing-defined-p'
since 'treesit-transpose-sexps' doesn't depend on the 'sexp' thing.
* lisp/textmodes/yaml-ts-mode.el (yaml-ts-mode):
Add 'treesit-thing-settings' with the 'list' thing,
but use it only for list commands, not sexp commands (bug#73404).
* lisp/treesit.el (treesit--forward-list-with-default): Improve to
better support the case when point is inside an opening/closing node
with a name longer than 1 character such as "#if" and "#endif".
(treesit-navigate-thing): No need to check for thing 'list'
since list commands now use other functions.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--thing-settings): Add
more preproc nodes such as #if/#endif and #ifdef/#endif and their
variants to the 'list' thing.
* doc/lispref/parsing.texi (User-defined Things): Mention new
functions 'treesit-forward-list', 'treesit-down-list',
'treesit-up-list', 'treesit-show-paren-data' that use the thing
'list' with the symbol property 'treesit-thing-symbol' (bug#73404).
* lisp/treesit.el: Put the property 'treesit-thing-symbol'
on the symbol 'list'.
(treesit--forward-list-with-default, treesit-down-list)
(treesit-up-list, treesit-navigate-thing)
(treesit-show-paren-data--categorize, treesit-major-mode-setup):
Replace 'sexp-list' with 'list'.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--thing-settings):
* lisp/progmodes/js.el (js-ts-mode):
* lisp/progmodes/ruby-ts-mode.el (ruby-ts-mode):
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-base-mode)
(tsx-ts-mode):
* lisp/textmodes/html-ts-mode.el (html-ts-mode):
Replace 'sexp-list' with 'list'.
* src/treesit.c (treesit_traverse_validate_predicate)
(treesit_traverse_match_predicate): Check if the 'pred'
symbol has the property 'Qtreesit_thing_symbol'.
(syms_of_treesit): New symbol 'Qtreesit_thing_symbol'.
(treesit--forward-list-with-default): New internal function
with body from 'treesit-forward-sexp-list' (bug#73404).
(treesit-forward-sexp-list, treesit-forward-list):
Replace body with a call to the shared implementation
'treesit--forward-list-with-default' using the
corresponding default function as an argument.
(treesit-show-paren-data--categorize): Use 'treesit-thing-defined-p'
to check if the thing exists for the language at POS before calling
'treesit-parent-until' that raises an error for an undefined thing.
(treesit-thing-defined-p): Add the alias signature to the docstring.
Use 'treesit-thing-next' and 'treesit-thing-prev' with
'treesit-node-named' to transpose named siblings that leaves the right
punctuation in anonymous nodes between named siblings. Fall back
to 'transpose-sexps-default-function' inside strings and comments.
(treesit-node-named): New helper function.
* lisp/treesit.el (treesit-explore-mode): We added the hook to
kill-buffer-hook, so make sure to remove it from
kill-buffer-hook, not post-command-hook.
* lisp/treesit.el (treesit-forward-sexp-list): Rewrite to support
the value of ARG more than 1. In this case every step moves forward
either over the next treesit-based list or over the syntax-based symbol.
(treesit-down-list, treesit-up-list): Rewrite to support the fallback
to the syntax-based navigation while inside the treesit-based list.
Also use a loop for ARG more than 1 (bug#73404).
Copy the implementation of treesit-parent-until here, and call
its treesit-node-match-p with the argument IGNORE-MISSING set to t
that causes it to fail silently instead of raising an error when
the predicate 'sexp-list' is not defined in an embedded language.
(treesit-major-mode-setup): Set show-paren-data-function to
treesit-show-paren-data only when 'sexp-list' thing is defined
that is used by treesit-show-paren-data--categorize (bug#75198).
* admin/tree-sitter/compat-template.html: Update template.
* admin/tree-sitter/treesit-admin.el:
(treesit-admin--builtin-language-sources): Add sources.
(treesit-admin--builtin-modes): New variable.
(treesit-admin--verify-major-mode-queries): Don't need to pass
LANGS argument anymore.
(treesit-admin-verify-major-mode-queries): Move, and use
treesit-admin--builtin-modes.
(treesit-admin--mode-languages): Set some variables so it gets
all the languages.
(treesit-admin--find-latest-compatible-revision): Also return
commit timestamp.
(treesit-admin--generate-compatibility-report): New parameter
EMACS-EXECUTABLES. Support generating report for multiple Emacs
versions.
* lisp/treesit.el (treesit--language-git-timestamp): New function.
