* lisp/treesit.el (treesit-explorer-anonymous-node)
(treesit-explorer-field-name): New face.
(treesit--explorer-draw-node): Use the new faces.
(treesit-explore-mode): Change playground to explorer.
Introduce a new command that aims to reindent code in a defun, or fill
a paragraph of text. The command uses treesit.el when available,
otherwise falls back to using syntax-ppss and regexps. Treesit.el
needs a new variable that is intended to be set by the major modes so
that this and other future functions can know what kind of node we are
looking at.
* doc/emacs/programs.texi: Mention the new command.
* etc/NEWS: Mention the new command.
* lisp/progmodes/c-ts-mode.el (c++-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/csharp-mode.el (csharp-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/java-ts-mode.el (java-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/js.el (js-ts-mode): Add regexp for the new variable.
* list/progmodes/prog-mode.el (prog-mode-map): Bind the new command by
default.
(prog-fill-reindent-defun): New command.
* lisp/progmodes/sh-script.el (bash-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-base-mode): Add
regexp for the new variable.
* lisp/treesit.el (treesit-text-type-regexp): New variable.
Before this change Ftreesit_query_capture doesn't convert character
position to byte position for BEG and END parameters. I observed
fontification issue in css files but couldn't figure out why, now I
know :-)
I decide to keep treesit--font-lock-query-expand-range, since it might
provide a escape hatch for problems we discover in the future, and it
should be very cheap so no downside of keeping it.
* lisp/textmodes/css-mode.el (css-ts-mode): Stop setting
treesit--font-lock-query-expand-range.
* lisp/treesit.el (treesit--font-lock-query-expand-range): Update
docstring.
* src/treesit.c (Ftreesit_query_capture): Convert BEG and END to byte
position. Also added parentheses wround "beg_byte - visible_beg" in
the call to ts_query_cursor_set_byte_range (i.e., style change).
This also fixes fontification problem with c-ts-mode--fontify-defun.
Now treesit-fontify-with-override clips the fontification region for
the user, so no need for (max start node-start) shenanigans anymore.
More importantly it doesn't fontify unless the region between
node-start and node-end intersects with the region between start and
end, which fixes the problem with c-ts-mode--fontify-defun.
* lisp/treesit.el (treesit-fontify-with-override): Add optional
parameter BOUND-START and BOUND-END. Wrap the function body in a
when-form.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--fontify-declarator)
(c-ts-mode--fontify-variable)
(c-ts-mode--fontify-defun)
(c-ts-fontify-error)
* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string): Use the
new signature.
treesit-comment-start/end is unnecessary because of
comment-start/end-skip, so they should be removed.
Cleanup and set comment-start/end-skip for tree-sitter C-like major
modes.
I replaced the [ \t]* part in comment-start-skip with (syntax
whitespace), which is what comment-end-skip uses. I also added
grouping in comment-start-skip to match that of comment-end-skip.
* lisp/progmodes/c-ts-mode.el (c-ts-mode)
(c++-ts-mode)
* lisp/progmodes/csharp-mode.el (csharp-ts-mode)
* lisp/progmodes/java-ts-mode.el (java-ts-mode)
* lisp/progmodes/js.el (js-ts-mode)
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-mode): Setup
comment-start/end-skip.
* lisp/treesit.el (treesit-comment-start)
(treesit-comment-end): Remove variables.
(treesit-simple-indent-presets): Use comment-start/end-skip instead.
This replaces font-lock-maximum-decoration and allows us to disable
the busiest fontification level by default.
* lisp/treesit.el (treesit-font-lock-level): New variable.
(treesit-font-lock-feature-list)
(treesit-font-lock-settings): Change docstring.
(treesit-font-lock-recompute-features): Use the new variable.
Before this change, treesit-beginning-of-defun skips nested defuns.
Now user can decide whether to skip nested defuns.
* lisp/treesit.el (treesit-search-forward-goto): Improve docstring.
(treesit-defun-prefer-top-level): New variable.
(treesit--defun-maybe-top-level): New function.
(treesit-beginning-of-defun)
(treesit-end-of-defun): Use treesit--defun-maybe-top-level.
* lisp/treesit.el (treesit--clip-ranges): New function.
(treesit-update-ranges): Now clips the range within (point-min)
and (point-max), so the new range we use are not out-of-range.
* lisp/treesit.el (treesit-max-buffer-size): Avoid overflow in
computing buffer-size limit. Account for 32-but systems built
"--with-wide-int". Extend doc string.
(treesit-ready-p): Compare the limit with the size of the buffer
in bytes, not in characters.
* src/treesit.c (treesit_check_buffer_size): Measure buffer size
in bytes.
1. Set activation of each feature (changes every feature)
2. Add/remove features (only change those explicitly configured by
ADD-LIST and REMOVE-LIST)
This is useful for enabling/disabling certain features for all
modes (without resetting others) by calling this function in
prog-mode-hook
* lisp/treesit.el (treesit-font-lock-recompute-features): See above
description.
treesit--font-lock-query-expand-range allows a major mode to fix
fontification problems temporarily before the parser can be fixed.
* lisp/treesit.el (treesit--font-lock-query-expand-range): New variable.
(treesit-font-lock-fontify-region): Use the new variable.
* lisp/textmodes/css-mode.el (css-ts-mode): Use the new variable.
So it turns out the slowness observed in bug#59415 is not due to the
size, but the strangely tall tree. Adjust the heuristic to DTRT:
don't enable the heuristic by default or when buffer is large, enable
when query is abnormally slow. We could do some clever thing that
calibrates a base reading for the query time instead of using a
hard-coded value, but it doesn't seem necessary.
* lisp/treesit.el (treesit--font-lock-fast-mode): New variable.
(treesit-font-lock-fontify-region): Don't activate heuristic by
default (reasons in comments). Measure the query time and activate
the fast mode if query time is long.
Previously applied heuristic 2 sometimes invalidates heuristic 1, add
a guard so it doesn't.
The new function is just for clearity of the code and has nothing to
do with the change itself.
* lisp/treesit.el (treesit--node-length): New function
(treesit-font-lock-fontify-region): Use the new function. Only do
heuristic 2 when the node is large.
* lisp/treesit.el (treesit--children-covering-range)
(treesit--children-covering-range-recurse): New functions. They are
not currently used but could be useful in the future, so I left them
in place.
(treesit-font-lock-fontify-region):
* lisp/treesit.el (treesit-font-lock-fontify-region): Use the result
of treesit-node-on instead of the root node.
Remove it since we are using separate major modes for tree-sitter and
native variant now.
* doc/lispref/parsing.texi (Tree-sitter major modes): Update manual.
* lisp/treesit.el (treesit-settings): Remove option.
(treesit--setting-for-mode): Remove function.
(treesit-ready-p): Don't check for user preference in treesit-settings.
The goal is to indent like this:
/* comment
comment --> This line aligns with the beginning of the first line
*/ --> This line aligns with the opening comment token
* lisp/treesit.el (treesit-comment-start)
(treesit-comment-end): New variables.
(treesit-simple-indent-presets): New preset comment-end,
comment-start, comment-start-skip
* lisp/progmodes/c-ts-mode.el (c-ts-mode--indent-styles)
(c-ts-mode)
* lisp/progmodes/java-ts-mode.el (java-ts-mode--indent-rules)
(java-ts-mode)
* lisp/progmodes/js.el (js--treesit-indent-rules)
(js-ts-mode)
* lisp/progmodes/ts-mode.el (ts-mode--indent-rules)
(ts-mode): Add identical indent rules to each mode, and set identical
treesit-comment-start/end's.
* doc/lispref/modes.texi (Parser-based Indentation)
* doc/lispref/parsing.texi (Tree-sitter major modes): Update manual.
I changed other presets to count all nodes a while ago, but forgot to
change 'match' too.
* lisp/treesit.el (treesit-simple-indent-presets): Change to count on
both named and anonymous nodes.
* doc/lispref/modes.texi (Parser-based Font Lock)
* lisp/treesit.el (treesit-font-lock-feature-list): Explain in more
detail, and change some features.
This mode is basically the tree-sitter playground[1] in Emacs:
displays the syntax tree with the source side-by-side, kept in sync in
real-time.
[1] https://tree-sitter.github.io/tree-sitter/playground
* doc/lispref/parsing.texi (Language Definitions): Mention in manual.
* lisp/treesit.el (treesit--explorer-buffer)
(treesit--explorer-source-buffer)
(treesit--explorer-language)
(treesit--explorer-refresh-timer)
(treesit--explorer-highlight-overlay)
(treesit--explorer-last-node): New variables.
* lisp/treesit.el (treesit--explorer--nodes-to-highlight)
(treesit--explorer-refresh)
(treesit--explorer-post-command)
(treesit--explorer-jump)
(treesit--explorer-highlight-node)
(treesit--explorer-draw-node): New functions.
(treesit--explorer-tree-mode)
(treesit-explore-mode): New modes.
Now nodes that are affected and changed during re-parse will be
correctly refontified. Nodes can change even if it corresponding text
wasn't edited: additional text can complete the parse tree and resolve
error nodes, for example.
* lisp/progmodes/python.el (python-mode): Create Python parser before
calling treesit-major-mode-setup.
* lisp/treesit.el (treesit--font-lock-notifier): New function.
(treesit-major-mode-setup): Register fontifier with every existing
parser.
We now have a better facility that can replace the contextual
hack. The C part is in the previous commit, and the Lisp part work
will be in the next commit.
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/progmodes/js.el (js--treesit-font-lock-settings)
* lisp/progmodes/python.el (python--treesit-settings)
* lisp/progmodes/ts-mode.el (ts-mode--font-lock-settings): Stop marking
contextual nodes.
* lisp/treesit.el (treesit-font-lock-contextual-post-process): Remove
function.
(treesit-font-lock-fontify-region): Remove code processing contextual
nodes.
* doc/lispref/parsing.texi (Retrieving Node): Update manual.
* lisp/treesit.el (treesit-node-at): Change semantic. It tries to
return the node that a user would expect in various circumstances.
* test/src/treesit-tests.el (treesit-node-at): New test.
