So it turns out the slowness observed in bug#59415 is not due to the
size, but the strangely tall tree. Adjust the heuristic to DTRT:
don't enable the heuristic by default or when buffer is large, enable
when query is abnormally slow. We could do some clever thing that
calibrates a base reading for the query time instead of using a
hard-coded value, but it doesn't seem necessary.
* lisp/treesit.el (treesit--font-lock-fast-mode): New variable.
(treesit-font-lock-fontify-region): Don't activate heuristic by
default (reasons in comments). Measure the query time and activate
the fast mode if query time is long.
Previously applied heuristic 2 sometimes invalidates heuristic 1, add
a guard so it doesn't.
The new function is just for clearity of the code and has nothing to
do with the change itself.
* lisp/treesit.el (treesit--node-length): New function
(treesit-font-lock-fontify-region): Use the new function. Only do
heuristic 2 when the node is large.
* lisp/treesit.el (treesit--children-covering-range)
(treesit--children-covering-range-recurse): New functions. They are
not currently used but could be useful in the future, so I left them
in place.
(treesit-font-lock-fontify-region):
* lisp/treesit.el (treesit-font-lock-fontify-region): Use the result
of treesit-node-on instead of the root node.
Remove it since we are using separate major modes for tree-sitter and
native variant now.
* doc/lispref/parsing.texi (Tree-sitter major modes): Update manual.
* lisp/treesit.el (treesit-settings): Remove option.
(treesit--setting-for-mode): Remove function.
(treesit-ready-p): Don't check for user preference in treesit-settings.
The goal is to indent like this:
/* comment
comment --> This line aligns with the beginning of the first line
*/ --> This line aligns with the opening comment token
* lisp/treesit.el (treesit-comment-start)
(treesit-comment-end): New variables.
(treesit-simple-indent-presets): New preset comment-end,
comment-start, comment-start-skip
* lisp/progmodes/c-ts-mode.el (c-ts-mode--indent-styles)
(c-ts-mode)
* lisp/progmodes/java-ts-mode.el (java-ts-mode--indent-rules)
(java-ts-mode)
* lisp/progmodes/js.el (js--treesit-indent-rules)
(js-ts-mode)
* lisp/progmodes/ts-mode.el (ts-mode--indent-rules)
(ts-mode): Add identical indent rules to each mode, and set identical
treesit-comment-start/end's.
* doc/lispref/modes.texi (Parser-based Indentation)
* doc/lispref/parsing.texi (Tree-sitter major modes): Update manual.
I changed other presets to count all nodes a while ago, but forgot to
change 'match' too.
* lisp/treesit.el (treesit-simple-indent-presets): Change to count on
both named and anonymous nodes.
* doc/lispref/modes.texi (Parser-based Font Lock)
* lisp/treesit.el (treesit-font-lock-feature-list): Explain in more
detail, and change some features.
This mode is basically the tree-sitter playground[1] in Emacs:
displays the syntax tree with the source side-by-side, kept in sync in
real-time.
[1] https://tree-sitter.github.io/tree-sitter/playground
* doc/lispref/parsing.texi (Language Definitions): Mention in manual.
* lisp/treesit.el (treesit--explorer-buffer)
(treesit--explorer-source-buffer)
(treesit--explorer-language)
(treesit--explorer-refresh-timer)
(treesit--explorer-highlight-overlay)
(treesit--explorer-last-node): New variables.
* lisp/treesit.el (treesit--explorer--nodes-to-highlight)
(treesit--explorer-refresh)
(treesit--explorer-post-command)
(treesit--explorer-jump)
(treesit--explorer-highlight-node)
(treesit--explorer-draw-node): New functions.
(treesit--explorer-tree-mode)
(treesit-explore-mode): New modes.
Now nodes that are affected and changed during re-parse will be
correctly refontified. Nodes can change even if it corresponding text
wasn't edited: additional text can complete the parse tree and resolve
error nodes, for example.
* lisp/progmodes/python.el (python-mode): Create Python parser before
calling treesit-major-mode-setup.
* lisp/treesit.el (treesit--font-lock-notifier): New function.
(treesit-major-mode-setup): Register fontifier with every existing
parser.
We now have a better facility that can replace the contextual
hack. The C part is in the previous commit, and the Lisp part work
will be in the next commit.
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/progmodes/js.el (js--treesit-font-lock-settings)
* lisp/progmodes/python.el (python--treesit-settings)
* lisp/progmodes/ts-mode.el (ts-mode--font-lock-settings): Stop marking
contextual nodes.
* lisp/treesit.el (treesit-font-lock-contextual-post-process): Remove
function.
(treesit-font-lock-fontify-region): Remove code processing contextual
nodes.
* doc/lispref/parsing.texi (Retrieving Node): Update manual.
* lisp/treesit.el (treesit-node-at): Change semantic. It tries to
return the node that a user would expect in various circumstances.
* test/src/treesit-tests.el (treesit-node-at): New test.
* lisp/treesit.el (treesit-font-lock-fontify-region): If the captured
node is outside of the region between START and END, don't fontify it.
Wrap fontification code in a when form.
* doc/lispref/modes.texi (Parser-based Indentation): Update manual.
* lisp/progmodes/js.el (js--treesit-indent-rules): Change all
occurance of ,js-indent-level to js-indent-level.
* lisp/progmodes/ts-mode.el (ts-mode--indent-rules): Change all
occurance of ,ts-mode-indent-offset to ts-mode-indent-offset.
* lisp/treesit.el (treesit-simple-indent-rules): Change docstring.
(treesit-simple-indent): Allow offset to be a variable.
(or (treesit-parser-create language)
(error "Cannot find a parser for %s" language))
is turned into
(treesit-parser-create language)
because treesit-parser-create never returns nil. (We used to use
treesit-get-parser which don't create parser automatically.)
* lisp/treesit.el (treesit-buffer-root-node): Replace error with signal.
* doc/lispref/parsing.texi (Multiple Languages): Extend and update
manual.
* lisp/treesit.el (treesit-range-functions): Remove variable.
(treesit-range-settings): New variable.
(treesit-range-rules): New function.
(treesit--merge-ranges): New function.
(treesit-update-ranges): Use treesit-range-settings instead of
treesit-range-functions.
(treesit-font-lock-rules): Fix docstring.
(treesit-indent)
(treesit-indent-region): Only update ranges in a region.
* test/src/treesit-tests.el (treesit-range): New test.
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string): Update
function to only fontify within the region.
* lisp/treesit.el (treesit-font-lock-rules): Update docstring.
(treesit-font-lock-fontify-region): Pass START and END to
fontification functions.
* lisp/progmodes/python.el: Remove function.
(python--treesit-settings): Capture contextual node.
* lisp/treesit.el (treesit--set-nonsticky):
(treesit-font-lock-contextual-post-process): New functions.
(treesit-font-lock-fontify-region): Change local variable START and
END to NODE-START and NODE-END, handle special capture name
"contextual".
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
Change from
(START END NODE OVERRIDE &rest _)
to
(NODE OVERRIDE &rest _)
START and END aren't used frequently enough to justify. If a
fontification function needs them, it can get them from NODE.
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string)
(python--treesit-fontify-string-end): Change signature.
* lisp/treesit.el (treesit-font-lock-rules): Update docstring.
(treesit-font-lock-fontify-region): Remove START and END arguments.
Rather than querying on the smallest node that spans START to END, we
query on the root node between START to END.
* lisp/progmodes/python.el (python--treesit-fontify-string): Accept
the string rather than the quote node.
(python--treesit-settings): Capture the string rather than the quote
node.
* lisp/treesit.el (treesit-font-lock-fontify-region): Query the root
node rather than the smallest node.
Before this change, we rebind font-lock-fontify-region-function and
call font-lock's function in tree-sitter's function, and the order of
fontification is tree-sitter, font-lock syntax, font-lock regexp.
Now we make font-lock's syntax function customizable and replace it
with tree-sitter's function, and the order of fontification is
tree-sitter, font-lock regexp.
* doc/lispref/modes.texi (Parser-based Font Lock): Reflect the change
in manual.
* lisp/font-lock.el (font-lock-fontify-syntactically-function): New
varaible.
(font-lock-default-fontify-region): Call
font-lock-fontify-syntactically-function rather.
(font-lock-fontify-syntactically-region): Rename to
font-lock-default-fontify-syntactically
(font-lock-default-fontify-syntactically): Rename to this.
* lisp/treesit.el (treesit-font-lock-fontify-region): Don't call
font-lock functions.
(treesit-font-lock-enable): Remove this function. It is not used even
before this change.
(treesit-major-mode-setup): Instead of binding
font-lock-fontify-region-function, now bind to
font-lock-fontify-syntactically-function. And we can let font-lock do
it's thing.
