1
Fork 0
mirror of git://git.sv.gnu.org/emacs.git synced 2025-12-19 12:20:17 -08:00
Commit graph

299 commits

Author SHA1 Message Date
Yuan Fu
e0e3299c01
; * lisp/treesit.el (treesit-update-ranges): Update all parsers. 2022-11-04 10:03:17 -07:00
Yuan Fu
784f53a215
; Replace error with signal in treesit.el
(or (treesit-parser-create language)
    (error "Cannot find a parser for %s" language))

is turned into

(treesit-parser-create language)

because treesit-parser-create never returns nil.  (We used to use
treesit-get-parser which don't create parser automatically.)

* lisp/treesit.el (treesit-buffer-root-node): Replace error with signal.
2022-11-04 10:01:51 -07:00
Yuan Fu
673a55780d
; Remove unused tree-sitter functions
They didn't turn out to be very useful.

* lisp/treesit.el (treesit-set-ranges)
(treesit-get-ranges): Remove functions.
2022-11-04 09:57:32 -07:00
Yuan Fu
8ad4c4009e
Add treesit-indent-error
* lisp/treesit.el (treesit-indent-error): New error.
(treesit--simple-indent-eval): Signal treesit-indent-error.
2022-11-04 09:57:11 -07:00
Yuan Fu
d9d66764e2
Revise tree-sitter facility for multi-language buffers
* doc/lispref/parsing.texi (Multiple Languages): Extend and update
manual.
* lisp/treesit.el (treesit-range-functions): Remove variable.
(treesit-range-settings): New variable.
(treesit-range-rules): New function.
(treesit--merge-ranges): New function.
(treesit-update-ranges): Use treesit-range-settings instead of
treesit-range-functions.
(treesit-font-lock-rules): Fix docstring.
(treesit-indent)
(treesit-indent-region): Only update ranges in a region.
* test/src/treesit-tests.el (treesit-range): New test.
2022-11-04 09:39:57 -07:00
Yuan Fu
b028d4f7df
; Tree-sitter: Don't refontify if entity contained in the region
* lisp/treesit.el (treesit-font-lock-contextual-post-process): Only
refontify when NODE-START < region START.
2022-11-03 19:37:55 -07:00
Yuan Fu
69ab588bb5
Pass region start and end to tree-sitter fontification functions
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string): Update
function to only fontify within the region.
* lisp/treesit.el (treesit-font-lock-rules): Update docstring.
(treesit-font-lock-fontify-region): Pass START and END to
fontification functions.
2022-11-02 17:03:03 -07:00
Yuan Fu
f331be1f07
Add handling of contextual entities in tree-sitter font-lock
* lisp/progmodes/python.el: Remove function.
(python--treesit-settings): Capture contextual node.
* lisp/treesit.el (treesit--set-nonsticky):
(treesit-font-lock-contextual-post-process): New functions.
(treesit-font-lock-fontify-region): Change local variable START and
END to NODE-START and NODE-END, handle special capture name
"contextual".
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
2022-11-02 16:47:53 -07:00
Yuan Fu
040991a469
Change signature of tree-sitter font-lock functions
Change from

(START END NODE OVERRIDE &rest _)

to

(NODE OVERRIDE &rest _)

START and END aren't used frequently enough to justify.  If a
fontification function needs them, it can get them from NODE.

* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.

* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string)
(python--treesit-fontify-string-end): Change signature.

* lisp/treesit.el (treesit-font-lock-rules): Update docstring.
(treesit-font-lock-fontify-region): Remove START and END arguments.
2022-11-02 11:26:38 -07:00
Yuan Fu
6d5b34d9de
; Tree-sitter font-lock debugging: print node type not language
* lisp/treesit.el (treesit-font-lock-fontify-region): Print node type.
2022-11-01 12:54:55 -07:00
Yuan Fu
f6e92035a7
Allow user to add/remove tree-sitter font-lock features
* lisp/treesit.el (treesit-font-lock-recompute-features): Add
ADD-LIST, REMOVE-LIST argument.
2022-11-01 12:54:55 -07:00
Yuan Fu
a3651d3237
Query on the root node in tree-sitter font-lock
Rather than querying on the smallest node that spans START to END, we
query on the root node between START to END.

* lisp/progmodes/python.el (python--treesit-fontify-string): Accept
the string rather than the quote node.
(python--treesit-settings): Capture the string rather than the quote
node.
* lisp/treesit.el (treesit-font-lock-fontify-region): Query the root
node rather than the smallest node.
2022-11-01 12:54:51 -07:00
Yuan Fu
eeeae5e9ee
Add an argument OVERRIDE to tree-sitter font-lock functions
* doc/lispref/modes.texi (Parser-based Font Lock): Reflect this change
in manual.
* lisp/progmodes/js.el (js--fontify-template-string): Add _OVERRIDE
argument.
* lisp/progmodes/python.el (python--treesit-fontify-string): Add
_OVERRIDE argument.
* lisp/treesit.el (treesit-font-lock-rules): Update docstring.
(treesit-fontify-with-override): New function.
(treesit-font-lock-fontify-region): Extract out into
treesit-fontify-with-override.
2022-10-31 15:03:49 -07:00
Yuan Fu
9fab83ed7a
; * lisp/treesit.el (tresit--font-lock-verbose): Fix typo. 2022-10-31 14:26:03 -07:00
Yuan Fu
18947103fa
Use tree-sitter for font-lock's syntactic function (optionally)
Before this change, we rebind font-lock-fontify-region-function and
call font-lock's function in tree-sitter's function, and the order of
fontification is tree-sitter, font-lock syntax, font-lock regexp.

Now we make font-lock's syntax function customizable and replace it
with tree-sitter's function, and the order of fontification is
tree-sitter, font-lock regexp.

* doc/lispref/modes.texi (Parser-based Font Lock): Reflect the change
in manual.
* lisp/font-lock.el (font-lock-fontify-syntactically-function): New
varaible.
(font-lock-default-fontify-region): Call
font-lock-fontify-syntactically-function rather.
(font-lock-fontify-syntactically-region): Rename to
font-lock-default-fontify-syntactically
(font-lock-default-fontify-syntactically): Rename to this.

* lisp/treesit.el (treesit-font-lock-fontify-region): Don't call
font-lock functions.
(treesit-font-lock-enable): Remove this function.  It is not used even
before this change.
(treesit-major-mode-setup): Instead of binding
font-lock-fontify-region-function, now bind to
font-lock-fontify-syntactically-function.  And we can let font-lock do
it's thing.
2022-10-31 00:23:31 -07:00
Yuan Fu
377ee8158b
Add tresit--font-lock-verbose
* lisp/treesit.el (tresit--font-lock-verbose): New variable.
(treesit-font-lock-fontify-region): Take tresit--font-lock-verbose
into account.  Return jit-lock-bounds.
2022-10-30 20:56:21 -07:00
Yuan Fu
52c8fdde16
Set jit-lock-contextually in treesit-major-mode-setup
* lisp/treesit.el (treesit-major-mode-setup): Set jit-lock-contextually.
2022-10-30 20:53:44 -07:00
Yuan Fu
f6483526cd
Fix byte-compile warning in treesit.el
Originally treesit-simple-indent-presets was

'((parent-is . (lambda (type)
                 `(lambda (n p b)
                    (equal (treesit-node-type p)
                           ,type)))))

Then I changed it to

`((parent-is . ,(byte-compile
                 (lambda (type)
                   (lambda (n p b)
                     (equal (treesit-node-type p)
                            type))))))

but that generates "function already compiled" warning when
byte-compiling treesit.el.  The latest form shouldn't have any problem
anymore.

* lisp/treesit.el (treesit-simple-indent-presets): Change byte-compile
to plain evaluation.
2022-10-29 14:50:52 -07:00
Yuan Fu
f6e3de3f63
Not return noindent in treesit-indent
* lisp/treesit.el (treesit-indent): Don't return noindent.
(treesit-defun-type-regexp): Return (nil . nil) rather than nil.
2022-10-29 14:49:31 -07:00
Yuan Fu
da87895df2
Add treesit-node-top-level
Merge treesit-node-top-level-p and treesit--find-top-level-match into
treesit-node-top-level.

* doc/lispref/parsing.texi (Retrieving Node): Add manual entry for it.
* lisp/progmodes/js.el (js--treesit-imenu-1): Replace
treesit-node-top-level-p with treesit-node-top-level.
* lisp/treesit.el (treesit-node-top-level-p)
(treesit--find-top-level-match): Remove functions.
(treesit-node-top-level): New function.
* lisp/treesit.el (treesit-beginning-of-defun)
(treesit-end-of-defun): Replace treesit--find-top-level-match with
treesit-node-top-level.
2022-10-29 14:47:05 -07:00
Yuan Fu
3a784d1321
Fix parent-bol preset for tree-sitter indent
* lisp/treesit.el (treesit-simple-indent-presets): Return point rather
than column.
2022-10-29 13:05:52 -07:00
Yuan Fu
a915d836fc
Keep cursor position in treesit-indent
* lisp/treesit.el (treesit-indent): Create a marker and jump back to
that marker after indentation.
2022-10-29 11:59:45 -07:00
Yuan Fu
ce5ae0117e
Refine handling of node at point in treesit-indent
* doc/lispref/modes.texi (Parser-based Indentation): Document the
semantic clearly.
* lisp/treesit.el (treesit-simple-indent-presets): Replace with
current-indentation.
(treesit--indent-1): Now NODE always starts at BOL, and parent is
always the smallest node above NODE that spans BOL.
(treesit-indent): Don't use save-excurtion, no one else does.
2022-10-29 09:43:50 -07:00
Yuan Fu
bde8e87cb7
Fix treesit-indent-region
* lisp/treesit.el (treesit-simple-indent-presets): Remove extra comma.
(treesit--indent-1): Return (ANCHOR . OFFSET) instead of column.
(treesit-indent): Accept (ANCHOR . OFFSET) from treesit--indent-1.
(treesit--indent-region-batch-size): Reduce to 400.
(treesit-indent-region): Put a marker on each line's ANCHOR and
compute the indentation with ANCHOR + OFFSET.  Precomputing column
gives wrong indentation when the ANHOR line doesn't have correct
indent yet.
2022-10-29 09:09:09 -07:00
Yuan Fu
f46eb3d3ca
Byte-compile treesit-simple-indent-presets
* lisp/treesit.el (treesit-simple-indent-presets): Byte-compile these
functions.
2022-10-28 17:56:05 -07:00
Yuan Fu
0c7a7df98e
Add treesit-indent-region
Now indenting a large buffer is reasonably fast.

* lisp/treesit.el (treesit-indent): Move meat to treesit--indent-1.
(treesit--indent-1)
(treesit-indent-region): New functions.
(treesit-major-mode-setup): Setup indent-region-function.
2022-10-28 17:54:43 -07:00
Yuan Fu
434fc2a22e
Make Emacs compile without tree-sitter library
* lisp/treesit.el: Add function declaration forms.
* src/emacs.c: Always include syms_of_treesit.
2022-10-28 16:46:10 -07:00
Stefan Kangas
4ba76224ea Minor doc fixes in treesit.el
* lisp/treesit.el (treesit-font-lock-rules)
(treesit-font-lock-settings): Minor doc fixes.
2022-10-28 16:01:01 +02:00
Yuan Fu
007d8c4859
; * lisp/treesit.el (treesit-simple-indent-presets): Add TODO. 2022-10-28 01:15:46 -07:00
Yuan Fu
3a44069753
Remove tree-sitter-indent's integration with cc-mode
Also minor improvement of tree-sitter-indent error messaging.

* lisp/progmodes/js.el (js--treesit-indent-rules): Fix it so it works
with the new presets (which expects regexps).
(js--treesit-cc-indent-rules): Remove variable.
(js-mode): Go back to js--treesit-indent-rules.
* lisp/treesit.el (treesit-indent): Don't report error message when
ANCHOR or OFFSET is nil: let treesit-simple-indent display the
message.
(treesit-simple-indent): Remove cc-mode integration.  Display error
message when no rules matches.
2022-10-28 01:15:45 -07:00
Stefan Kangas
5868580e94 ; * lisp/treesit.el: Minor copy-edits of documentation. 2022-10-27 08:08:30 +02:00
Yuan Fu
62c8c8e51a
Plug tree-sitter-simple-indent into c-offset-alist
Now tree-sitter indentation can produce a cc-engine syntax symbol and
use c-offset-alist to compute the offset.  Catch: line-up functions
don't work with tree-sitter.

* lisp/progmodes/js.el (js--treesit-cc-indent-rules): New variable.
(js-mode): Use cc-indent rules by default.
* lisp/treesit.el (treesit-simple-indent-presets): Consider types as
regexp now.  New matchers: n-p-gp, field-is, top-level, catch-all.
New anchors: nth-sibling, grand-parent, and, or, not, list.
first-sibling now returns the actual first sibling rather than the
first named sibling.
2022-10-26 18:41:29 -07:00
Yuan Fu
5532ae81cf
* lisp/treesit.el (treesit-node-top-level-p): New argument TYPE. 2022-10-26 18:31:07 -07:00
Yuan Fu
76b86d9853
Generalize js--treesit-imenu-top-level-p
* lisp/progmodes/js.el (js--treesit-imenu-top-level-p): Move.
(js--treesit-font-lock-settings): Fix usage.
* lisp/treesit.el (treesit-node-top-level-p): Move to here.
2022-10-26 18:06:07 -07:00
Yuan Fu
734df28368
Fix tree-sitter navigation
* lisp/treesit.el (treesit-search-forward-goto): Track current
position with a local variable instead of (point).  Clean up the
condition forms.
(treesit-beginning-of-defun): Fix traverse direction.
2022-10-26 15:34:02 -07:00
Yuan Fu
5070278539
* lisp/treesit.el (treesit-simple-indent-rules): Make buffer-local. 2022-10-25 14:27:43 -07:00
Yuan Fu
e5043db2f1
Fix treesit-language-at
* lisp/treesit.el (treesit-language-at-point-function): New local
variable.
(treesit-language-at): Change to use
treesit-language-at-point-function rather than trying each parser
one-by-one.
* doc/lispref/parsing.texi (Multiple Languages): Update manual.
2022-10-25 14:26:41 -07:00
Yuan Fu
7c5d434833
New tree-sitter toggle scheme
This version: central variable, everything controlled by
treesit-settings.  Major mode sets up tree-sitter/non-tree-sitter
in a conditional branch, based on the setting.

* lisp/treesit.el (treesit-settings): New option.
(treesit-defun-type-regexp): Change docstring.
(treesit-mode-supported)
(treesit-required-languages)
(treesit--local-variable-backup): Remove variables.
(treesit--backup-local-variable)
(treesit-mode)
(global-treesit-mode--turn-on)
(global-treesit-mode): Remove functions.
(treesit--setting-for-mode): New function.
(treesit-ready-p): New argument MODE, changed REPORT to QUIET, and
LANGUAGEs to LANGUAGE (now it can be a single symbol or a list of
them).
(treesit-major-mode-setup): New function.  Mostly comes from
treesit-mode.

* test/src/treesit-tests.el (treesit-misc): New test.

* lisp/progmodes/python.el (python-mode): Move some setup code into
the conditional branch at the end.

* lisp/progmodes/js.el (js-json-mode)
(js-mode): Move some setup code into the conditional branch at the
end.

* lisp/progmodes/ts-mode.el: Move tree-sitter setup into the
conditional branch.
2022-10-25 14:07:47 -07:00
Yuan Fu
6203b9db34
* lisp/treesit.el (treesit-node-at): Remove argument LARGEST.
LARGEST returns the root node at BOB, finding the smallest node at
point and search upward is a better practice.
2022-10-23 18:54:37 -07:00
Yuan Fu
1787e651d3
* lisp/treesit.el (treesit-parser-range-on): New function. 2022-10-23 18:51:35 -07:00
Yuan Fu
c120b0ebdb
Fix treesit-beginning/end-of-defun
Now they always move to the top-level defun (i.e., skips nested ones).

* lisp/treesit.el (treesit--find-top-level-match): New function
(treesit-beginning-of-defun): Goes to the top-level match.
(treesit-end-of-defun): Simply move to the end of current match.
2022-10-23 18:47:02 -07:00
Yuan Fu
6cf2a9c55d
Make treesit-search-forward-goto accept a NODE argument
With NODE argument we can do

(setq node (treesit-search-forward-goto node))

And we can choose what node to pass to it (maybe we want to pass it
the largest node at point, rather than the smallest node, and in case
of multiple parsers, we can choose which parser to use).

* doc/lispref/parsing.texi (Retrieving Node): Update manual.
* lisp/treesit.el (treesit-search-forward-goto): Accept a NODE
argument.
2022-10-23 18:29:02 -07:00
Yuan Fu
524d10c86b
Make treesit-node-at return the last leaf node at EOB
* doc/lispref/parsing.texi (Retrieving Node): Update manual.
* lisp/treesit.el (treesit-node-at): Add a (when node) form just to be
explicit, return the last leaf node if there is no node after POS.
2022-10-23 18:23:17 -07:00
Yuan Fu
e45d8dbeea
; * lisp/treesit.el (treesit-range-functions): Fix typo. 2022-10-23 16:45:54 -07:00
Yuan Fu
3114eb8078
* lisp/treesit.el (treesit-node-at): Add LARGEST argument. 2022-10-23 16:16:27 -07:00
Yuan Fu
fa9bede36c
Change function signiture of treesit search functions
Justification: We want to make the SIDE argument in
treesit-search-forward-goto optional, so I changed it to START.
It makes more sense for BACKWARD to follow START so two common
case, search for end forward and beginning backwards are

(treesit-search-forward-goto node pred)
(treesit-search-forward-goto node pred t t)

Then since we swapped BACKWARD and ALL for
treesit-search-forward-goto, it's best to swap them for
treesit-search-forward and treesit-search-subtree, too.  And BACKWARD
will probably be used more frequently than ALL anyway.

* doc/lispref/parsing.texi (Retrieving Node): Resolve FIXME and update
function signitures.
* lisp/treesit.el (treesit-search-forward-goto): Change SIDE to
START, swap BACKWARD and ALL.
(treesit-beginning-of-defun)
(treesit-end-of-defun): Update use of treesit-search-forward-goto
* src/treesit.c (Ftreesit_search_subtree)
(Ftreesit_search_forward): Swap BACKWARD and ALL.
2022-10-23 15:19:00 -07:00
Yuan Fu
57b904f4ba
Fix infinite loop in treesit-search-forward-goto
* lisp/treesit.el (treesit-search-forward-goto): Remove UP argument.
* src/treesit.c (treesit_traverse_child_helper): New function.
(treesit_search_forward): Remove UP_ONLY and SKIP_START argument.
Don't traverse subtree of START.  And after we've found the next
sibling/parent, go down to the first leaf node.  Also change recursion
to loop.
(Ftreesit_search_forward): Change docstring, remove UP argument.
2022-10-22 22:50:56 -07:00
Eli Zaretskii
6f28810f6b Clean up tree-sitter sections of the ELisp manual
* doc/lispref/parsing.texi (Parsing Program Source):
* doc/lispref/modes.texi (Font Lock Mode)
(Parser-based Font Lock): Fix wording, punctuation, and markup.
Add index entries.

* lisp/treesit.el (treesit-node-at, treesit-language-at): Rename
argument POINT to POS.
2022-10-22 18:48:42 +03:00
Yuan Fu
7c750343be
* lisp/treesit.el (treesit-font-lock-enable): Add function back. 2022-10-21 18:50:24 -07:00
Yuan Fu
5159789e55
Revise the toggle scheme of tree-sitter (again)
Now instead of a toggle function (major-mode-backend-function), we let
major mode set local variables like treesit-font-lock-settings,
treesit-imenu-function, then treesit-mode takes care of activating
those things (clearing font-lock-keywords, setting
imenu-create-index-function to treesit-imenu-function, etc).

js.el and python.el: I've returned js-mode and python-mode to exactly
what they were before tree-sitter change, plus lines at the end
setting up tree-sitter variables.  Sorry about all these fuss :-D

* lisp/treesit.el (treesit-mode-inhibit-message): Remove option.
(major-mode-backend-function)
(treesit-remapped-major-mode-alist): Remove variables.
(treesit-mode): Move down to the end of buffer.  Do more things.
(global-treesit-mode): Move down to the end of buffer.  Don't handle
major-mode-remap-alist anymore.
(global-treesit-mode--turn-on): Move down to the end of buffer.
(treesit-ready-p): Move down to the end of buffer.  Changed signature.
(treesit-font-lock-enable): Remove function.
(treesit-defun-type-regexp): New variable.
(treesit-beginning-of-defun)
(treesit-end-of-defun): New function.
(treesit-imenu-function): New variable.
(treesit-mode-supported)
(treesit-required-languages)
(treesit--local-variable-backup): New variables.
(treesit--backup-local-variable): New function

* lisp/progmodes/js.el (js-use-tree-sitter): Remove option.
(js--treesit-defun-type-regexp): Remove variable.  (Now set inline in
js-mode.)
(js--treesit-beginning-of-defun)
(js--treesit-end-of-defun): Remove functions.  (Now use
treesit-beginning/end-of-defun.)
(js--backend-toggle)
(js--json-backend-toggle): Remove function.
(js-mode)
(js-json-mode): Restore back to before tree-sitter changes.  Add
tree-sitter setup at the end.

* lisp/progmodes/python.el (python--backend-toggle): Remove function.
(python-mode): Restore back to before tree-sitter changes.  Add
tree-sitter setup at the end.

* lisp/progmodes/ts-mode.el (ts-mode--font-lock-settings): Use
js--fontify-template-string.
(ts-mode--fontify-template-string): Remove function (because we can
just use js--fontify-template-string).
(ts-mode--defun-type-regexp): Remove variable (now set inline in
ts-mode).
(ts-mode--beginning-of-defun)
(ts-mode--end-of-defun): Remove functions (now using
treesit-beginning/end-of-defun).
(ts-mode): Setup tree-sitter variables and then turn on treesit-mode
or move to js-mode.
2022-10-19 16:44:04 -07:00