* src/lisp.h: Do not include stdalign.h, as Gnulib now
approximates C23 (where stdalign.h does nothing) and config.h
already includes stdalign.h as needed on pre-C23 platforms.
When Emacs is compiled with AddressSanitizer support, enable
poisoning/unpoisoning freed/unused Lisp objects and other internal
memory management structures. If enabled, this will mark freed bytes
that have been put on free lists for future use and initially
allocated memory blocks/chunks as "poisoned", triggering an ASan error
if they are accessed improperly. Structures are unpoisoned when they
have been taken off their respective free lists. Additionally, add
optional macros for performing unaligned loads, which when enabled by
defining USE_SANITIZER_UNALIGNED_LOAD will use ASan provided functions
for loading from unaligned addresses, which may help catch bugs that
AddressSanitizer might otherwise miss.
* configure.ac: Check for the existence of address and common
sanitizer API headers.
* src/lisp.h (UNALIGNED_LOAD_SIZE): New macro. If enabled, and the
necessary sanitizer API is available, define it to
__sanitizer_unaligned_load(64|32) depending on the word size of the
architecture.
* src/fns.c [HAVE_FAST_UNALIGNED_ACCESS] (Fstring_lessp): Use
'UNALIGNED_LOAD_SIZE' to perform unaligned loads from the two strings.
* src/alloc.c (ASAN_POISON_ABLOCK, ASAN_UNPOISON_ABLOCK)
(ASAN_POISON_INTERVAL_BLOCK, ASAN_UNPOISON_INTERVAL_BLOCK)
(ASAN_POISON_INTERVAL, ASAN_UNPOISON_INTERVAL)
(ASAN_PREPARE_DEAD_SDATA, ASAN_PREPARE_LIVE_SDATA)
(ASAN_POISON_SBLOCK_DATA, ASAN_POISON_STRING_BLOCK)
(ASAN_UNPOISON_STRING_BLOCK, ASAN_POISON_STRING)
(ASAN_UNPOISON_STRING, ASAN_POISON_FLOAT_BLOCK)
(ASAN_UNPOISON_FLOAT_BLOCK, ASAN_POISON_FLOAT)
(ASAN_UNPOISON_FLOAT, ASAN_POISON_CONS_BLOCK)
(ASAN_POISON_CONS, ASAN_UNPOISON_CONS)
(ASAN_POISON_VECTOR_CONTENTS, ASAN_UNPOISON_VECTOR_CONTENTS)
(ASAN_UNPOISON_VECTOR_BLOCK, ASAN_POISON_SYMBOL_BLOCK)
(ASAN_UNPOISON_SYMBOL_BLOCK, ASAN_POISON_SYMBOL)
(ASAN_UNPOISON_SYMBOL) [ADDRESS_SANITIZER]: New macros. When
address sanitization is enabled, define them to poison/unpoison
objects.
(lisp_align_malloc): Poison newly allocated blocks on `free_ablock',
unpoison ablocks taken from it respectively.
(lisp_align_free): Poison individual ablocks when they are put on the
free list, unpoison them when an entire `ablocks' chunk is being
freed.
(make_interval): Poison interval blocks on initial allocation,
unpoison individual intervals on allocation and removal from
`interval_free_list'.
(sweep_intervals): Unpoison interval blocks before sweeping, poison
dead/unmarked intervals.
(allocate_string): Poison string blocks on initial allocation,
unpoison Lisp_Strings on removal from the free list.
(allocate_string_data): Poison `sblock' data on initial allocation,
unpoison individual `sdata' contents on allocation or removal from the
free list. Call `ASAN_PREPARE_LIVE_SDATA' on the new `sdata' struct.
(sweep_strings): Unpoison string blocks before sweeping them,
poisoning dead strings and their sdata afterwards.
(compact_small_strings): Call `ASAN_PREPARE_LIVE_DATA' on the `sdata'
to where compacted strings to moved to.
(pin_string): Call `ASAN_PREPARE_DEAD_SDATA' on `old_sdata'.
(make_float): Poison float blocks on allocation, unpoisoning
individual Lisp_Floats on allocation or removal from
`float_free_list'.
(sweep_floats): Unpoison float blocks before sweeping, poison
dead/unmarked floats.
(free_cons): Poison `ptr'.
(Fcons): Poison cons blocks on allocation, unpoisoning individual
Lisp_Cons on allocation or removal from `cons_free_list'.
(sweep_conses): Poison dead/unmarked conses.
(setup_free_list): Poison vectors put on `vector_free_lists'.
(allocate_vector_from_block): Unpoison vectors taken from the free
list, poison excess vector bytes when vectors allocated from the free
list are larger than requested.
(sweep_vectors): Unpoison vector blocks before sweeping them.
(Fmake_symbol): Poison symbol blocks on initial allocation,
unpoisoning individual Lisp_Symbols on allocation or removal from
`symbol_free_list'.
(sweep_symbols): Unpoison symbol blocks before sweeping, poisoning
dead/unmarked symbols.
(live_string_holding, live_cons_holding, live_symbol_holding)
(live_float_holding): When compiling with address sanitization and GC
poisoning enabled, return NULL if the passed address is poisoned, or
if the Lisp object it resides in is poisoned, avoiding a
use-after-poison trigger if these functions are called on a pointer
that might be referring to a now dead/swept object.
* etc/DEBUG: Add information about enabling ASan memory poisoning.
* src/editfns.c: (narrowing_locks): New alist to hold the narrowing
locks and their buffers.
(narrowing_lock_get_bound, narrowing_lock_peek_tag)
(narrowing_lock_push, narrowing_lock_pop): New functions to access
and update 'narrowing_locks'.
(reset_outermost_narrowings, unwind_reset_outermost_narrowing):
Functions moved from src/xdisp.c, and rewritten with the above
functions.
(Fwiden): Use the above functions. Update docstring.
(Fnarrow_to_region, Fnarrowing_lock, Fnarrowing_unlock): Use the above
functions.
(syms_of_editfns): Remove the 'narrowing-locks' variable.
* src/lisp.h: Make 'reset_outermost_narrowings' externally visible.
* src/xdisp.c (reset_outermost_narrowings)
unwind_reset_outermost_narrowing): Functions moved to src/editfns.c.
* lisp/subr.el (with-locked-narrowing): Improved macro, with a helper
function.
Generalize fast_c_string_match_ignore_case into
fast_c_string_match_internal. And Make fast_c_string and
fast_c_string_match_ignore_case use fast_c_string_match_internal.
* src/lisp.h (fast_c_string_match_internal)
(fast_c_string_match): New declaration.
(fast_c_string_match_ignore_case): Change to thin wrapper.
* src/search.c (fast_c_string_match_internal): New function.
The most obvious problem was the lack of support for
`insert-before-markers`, but the behavior was also different in a few
other cases.
* src/itree.h (itree_insert_gap):
* src/itree.c (itree_insert_gap): Add `before_markers` arg.
* src/lisp.h (adjust_overlays_for_insert):
* src/buffer.c (adjust_overlays_for_insert): Add `before_markers` arg.
* src/insdel.c (adjust_markers_for_replace, adjust_markers_for_insert)
(adjust_markers_for_delete): Adjust overlays directly from here.
(insert_1_both, insert_from_string_1, insert_from_gap)
(insert_from_buffer_1, adjust_after_replace, replace_range)
(replace_range_2, del_range_2): Don't adjust overlays explicitly here
any more.
* test/src/buffer-tests.el (test-overlay-insert-before-markers-empty)
(test-overlay-insert-before-markers-non-empty): New tests.
Problem reported by Tino Calancha (Bug#58960).
* src/callproc.c (call_process): Initialize SIGCHLD handling
before possibly creating a child with emacs_span. This need not
be in the critical section that calls emacs_spawn, so do it
outside the critical section.
* src/process.c (child_signal_init): Now extern.
This fixes bug #58739. Make subr-arity return, e.g., (12 . 12) rather than
(12 . many) for a function with a fixed number of arguments more than 8.
* lisp/emacs-lisp/comp.el (comp-prepare-args-for-top-level): Only return a cdr
of 'many when there are &rest arguments.
* src/eval.c (eval_sub): Also check for a fixed number of args over 8 when
using the nargs + *args calling convention.
(funcall_subr): Also check numargs <= 8 before using the fixed args calling
convention. Include the case numargs > 8 in the aMany calling convention.
* src/lisp.h (DEFUN): Amend the comment about MANY.
This removes the HAVE_PDUMPER || HAVE_UNEXEC condition
of gflags.will_dump_, which gets in the way when Emacs
is built without any dumping method. (Bug#58708)
* src/frame.c (do_switch_frame): Move call to `get-mru-window` outside
of the "critical region" where (selected-window) != (frame-selected-window).
* src/minibuf.c (move_minibuffers_onto_frame): Add arg `frame`.
Use it instead of `selected_frame`. Adjust all callers.
* src/lisp.h (move_minibuffers_onto_frame): Adjust declaration.
Move args between `build_overlay` and `add_buffer_overlay`,
to try and keep buffer positions together with their buffer.
Be more strict in the `otick` values passed to `interval_tree_insert`.
Move a few things around to try and reduce dependencies through `.h` files.
Fix a thinko bug in `check_tree`.
* src/alloc.c (build_overlay): Remove `begin` and `end` args.
* src/buffer.c (add_buffer_overlay): Move from `buffer.h`.
Add `begin` and `end` args.
(copy_overlays): Adjust accordingly.
(Fmake_overlay): Use BUF_BEG and BUF_Z; adjust call to `build_overlay`
and `add_buffer_overlay`.
(Fmove_overlay): Use BUF_BEG and BUF_Z; Use the new `begin` and `end`
args of `add_buffer_overlay` so we don't need to use
`interval_node_set_region` when moving to a new buffer.
(remove_buffer_overlay, set_overlay_region): Move from `buffer.h`.
* src/buffer.h (set_overlay_region, add_buffer_overlay)
(remove_buffer_overlay): Move to `buffer.c`.
(build_overlay): Move from `lisp.h`.
(maybe_alloc_buffer_overlays): Delete function (inline into its only
caller).
* src/itree.c (interval_tree_insert): Move declaration `from buffer.h`.
(check_tree): Fix initial offset in call to `recurse_check_tree`.
Remove redundant check of the `limit` value.
(interval_node_init): Remove `begin` and `end` args.
(interval_tree_insert): Mark it as static.
Assert that the new node's `otick` should already be uptodate and its
new parent as well.
(itree_insert_node): New function.
(interval_tree_insert_gap): Assert the otick of the removed+added nodes
were uptodate and mark them as uptodate again after adjusting
their positions.
(interval_tree_inherit_offset): Check that the parent is at least as
uptodate as the child.
* src/lisp.h (build_overlay): Move to `buffer.h`.
* src/itree.h (interval_node_init): Adjust accordingly.
(interval_tree_insert): Remove declaration.
(itree_insert_node): New declaration.
* src/lisp.h (alignas): C23 is removing the __alignas_is_defined
macro, so do not rely on it. Instead, do not define alignas if C23
or later, or if C++11 or later.
* src/editfns.c (Fnarrowing_lock, Fnarrowing_unlock,
narrow_to_region_locked, unwind_narrow_to_region_locked):
New functions.
(Fnarrow_to_region, Fwiden): Adapt, and make it possible to use
these functions within the bounds of the locked narrowing.
(syms_of_editfns): Change the name of the variable, make it
buffer-local, and add the two Snarrowing_lock and Snarrowing_unlock
subroutines.
* src/lisp.h: Prototype of 'narrow_to_region_locked'.
* src/xdisp.c (handle_fontified_prop):
* src/keyboard.c (safe_run_hooks_maybe_narrowed): Use
'narrow_to_region_locked'.
* lisp/subr.el (with-locked-narrowing): New macro.
Setting backtrace-on-redisplay-error to non-nil enables the generation of a
Lisp backtrace in buffer *Redisplay-trace* following an error in Lisp called
from redisplay.
* doc/lispref/debugging.texi (Debugging Redisplay): New subsection.
(Error Debugging): Reference to the new subsection.
* etc/NEWS: New entry for the new facility.
* src/eval.c (redisplay_deep_handler): New variable.
(init_eval): Initialize redisplay_deep_handler.
(call_debugger): Don't throw to top-level after calling debug-early
(internal_condition_case_n): "Bind" redisplay_deep_handler to the current
handler.
(backtrace_yet): New boolean variable.
(signal_or_quit): New code section to handle Lisp errors occurring in
redisplay.
(syms_of_eval): New DEFVAR_BOOL backtrace-on-redisplay-error.
* src/keyboard.c (command_loop_1): Set backtrace_yet to false each time around
the loop.
(safe_run_hooks_error): Allow args to be up to four Lisp_Objects long.
(safe_run_hooks_2): New function.
* src/lisp.h (top level): declare as externs backtrace_yet and
safe_run_hooks_2.
* src/xdisp.c (run_window_scroll_functions): Replace a call to
run_hook_with_args_2 with one to safe_run_hooks_2.
* src/lisp.h (emacs_spawn):
* src/callproc.c (emacs_spawn): Add PTY_IN and PTY_OUT arguments to
specify which streams should be set up as a PTY.
(call_process): Adjust call to 'emacs_spawn'.
* src/process.h (Lisp_Process): Replace 'pty_flag' with 'pty_in' and
'pty_out'.
* src/process.c (is_pty_from_symbol): New function.
(make-process): Allow :connection-type to be a cons cell, and allow
using a stderr process with a PTY for stdin/stdout.
(create_process): Handle creating a process where only one of stdin or
stdout is a PTY.
* lisp/eshell/esh-proc.el (eshell-needs-pipe, eshell-needs-pipe-p):
Remove.
(eshell-gather-process-output): Use 'make-process' and set
':connection-type' as needed by the value of 'eshell-in-pipeline-p'.
* lisp/net/tramp.el (tramp-handle-make-process):
* lisp/net/tramp-adb.el (tramp-adb-handle-make-process):
* lisp/net/tramp-sh.el (tramp-sh-handle-make-process): Don't signal an
error when ':connection-type' is a cons cell.
* test/src/process-tests.el
(process-test-sentinel-wait-function-working-p): Allow passing PROC
in, and rework into...
(process-test-wait-for-sentinel): ... this.
(process-test-sentinel-accept-process-output)
(process-test-sentinel-sit-for, process-test-quoted-batfile)
(process-test-stderr-filter): Use 'process-test-wait-for-sentinel'.
(make/process/test-connection-type): New function.
(make-process/connection-type/pty, make-process/connection-type/pty-2)
(make-process/connection-type/pipe)
(make-process/connection-type/pipe-2)
(make-process/connection-type/in-pty)
(make-process/connection-type/out-pty)
(make-process/connection-type/pty-with-stderr-buffer)
(make-process/connection-type/out-pty-with-stderr-buffer): New tests.
* test/lisp/eshell/esh-proc-tests.el (esh-proc-test--detect-pty-cmd):
New variable.
(esh-proc-test/pipeline-connection-type/no-pipeline)
(esh-proc-test/pipeline-connection-type/first)
(esh-proc-test/pipeline-connection-type/middle)
(esh-proc-test/pipeline-connection-type/last): New tests.
* doc/lispref/processes.texi (Asynchronous Processes): Document new
':connection-type' behavior.
(Output from Processes): Remove caveat about ':stderr' forcing
'make-process' to use pipes.
* etc/NEWS: Announce this change (bug#56025).
* src/editfns.c (narrow_to_region_internal): New function, which
contains the body previously in 'Fnarrow_to_region' but accepts
a third argument.
(Fnarrow_to_region): Use the new function. Update the docstring.
(Fwiden): Update the docstring.
* src/lisp.h: Prototype of the new function.
* src/xdisp.c (handle_fontified_prop): Use the new function instead
of 'Fnarrow_to_region'.
* src/process.c (Finternal_default_process_filter):
* src/lread.c (readevalloop): Remove the third argument to
'Fnarrow_to_region'.
* src/bytecode.c (exec_byte_code):
* lisp/emacs-lisp/comp.el (comp-limplify-lap-inst):
* lisp/emacs-lisp/bytecomp.el: Restore the statu quo ante.
* etc/NEWS: Remove the entry about the new optional argument.
* doc/lispref/positions.texi (Narrowing): Update the documentation.
* src/keyboard.c (safe_run_hooks_maybe_narrowed): New function.
(command_loop_1): Use it for 'pre-command-hook' and 'post-command-hook'.
(syms_of_keyboard): Update docstrings of 'pre-command-hook' and
'post-command-hook'.
* src/lisp.h: Prototype of the new function.
* src/xdisp.c (redisplay_window): Increase the threshold above which
long lines detection is performed in the buffer. This should avoid
triggering that detection for most simple editing operations.
* src/lisp.h (modiff_incr): Explain why the counter is incremented
logarithmically.
* src/buffer.h (struct buffer_text): Adapt the comment about the
'modiff' field accordingly.
* src/buffer.c (modify_overlay): Increase the counter by 1 instead of
the size of the buffer section on which the overlay is placed.
* etc/NEWS: Small improvement.
* src/lisp.h (modiff_incr): Add a parameter to 'modiff_incr' to record
the extent of the modification.
* src/insdel.c (insert_1_both, insert_from_string_1, insert_from_gap,
insert_from_buffer_1, adjust_after_replace, replace_range,
replace_range_2, del_range_2, modify_text): Add an argument to each
call to 'modiff_incr'.
* src/textprop.c (modify_text_properties): Add an argument to the call
to 'modiff_incr'.
* src/buffer.c (Frestore_buffer_modified_p, Fbuffer_swap_text,
modify_overlay): Add an argument to each call to 'modiff_incr'.
* src/xdisp.c (redisplay_window): Use the improved version of 'MODIFF'
for the heuristic.
* src/buffer.h (struct buffer): New field 'long_line_optimizations_p'.
* src/buffer.c (syms_of_buffer): New variable 'long-line-threshold'.
(reset_buffer): Initialize the 'long_line_optimizations_p' field.
(Fbuffer_swap_text): Handle it.
* src/xdisp.c (redisplay_window): Set 'long_line_optimizations_p' when
a buffer contains long lines.
(init_iterator): Use 'long_line_optimizations_p'.
(get_narrowed_begv): Update.
(SET_WITH_NARROWED_BEGV): New macro.
(unwind_narrowed_begv): New internal function used by the new macro.
(back_to_previous_line_start, get_visually_first_element,
move_it_vertically_backward): Use the new macro.
* src/search.c (find_newline1): Make it externally visible.
* src/lisp.h: Make 'find_newline1' externally visible.
* src/dispextern.h (struct it): Update comment. Remove the
'WITH_NARROWED_BEGV' macro.
* etc/NEWS: Mention the 'long-line-threshold' variable.
* src/lisp.h (STRING_SET_MULTIBYTE): Make it into a function.
* src/composite.c (Fcomposition_get_gstring):
Prefer `make_multibyte_string` over Fconcat+STRING_SET_MULTIBYTE.
* lisp/simple.el (primitive-undo): If the visited-modtime of the
indirect buffer's file is bogus, use the modtime of the file
visited by its base buffer.
* src/undo.c (record_first_change): Call
'buffer_visited_file_modtime' with the correct buffer, instead of
always calling 'Fvisited_file_modtime', which returns possibly
bogus values for indirect buffers.
* src/fileio.c (Fset_visited_file_modtime): Signal a meaningful
error for indirect buffers.
(buffer_visited_file_modtime): New function, with implementation
taken from 'Fvisited_file_modtime'.
(Fvisited_file_modtime): Call 'buffer_visited_file_modtime'.
* src/lisp.h: Add prototype for 'buffer_visited_file_modtime'.
(Bug#56397)
* src/lisp.h (lisp_h_BASE2_EQ, BASE2_EQ): New macros and functions.
* src/timefns.c (tzlookup, Fdecode_time): Use them.
(Ftime_convert): Convert to symbol once, instead of many times.
No intergration/interaction with the new type, just adding it.
* lisp/emacs-lisp/cl-preloaded.el (cl--typeof-types): Add new type.
* src/alloc.c (cleanup_vector): Add gc for the new type.
* src/data.c (Ftype_of): Add switch case for the new type.
(syms_of_data): Add symbols for the new type.
* src/lisp.h (DEFINE_GDB_SYMBOL_BEGIN): Add new type.
* src/treesit.c (Ftreesit_compiled_query_p): New function.
(syms_of_treesit): Add symbol for the new type.
* src/treesit.h (struct Lisp_TS_Query): New struct.
(TS_COMPILED_QUERY_P, XTS_COMPILED_QUERY, CHECK_TS_COMPILED_QUERY):
New macros.
* src/print.c (print_vectorlike): Add printing for the new type.
This fixes errors caused by invalid error traps being left on
the error handler stack if an IO error causes a non-local exit
out of the protected code, and another crash caused by
delete_frame trying to read async input.
* src/eval.c (unwind_to_catch, push_handler_nosignal): Save and
restore the X error handler stack.
* src/lisp.h (struct handler): [HAVE_X_WINDOWS]: New field
`x_error_handler_depth'.
* src/xterm.c (struct x_error_message_stack): Make string a
regular string.
(x_unwind_errors_to): New function.
(x_error_catcher, x_catch_errors_with_handler)
(x_uncatch_errors_after_check, x_uncatch_errors): Update the
stack depth.
(x_check_errors): Stop manually unwinding since unwind_to_catch
now does that for us.
(x_had_errors_p, x_clear_errors): Update for new type of
`string'.
(x_connection_closed): Block input between just before
delete_frame to when the terminal is unlinked.
* src/xterm.h: Update prototypes.
* src/lisp.h (CHAR_TABLE_REF_ASCII): Refactor as a straightforward
for-loop. Redo an if-then-else to be an (!if)-else-then as this
is a bit cleaner, and it also works around GCC bug 105755.
Restructure the reader to be nonrecursive so that it is not limited by
the C stack or crashes Emacs when reading deeply nested data.
This also improves performance.
A few minor bugs were fixed:
- (a .{NBSP}b) where {NBSP} is a non-breaking space (U+00A0) is now
the dotted pair (a . b), not the 3-element list (a \. b), since U+00A0
is treated as whitespace everywhere else.
- #_ with no symbol following is now equivalent to ## (empty interned
symbol), not #: (empty uninterned symbol).
* src/alloc.c (garbage_collect): Call mark_lread.
* src/lread.c (readevalloop): Use read0 instead of read_list.
(stackbufsize): Increase to 1024, now that read0 isn't recursive.
(invalid_radix_integer): Buffer overflow check.
(read1, read_list, read_vector): Remove.
(read_char_literal, read_string_literal)
(hash_table_from_plist, record_from_list, vector_from_rev_list)
(bytecode_from_rev_list, char_table_from_rev_list)
(sub_char_table_from_rev_list, string_props_from_rev_list)
(read_bool_vector, skip_lazy_string, symbol_char_span)
(skip_space_and_comments)
(enum read_entry_type, struct read_stack_entry, struct read_stack)
(rdstack, mark_lread, read_stack_top, read_stack_pop)
(read_stack_empty_p, grow_read_stack, read_stack_push): New.
(read0): Rewrite to be nonrecursive.
* test/src/lread-tests.el (lread-deeply-nested, lread-misc): New tests.
