`struct composition` kept an index into the internal `key_and_value` array
of hash tables, which only worked because of details of how
hash-tables are handled. Replace it with a reference to the
key stored at that location in the hash-table, which saves us an
indirection while at it.
* src/composite.h (struct composition): Replace `hash_index` with
the actual `key`.
(COMPOSITION_KEY): Simplify accordingly.
(mark_composite): Declare.
* src/composite.c (get_composition_id): Adjust accordingly.
(mark_composite): New function.
* src/charset.c (mark_charset): Uncomment.
* src/lisp.h (mark_charset): Declare.
* src/alloc.c (garbage_collect): Call `mark_charset` and `mark_composite`.
* src/pdumper.c (hash_table_contents): Remove invalid comment, since
compositions aren't dumped.
This removes hacks from code that had to be careful not to use
Qunbound as a hash table key, at the cost of a minor hack in
the GC marker.
* src/lisp.h (INVALID_LISP_VALUE, HASH_UNUSED_ENTRY_KEY):
Define as a null-pointer float.
* src/alloc.c (process_mark_stack): Add hack to ignore that value.
* src/pdumper.c (dump_object_needs_dumping_p)
(pdumper_init_symbol_unbound, pdumper_load):
* src/print.c (PRINT_CIRCLE_CANDIDATE_P): Remove hacks for Qunbound.
Be more systematic about putting space before paren in calls,
and in avoiding unnecessary parentheses in macros.
This was partly inspired by my wading through gcc -E output
while debugging something else, and seeing too many parens.
This patch does not change the generated .o files on my platform.
This saves several words in the hash table object at the cost of an
indirection at runtime. This seems to be a gain in overall
performance.
FIXME: We cache hash test objects in a rather clumsy way. A better
solution is sought.
* src/lisp.h (struct Lisp_Hash_Table): Use a pointer to the test
struct. All references adapted.
* src/alloc.c (garbage_collect):
* src/fns.c (struct hash_table_user_test, hash_table_user_tests)
(mark_fns, get_hash_table_user_test): New state for caching test
structs, and functions managing it.
Using xmalloc for allocating these arrays is much cheaper than using
Lisp vectors since they are no longer marked or swept by the GC, and
deallocated much sooner. This makes GC faster and less frequent, and
improves temporal locality.
Zero-sized tables use NULL for their (0-length) vectors except the
index vector which has size 1 and uses a shared constant static vector
since it cannot be modified anyway. This makes creation and
destruction of zero-sized hash tables very fast; they consume no
memory outside the base object.
* src/lisp.h (struct Lisp_Hash_Table): Retype the index, next, hash
and key_and_value vectors from Lisp_Object to appropriately typed
arrays (although hash values are still stored as Lisp fixnums). Add
explicit table_size and index_size members. All users updated.
* src/alloc.c (gcstat): Add total_hash_table_bytes.
(hash_table_allocated_bytes): New.
(cleanup_vector): Free hash table vectors when sweeping
the object.
(hash_table_alloc_bytes, hash_table_free_bytes): New.
(sweep_vectors): Update gcstat.total_hash_table_bytes.
(total_bytes_of_live_objects): Use it.
(purecopy_hash_table): Adapt allocation of hash table vectors.
(process_mark_stack): No more Lisp slots in the struct to trace.
* src/fns.c (empty_hash_index_vector): New.
(allocate_hash_table): Allocate without automatically GCed slots.
(alloc_larger_vector): Remove.
(make_hash_table, copy_hash_table, maybe_resize_hash_table):
Adapt vector allocation and initialisation.
* src/pdumper.c (hash_table_freeze, hash_table_thaw, dump_hash_table)
(dump_hash_table_contents):
Adapt dumping and loading to field changes.
This avoids any extra allocation for such vectors, including empty
tables read by the Lisp reader, and provides extra safety essentially
for free.
* src/fns.c (make_hash_table): Allow tables to be 0-sized. The index
will always have at least one entry, to avoid extra look-up costs.
* src/alloc.c (process_mark_stack): Don't mark pure objects,
because empty vectors are pure.
This takes less space (saves an entire word) and is more type-safe.
No change in behaviour.
* src/lisp.h (hash_table_weakness_t): New.
(struct Lisp_Hash_Table): Replace Lisp object `weak` with enum
`weakness`.
* src/fns.c
(keep_entry_p, hash_table_weakness_symbol): New.
(make_hash_table): Retype argument. All callers updated.
(sweep_weak_table, Fmake_hash_table, Fhash_table_weakness):
* src/alloc.c (purecopy_hash_table, purecopy, process_mark_stack):
* src/pdumper.c (dump_hash_table):
* src/print.c (print_object): Use retyped field.
The profiler stored data being collected in Lisp hash tables but
relied heavily on their exact internal representation, which made it
difficult and error-prone to change the hash table implementation.
In particular, the profiler has special run-time requirements that are
not easily met using standard Lisp data structures: accesses and
updates are made from async signal handlers in almost any messy
context you can think of and are therefore very constrained in what
they can do.
The new profiler tables are designed specifically for their purpose
and are more efficient and, by not being coupled to Lisp hash tables,
easier to keep safe.
The old profiler morphed internal hash tables to ones usable from Lisp
and thereby made them impossible to use internally; now export_log
just makes new hash table objects for Lisp. The Lisp part of the
profiler remains entirely unchanged.
* src/alloc.c (garbage_collect): Mark profiler tables.
* src/eval.c (get_backtrace): Fill an array of Lisp values instead of
a Lisp vector.
* src/profiler.c (log_t): No longer a Lisp hash table but a custom
data structure: a fully associative fixed-sized cache that maps
fixed-size arrays of Lisp objects to counts.
(make_log): Build new struct.
(mark_log, free_log, get_log_count, set_log_count, get_key_vector)
(log_hash_index, remove_log_entry, trace_equal, trace_hash)
(make_profiler_log, free_profiler_log, mark_profiler): New.
(cmpfn_profiler, hashtest_profiler, hashfn_profiler)
(syms_of_profiler_for_pdumper): Remove.
(approximate_median, evict_lower_half, record_backtrace, export_log)
(Fprofiler_cpu_log, Fprofiler_memory_log, syms_of_profiler):
Adapt to the new data structure.
This code was introduced in 2014 to catch a GC bug that, according to
Paul Eggert in 2019, "seems to have been fixed" (see 2b552f3489
2019-08-21 "Don’t debug fset by default"). It has been marked
obsolete since that time, and no one has mentioned it on our mailing
lists since. Let's just get rid of it.
* src/alloc.c
(SUSPICIOUS_OBJECT_CHECKING) [ENABLE_CHECKING]: Don't define.
(suspicious_free_record, suspicious_objects, suspicious_object_index)
(suspicious_free_history, suspicious_free_history_index)
(note_suspicious_free) [SUSPICIOUS_OBJECT_CHECKING]: Delete.
(find_suspicious_object_in_range)
(detect_suspicious_free): Delete functions.
(cleanup_vector)
(allocate_vectorlike): Don't call above deleted functions.
(Fsuspicious_object): Delete DEFUN.
(syms_of_alloc) <Ssuspicious_object>: Delete defsubr.
Instead of scanning vector_free_lists from the appropriate size until
we find a nonempty bucket, start at the last bucket where we last put
something in. This may favour splitting larger vectors than necessary
but in general saves a lot of time in the allocation of small vectors.
Original patch by Ihor Radchenko.
* src/alloc.c (last_inserted_vector_free_idx): New variable.
(setup_on_free_list): Set it.
(allocate_vector_from_block): Use it.
(sweep_vectors): Reset it.
The latter half of vector_free_lists was never used in any meaningful
way but it did require traversal during allocation and GC. Reduce it
to sizes we actually allocate, with a bucket for bigger ones.
* src/alloc.c (VECTOR_MAX_FREE_LIST_INDEX): Rename to...
(VECTOR_FREE_LIST_ARRAY_SIZE): ... this and adjust its value.
(vector_free_lists): Use new, smaller size.
(setup_on_free_list, allocate_vector_from_block):
Adapt to new vector_free_lists size.
(pseudovector_nbytes): New function extracted from...
(vectorlike_nbytes): ...here.
* src/alloc.c: The size of a vector block is bound by the number of
words, not bytes, represented by the pseudovector header RESTSIZE
field, because that limits how big a PVEC_FREE object can be.
* src/alloc.c (NEAR_STACK_TOP): Hoist from here ...
* src/thread.h: ... to here.
* src/print.c (print_object): Use NEAR_STACK_TOP instead of raw
buffer address. This is more natural, and pacifies GCC 13.
Running arbitrary ELisp code from an atimer is still dangerous,
at least because the regexp engine is not-reentrant, so let's patch up
the case we bumped into. There are probably many other such holes :-(
* src/alloc.c (garbage_collection_inhibited): Make it non-static.
* src/xdisp.c (garbage_collection_inhibited): Declare it.
(set_message, clear_message): Use it as a proxy for "we're in
a dangerous context like within `probably_quit`".
Originally discovered by Tyler Dodge in his article "Significant Garbage
Collection Improvement For Emacs".
While testing this change on Archlinux system with Intel i5-7200U CPU,
average time of garbage collection gets reduced by ≈25%. Other users
report improvements up to 50%. While monitoring PSS of emacs with and
without customizations loaded before and after the patch, no
statistically significant differences were discovered. So overall, this
change is a win.
* src/alloc.c (BLOCK_ALIGN): increase from 1024 to 32768.
07f3236133 * src/profiler.c (malloc_probe): Make it safe for GC (bug...
1f1d36fa80 * lisp/emacs-lisp/debug-early.el (debug-early-backtrace):...
99df815c15 Revert "Don't disable eldoc when doing edebug"
0a4b1c0102 ; Eglot: improve bug-reference-url-format/bug-reference-u...
40c9fc8e3b Eglot: work around Tramp instability bug#61350
4a6db125b9 Fix treesit-indent-region
7ef9a8210c Replace C++ comments with C style equivalents
* java/org/gnu/emacs/EmacsDrawLine.java: Fix this again. Gosh,
how does Android do this.
* java/org/gnu/emacs/EmacsNoninteractive.java (main): Port to
Android 2.3.3.
* java/org/gnu/emacs/EmacsSdk11Clipboard.java
(EmacsSdk11Clipboard): Port to Android 4.0.3.
* java/org/gnu/emacs/EmacsService.java (getClipboardManager):
New function.
* src/alloc.c (find_string_data_in_pure): Fix Android alignment
issue.
* src/android-emacs.c (main): Port to Android 4.4.
* src/android.c (initEmacs): Align stack to 32 bytes, so it ends
up aligned to 16 even though gcc thinks the stack is already
aligned to 16 bytes.
* src/callproc.c (init_callproc): Use /system/bin/sh instead of
/bin/sh by default.
* configure.ac (ANDROID_CFLAGS): Add sfnt-related font objects
to ANDROID_OBJ when not building stubs.
* lisp/startup.el (android-fonts-enumerated): New variable.
(normal-top-level): Set it. Also enumerate fonts as early as
possible upon startup.
* src/alloc.c (cleanup_vector): Only finalize Android font
entities.
(garbage_collect): Mark sfntfont.c.
* src/android.c (struct android_emacs_drawable): New field
`damage_rect'.
(android_init_emacs_drawable): Initialize
Lorg/gnu/emacs/EmacsDrawable;#damageRect(Landroid/graphics/rect;)V.
(android_create_gc): Initialize cached GC fields.
(android_free_gc): Free cached GC clip rectangles.
(android_change_gc): Cache fields as appropriate.
(android_set_clip_rectangles): Set cached clip rectangles for
easy access from C.
(android_get_gc_values): Use cached values.
(android_get_image): Remove obsolete comment.
(android_lock_bitmap, android_damage_window): New functions that
don't parallel anything on X.
* src/android.h: Update prototypes.
* src/androidfns.c (android_default_font_parameter): Set Droid
Sans Mono as the default monospace font.
(Fx_create_frame): Register the sfntfont driver.
* src/androidgui.h (struct android_gc): Add C side caches for
clip rectangles and the foreground and background colors.
* src/androidterm.h: Update prototypes.
* src/dispextern.h (struct gui_box): New struct.
(gui_union_rectangles): New function.
* src/emacs.c (android_emacs_init): Initialize Android font
stuff late.
* src/font.c (font_make_entity): Clear `is_android' field on
Android.
(font_make_entity_android): Set `is_android' field.
* src/font.h (struct font_entity): New field `is_android'.
* src/print.c (print_vectorlike): Don't print private data,
which could include Lisp_Misc.
* src/sfnt.c (sfnt_read_cmap_format_0, sfnt_read_cmap_format_2)
(sfnt_read_cmap_format_4, sfnt_read_cmap_format_6)
(sfnt_read_cmap_format_8, sfnt_read_cmap_format_12): Remove
buggy pragmas.
(sfnt_lookup_glyph_4_1): New function.
(sfnt_lookup_glyph_4): Handle malformed lookup tables found on
Android.
(sfnt_lookup_glyph): Fix overflow problems in glyph checks.
(sfnt_read_glyph): Handle empty glyphs. This implements some
behavior which everyone else seems to as well, but I can't find
documented in the TrueType Reference Manual.
(sfnt_free_glyph): Export correctly.
(sfnt_transform_coordinates): Make faster.
(sfnt_lerp_half): Fix lerping in some cases.
(sfnt_decompose_glyph): Handle empty glyphs correctly. Close
contours manually instead of waiting for the edge building
process to do that. This lets curves be handled correctly.
(struct sfnt_build_glyph_outline_context): Move internal struct
back to sfnt.c.
(sfnt_build_append): Fix detection of initial entry.
(sfnt_curve_to_and_build_1): Fix De Casteljau implementation.
(sfnt_curve_to_and_build): Use fixed point arithmetic to scale
outlines.
(sfnt_build_glyph_outline): Clear reference counts. Use fixed
point arithmetic.
(sfnt_prepare_raster): Align rasters to 4 bytes,
SFNT_POLY_ALIGNMENT. Fix calculation of offx and offy.
(sfnt_step_edge_by): Step edge by previously computed step_x.
(sfnt_build_outline_edges): Adjust for already closed contours.
Ignore edges abandoned after grid fit. Also precompute step_x
to avoid multiplication on each span rastered.
(sfnt_poly_edges): Improve alignment.
(sfnt_fill_span): Rewrite to avoid control flow in while loop.
(sfnt_poly_span): Remove unnecessary code.
(sfnt_raster_glyph_outline): Use raster stride instead of width.
(sfnt_test_edge, sfnt_test_raster, main): Improve debugging
code.
* src/sfnt.h (struct sfnt_glyph_outline): Add refcount field to
outline.
(struct sfnt_build_glyph_outline_context): Remove private
struct.
(struct sfnt_raster): Add refcount field to raster.
(struct sfnt_edge): Improve doc. Add `source_x' field used when
built with TEST.
(SFNT_CEIL_FIXED): New macro.
* src/sfntfont-android.c (sfntfont_android_saturate32)
(sfntfont_android_scale32, sfntfont_android_mul8x2)
(sfntfont_android_blend, U255TO256)
(sfntfont_android_composite_bitmap, sfntfont_android_union_boxes)
(sfntfont_android_put_glyphs, sfntfont_android_get_cache): New
functions.
(android_sfntfont_driver): New font driver.
(Fandroid_enumerate_fonts): New function.
(syms_of_sfntfont_android_for_pdumper, init_sfntfont_android)
(syms_of_sfntfont_android): Initialize default fonts, special
family mapping and font driver.
* src/sfntfont.c (struct sfnt_font_desc): New fields
`char_cache', `cmap_invalid' and `subtable'.
(sfnt_setup_coding_system): Improve commentary. Add default
branch. Fix return value.
(sfnt_safe_encode_coding_object_1)
(sfnt_safe_encode_coding_object_2):
(sfnt_safe_encode_coding_object): Use decode_coding_object
instead of encode_coding_object.
(sfnt_decode_font_string): Adjust for rename.
(sfnt_decode_foundry_name): New function.
(sfnt_weight_descriptions, sfnt_slant_descriptions)
(sfnt_width_descriptions): Fix definitions.
(sfnt_parse_style): Make function work.
(sfnt_enum_font): Initialize designer, char-cache and subtable
platform ID.
(sfntfont_charset_for_name, mark_sfntfont)
(sfntfont_charset_for_cmap): New functions.
(syms_of_sfntfont): New variable `sfnt-default-family-alist'.
* src/sfntfont.h (_SFNTFONT_H_): Update prototypes.
* src/xdisp.c (gui_union_rectangles): New function.
cae528457c ; Add 2023 to copyright years.
b394359261 Improve documentation of 'isearch-open-overlay-temporary'
ab3210e709 Document 'use-package' in the 2 main manuals
# Conflicts:
# etc/refcards/ru-refcard.tex
# lib/explicit_bzero.c
# m4/explicit_bzero.m4
0754173c92 ; Fix docstring
014232d384 Eglot: eglot--servers-by-xrefed-file doesn't need to be v...
bbe35c280c Prevent stale servers when using eglot-extend-to-xref
9bf13a3fb9 ; * src/alloc.c (Fmemory_info): Doc fix.
baa33a5c05 * etc/tutorials/TUTORIAL.cn: Fix typos. (Bug#60221)
98d7f76b45 Merge branch 'emacs-29' of git.savannah.gnu.org:/srv/git/...
a488a6870a Add alias for removed font-lock function
02e046566e Set beginning/end-of-defun-function in treesit-major-mode...
7dea58b88d Add treesit-defun-at-point and fix c-ts-mode-indent-defun
69123d4aa4 ; Fix treesit--defuns-around
05d8310fb5 Use the new keyword ':repeat' in repeatable keymaps.
When Emacs is compiled with AddressSanitizer support, enable
poisoning/unpoisoning freed/unused Lisp objects and other internal
memory management structures. If enabled, this will mark freed bytes
that have been put on free lists for future use and initially
allocated memory blocks/chunks as "poisoned", triggering an ASan error
if they are accessed improperly. Structures are unpoisoned when they
have been taken off their respective free lists. Additionally, add
optional macros for performing unaligned loads, which when enabled by
defining USE_SANITIZER_UNALIGNED_LOAD will use ASan provided functions
for loading from unaligned addresses, which may help catch bugs that
AddressSanitizer might otherwise miss.
* configure.ac: Check for the existence of address and common
sanitizer API headers.
* src/lisp.h (UNALIGNED_LOAD_SIZE): New macro. If enabled, and the
necessary sanitizer API is available, define it to
__sanitizer_unaligned_load(64|32) depending on the word size of the
architecture.
* src/fns.c [HAVE_FAST_UNALIGNED_ACCESS] (Fstring_lessp): Use
'UNALIGNED_LOAD_SIZE' to perform unaligned loads from the two strings.
* src/alloc.c (ASAN_POISON_ABLOCK, ASAN_UNPOISON_ABLOCK)
(ASAN_POISON_INTERVAL_BLOCK, ASAN_UNPOISON_INTERVAL_BLOCK)
(ASAN_POISON_INTERVAL, ASAN_UNPOISON_INTERVAL)
(ASAN_PREPARE_DEAD_SDATA, ASAN_PREPARE_LIVE_SDATA)
(ASAN_POISON_SBLOCK_DATA, ASAN_POISON_STRING_BLOCK)
(ASAN_UNPOISON_STRING_BLOCK, ASAN_POISON_STRING)
(ASAN_UNPOISON_STRING, ASAN_POISON_FLOAT_BLOCK)
(ASAN_UNPOISON_FLOAT_BLOCK, ASAN_POISON_FLOAT)
(ASAN_UNPOISON_FLOAT, ASAN_POISON_CONS_BLOCK)
(ASAN_POISON_CONS, ASAN_UNPOISON_CONS)
(ASAN_POISON_VECTOR_CONTENTS, ASAN_UNPOISON_VECTOR_CONTENTS)
(ASAN_UNPOISON_VECTOR_BLOCK, ASAN_POISON_SYMBOL_BLOCK)
(ASAN_UNPOISON_SYMBOL_BLOCK, ASAN_POISON_SYMBOL)
(ASAN_UNPOISON_SYMBOL) [ADDRESS_SANITIZER]: New macros. When
address sanitization is enabled, define them to poison/unpoison
objects.
(lisp_align_malloc): Poison newly allocated blocks on `free_ablock',
unpoison ablocks taken from it respectively.
(lisp_align_free): Poison individual ablocks when they are put on the
free list, unpoison them when an entire `ablocks' chunk is being
freed.
(make_interval): Poison interval blocks on initial allocation,
unpoison individual intervals on allocation and removal from
`interval_free_list'.
(sweep_intervals): Unpoison interval blocks before sweeping, poison
dead/unmarked intervals.
(allocate_string): Poison string blocks on initial allocation,
unpoison Lisp_Strings on removal from the free list.
(allocate_string_data): Poison `sblock' data on initial allocation,
unpoison individual `sdata' contents on allocation or removal from the
free list. Call `ASAN_PREPARE_LIVE_SDATA' on the new `sdata' struct.
(sweep_strings): Unpoison string blocks before sweeping them,
poisoning dead strings and their sdata afterwards.
(compact_small_strings): Call `ASAN_PREPARE_LIVE_DATA' on the `sdata'
to where compacted strings to moved to.
(pin_string): Call `ASAN_PREPARE_DEAD_SDATA' on `old_sdata'.
(make_float): Poison float blocks on allocation, unpoisoning
individual Lisp_Floats on allocation or removal from
`float_free_list'.
(sweep_floats): Unpoison float blocks before sweeping, poison
dead/unmarked floats.
(free_cons): Poison `ptr'.
(Fcons): Poison cons blocks on allocation, unpoisoning individual
Lisp_Cons on allocation or removal from `cons_free_list'.
(sweep_conses): Poison dead/unmarked conses.
(setup_free_list): Poison vectors put on `vector_free_lists'.
(allocate_vector_from_block): Unpoison vectors taken from the free
list, poison excess vector bytes when vectors allocated from the free
list are larger than requested.
(sweep_vectors): Unpoison vector blocks before sweeping them.
(Fmake_symbol): Poison symbol blocks on initial allocation,
unpoisoning individual Lisp_Symbols on allocation or removal from
`symbol_free_list'.
(sweep_symbols): Unpoison symbol blocks before sweeping, poisoning
dead/unmarked symbols.
(live_string_holding, live_cons_holding, live_symbol_holding)
(live_float_holding): When compiling with address sanitization and GC
poisoning enabled, return NULL if the passed address is poisoned, or
if the Lisp object it resides in is poisoned, avoiding a
use-after-poison trigger if these functions are called on a pointer
that might be referring to a now dead/swept object.
* etc/DEBUG: Add information about enabling ASan memory poisoning.
