On my platform, this sped up (make-string 4000000000 0) from 2.5
to 0.015 seconds (not that people should want to do this much :-).
* src/alloc.c (allocate_string_data): New arg CLEARIT.
Callers changed.
(Fmake_string): Prefer calloc to malloc+memset when allocating a
large string of NUL bytes.
(make_clear_string): New function.
(make_uninit_string): Use it.
(make_clear_multibyte_string): New function.
(make_uninit_multibyte_string): Use it.
* src/module-env-28.h: Add new module environment functions to
module environment for Emacs 28.
* src/emacs-module.h.in: Document that 'emacs_finalizer' also works
for function finalizers.
* src/emacs-module.c (CHECK_MODULE_FUNCTION): New function.
(struct Lisp_Module_Function): Add finalizer data member.
(module_make_function): Initialize finalizer.
(module_get_function_finalizer)
(module_set_function_finalizer): New module environment functions.
(module_finalize_function): New function.
(initialize_environment): Initialize new environment functions.
* src/alloc.c (cleanup_vector): Call potential module function
finalizer during garbage collection.
* test/data/emacs-module/mod-test.c (signal_error): New helper
function.
(memory_full): Use it.
(finalizer): New example function finalizer.
(Fmod_test_make_function_with_finalizer)
(Fmod_test_function_finalizer_calls): New test module functions.
(emacs_module_init): Define them.
* test/src/emacs-module-tests.el (module/function-finalizer): New unit
test.
* doc/lispref/internals.texi (Module Functions): Document new
functionality.
(Module Misc): Move description of 'emacs_finalizer' type to 'Module
Functions' node, and add a reference to it.
* etc/NEWS: Mention new functionality.
Prefer calloc to malloc+memset when allocating large zeroed objects.
This avoids page thrashing when (make-vector 1000000000 nil)
allocates a large nil vector, as Emacs need not touch the
vector’s pages. This wins on platforms like GNU/Linux where
calloc can fiddle with page tables to create a block of memory
that is lazily zeroed.
* src/alloc.c (lisp_malloc, lmalloc, allocate_vectorlike):
New arg CLEARIT to tell callee whether to use malloc or calloc.
All callers changed.
(allocate_clear_vector, allocate_nil_vector): New functions.
* src/alloc.c (xzalloc, make_vector):
* src/lisp.h (make_nil_vector):
Prefer calloc to malloc + memset(...,0,...).
* doc/lispref/internals.texi (Garbage Collection)
(Pure Storage):
* src/alloc.c (Fgarbage_collect): Update the documentation of
pure-space overflow for when pdumper is used. (Bug#38492)
The old calculation relied on a hodgpodge of partly updated GC
stats to find a number to multiply gc-cons-percentage by.
The new one counts data found by the previous GC, plus half of
the data allocated since then; this is more systematic albeit
still ad hoc.
* src/alloc.c (consing_until_gc, gc_threshold, consing_threshold):
Now EMACS_INT, not intmax_t.
(HI_THRESHOLD): New macro.
(tally_consing): New function.
(make_interval, allocate_string, allocate_string_data)
(make_float, free_cons, allocate_vectorlike, Fmake_symbol): Use it.
(allow_garbage_collection, inhibit_garbage_collection)
(consing_threshold, garbage_collect):
Use HI_THRESHOLD rather than INTMAX_MAX.
(consing_threshold): New arg SINCE_GC. All callers changed.
(bump_consing_until_gc): Return new consing_until_gc, instead of
nil. All callers changed. Don’t worry about overflow since we
now saturate at HI_THRESHOLD. Guess that half of
recently-allocated objects are still alive, instead of relying on
the previous (even less-accurate) hodgepodge.
(maybe_garbage_collect): New function.
(garbage_collect): Work even if a finalizer disables or enables
memory profiling. Do not use malloc_probe if GC reclaimed nothing.
* src/lisp.h (maybe_gc): Call maybe_garbage_collect instead
of garbage_collect.
* src/alloc.c (make_interval, allocate_string, make_float)
(free_cons, Fcons, setup_on_free_list)
(allocate_vector_from_block, Fmake_symbol):
Do not update gcstat, since it is for statistics from the most
recent GC, not for a partially-updated hodgepodge.
(sweep_vectors): Update gcstat, since setup_on_free_list
no longer does.
(garbage_collect_1): Rename to garbage_collect and adopt its API.
Remove the old garbage_collect, which is no longer needed.
All callers changed.
* src/alloc.c (memory_full): Just report "memory exhausted" if
failure occurs during initialization, since fancier recovery
schemes are not likely to work when not initialized.
* src/emacs.c (dump_error_to_string): Accept int, not enum
pdumper_load_result, since the result might not fit in the
enum. Use strerror if it was derived from errno. This is for
better diagnostics of pdump load failures.
(load_pdump_find_executable): Return char *, not enum. 2nd
arg is now pointer to buffer size, rather than pointer to
pointer to buffer. All callers changed. Use Emacs allocator
since they should now be OK even during early startup.
Use check_executable instead access, to use effective rather
than real permissions.
(load_pdump): Return void since callers ignore result.
Use int where enum could be too narrow. Use heap rather
than stack for possibly-long string. Prefer ptrdiff_t to
size_t.
* src/fileio.c (check_executable): Now extern.
* src/pdumper.c (pdumper_load): Return int that may have
errno added to it, for better diagnostics when loads fail.
Problem reported by Michael Heerdegen (Bug#37321).
* src/alloc.c (gc_threshold): New static var.
(bump_consing_until_gc): Change args from DIFF to THRESHOLD and
PERCENTAGE. All uses changed. When accounting for a changed
gc-cons-percentage, do not assume that total_bytes_of_live_objects
returns the same value now that it did the last time we were
called.
Problem reported by Joseph Mingrone (Bug#37006#72).
* src/alloc.c (watch_gc_cons_threshold)
(watch_gc_cons_percentage):
Don’t try to store an intmax_t into an int.
Redo to make the code clearer.
(watch_gc_cons_percentage):
Use gc_cons_threshold, not consing_until_gc.
* src/alloc.c (mark_maybe_pointer):
* src/pdumper.h (pdumper_object_p_precise):
Use pdumper_valid_object_type_p.
* src/pdumper.c (pdumper_find_object_type_impl):
* src/pdumper.h (pdumper_find_object_type):
Return int, not enum Lisp_Type. All callers changed.
* src/pdumper.h (PDUMPER_NO_OBJECT): Do not cast -1 to enum
Lisp_Type; in theory, C18 says this could yield 7, which would
mean PDUMPER_NO_OBJECT == Lisp_Float (!).
(pdumper_valid_object_type_p): New function.
Add watchers for gc-cons-threshold and gc-cons-percentage
that update consing_until_gc accordingly.
Suggested by Eli Zaretskii (Bug#37006#52).
* src/alloc.c (consing_threshold, bump_consing_until_gc)
(watch_gc_cons_threshold, watch_gc_cons_percentage):
New functions.
(garbage_collect_1): Use consing_threshold.
(syms_of_alloc): Arrange to watch gc-cons-threshold and
gc-cons-percentage.
This GC bug seems to have been fixed, so the check is no longer
needed in production code. From a suggestion by Pip Cet in:
https://lists.gnu.org/r/emacs-devel/2019-08/msg00316.html
* src/alloc.c (SUSPICIOUS_OBJECT_CHECKING) [!ENABLE_CHECKING]:
Do not define.
(find_suspicious_object_in_range, detect_suspicious_free):
Expand to proper dummy expressions if !SUSPICIOUS_OBJECT_CHECKING.
* src/data.c (Ffset): Convert test to an eassert.
This uses ‘const’ to be better at catching bugs that
mistakenly attempt to modify a bignum value.
Lisp bignums are supposed to be immutable.
* src/alloc.c (make_pure_bignum):
* src/fns.c (sxhash_bignum):
Accept Lisp_Object instead of struct Lisp_Bignum *, as that’s
simpler now. Caller changed.
* src/bignum.h (bignum_val, xbignum_val): New inline functions.
Prefer them to &i->value and XBIGNUM (i)->value, since they
apply ‘const’ to the result.
* src/timefns.c (lisp_to_timespec): Use mpz_t const *
to point to a bignum value.
* src/alloc.c (free_cons):
* src/casefiddle.c (do_casify_multibyte_string):
* src/editfns.c (styled_format):
* src/image.c (png_load_body):
Remove recent workarounds for INT_ADD_WRAPV bugs since
the bugs have been fixed (Bug#37006).
This builds on the previous patch.
* src/alloc.c (consing_until_gc): Now of type intmax_t,
since gc-cons-threshold can be up to INTMAX_MAX. All uses changed.
* src/lisp.h (CONSING_CT_MAX, consing_ct): Remove.
Suggested by Eli Zaretskii (Bug#37006#46).
* src/alloc.c (consing_until_gc): Now of type consing_ct.
All uses changed, so gc-cons-threshold no longer saturates
against OBJECT_CT_MAX.
(object_ct): Move typedef here from lisp.h.
* src/lisp.h (consing_ct, CONSING_CT_MAX): New type and macro.
(OBJECT_CT_MAX): Remove. Replace all uses with CONSING_CT_MAX.
Problem reported by Eli Zaretskii (Bug#37006#25).
* src/alloc.c (garbage_collect_1): Fix typo in threshold calc.
Go back to dividing by 10 since the numerator’s a constant now.
Problem introduced in 2019-07-21T02:40:03Z!eggert@cs.ucla.edu.
* src/alloc.c (free_cons):
* src/casefiddle.c (do_casify_multibyte_string):
* src/editfns.c (styled_format):
* src/image.c (png_load_body):
Use signed arguments to INT_MULTIPLY_WRAPV etc. This doesn’t fix
any bugs, but GCC emits better code when all args are signed.
Also, this removes the need for an if in free_cons (Bug#37006).
* src/alloc.c (c_symbol_p): Move from here ...
* src/lisp.h (c_symbol_p): ... to here, and make it more portable
to hypothetical platforms where pointers are wider than ptrdiff_t.
* src/pdumper.c (dump_builtin_symbol_p): Use c_symbol_p.
Say that pdumping cannot redump unless -batch is used. Say that
the traditional unexec dumping method is by default not available,
and is deprecated. Don't call dump files "portable", as dump files
are not any more portable than the Emacs executables themselves.
Just call them "dump files". Similar, prefer "portable dumper"
(since the dumper code is portable) to "portable dumping" (since
the dump file is not). Be more systematic about calling them
"dump files" instead of "dumped images" or whatnot.
Problem reported by Pip Cet in:
https://lists.gnu.org/r/emacs-devel/2019-07/msg00520.html
* src/alloc.c (inhibit_garbage_collection): Use new function.
(allow_garbage_collection): Accept intmax_t, not pointer.
* src/eval.c (default_toplevel_binding, do_one_unbind)
(backtrace_eval_unrewind, Fbacktrace__locals, mark_specpdl):
Support SPECPDL_UNWIND_INTMAX.
(record_unwind_protect_excursion): New function.
* src/lisp.h (enum specbind_tag): New constant SPECPDL_UNWIND_INTMAX.
(union specbinding): New member unwind_intmax.
* src/alloc.c (allow_garbage_collection)
(inhibit_garbage_collection): Temporarily bump
consing_until_gc, to improve performance of maybe_gc while
garbage collection is inhibited. Suggested by Stefan Monnier in:
https://lists.gnu.org/r/emacs-devel/2019-07/msg00511.html
* src/alloc.c (pure_alloc): Clear any heap-allocated storage.
This is simpler than auditing all the callers to make sure
they don’t assume pure memory is cleared memory, and the
performance implication is nonexistent except when Emacs
is misconfigured. Also, add an assertion to catch
caller misuse when pure space is exhausted.
* src/fns.c (restore_mutability)
(hash_table_user_defined_call): New functions.
(cmpfn_user_defined, hashfn_user_defined): Use them.
(make_hash_table, copy_hash_table):
Mark new hash table as mutable.
(check_mutable_hash_table): New function.
(Fclrhash, Fputhash, Fremhash): Use it instead of CHECK_IMPURE.
* src/lisp.h (struct hash_table_test): User-defined functions
now take pointers to struct Lisp_Hash_Table, not to struct
hash_table_test. All uses changed.
(struct Lisp_Hash_Table): New member ‘mutable’.
* src/pdumper.c (dump_hash_table): Copy it.
* test/src/fns-tests.el (test-hash-function-that-mutates-hash-table):
New test, which tests for the bug.
Without this patch, there are unlikely ways that garbage
collection could occur (sometimes causing undefined behavior)
even when inhibit_garbage_collection is in effect.
* src/alloc.c (garbage_collection_inhibited): New var.
(pure_alloc): Increment it if pure space is exhausted, so that
garbage_collect_1 no longer needs to inspect
pure_bytes_used_before_overflow.
(allow_garbage_collection): New function.
(inhibit_garbage_collection): Increment the new variable rather
than specbinding a user variable.
(garbage_collect_1): Do not garbage collect if the new variable
is set, rather than if pure_bytes_used_before_overflow is set.
* src/alloc.c (consing_until_gc): New variable, replacing the
combination of consing_since_gc and gc_relative_threshold.
All uses changed.
(byte_ct): Move decl here from lisp.h.
(memory_full_cons_threshold): New an enum constant.
(free_cons): Check for integer overflow in
statistics calculation.
* src/lisp.h (object_ct): Move decl here from alloc.c.
(OBJECT_CT_MAX): New macro.
(maybe_gc): Simplify accordingly.
* src/lisp.h (struct Lisp_Hash_Table): Rename ‘pure’ member to
‘purecopy’, as the old name was quite confusing (it did not
mean the hash table was pure). All uses changed.
This speeds up ‘make compile-always’ by 0.1% on my platform.
Suggested by Pip Cet in:
https://lists.gnu.org/r/emacs-devel/2019-07/msg00257.html
* src/.gdbinit (pwinx, pgx, xbuffer, xprintstr):
Output dead_object () as "DEAD".
* src/alloc.c (Vdead, DEADP): Remove.
All uses replaced by dead_object () / deadp.
(deadp): New function.
(init_alloc_once_for_pdumper): Remove no-longer-needed
initialization.
* src/lisp.h (dead_object): New function.
Without this fix, Emacs can get into a tight loop reporting
a range error when calculating timestamps.
* doc/lispref/numbers.texi (Integer Basics):
* src/alloc.c (syms_of_alloc): Document this.
* src/bignum.c (make_bignum_bits): Always allow bignums
of at least twice the width of (u)intmax_t.
The GCC + valgrind fix caused the m68k build to fail (Bug#35711).
Simplify string allocation a bit to make similar problems less
likely in the future.
* src/alloc.c (sdata, SDATA_NBYTES, SDATA_DATA) [GC_CHECK_STRING_BYTES]:
Use the same implementation as with !GC_CHECK_STRING_BYTES,
as the special case is no longer needed.
(SDATA_ALIGN): New constant.
(SDATA_SIZE): Remove this macro, replacing with ...
(sdata_size): ... this new function. All uses changed.
Properly account for sizes and alignments even in the m68k case,
and even if GC_CHECK_STRING_BYTES is not defined.
* src/alloc.c: No need to include stdalign.h; it’s pervasive.
(GC_STRING_OVERRUN_COOKIE_SIZE): Align to sdata’s alignment,
so that the code works even if alignof (sdata) exceeds 8.
Don’t require the cookie size to be 8, as this overly fattens
32-bit platforms and one DEADBEEF should be enough.
(GC_STRING_EXTRA): Omit now-unnecessary ‘verify’.
(allocate_string_data): Omit unnecessary cast.
Issue found by Clang’s UBSan.
* src/alloc.c (GC_STRING_OVERRUN_COOKIE_SIZE): Increase to 8.
(string_overrun_cookie): Extend accordingly.
(GC_STRING_EXTRA): Ensure that it’s properly aligned for ‘sdata’.
(allocate_string_data): Verify that ‘sdata’ blocks remain aligned.
It doesn’t work anymore, and these days ‘gcc -fsanitize=address’
does a better job anyway.
* configure.ac: Remove the option.
* configure.ac (ac_xmalloc_overrun, XMALLOC_OVERRUN_CHECK):
* src/alloc.c (XMALLOC_OVERRUN_CHECK_OVERHEAD)
(XMALLOC_OVERRUN_CHECK_SIZE, XMALLOC_OVERRUN_SIZE_SIZE)
(xmalloc_overrun_check_header, xmalloc_overrun_check_trailer)
(xmalloc_put_size, xmalloc_get_size, overrun_check_malloc)
(overrun_check_realloc, overrun_check_free):
Remove. All uses removed.
* etc/NEWS: Mention this.
Because XPNTR now uses ATTRIBUTE_NO_SANITIZE_UNDEFINED,
it can be simplified.
* src/alloc.c (macro_PNTR_ADD, PNTR_ADD, macro_XPNTR): Remove.
(XPNTR): Open-code rather than using the removed macros and
functions. Also, simplify by using LISP_WORD_TAG.
