This fixes the only remaining GCC diagnostics when emacs-28 is
configured with --enable-gcc-warnings. It does so by adding
ATTRIBUTE_RETURNS_NONNULL so that GCC knows certain functions
return nonnull. It also arranges for three of those functions to
always return nonnull; I thought these functions already were
doing so, but apparently not, and it is conceivable (though I
haven’t checked this) that changing these functions to always
return nonnull even on non-GNU platforms may fix unlikely
portability bugs elsewhere in Emacs. I used GCC 11.2.1 20210728
(Red Hat 11.2.1-1) on x86-64 when checking the diagnostics.
* configure.ac: Invoke gl_EEMALLOC before gl_INIT, in case
the regex code doesn't invoke gl_EEMALLOC; needed for src/alloc.c’s
use of MALLOC_0_IS_NONNULL.
* src/alloc.c (xmalloc, xzalloc, xrealloc): Don’t worry about the
special case where SIZE == 0, since lmalloc and lrealloc now
return null only on allocation failure.
(lmalloc, lrealloc): Return null only on allocation failure,
instead of having special cases that treat malloc (0) and
realloc (X, 0) as successes even when they return null.
* src/lisp.h: Add ATTRIBUTE_RETURNS_NONNULL to a few functions
that always return nonnull pointers, so that gcc -fanalyzer
does not issue diagnostics like “alloc.c: In function
‘allocate_vector_block’: alloc.c:2985:15: warning: dereference of
possibly-NULL ‘block’ [CWE-690] [-Wanalyzer-possible-null-dereference]”
as per <https://cwe.mitre.org/data/definitions/690.html>.
* src/alloc.c (lisp_malloc): Document that NBYTES must be
positive, and omit a needless runtime check. This pacifies a
false alarm with gcc 11.2.1 -Wanalyzer-possible-null-dereference.
Prevent some false-positives in conservative GC marking.
This doesn’t fix any correctness bugs; it’s merely to
reclaim some memory instead of keeping it unnecessarily.
* src/alloc.c (mark_maybe_pointer): New arg SYMBOL_ONLY.
All callers changed. Check that the pointer’s tag, if any,
matches the pdumper-reported type.
* src/alloc.c (mark_maybe_pointer): Also mark pointers
to pdumper objects, even when the pointers are tagged.
Add a FIXME saying why this isn’t enough.
Simplify closure creation by calling a single function at run time
instead of putting it together from small pieces. This is faster
(by about a factor 2), takes less space on disk and in memory, and
makes internal functions somewhat readable in disassembly listings again.
This is done by creating a prototype function at compile-time whose
closure variables are placeholder values V0, V1... which can be seen
in the disassembly. The prototype is then cloned at run time using
the new make-closure function that replaces the placeholders with
the actual closure variables.
* lisp/emacs-lisp/bytecomp.el (byte-compile-make-closure):
Generate call to make-closure from a prototype function.
* src/alloc.c (Fmake_closure): New function.
(syms_of_alloc): Defsubr it.
* src/data.c (syms_of_data): Defsym byte-code-function-p.
* src/lisp.h (struct Lisp_Subr): Add 'type' field.
(SUBR_TYPE): New inline accessor.
* src/pdumper.c (dump_subr): Update for 'type' field.
* src/data.c (Fsubr_type): New primitive.
(syms_of_data): Update.
* src/comp.c (ABI_VERSION): Bump new ABI version.
(make_subr): Set type.
(Fcomp__register_lambda, Fcomp__register_subr)
(Fcomp__late_register_subr): Receive and pass subr type to
'make_subr'.
* src/alloc.c (mark_object): Mark subr type.
* lisp/emacs-lisp/comp.el (comp-func): Change slot type into mvar.
(comp-emit-for-top-level, comp-emit-lambda-for-top-level): Pass
type mvar to subr register functions.
(comp-compute-function-type): Fix-up subr type mvars.
* test/src/comp-tests.el (comp-tests-check-ret-type-spec): Use
`subr-type'.
The latest Gnulib merge brought in free-posix, which causes 'free'
to preserve errno. This lets us simplify some Emacs code that
calls 'free'.
* admin/merge-gnulib (GNULIB_MODULES): Add free-posix.
This module is pulled in by canonicalize-lgpl anyway,
so we might as well rely on it.
* lib-src/emacsclient.c (get_current_dir_name):
Sync better with src/sysdep.c.
* lib-src/etags.c (process_file_name, etags_mktmp):
* lib-src/update-game-score.c (unlock_file):
* src/fileio.c (file_accessible_directory_p):
* src/sysdep.c (get_current_dir_name_or_unreachable):
Simplify by assuming that 'free' preserves errno.
* src/alloc.c (malloc_unblock_input):
Preserve errno, so that xfree preserves errno.
* src/sysdep.c (get_current_dir_name_or_unreachable):
Simplify by using strdup instead of malloc+memcpy.
No need for realloc (and the old code leaked memory anyway on
failure); just use free+malloc.
We used to store module runtime and environment pointers in the static
lists Vmodule_runtimes and Vmodule_environments. However, this is
incorrect because these objects have to be kept per-thread. With this
naive approach, interleaving module function calls in separate threads
leads to environments being removed in the wrong order, which in turn
can cause local module values to be incorrectly garbage-collected.
The fix isn't completely trivial: specbinding the lists wouldn't work
either, because then the garbage collector wouldn't find the
environments in other threads than the current ones, again leading to
objects being garbage-collected incorrectly. While introducing custom
pseudovector types would fix this, it's simpler to put the runtime and
environment pointers into the specbinding list as new specbinding
kinds. This works since we need to unwind them anyway, and we only
ever treat the lists as a stack. The thread switching machinery
ensures that the specbinding lists are thread-local, and that all
elements of the specbinding lists in all threads are marked during
garbage collection.
Module assertions now have to walk the specbinding list for the
current thread, which is more correct since they now only find
environments for the current thread. As a result, we can now remove
the faulty Vmodule_runtimes and Vmodule_environments variables
entirely.
Also add a unit test that exemplifies the problem. It interleaves two
module calls in two threads so that the first call ends while the
second one is still active. Without this change, this test triggers
an assertion failure.
* src/lisp.h (enum specbind_tag): Add new tags for module runtimes and
environments.
* src/eval.c (record_unwind_protect_module): New function to record a
module object in the specpdl list.
(do_one_unbind): Unwind module objects.
(backtrace_eval_unrewind, default_toplevel_binding, lexbound_p)
(Fbacktrace__locals): Deal with new specbinding types.
(mark_specpdl): Mark module environments as needed.
* src/alloc.c (garbage_collect): Remove call to 'mark-modules'.
Garbage collection of module values is now handled as part of marking
the specpdl of each thread.
* src/emacs-module.c (Fmodule_load, funcall_module): Use specpdl to
record module runtimes and environments.
(module_assert_runtime, module_assert_env, value_to_lisp): Walk
through specpdl list instead of list variables.
(mark_module_environment): Rename from 'mark_modules'. Don't attempt
to walk though current thread's environments only, since that would
miss other threads.
(initialize_environment, finalize_environment): Don't change
Vmodule_environments variable; environments are now in the specpdl
list.
(finalize_environment_unwind, finalize_runtime_unwind): Make 'extern'
since do_one_unbind now calls them.
(finalize_runtime_unwind): Don't change Vmodule_runtimes variable;
runtimes are now in the specpdl list.
(syms_of_module): Remove Vmodule_runtimes and Vmodule_environments.
* test/data/emacs-module/mod-test.c (Fmod_test_funcall): New test
function.
(emacs_module_init): Bind it.
* test/src/emacs-module-tests.el (emacs-module-tests--variable): New
helper type to guard access to state in a thread-safe way.
(emacs-module-tests--wait-for-variable)
(emacs-module-tests--change-variable): New helper functions.
(emacs-module-tests/interleaved-threads): New unit test.
When recompiling remove the corresponding stale elns found in the
`comp-eln-load-path'.
When removing a package remove the corresponding elns too.
On Windows both of these are performed only when possible, when it's
not the file is renamed as .eln.old and a last attempt to remove this
is performed closing the Emacs session. When a file being deleted was
loaded by multiple Emacs sessions the last one being closed should
delete it.
* lisp/emacs-lisp/comp.el (comp-clean-up-stale-eln): New function.
(comp-delete-or-replace-file): Rename from
`comp--replace-output-file' and update so it can be used for
replacing or deleting shared libs safetly.
* lisp/emacs-lisp/package.el (package--delete-directory): When
native compiled just call `comp-clean-up-stale-eln' for each
eln file we want to clean-up.
* src/alloc.c (cleanup_vector): Call directly the dynlib_close.
* src/comp.c (syms_of_comp): Update for comp_u->cfile removal.
Make 'all_loaded_comp_units_h' key-value weak as now the key will
be the filename.
(load_comp_unit): Register the compilation unit only when the load
is fully completed.
(register_native_comp_unit): Make the key of
all_loaded_comp_units_h the load filename.
(eln_load_path_final_clean_up): New function.
(dispose_comp_unit)
(finish_delayed_disposal_of_comp_units)
(dispose_all_remaining_comp_units)
(clean_package_user_dir_of_old_comp_units): Remove.
(Fcomp__compile_ctxt_to_file): Update for
`comp--replace-output-file' -> `comp-delete-or-replace-file'
rename.
* src/comp.h (dispose_comp_unit)
(finish_delayed_disposal_of_comp_units)
(dispose_all_remaining_comp_units)
(clean_package_user_dir_of_old_comp_units): Remove.
(eln_load_path_final_clean_up): Add.
(struct Lisp_Native_Comp_Unit): Remove cfile field.
* src/emacs.c (Fkill_emacs): Call 'eln_load_path_final_clean_up'.
* src/pdumper.c (dump_do_dump_relocation): Do not set comp_u->cfile.
The report that they broke macOS was a false alarm, as the
previous commit was also broken (Bug#43152#62).
* src/alloc.c (live_string_holding, live_cons_holding)
(live_symbol_holding):
Count only pointers that point to a struct component,
or are a tagged pointer to the start of the struct.
Exception: for non-bool-vector pseudovectors,
count any pointer past the header, since it’s too much
of a pain to write code for every pseudovector.
(live_float_holding, live_vector_pointer):
New functions, which are similar about counting pointers.
(live_float_p, live_large_vector_holding)
(live_small_vector_pointer, mark_maybe_pointer): Use them.
(mark_maybe_object, mark_maybe_objects): Remove,
and remove all callers; mark_maybe_pointer now suffices.
(mark_objects): New function.
* src/alloc.c (mark_vectorlike, mark_face_cache):
* src/eval.c (mark_specpdl):
* src/fringe.c (mark_fringe_data):
* src/keyboard.c (mark_kboards):
Simplify by using mark_objects.
* src/lisp.h (SAFE_ALLOCA_LISP_EXTRA):
Clear any Lisp_Object arrays large enough to not fit into the stack,
so that GC need not worry about whether they contain objects.
* src/alloc.c (live_string_holding, live_cons_holding)
(live_symbol_holding, live_large_vector_holding)
(live_small_vector_holding):
Go back to old approach of treating every would-be pointer to any
byte in the object (though not to just past the object end) as
addressing the object.
(live_float_p): Require that the would-be float point
to the start of the Lisp_Float, and not anywhere else.
(live_vector_pointer, live_float_holding, mark_objects):
Remove. All uses removed.
(mark_maybe_object, mark_maybe_objects):
Bring back these functions.
* src/lisp.h (SAFE_ALLOCA_LISP_EXTRA): Do not clear the
new slots, as they're now checked via mark_maybe_objects,
not via mark_objects.
* src/alloc.c (mark_maybe_object, mark_maybe_objects): Remove.
(mark_objects): New function.
* src/eval.c (mark_specpdl): Use mark_objects instead of
mark_maybe_objects, since the array now has only valid Lisp objects.
* src/lisp.h (SAFE_ALLOCA_LISP_EXTRA): When allocating a large
array, clear it so that it contains only valid Lisp objects. This
is simpler and safer, and does not hurt performance significantly
on my usual benchmark as the code is executed so rarely.
This lets Emacs avoid marking some garbage as if it were in use.
On one test platform (RHEL 7.8, Intel Xeon Silver 4116) it
sped up ‘cd lisp; make compile-always’ by a bit over 1%.
* src/alloc.c (live_string_holding, live_cons_holding)
(live_symbol_holding, live_large_vector_holding)
(live_small_vector_holding):
Count only pointers that point to a struct component,
or are a tagged pointer to the start of the struct.
Exception: for non-bool-vector pseudovectors,
count any pointer past the header, since it’s too much
of a pain to write code for every pseudovector.
(live_vector_pointer): New function.
* src/alloc.c (mark_memory): Do not bother using mark_maybe_object
on the stack, since mark_maybe_pointer now marks everything that
mark_maybe_object would.
On --with-wide-int platforms where Lisp_Object can be
put into non-adjacent registers, mark_maybe_pointer failed
to mark a float whose only reference was as a tagged pointer.
* src/alloc.c (live_float_holding): New function,
a generalization of the old live_float_p.
(live_float_p): Use it.
(mark_maybe_pointer): Use live_float_holding, not live_float_p.
GCC has removed the -fcheck-pointer bounds option, and the Linux
kernel has also removed support for Intel MPX, so there’s no point
to keeping this debugging option within Emacs.
* src/bytecode.c (BYTE_CODE_THREADED):
* src/lisp.h (DEFINE_LISP_SYMBOL, XSYMBOL, make_lisp_symbol):
Assume __CHKP__ is not defined.
* src/ptr-bounds.h: Remove. All uses of ptr_bounds_clip,
ptr_bounds_copy, ptr_bounds_init, ptr_bounds_set removed.
* src/alloc.c (mark_maybe_object):
Use simpler way to avoid -fsanitize=undefined false alarms,
by converting the word tag to intptr_t first.
Omit now-unnecessary runtime overflow check.
(mark_memory): Work even if UINTPTR_MAX <= INT_MAX (!).
