Work around a bug in GnuTLS 3.7.11 and earlier: when built
statically, its mistakenly exports symbols hash_lookup and
hash_string, which collide with Emacs symbols of the same name,
preventing temacs from linking statically. Problem reported by
Greg A. Woods (Bug#77476).
Because GnuTLS never uses hash_lookup or hash_string this issue
ordinarily doesn’t seem to prevent temacs from linking to GnuTLS
on GNU/Linux, as it’s linked dynamically and the dynamic linker
never needs to resolve references to either symbol. However, I
suppose a clash or bug could occur even with dynamic linking if
Emacs later loads a module that uses either symbol.
Although GnuTLS should be fixed, Emacs should link statically to
current and older GnuTLS versions in the meantime, and it should
avoid potential problems with dynamic linking. Renaming the two
clashing names is an easy way to do this. For consistency with
the new name for hash_lookup, also rename hash_lookup_with_hash
and hash_lookup_get_hash.
* src/fns.c (hash_find_with_hash): Rename from hash_lookup_with_hash.
(hash_find): Rename from hash_lookup.
(hash_find_get_hash): Rename from hash_lookup_get_hash.
(hash_char_array): Rename from hash_string.
All uses changed.
This clarifies that the error isn't talking about an editing buffer.
* src/emacs-module.c (module_memory_buffer_too_small): Rename from
'module_buffer_too_small'.
(module_copy_string_contents, module_extract_big_integer): Adapt
callers.
(syms_of_module): Rename symbol 'buffer-too-small' to
'memory-buffer-too-small'.
The convention used by 'aref' and friends is that for
'args-out-of-range', the error data is a list (SEQ INDEX). Use the same
convention for the vector-related module functions.
* src/emacs-module.c (check_vec_index): Use vector and index as error
data.
'args-out-of-range' means that some index argument isn't valid for a
given sequence/range, which isn't the case here. Instead, define a new
error symbol to mean "user-supplied buffer is too small." Since we
never specified nor tested which error symbol was signalled in this
case, changing it shouldn't cause severe breakages.
* src/emacs-module.c (module_buffer_too_small): New helper function.
(module_copy_string_contents, module_extract_big_integer): Use it.
(syms_of_module): Define 'buffer-too-small' error symbol.
The previous approach would incorrectly invalidate the returned module
values if another non-local exit occurred while dealing with a non-local
exit. See Bug#65796. Instead, allocate the values from the usual
environment storage, and return statically-allocated objects if that
fails.
* src/emacs-module.c (struct emacs_env_private): Turn non-local exit
symbol and data into normal Lisp objects.
(initialize_environment): Initialize them.
(mark_module_environment): Prevent them from being garbage-collected.
(module_signal_or_throw, module_non_local_exit_signal_1)
(module_non_local_exit_throw_1): Adapt uses.
(value_to_lisp): No longer scan for them with module assertions enabled.
(module_out_of_memory_signal, module_out_of_memory_data): New
statically-allocated module values to return in case of allocation
failure.
(syms_of_module): Initialize them.
(module_non_local_exit_get): Allocate module values normally. If that
fails, return statically-allocated values.
* doc/lispref/internals.texi (Module Nonlocal): Document new behavior.
This fix is also prompted by Emacs bug#71744.
* src/emacs-module.c: Do not ignore -Wclobbered.
(MODULE_HANDLE_NONLOCAL_EXIT): Fix violations of the C standard,
where setjmp clobbered env and internal_cleanup.
(module_extract_big_integer) [GCC_LINT && __GNUC__ && !__clang__]:
Work around GCC -Wclobbered false positive for ‘sign’.
* src/comp.c (declare_imported_func, emit_simple_limple_call)
(declare_lex_function, compile_function):
* src/emacs-module.c (funcall_module):
* src/fns.c (Fstring_distance):
* src/font.c (font_sort_entities):
* src/haikumenu.c (digest_menu_items, haiku_menu_show):
* src/pgtkselect.c (Fpgtk_register_dnd_targets):
* src/xfns.c (Fx_begin_drag):
* src/xmenu.c (x_menu_show):
* src/xterm.c (x_dnd_compute_toplevels, handle_one_xevent)
(x_term_init):
Prefer SAFE_NALLOCA to doing size multiplication by hand, to catch
unlikely integer overflows.
* src/comp.c (emit_simple_limple_call): Fix bug where
SAFE_FREE was called too early, leading to unlikely
use of freed storage.
* src/xterm.c (handle_one_xevent): Remove side effects
from SAFE_ALLOCA args, as the args are evaluated twice.
This leads to simpler code in the users, and more efficient machine
code because we don't repeatedly need to fetch the `table_size`
and `key_and_value` fields of the hash table object.
* src/lisp.h (DOHASH): Rewrite.
* src/composite.c (composition_gstring_lookup_cache): Simplify.
(composition_gstring_cache_clear_font):
* src/print.c (print):
* src/pdumper.c (hash_table_contents):
* src/minibuf.c (Ftest_completion):
* src/json.c (lisp_to_json_nonscalar_1):
* src/emacs-module.c (module_global_reference_p):
* src/comp.c (compile_function, Fcomp__compile_ctxt_to_file):
* src/fns.c (Fmaphash): Adjust to new calling convention.
This saves several words in the hash table object at the cost of an
indirection at runtime. This seems to be a gain in overall
performance.
FIXME: We cache hash test objects in a rather clumsy way. A better
solution is sought.
* src/lisp.h (struct Lisp_Hash_Table): Use a pointer to the test
struct. All references adapted.
* src/alloc.c (garbage_collect):
* src/fns.c (struct hash_table_user_test, hash_table_user_tests)
(mark_fns, get_hash_table_user_test): New state for caching test
structs, and functions managing it.
This improves performance in several ways. Separate functions are
used depending on whether the caller has a hash value computed or not.
* src/fns.c (hash_lookup_with_hash, hash_lookup_get_hash): New.
(hash_lookup): Remove hash return argument.
All callers adapted.
hash_lookup_with_hash hash_hash_t arg
This improves typing, saves pointless tagging and untagging, and
prepares for further changes. The new typedef hash_hash_t is an alias
for EMACS_UINT, and hash values are still limited to the fixnum range.
We now use hash_unused instead of Qnil to mark unused entries.
* src/lisp.h (hash_hash_t): New typedef for EMACS_UINT.
(hash_unused): New constant.
(struct hash_table_test): `hashfn` now returns
hash_hash_t. All callers and implementations changed.
(struct Lisp_Hash_Table): Retype hash vector to an array of
hash_hash_t. All code using it changed accordingly.
(HASH_HASH, hash_from_key):
* src/fns.c (set_hash_index_slot, hash_index_index)
(hash_lookup_with_hash, hash_lookup_get_hash, hash_put):
(hash_lookup, hash_put): Retype hash value arguments
and return values. All callers adapted.
These parameters have no visible semantics and are hardly ever used,
so just use the default values for all hash tables. This saves
memory, shrinks the external representation, and will improve
performance.
* src/fns.c (std_rehash_size, std_rehash_threshold): New.
(hash_index_size): Use std_rehash_threshold. Remove table argument.
All callers updated.
(make_hash_table): Remove rehash_size and rehash_threshold args.
All callers updated.
(maybe_resize_hash_table)
(Fhash_table_rehash_size, Fhash_table_rehash_threshold):
Use std_rehash_size and std_rehash_threshold.
(Fmake_hash_table): Ignore :rehash-size and :rehash-threshold args.
* src/lisp.h (struct Lisp_Hash_Table):
Remove rehash_size and rehash_threshold fields.
(DEFAULT_REHASH_THRESHOLD, DEFAULT_REHASH_SIZE): Remove.
* src/lread.c (hash_table_from_plist): Don't read rehash-size or
rehash-threshold.
(syms_of_lread): Remove unused symbols.
* src/print.c (print_object): Don't print rehash-size or rehash-threshold.
* src/pdumper.c (dump_hash_table): Don't dump removed fields.
This takes less space (saves an entire word) and is more type-safe.
No change in behaviour.
* src/lisp.h (hash_table_weakness_t): New.
(struct Lisp_Hash_Table): Replace Lisp object `weak` with enum
`weakness`.
* src/fns.c
(keep_entry_p, hash_table_weakness_symbol): New.
(make_hash_table): Retype argument. All callers updated.
(sweep_weak_table, Fmake_hash_table, Fhash_table_weakness):
* src/alloc.c (purecopy_hash_table, purecopy, process_mark_stack):
* src/pdumper.c (dump_hash_table):
* src/print.c (print_object): Use retyped field.
Qunbound is used for many things; using a predicate and constant for
the specific purpose of unused hash entry keys allows us to locate
them and make changes much more easily.
* src/lisp.h (HASH_UNUSED_ENTRY_KEY, hash_unused_entry_key_p):
New constant and function.
* src/comp.c (compile_function, Fcomp__compile_ctxt_to_file):
* src/composite.c (composition_gstring_cache_clear_font):
* src/emacs-module.c (module_global_reference_p):
* src/fns.c (make_hash_table, maybe_resize_hash_table, hash_put)
(hash_remove_from_table, hash_clear, sweep_weak_table, Fmaphash):
* src/json.c (lisp_to_json_nonscalar_1):
* src/minibuf.c (Ftry_completion, Fall_completions, Ftest_completion):
* src/print.c (print, print_object):
Use them.
* configure.ac: Default modules to on. Remove check for
__attribute__((cleanup)). However, keep the new `ifavailable'
value for systems without dlopen.
* src/emacs-module.c (MODULE_HANDLE_NONLOCAL_EXIT): Don't rely
on cleanup attribute and correctly reset handlerlist upon
longjmp.
(MODULE_INTERNAL_CLEANUP): New macro.
(module_make_global_ref, module_free_global_ref)
(module_make_function, module_get_function_finalizer)
(module_set_function_finalizer, module_make_interactive)
(module_funcall, module_intern, module_type_of)
(module_extract_integer, module_make_integer, module_extract_float)
(module_make_float, module_copy_string_contents)
(module_make_string, module_make_unibyte_string)
(module_make_user_ptr, module_get_user_ptr, module_set_user_ptr)
(module_get_user_finalizer, module_set_user_finalizer)
(module_vec_set, module_vec_get, module_vec_size)
(module_process_input, module_extract_time, module_make_time)
(module_extract_big_integer, module_make_big_integer)
(module_open_channel): Call MODULE_INTERNAL_CLEANUP prior to
returning.
* INSTALL.android: Port to MIPS.
* configure.ac (modules): Default to ifavailable.
Write actual test for __attribute__((cleanup)).
* m4/ndk-build.m4: Recognize mips and mips64.
* src/emacs-module.c: Remove broken HAS_ATTRIBUTE test.
C23 is adding the C++ keywords bool, true, and false;
prefer them to <stdbool.h> if they are available.
* admin/merge-gnulib (GNULIB_MODULES):
Add stdbool, which emulates C23 on pre-C23 platforms.
(AVOIDED_MODULES): Remove stdbool; Gnulib has renamed
this module to stdbool-c99 and nobody uses it so it does
not need to be avoided.
* m4/c-bool.m4: New file, from Gnulib stdbool module.
* lib-src/seccomp-filter.c, src/conf_post.h, src/dynlib.h:
* src/emacs-module.c, src/nsterm.m, src/systhread.h:
* test/src/emacs-module-resources/mod-test.c:
Use the C23 style and use bool without including <stdbool.h>.
The Gnulib stdbool module causes config.h to include stdbool.h
on pre-C23 platforms.
* src/emacs-module.h.in:
Don’t include <stdbool.h> if C23 or later, or if
it has already been included.
* doc/lispref/commands.texi (Using Interactive): Document the
extended `interactive' form.
* doc/lispref/loading.texi (Autoload): Document list-of-modes
form.
* lisp/emacs-lisp/autoload.el (make-autoload): Pick the list of
modes from `interactive' out of the functions.
* lisp/emacs-lisp/bytecomp.el (byte-compile-lambda): Allow for the
extended `interactive' form.
* src/callint.c (Finteractive): Document the extended form.
* src/data.c (Finteractive_form): Return the interactive form in
the old format (even when there's an extended `interactive') to
avoid having other parts of Emacs be aware of this.
(Fcommand_modes): New defun.
* src/emacs-module.c (GCALIGNED_STRUCT): Allow for modules to
return command modes.
* src/lisp.h: New function module_function_command_modes.
When creating multibyte or unibyte strings, we should guarantee the
following invariants:
- When creating empty strings, a NULL data pointer should be allowed.
This often arises in practice if the string length isn't known in
advance, and we don't want to unnecessarily trigger undefined
behavior. Since functions like memcpy might not accept NULL
pointers, use the canonical empty string objects in this case.
- Nonzero strings should be guaranteed to be unique and mutable.
These are the same guarantees expected from Lisp functions such as
'make-string' or 'unibyte-string'. On the other hand, empty strings
might not be unique.
* src/emacs-module.c (module_make_string)
(module_make_unibyte_string): Correctly handle empty strings.
* test/src/emacs-module-resources/mod-test.c (Fmod_test_make_string):
New test function.
(emacs_module_init): Expose it.
* test/src/emacs-module-tests.el (mod-test-make-string/empty)
(mod-test-make-string/nonempty): New unit tests.
* doc/lispref/internals.texi (Module Values): Document properties and
corner cases for strings.
We used to store module runtime and environment pointers in the static
lists Vmodule_runtimes and Vmodule_environments. However, this is
incorrect because these objects have to be kept per-thread. With this
naive approach, interleaving module function calls in separate threads
leads to environments being removed in the wrong order, which in turn
can cause local module values to be incorrectly garbage-collected.
The fix isn't completely trivial: specbinding the lists wouldn't work
either, because then the garbage collector wouldn't find the
environments in other threads than the current ones, again leading to
objects being garbage-collected incorrectly. While introducing custom
pseudovector types would fix this, it's simpler to put the runtime and
environment pointers into the specbinding list as new specbinding
kinds. This works since we need to unwind them anyway, and we only
ever treat the lists as a stack. The thread switching machinery
ensures that the specbinding lists are thread-local, and that all
elements of the specbinding lists in all threads are marked during
garbage collection.
Module assertions now have to walk the specbinding list for the
current thread, which is more correct since they now only find
environments for the current thread. As a result, we can now remove
the faulty Vmodule_runtimes and Vmodule_environments variables
entirely.
Also add a unit test that exemplifies the problem. It interleaves two
module calls in two threads so that the first call ends while the
second one is still active. Without this change, this test triggers
an assertion failure.
* src/lisp.h (enum specbind_tag): Add new tags for module runtimes and
environments.
* src/eval.c (record_unwind_protect_module): New function to record a
module object in the specpdl list.
(do_one_unbind): Unwind module objects.
(backtrace_eval_unrewind, default_toplevel_binding, lexbound_p)
(Fbacktrace__locals): Deal with new specbinding types.
(mark_specpdl): Mark module environments as needed.
* src/alloc.c (garbage_collect): Remove call to 'mark-modules'.
Garbage collection of module values is now handled as part of marking
the specpdl of each thread.
* src/emacs-module.c (Fmodule_load, funcall_module): Use specpdl to
record module runtimes and environments.
(module_assert_runtime, module_assert_env, value_to_lisp): Walk
through specpdl list instead of list variables.
(mark_module_environment): Rename from 'mark_modules'. Don't attempt
to walk though current thread's environments only, since that would
miss other threads.
(initialize_environment, finalize_environment): Don't change
Vmodule_environments variable; environments are now in the specpdl
list.
(finalize_environment_unwind, finalize_runtime_unwind): Make 'extern'
since do_one_unbind now calls them.
(finalize_runtime_unwind): Don't change Vmodule_runtimes variable;
runtimes are now in the specpdl list.
(syms_of_module): Remove Vmodule_runtimes and Vmodule_environments.
* test/data/emacs-module/mod-test.c (Fmod_test_funcall): New test
function.
(emacs_module_init): Bind it.
* test/src/emacs-module-tests.el (emacs-module-tests--variable): New
helper type to guard access to state in a thread-safe way.
(emacs-module-tests--wait-for-variable)
(emacs-module-tests--change-variable): New helper functions.
(emacs-module-tests/interleaved-threads): New unit test.
* doc/lispref/internals.texi (Module Values): Document
make_unibyte_string (bug#34873).
* src/emacs-module.c (module_make_unibyte_string): New function.
(initialize_environment): Export it.
* src/module-env-25.h: Define it.
* test/data/emacs-module/mod-test.c (Fmod_test_return_unibyte):
Test it.
* test/src/emacs-module-tests.el (module/unibyte): Test it.
* src/module-env-28.h: Add field for 'make_interactive' function.
* src/emacs-module.c (Lisp_Module_Function): Add new field holding the
interactive form.
(allocate_module_function): Adapt to structure layout change.
(module_make_interactive, module_function_interactive_form): New
functions.
(initialize_environment): Use them.
* src/eval.c (Fcommandp):
* src/data.c (Finteractive_form): Also handle interactive module
functions.
* test/data/emacs-module/mod-test.c (Fmod_test_identity): New test
function.
(emacs_module_init): Create two interactive module test functions.
* test/src/emacs-module-tests.el (module/interactive/return-t)
(module/interactive/return-t-int, module/interactive/identity):
New unit tests.
* doc/lispref/internals.texi (Module Functions): Document new
function. Rework paragraph about wrapping module functions, as the
example no longer applies.
* etc/NEWS: Document new facility.
* configure.ac: Use AC_CHECK_FUNCS_ONCE for __lsan_ignore_object.
* src/buffer.c, src/data.c, src/emacs-module.c, src/regex-emacs.c:
* src/search.c: Use __lsan_ignore_object unconditionally, and don’t
include sanitizer/lsan_interface.h.
* src/lisp.h (__lsan_ignore_object): Provide a dummy in the
typical case where leak sanitization is not available.
* src/emacs-module.c (initialize_environment): Call
__lsan_ignore_object only if HAVE_SANITIZER_LSAN_INTERFACE_H is
undefined. This fixes compilation on systems that don't have
__lsan_* functions.
We intentionally leak some objects. Prevent the ASan leak detector
from raising false alarms in these cases.
* configure.ac: Search for lsan_interface.h header.
* src/data.c (make_blv): Allow leaking of buffer-local values.
* src/buffer.c (enlarge_buffer_text): Allow leaking of buffer text.
* src/emacs-module.c (Fmodule_load, initialize_environment): Allow
intentional leak of runtime and environment objects if module
assertions are enabled.
We can't just use a hash lookup because a global and a local reference
might refer to the same Lisp object.
* src/emacs-module.c (module_free_global_ref): More precise check for
global liveness.
* test/data/emacs-module/mod-test.c (Fmod_test_globref_invalid_free):
New test module function.
(emacs_module_init): Export it.
* test/src/emacs-module-tests.el
(module--test-assertions--globref-invalid-free): New unit test.
