* doc/lispref/internals.texi (Module Values): Document
make_unibyte_string (bug#34873).
* src/emacs-module.c (module_make_unibyte_string): New function.
(initialize_environment): Export it.
* src/module-env-25.h: Define it.
* test/data/emacs-module/mod-test.c (Fmod_test_return_unibyte):
Test it.
* test/src/emacs-module-tests.el (module/unibyte): Test it.
* src/module-env-28.h: Add field for 'make_interactive' function.
* src/emacs-module.c (Lisp_Module_Function): Add new field holding the
interactive form.
(allocate_module_function): Adapt to structure layout change.
(module_make_interactive, module_function_interactive_form): New
functions.
(initialize_environment): Use them.
* src/eval.c (Fcommandp):
* src/data.c (Finteractive_form): Also handle interactive module
functions.
* test/data/emacs-module/mod-test.c (Fmod_test_identity): New test
function.
(emacs_module_init): Create two interactive module test functions.
* test/src/emacs-module-tests.el (module/interactive/return-t)
(module/interactive/return-t-int, module/interactive/identity):
New unit tests.
* doc/lispref/internals.texi (Module Functions): Document new
function. Rework paragraph about wrapping module functions, as the
example no longer applies.
* etc/NEWS: Document new facility.
* test/src/emacs-module-tests.el (mod-test-globref-reordered): New
unit test.
* test/data/emacs-module/mod-test.c (Fmod_test_globref_reordered): New
test module function.
(emacs_module_init): Export it.
This backports commit 9f01ce6327 from master. Since the bug isn’t
present on emacs-27, just backport the new test case.
* test/data/emacs-module/mod-test.c (Fmod_test_globref_invalid_free):
New test module function.
(emacs_module_init): Export it.
* test/src/emacs-module-tests.el
(module--test-assertions--globref-invalid-free): New unit test.
This backports commit 6355a3ec62 from master. Since the bug isn’t
present in emacs-27, just backport the test case.
* test/data/emacs-module/mod-test.c
(Fmod_test_invalid_store_copy): New test module function.
(emacs_module_init): Export it.
* test/src/emacs-module-tests.el
(module--test-assertions--load-non-live-object-with-global-copy):
New unit test.
We can't just use a hash lookup because a global and a local reference
might refer to the same Lisp object.
* src/emacs-module.c (module_free_global_ref): More precise check for
global liveness.
* test/data/emacs-module/mod-test.c (Fmod_test_globref_invalid_free):
New test module function.
(emacs_module_init): Export it.
* test/src/emacs-module-tests.el
(module--test-assertions--globref-invalid-free): New unit test.
We can't simply look up the Lisp object in the global reference table
because an invalid local and a valid global reference might refer to
the same object. Instead, we have to test the address of the global
reference against the stored references.
* src/emacs-module.c (module_global_reference_p): New helper function.
(value_to_lisp): Use it.
* test/data/emacs-module/mod-test.c
(Fmod_test_invalid_store_copy): New test module function.
(emacs_module_init): Export it.
* test/src/emacs-module-tests.el
(module--test-assertions--load-non-live-object-with-global-copy):
New unit test.
Instead of doing GMP by hand, use the Gnulib libgmp module.
* .gitignore: Add lib/gmp.h.
* admin/merge-gnulib (GNULIB_MODULES): Add libgmp.
* configure.ac (GMP_LIB, GMP_OBJ): Remove. Gnulib uses the name
LIB_GMP, so all uses changed. All uses of GMP_OBJ removed.
(HAVE_GMP): Set this from Gnulib’s variables.
* lib/gnulib.mk.in, m4/gnulib-comp.m4: Regenerate.
* lib/mini-gmp-gnulib.c, lib/mini-gmp.c, lib/mini-gmp.h, m4/libgmp.m4:
New files, copied from Gnulib.
* src/bignum.h, test/data/emacs-module/mod-test.c:
Include gmp.h unconditionally.
* src/mini-gmp-emacs.c, src/mini-gmp.c, src/mini-gmp.h:
Remove. This moves these files from src to lib, and
updates them to the current GMP version.
* test/Makefile.in (GMP_H): New macro.
($(test_module)): Use it to decide whether to compile
mini-gmp-gnulib.c too.
These changes let the code compile and produce a valid DLL, but the
test hangs. It looks like the hang is in Fdelete_process, when it
closes one of the descriptors of the pipe process.
In addition, this use of the pipe process cannot currently work
on MS-Windows, since make-pipe-process doesn't set up the reader
thread to read from the Emacs's side of the pipe, so the select
emulation doesn't know there's stuff to read from that pipe.
* test/data/emacs-module/mod-test.c [WINDOWSNT]: Include
windows.h.
(ALIGN_STACK) [!__x86_64__]: Define for 32-bit builds.
(sleep_for_half_second): New function.
(write_to_pipe): Declare return type differently for WINDOWSNT.
Call sleep_for_half_second.
(Fmod_test_async_pipe) [WINDOWSNT]: Use _beginthread as substitute
for pthread_create.
(invalid_finalizer): Replace non_ASCII character in a comment.
* test/src/emacs-module-tests.el (module/async-pipe): Skip on
MS-Windows, as the test fails and then hangs.
A common complaint about the module API is that modules can't
communicate asynchronously with Emacs. While it isn't possible to
call arbitrary Emacs functions asynchronously, writing to a pipe
should always be fine and is a pretty low-hanging fruit.
This patch implements a function that adapts an existing pipe
process. That way, users can use familiar tools like process filters
or 'accept-process-output'.
* src/module-env-28.h: Add 'open_channel' module function.
* src/emacs-module.c (module_open_channel): Provide definition for
'open_channel'.
(initialize_environment): Use it.
* src/process.c (open_channel_for_module): New helper function.
(syms_of_process): Define necessary symbol.
* test/src/emacs-module-tests.el (module/async-pipe): New unit test.
* test/data/emacs-module/mod-test.c (signal_system_error): New helper
function.
(signal_errno): Use it.
(write_to_pipe): New function running in the background.
(Fmod_test_async_pipe): New test module function.
(emacs_module_init): Export it.
* doc/lispref/internals.texi (Module Misc): Document new module
function.
* doc/lispref/processes.texi (Asynchronous Processes): New anchor
for pipe processes.
* etc/NEWS: Document 'open_channel' function.
This is especially useful in cases where modules only use a single
entry point and use the data to dispatch to the actual function. Such
a design is common for languages such as Go and C++.
* src/emacs-module.c (module_function_data): New function.
* src/print.c (print_vectorlike): Use it to print module function data
if not NULL.
(print_object): Adapt size of buffer.
* test/data/emacs-module/mod-test.c (emacs_module_init): Pass some
non-NULL data to ‘mod-test-sum’.
(Fmod_test_sum): Check that correct data is passed through.
* test/src/emacs-module-tests.el (mod-test-sum-test)
(module-function-object): Adapt unit tests.
* src/module-env-28.h: Add new module environment functions to
module environment for Emacs 28.
* src/emacs-module.h.in: Document that 'emacs_finalizer' also works
for function finalizers.
* src/emacs-module.c (CHECK_MODULE_FUNCTION): New function.
(struct Lisp_Module_Function): Add finalizer data member.
(module_make_function): Initialize finalizer.
(module_get_function_finalizer)
(module_set_function_finalizer): New module environment functions.
(module_finalize_function): New function.
(initialize_environment): Initialize new environment functions.
* src/alloc.c (cleanup_vector): Call potential module function
finalizer during garbage collection.
* test/data/emacs-module/mod-test.c (signal_error): New helper
function.
(memory_full): Use it.
(finalizer): New example function finalizer.
(Fmod_test_make_function_with_finalizer)
(Fmod_test_function_finalizer_calls): New test module functions.
(emacs_module_init): Define them.
* test/src/emacs-module-tests.el (module/function-finalizer): New unit
test.
* doc/lispref/internals.texi (Module Functions): Document new
functionality.
(Module Misc): Move description of 'emacs_finalizer' type to 'Module
Functions' node, and add a reference to it.
* etc/NEWS: Mention new functionality.
* test/data/emacs-module/mod-test.c (extract_big_integer): Fix
calculation of size of 'magnitude' when 'emacs_limb_t' is wider
than 'unsigned long'.
* test/src/emacs-module-tests.el
(module--test-assertions--load-non-live-object)
(module--test-assertions--call-emacs-from-gc): On MS-Windows,
check also mod-test-emacs with ".exe" appended, before
skipping the tests.
As described in the new comment added to emacs-module.c, using GMP
directly in the module interface has significant downsides: it couples
the module interface directly to the implementation and requires
module authors to link their module against the same GMP library as
Emacs itself, which is often difficult and an unnecessary burden. By
picking a representation for the magnitude that often matches the one
used by GMP, we can avoid overhead when converting from and to GMP in
most cases.
Loading the test module in test/data/emacs-module and evaluating
(dotimes (_ 10000)
(mod-test-double (* 2 most-negative-fixnum)))
under Callgrind shows that on my (GNU/Linux) machine Emacs only spends
10% of the CPU time of mod-test-double in mpz_import and mpz_export
combined, even though that function does little else. (By contrast,
30% is spent in allocate_pseudovector.)
* src/emacs-module.h.in: Don't check EMACS_MODULE_GMP. Don't include
gmp.h. Remove emacs_mpz structure. Instead, define type alias
emacs_limb_t and macro EMACS_LIMB_MAX.
* src/module-env-27.h: Change interface of extract_big_integer and
make_big_integer to take a sign-magnitude representation instead of
mpz_t.
* src/emacs-module.c: Don't check EMACS_MODULE_GMP or
EMACS_MODULE_HAVE_MPZ_T. Add a comment about the chosen
implementation.
(module_extract_big_integer, module_make_big_integer): Reimplement
without using mpz_t in the interface.
* doc/lispref/internals.texi (Module Values): Adapt function
documentation and example. Stop mentioning GMP and EMACS_MODULE_GMP.
* test/data/emacs-module/mod-test.c: Don't define EMACS_MODULE_GMP or
EMACS_MODULE_HAVE_MPZ_T.
(memory_full, extract_big_integer, make_big_integer): New helper
functions, identical to example in the Info documentation.
(Fmod_test_nanoseconds, Fmod_test_double): Adapt to new interface.
* test/data/emacs-module/mod-test.c (NDEBUG): Undef, to prevent
GCC from complaining “error: unused variable ‘dummy’” and failing
to build the test, if --enable-gcc-warnings.
* test/data/emacs-module/mod-test.c (Fmod_test_nanoseconds)
[__MINGW32__]: Use _Static_assert, not static_assert, as the
latter is not available in mingw.org's MinGW's assert.h.
Add support for a new preprocessor macro EMACS_MODULE_HAVE_MPZ_T to
emacs-module.h. If this macro is defined, assume that mpz_t is
already defined and don’t include gmp.h.
Don’t document the new macro for now, as it’s unclear whether we want
to support this in modules outside the Emacs tree.
* src/emacs-module.h.in: Allow user to prevent inclusion of gmp.h.
* src/emacs-module.c: Use mini-gmp if GMP is unavailable. Don’t
include gmp.h.
* src/lisp.h: Don’t require gmp.h. It’s not needed for lisp.h.
* test/Makefile.in (GMP_LIB, GMP_OBJ): New variables.
($(test_module)): Use them.
* test/data/emacs-module/mod-test.c: Use mini-gmp if GMP is unavailable.
* doc/lispref/internals.texi (Module Values): Clarify that the
truncation is towards negative infinity.
* test/data/emacs-module/mod-test.c (Fmod_test_nanoseconds): Add test
function.
(emacs_module_init): Define it.
* test/src/emacs-module-tests.el (mod-test-nanoseconds): New unit test.
* src/module-env-27.h: Add new module functions to convert big
integers.
* src/emacs-module.h.in (emacs_mpz): Define if GMP is available.
* src/emacs-module.c (module_extract_big_integer)
(module_make_big_integer): New functions.
(initialize_environment): Use them.
* test/data/emacs-module/mod-test.c (Fmod_test_double): New test
function.
(emacs_module_init): Define it.
* test/src/emacs-module-tests.el (mod-test-double): New unit test.
* doc/lispref/internals.texi (Module Values): Document new functions.
Time values are a fundamental data type, and such conversions are hard
to implement within modules because of the various forms of time
values in Emacs Lisp. Adding dedicated conversion functions can
significantly simplify module code dealing with times.
This approach uses nanosecond precision. While Emacs in theory has
support for higher-precision time values, in practice most languages
and standards, such as POSIX, C, Java, and Go, have settled on
nanosecond-precision integers to represent time.
* src/emacs-module.h.in: Add header for struct timespec.
* src/module-env-27.h: Add module functions for time conversion.
* src/emacs-module.c (module_extract_time, module_make_time): New
functions.
(initialize_environment): Use them.
* test/data/emacs-module/mod-test.c (Fmod_test_add_nanosecond): New
test function.
(emacs_module_init): Define it.
* test/src/emacs-module-tests.el (mod-test-add-nanosecond/valid)
(mod-test-add-nanosecond/nil, mod-test-add-nanosecond/invalid): New
unit tests.
* doc/lispref/internals.texi (Module Values): Document time
conversion functions.
pending_signals is often set if no quit is pending. This results in
bugs in module code if the module returns but no quit is actually
pending.
As a better alternative, add a new process_input environment function
for Emacs 27. That function processes signals (like maybe_quit).
* configure.ac: Add module snippet for Emacs 27.
* src/module-env-27.h: New file.
* src/emacs-module.h.in: Add process_input function to environment
interface.
* src/emacs-module.c (module_should_quit): Use QUITP macro to check
whether the caller should quit.
(module_process_input): New function.
(initialize_environment): Use it.
* src/eval.c: Remove obsolete comment.
* test/data/emacs-module/mod-test.c (signal_wrong_type_argument)
(signal_errno): New helper functions.
(Fmod_test_sleep_until): New test module function.
* test/src/emacs-module-tests.el (mod-test-sleep-until): New unit
test.
* doc/lispref/internals.texi (Module Misc): Document process_input.
54fb383 (origin/emacs-26) Fix detection of freed emacs_values (Bug#32...
769d0cd ; Fix out-of-tree build for mod-test.so
9a1329e Avoid crashes with very wide TTY frames on MS-Windows
9a613d3 Prevent `modify-file-local-variable-prop-line' from adding ex...
624e7dc Update GNOME bugtracker URLs
51ef6d5 Clarify in the Emacs manual that ChangeLog files are not used
6e08019 Recognize codepage 65001 as a valid encoding
1a350d7 ; * etc/NEWS: Fix format of first lines of some entries.
22d1f53 Avoid compilation warning in nt/addpm.c
7bc9ce7 Fix duplicate custom group names in bibtex.el
a9cf938 Fix outdated text in the Calc manual
Conflicts:
etc/NEWS
etc/PROBLEMS
src/emacs-module.c
src/gtkutil.c
src/image.c
src/xterm.c
test/Makefile.in
* src/emacs-module.c (module_free_global_ref): Compare a value to be
freed with all entries of the list.
* test/data/emacs-module/mod-test.c (Fmod_test_globref_free): New
function.
(emacs_module_init): Make it accessible from Lisp.
* test/src/emacs-module-tests.el (mod-test-globref-free-test): New
test which uses it.
This puts functions defined in the module into the 'load-history'.
* test/data/emacs-module/mod-test.c (bind_function): Use 'defalias'
instead of 'fset'.
* test/src/emacs-module-tests.el (module/describe-function-1): Adapt
unit test.
* test/Makefile.in (abs_top_srcdir): Add variable, needed by
CPPFLAGS.
* test/data/emacs-module/mod-test.c: Include <limits.h>.
(pT, pZ, T_TYPE, Z_TYPE): Compatibility macros, for systems that
don't support %td and %zu format specs.
(emacs_module_init): Use compatibility macros to make the error
messages print meaningful values (and avoid compiler warnings).
Most of this change is to boilerplate commentary such as license URLs.
This change was prompted by ftp://ftp.gnu.org's going-away party,
planned for November. Change these FTP URLs to https://ftp.gnu.org
instead. Make similar changes for URLs to other organizations moving
away from FTP. Also, change HTTP to HTTPS for URLs to gnu.org and
fsf.org when this works, as this will further help defend against
man-in-the-middle attacks (for this part I omitted the MS-DOS and
MS-Windows sources and the test tarballs to keep the workload down).
HTTPS is not fully working to lists.gnu.org so I left those URLs alone
for now.
It's technically possible to write a user pointer finalizer that calls
into Emacs module functions. This would be disastrous because it
would allow arbitrary Lisp code to run during garbage collection.
Therefore extend the module assertions to check for this case.
* src/emacs-module.c (module_assert_thread): Also check whether a
garbage collection is in progress.
* test/data/emacs-module/mod-test.c (invalid_finalizer)
(Fmod_test_invalid_finalizer): New test module functions.
(emacs_module_init): Register new test function.
* test/src/emacs-module-tests.el (module--test-assertion)
(module--with-temp-directory): New helper macros.
(module--test-assertions--load-non-live-object): Rename existing
unit test, use helper macros.
(module--test-assertions--call-emacs-from-gc): New unit test.
The test/data/emacs-module/Makefile only built a single target, and
inlining it into test/Makefile simplifies dependency tracking and
reduces code duplication.
* configure.ac: Don't build test/data/emacs-module/Makefile.
* Makefile.in ($(test_module)): Inline compilation.
(clean): Also clean test module outputs.
This makes it easier to pass compilation flags around.
* configure.ac: Also build test module Makefile.
* test/data/emacs-module/Makefile.in: New makefile template.
* test/Makefile.in ($(test_module)): No longer necessary to pass
@MODULES_SUFFIX@ around.
* .gitignore: Test module Makefile can now be ignored.
Add a new command-line option '-module-assertions' that users can
enable developing or debugging a module. If this option is present,
Emacs performs additional checks to verify that modules fulfill their
requirements. These checks are expensive and crash Emacs if modules
are invalid, so disable them by default.
This is a command-line option instead of an ordinary variable because
changing it while Emacs is running would cause data structure
imbalances.
* src/emacs.c (main): New command line option '-module-assertions'.
* src/emacs-module.c (module_assert_main_thread)
(module_assert_runtime, module_assert_env, module_assert_value):
New functions to assert module requirements.
(syms_of_module): New uninterned variable 'module-runtimes'.
(init_module_assertions, in_main_thread, module_abort): New helper
functions.
(initialize_environment): Initialize value list. If assertions are
enabled, use a heap-allocated environment object.
(finalize_environment): Add assertion that environment list is never
empty.
(finalize_runtime_unwind): Pop module runtime object stack.
(value_to_lisp): Assert that the value is valid.
(lisp_to_value): Record new value if assertions are enabled.
(mark_modules): Mark allocated object list.
(MODULE_FUNCTION_BEGIN_NO_CATCH)
(module_non_local_exit_check, module_non_local_exit_clear)
(module_non_local_exit_get, module_non_local_exit_signal)
(module_non_local_exit_throw): Assert thread and environment.
(module_get_environment): Assert thread and runtime.
(module_make_function, module_funcall, module_intern)
(module_funcall, module_make_integer, module_make_float)
(module_make_string, module_make_user_ptr, module_vec_get)
(funcall_module, Fmodule_load): Adapt callers.
(module_make_global_ref): If assertions are enabled, use the global
environment to store global values.
(module_free_global_ref): Remove value from global value list.
* test/Makefile.in (EMACSOPT): Enable module assertions when testing
modules.
* test/data/emacs-module/mod-test.c (Fmod_test_invalid_store)
(Fmod_test_invalid_load): New functions to test module assertions.
(emacs_module_init): Bind the new functions.
* test/src/emacs-module-tests.el (mod-test-emacs): New constant for
the Emacs binary file.
(mod-test-file): New constant for the test module file name.
(module--test-assertions): New unit test.
Instead of a lambda, create a new type containing all data required to
call the function, and support it in the evaluator. Because this type
now also needs to store the function documentation, it is too big for
Lisp_Misc; use a pseudovector instead. That also has the nice benefit
that we don't have to add special support to the garbage collector.
Since the new type is user-visible, give it a predicate.
Now we can easily support 'help-function-args' and 'func-arity'; add
unit tests for these.
* src/lisp.h (allocate_module_function, MODULE_FUNCTIONP)
(XMODULE_FUNCTION): New pseudovector type 'module function'.
* src/eval.c (FUNCTIONP): Also treat module functions as functions.
(funcall_lambda, Ffuncall, eval_sub): Add support for calling module
functions.
(Ffunc_arity): Add support for detecting the arity of module
functions.
* src/emacs-module.c (module_make_function): Adapt to new structure.
Return module function object directly instead of wrapping it in a
lambda; remove FIXME.
(funcall_module): New function to call module functions. Replaces
`internal--module-call' and is called directly from eval.c.
(syms_of_module): Remove internal helper function, which is no longer
needed.
(module_function_arity): New helper function.
* src/data.c (Ftype_of): Adapt to new implementation.
(Fmodule_function_p, syms_of_data): New user-visible function. Now
that module functions are first-class objects, they deserve a
predicate. Define it even if not compiled with --enable-modules so
that Lisp code doesn't have to check for the function's existence.
* src/doc.c (Fdocumentation): Support module functions.
* src/print.c (print_object): Adapt to new implementation.
* src/alloc.c (mark_object): Specialized garbage collector support is
no longer needed.
* lisp/help.el (help-function-arglist): Support module functions.
While there, simplify the arity calculation by using `func-arity',
which does the right thing for all kinds of functions.
* test/data/emacs-module/mod-test.c: Amend docstring so we can test
the argument list.
* test/src/emacs-module-tests.el (mod-test-sum-docstring): Adapt to
new docstring.
(mod-test-non-local-exit-signal-test): Because `internal--module-call'
is gone, the backtrace has changed and no longer leaks the
implementation.
(module--func-arity): New test for `func-arity'.
(module--help-function-arglist): New test for `help-function-arglist'.
* test/Makefile.in (ELFILES): Exclude module test if modules aren't
configured.
(EMACS_TEST_DIRECTORY): Expand test directory so that it's set
correctly even if Emacs changes the current directory.
($(srcdir)/src/emacs-module-tests.log)
($(test_module)): Proper dependency tracking for test module.
* test/data/emacs-module/Makefile (ROOT): Adapt to new location.
Remove 'check' target and EMACS variable, which are no longer
necessary.
(SO): Change to include period.
* test/src/emacs-module-tests.el (mod-test): Use EMACS_TEST_DIRECTORY
environment variable to reliably find test data.
* configure.ac (HAVE_MODULES, MODULES_SUFFIX): Add necessary
substitutions.
