Add a new command-line option '-module-assertions' that users can
enable developing or debugging a module. If this option is present,
Emacs performs additional checks to verify that modules fulfill their
requirements. These checks are expensive and crash Emacs if modules
are invalid, so disable them by default.
This is a command-line option instead of an ordinary variable because
changing it while Emacs is running would cause data structure
imbalances.
* src/emacs.c (main): New command line option '-module-assertions'.
* src/emacs-module.c (module_assert_main_thread)
(module_assert_runtime, module_assert_env, module_assert_value):
New functions to assert module requirements.
(syms_of_module): New uninterned variable 'module-runtimes'.
(init_module_assertions, in_main_thread, module_abort): New helper
functions.
(initialize_environment): Initialize value list. If assertions are
enabled, use a heap-allocated environment object.
(finalize_environment): Add assertion that environment list is never
empty.
(finalize_runtime_unwind): Pop module runtime object stack.
(value_to_lisp): Assert that the value is valid.
(lisp_to_value): Record new value if assertions are enabled.
(mark_modules): Mark allocated object list.
(MODULE_FUNCTION_BEGIN_NO_CATCH)
(module_non_local_exit_check, module_non_local_exit_clear)
(module_non_local_exit_get, module_non_local_exit_signal)
(module_non_local_exit_throw): Assert thread and environment.
(module_get_environment): Assert thread and runtime.
(module_make_function, module_funcall, module_intern)
(module_funcall, module_make_integer, module_make_float)
(module_make_string, module_make_user_ptr, module_vec_get)
(funcall_module, Fmodule_load): Adapt callers.
(module_make_global_ref): If assertions are enabled, use the global
environment to store global values.
(module_free_global_ref): Remove value from global value list.
* test/Makefile.in (EMACSOPT): Enable module assertions when testing
modules.
* test/data/emacs-module/mod-test.c (Fmod_test_invalid_store)
(Fmod_test_invalid_load): New functions to test module assertions.
(emacs_module_init): Bind the new functions.
* test/src/emacs-module-tests.el (mod-test-emacs): New constant for
the Emacs binary file.
(mod-test-file): New constant for the test module file name.
(module--test-assertions): New unit test.
Fix a FIXME in emacs-module.c. Put the printing into print.c, like
other types.
* src/print.c (print_vectorlike): Add code to print module functions.
* src/emacs-module.c (funcall_module): Stop calling
'module_format_fun_env'. Now that module functions are first-class
objects, they can be added to signal data directly.
(module_handle_signal): Remove now-unused function
'module_format_fun_env'.
* test/src/emacs-module-tests.el (mod-test-sum-test): Adapt unit test.
* src/eval.c (funcall_lambda): Adapt call to changed signature of
'funcall_module'.
Previously styled_format overwrite the argument vector. This is no
longer possible because there might be more than one specification per
argument. Use the existing auxiliary info array instead.
* src/editfns.c (styled_format): Record arguments in the info
structure instead of overwriting them.
* test/src/editfns-tests.el (format-with-field): Add unit test.
* doc/lispref/strings.texi (Formatting Strings):
Don’t allow mixing numbered with unnumbered format specs.
* src/editfns.c (styled_format): Don’t bother checking for field 0,
since it doesn’t crash and the behavior is not specified.
* test/src/editfns-tests.el (format-with-field): Adjust tests to
match current doc. Add more tests for out-of-range fields.
* src/editfns.c (styled_format): Allow field numbers in a %% spec.
No need for a special diagnostic for field numbers greater than
PTRDIFF_MAX. Reword diagnostic for field 0.
* test/src/editfns-tests.el (format-with-field): Adjust to match.
A field number explicitly specifies the argument to be formatted.
This is especially important for potential localization work, since
grammars of various languages dictate different word orders.
* src/editfns.c (Fformat): Update documentation.
(styled_format): Implement field numbers.
* doc/lispref/strings.texi (Formatting Strings): Document field numbers.
* lisp/emacs-lisp/bytecomp.el (byte-compile-format-warn): Adapt.
* test/src/editfns-tests.el (format-with-field): New unit test.
Instead of a lambda, create a new type containing all data required to
call the function, and support it in the evaluator. Because this type
now also needs to store the function documentation, it is too big for
Lisp_Misc; use a pseudovector instead. That also has the nice benefit
that we don't have to add special support to the garbage collector.
Since the new type is user-visible, give it a predicate.
Now we can easily support 'help-function-args' and 'func-arity'; add
unit tests for these.
* src/lisp.h (allocate_module_function, MODULE_FUNCTIONP)
(XMODULE_FUNCTION): New pseudovector type 'module function'.
* src/eval.c (FUNCTIONP): Also treat module functions as functions.
(funcall_lambda, Ffuncall, eval_sub): Add support for calling module
functions.
(Ffunc_arity): Add support for detecting the arity of module
functions.
* src/emacs-module.c (module_make_function): Adapt to new structure.
Return module function object directly instead of wrapping it in a
lambda; remove FIXME.
(funcall_module): New function to call module functions. Replaces
`internal--module-call' and is called directly from eval.c.
(syms_of_module): Remove internal helper function, which is no longer
needed.
(module_function_arity): New helper function.
* src/data.c (Ftype_of): Adapt to new implementation.
(Fmodule_function_p, syms_of_data): New user-visible function. Now
that module functions are first-class objects, they deserve a
predicate. Define it even if not compiled with --enable-modules so
that Lisp code doesn't have to check for the function's existence.
* src/doc.c (Fdocumentation): Support module functions.
* src/print.c (print_object): Adapt to new implementation.
* src/alloc.c (mark_object): Specialized garbage collector support is
no longer needed.
* lisp/help.el (help-function-arglist): Support module functions.
While there, simplify the arity calculation by using `func-arity',
which does the right thing for all kinds of functions.
* test/data/emacs-module/mod-test.c: Amend docstring so we can test
the argument list.
* test/src/emacs-module-tests.el (mod-test-sum-docstring): Adapt to
new docstring.
(mod-test-non-local-exit-signal-test): Because `internal--module-call'
is gone, the backtrace has changed and no longer leaks the
implementation.
(module--func-arity): New test for `func-arity'.
(module--help-function-arglist): New test for `help-function-arglist'.
If dladdr(3) isn't available or didn't work, the printed
representation of a module function will not include the file name,
but only the address. Make the tests pass in that case.
* test/src/emacs-module-tests.el (module-function-object): Fix match for
module function printed representation
* src/lread.c (load_warn_old_style_backquotes, Fload, read1)
(syms_of_lread): Rename `old-style-backquotes' to
`lread--old-style-backquotes', and clarify that it's for internal
use only.
* lisp/emacs-lisp/bytecomp.el (byte-compile-from-buffer): Rename
variable.
* test/src/lread-tests.el (lread-tests--old-style-backquotes): Add
unit test.
* emacs-lisp/bytecomp-tests.el
(bytecomp-tests--old-style-backquotes): Add unit test.
This test fails in my local machine because the data files
are compiled, and the test doesn't expect that.
* test/src/lread-tests.el (lread-test-bug26837): Match a suffix
ending with '.elc' when the data files are compiled.
* src/lread.c (readevalloop): Fix the "whole buffer" check to
operate in the correct buffer.
(Feval_buffer): Move point back to the start after checking
for lexical binding.
* test/src/lread-tests.el (lread-test-bug26837): New test.
* test/data/somelib.el, test/data/somelib2.el: New test data files.
This resolves a couple of FIXMEs in emacs-module.c.
* src/lisp.h (MODULE_FUNCTIONP, XMODULE_FUNCTION): New functions.
* src/alloc.c (make_module_function): New function.
(mark_object): GC support.
* src/data.c (Ftype_of, syms_of_data): Handle module function type.
* src/print.c (print_object): Print support for new type.
* src/emacs-module.c (module_make_function, Finternal_module_call):
Use new module function type, remove FIXMEs.
(module_format_fun_env): Adapt and give it external linkage.
* test/src/emacs-module-tests.el (module-function-object): Add unit
test.
* src/lread.c (load_warn_unescaped_character_literals, Fload, read1)
(syms_of_lread): Warn if unescaped character literals are
found (Bug#20152).
* lisp/emacs-lisp/bytecomp.el (byte-compile-from-buffer): Check for
unescaped character literals during byte compilation.
* test/src/lread-tests.el (lread-tests--unescaped-char-literals): New
unit test.
(lread-tests--with-temp-file, lread-tests--last-message): Helper
functions for unit test.
* test/lisp/emacs-lisp/bytecomp-tests.el
(bytecomp-tests--unescaped-char-literals): New unit test.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--with-temp-file):
Helper macro for unit test.
* test/src/editfns-tests.el (format-time-string-with-zone)
(format-time-string-with-outlandish-zone): Don’t format
timestamps near the Epoch, as this runs into bugs on MS-Windows,
and we don’t want to worry about those bugs.
* test/Makefile.in (ELFILES): Exclude module test if modules aren't
configured.
(EMACS_TEST_DIRECTORY): Expand test directory so that it's set
correctly even if Emacs changes the current directory.
($(srcdir)/src/emacs-module-tests.log)
($(test_module)): Proper dependency tracking for test module.
* test/data/emacs-module/Makefile (ROOT): Adapt to new location.
Remove 'check' target and EMACS variable, which are no longer
necessary.
(SO): Change to include period.
* test/src/emacs-module-tests.el (mod-test): Use EMACS_TEST_DIRECTORY
environment variable to reliably find test data.
* configure.ac (HAVE_MODULES, MODULES_SUFFIX): Add necessary
substitutions.
"Fix ert-tests when running compiled" 2016-12-06 accidentally
introduced a dependency on `bytecomp' into `ert'. As mentioned in
"Avoid ert test failures" 2017-04-18, the accidental dependency of ert
on bytecomp was masked by loading other libraries until recently.
* lisp/emacs-lisp/ert.el (ert--expand-should-1): Only use
`byte-compile-macro-environment' if it's bound.
* test/src/eval-tests.el: Add defvar for dynamic variable
`byte-compile-debug'.
Since copy-sequence seems to be needed anyway for records, have it
work on records, and remove copy-record as being superfluous.
* doc/lispref/records.texi (Records, Record Functions):
* lisp/emacs-lisp/cl-macs.el (cl-defstruct):
* lisp/emacs-lisp/eieio.el (make-instance, clone):
* test/src/alloc-tests.el (record-3):
Use copy-sequence, not copy-record, to copy records.
* doc/lispref/sequences.texi (Sequence Functions)
(Array Functions): Document that aref and copy-sequence
work on records.
* etc/NEWS: Omit copy-record.
* src/alloc.c (Fcopy_record): Remove.
* src/data.c (Faref): Document that arg can be a record.
* src/fns.c (Fcopy_sequence): Copy records, too.
In Greek, a sigma character has two lower case forms which depend on
their position in the word. Implement logic determining it.
* src/casefiddle.c (struct casing_context, case_character_impl): Don’t
assume inword is true when flag is CASE_UP and false when flag is
CASE_DOWN. For final sigma detection we need this information tracked
reliably;.
(CAPITAL_SIGMA, SMALL_SIGMA, SMALL_FINAL_SIGMA): New macros defining
Unicode code point of different forms of sigma letter.
(case_character): Implement support for final sigma casing.
(do_casify_multibyte_string, do_casify_multibyte_region): Update after
changes to case_character.
* test/src/casefiddle-tests.el (casefiddle-tests-casing): Add test
cases for final sigma.
Implement unconditional special casing rules defined in Unicode standard.
Among other things, they deal with cases when a single code point is
replaced by multiple ones because single character does not exist (e.g.
‘fi’ ligature turning into ‘FL’) or is not commonly used (e.g. ß turning
into SS).
* admin/unidata/SpecialCasing.txt: New data file pulled from Unicode
standard distribution.
* admin/unidata/README: Mention SpecialCasing.txt.
* admin/unidata/unidata-get.el (unidata-gen-table-special-casing,
unidata-gen-table-special-casing--do-load): New functions generating
‘special-uppercase’, ‘special-lowercase’ and ‘special-titlecase’
character Unicode properties built from the SpecialCasing.txt Unicode
data file.
* src/casefiddle.c (struct casing_str_buf): New structure for
representing short strings used to handle one-to-many character
mappings.
(case_character_imlp): New function which can handle one-to-many
character mappings.
(case_character, case_single_character): Wrappers for the above
functions. The former may map one character to multiple (or no)
code points while the latter does what the former used to do (i.e.
handles one-to-one mappings only).
(do_casify_natnum, do_casify_unibyte_string,
do_casify_unibyte_region): Use case_single_character.
(do_casify_multibyte_string, do_casify_multibyte_region): Support new
features of case_character.
* (do_casify_region): Updated to reflact do_casify_multibyte_string
changes.
(casify_word): Handle situation when one character-length of a word
can change affecting where end of the word is.
(upcase, capitalize, upcase-initials): Update documentation to mention
limitations when working on characters.
* test/src/casefiddle-tests.el (casefiddle-tests-char-properties):
Add test cases for the newly introduced character properties.
(casefiddle-tests-casing): Update test cases which are now passing.
* test/lisp/char-fold-tests.el (char-fold--ascii-upcase,
char-fold--ascii-downcase): New functions which behave like old ‘upcase’
and ‘downcase’.
(char-fold--test-match-exactly): Use the new functions. This is needed
because otherwise fi and similar characters are turned into their multi-
-character representation.
* doc/lispref/strings.texi: Describe issue with casing characters versus
strings.
* doc/lispref/nonascii.texi: Describe the new character properties.
* src/casefiddle.c (struct casing_context, prepare_casing_context): Add
titlecase_char_table member. It’s set to the ‘titlecase’ Unicode
property table if capitalisation has been requested.
(case_character): Make use of the titlecase_char_table to title-case
initial characters when capitalising.
* test/src/casefiddle-tests.el (casefiddle-tests--characters,
casefiddle-tests-casing): Update test cases which are now passing.
* src/alloc.c (allocate_record): New function.
(Fmake_record, Frecord, Fcopy_record): New functions.
(syms_of_alloc): defsubr them.
(purecopy): Work with records.
* src/data.c (Ftype_of): Return slot 0 for record objects, or type
name if record's type holds class.
(Frecordp): New function.
(syms_of_data): defsubr it. Define `Qrecordp'.
(Faref, Faset): Work with records.
* src/fns.c (Flength): Work with records.
* src/lisp.h (prec_type): Add PVEC_RECORD.
(RECORDP, CHECK_RECORD, CHECK_RECORD_TYPE): New functions.
* src/lread.c (read1): Add syntax for records.
* src/print.c (PRINT_CIRCLE_CANDIDATE_P): Add RECORDP.
(print_object): Add syntax for records.
* test/lisp/emacs-lisp/cl-print-tests.el (cl-print-tests-2):
New test.
* test/src/alloc-tests.el (record-1, record-2, record-3):
New tests.
* doc/lispref/elisp.texi, doc/lispref/objects.texi,
doc/lispref/records.texi: Add documentation for records.
This catches some problems with integer overflow and races
that I noticed in inotify.c after reviewing the changes
installed to fix Bug#26126.
* src/fns.c, src/lisp.h (equal_no_quit): Now extern.
* src/inotify.c (aspect_to_inotifymask):
Check for cycles and for improper lists.
(make_lispy_mask, lispy_mask_match_p): Remove.
All callers changed to use INTEGER_TO_CONS and CONS_TO_INTEGER.
(inotifyevent_to_event, add_watch):
Don’t assume watch descriptors and cookies fit in fixnums.
(add_watch): Use assoc_no_quit, not Fassoc.
Avoid integer overflow in (very!) long-running processes where
the Emacs watch ID could overflow. Avoid some duplicate code.
(find_descriptor): New function.
(remove_descriptor): First arg is now the returned value from
find_descriptor, rather than the descriptor. This way, the
value can be removed without calling Fdelete, which might quit.
Wait until the end (when watch_list is consistent) before signaling
any errors.
(remove_watch, inotify_callback):
Use find_descriptor to avoid the need for Fdelete.
(inotify_callback): Use simpler tests for ioctl failure.
Free temporary buffer if signaled, and put it on the stack if small.
Use ssize_t to index through read results, to avoid a cast.
(valid_watch_descriptor): New function, with a tighter check.
(Finotify_rm_watch, Finotify_valid_p): Use it.
(Finotify_valid_p): Use assoc_no_quit and ass_no_quit instead
of Fassoc. Do not assume the first assoc succeeds.
* test/src/inotify-tests.el (inotify-valid-p-simple):
Add inotify-valid-p tests, some of which dump core without
the fixes noted above.
Problem reported by Glenn Morris in:
http://lists.gnu.org/archive/html/emacs-devel/2017-03/msg00147.html
* src/data.c (minmax_driver): Convert any marker result to an
integer, since some callers assume this.
* test/src/data-tests.el (data-tests-max, data-tests-min):
Test for this.
* doc/lispref/numbers.texi (Comparison of Numbers):
* etc/NEWS: Document this.
* src/data.c (Amax, Amin): Remove constants. All uses removed.
(minmax_driver): New function.
(Fmax, Fmin): Use it instead of arith_driver.
* test/src/data-tests.el (data-tests-max, data-tests-min): New tests.
* etc/NEWS: Document this.
* src/data.c (store_symval_forwarding):
* src/sound.c (parse_sound):
Do not botch NaN comparison.
* src/data.c (cons_to_unsigned, cons_to_signed):
Signal an error if a floating-point arg is not integral.
* src/data.c (cons_to_unsigned, cons_to_signed):
* src/fileio.c (file_offset):
Use simpler overflow check.
* src/dbusbind.c (xd_extract_signed, xd_extract_unsigned):
Avoid rounding error in overflow check.
(Fcar_less_than_car): Use arithcompare directly.
* test/src/charset-tests.el: New file.
* admin/merge-gnulib (GNULIB_MODULES): Add count-leading-zeros.
* etc/NEWS: Document the change.
* lib/count-leading-zeros.c, lib/count-leading-zeros.h:
* m4/count-leading-zeros.m4: New files, copied from Gnulib.
* lib/gnulib.mk, m4/gnulib-comp.m4: Regenerate.
* src/floatfns.c: Include count-leading-zeros.h.
(Flogb): Do not convert fixnum to float before taking the log,
as the rounding error can cause the answer to be off by 1.
* src/lisp.h (EMACS_UINT_WIDTH): New constant.
* test/src/floatfns-tests.el (logb-extreme-fixnum): New test.
* etc/NEWS: Document this.
* src/bytecode.c (exec_byte_code):
* src/data.c (arithcompare):
Do not lose information when comparing floats to integers.
* test/src/data-tests.el (data-tests-=, data-tests-<)
(data-tests->, data-tests-<=, data-tests->=):
Test this.
Without this fix, (ceiling most-negative-fixnum -1.0) returns
most-negative-fixnum instead of correctly signaling range-error,
and similarly for floor, round, and truncate.
* configure.ac (trunc): Add a check, since Gnulib’s doc says
‘trunc’ is missing from MSVC 9. The Gnulib doc says ‘trunc’ is
also missing from some other older operating systems like Solaris
9 which I know we don’t care about any more, so MSVC is the only
reason to worry about ‘trunc’ here.
* src/editfns.c (styled_format): Formatting a float with %c is now an
error. The old code did not work in general, because FIXNUM_OVERFLOW_P
had rounding errors. Besides, the "if (FLOATP (...))" was in there
only as a result of my misunderstanding old code that I introduced
2011. Although %d etc. is sometimes used on floats that represent
huge UIDs or PIDs etc. that do not fit in fixnums, this cannot
happen with characters.
* src/floatfns.c (rounding_driver): Rework to do the right thing
when the intermediate result equals 2.305843009213694e+18, i.e.,
is exactly 1 greater than MOST_POSITIVE_FIXNUM on a 64-bit host.
Simplify so that only one section of code checks for overflow,
rather than two.
(double_identity): Remove. All uses changed to ...
(emacs_trunc): ... this new function. Add replacement for
platforms that lack ‘trunc’.
* src/lisp.h (FIXNUM_OVERFLOW_P, make_fixnum_or_float):
Make it clear that the arg cannot be floating point.
* test/src/editfns-tests.el (format-c-float): New test.
* test/src/floatfns-tests.el: New file, to test for this bug.
A compatibility digraph characters, such as Dž, are neither upper- nor
lower-case. At the moment however, those are reported as upper-case¹
despite the fact that they change when upper-cased.
Stop checking if a character is upper-case before trying to up-case it
so that title-case characters are handled correctly. This fixes one of
the issues mentioned in bug#24603.
¹ Because they change when converted to lower-case. Notice an asymmetry
in that for a character to be considered lower-case it must not be
upper-case (plus the usual condition of changing when upper-cased).
* src/buffer.h (upcase1): Delete.
(upcase): Change to upcase character unconditionally just like downcase
does it. This is what upcase1 was.
* src/casefiddle.c (casify_object, casify_region): Use upcase instead
of upcase1 and don’t check !uppercasep(x) before calling upcase.
* src/keyboard.c (read_key_sequence): Don’t check if uppercase(x), just
downcase(x) and see if it changed.
* test/src/casefiddle-tests.el (casefiddle-tests--characters,
casefiddle-tests-casing): Update test cases which are now passing.
Use Unicode data to generate case tables instead of mostly repeating
them in lisp code. Do that in a way which maps ‘Dz’ (and similar)
digraph to ‘dz’ when down- and ‘DZ’ when upcasing.
https://debbugs.gnu.org/cgi/bugreport.cgi?msg=89;bug=24603 lists all
changes to syntax table and case tables introduced by this commit.
* lisp/international/characters.el: Remove case-pairs defined with
explicit Lisp code and instead use Unicode character properties.
* test/src/casefiddle-tests.el (casefiddle-tests--characters,
casefiddle-tests-casing): Update test cases which are now working
as they should.
Fixes cases marked FIXME upcoming in followup commits.
* test/src/casefiddle-tests.el (casefiddle-tests-char-properties,
casefiddle-tests-case-table, casefiddle-tests-casing-character,
casefiddle-tests-casing, casefiddle-tests-casing-byte8,
casefiddle-tests-casing-byte8-with-changes): New tests.
(casefiddle-tests--test-casing): New helper function for runnig
some of the tests.
Problem reported by Eli Zaretskii (Bug#25606#62).
* test/src/fns-tests.el (test-cycle-lax-plist-get)
(test-cycle-plist-put, test-cycle-lax-plist-put):
Fix tests to match behavior.
Characters having both open paren syntax and comment start syntax were
being detected as open parens even when they should have been part a
comment starter (Bug#24870).
* src/syntax.c (in_2char_comment_start): New function, extracted from
`scan_sexps_forward'.
(scan_sexps_forward): Add check for a 2-char comment starter before the
loop. Inside the loop, do that check after incrementing the 'from'
character index. Move the single char comment syntax cases into the
switch instead of special casing them before.
* test/src/syntax-tests.el (parse-partial-sexp-paren-comments):
(parse-partial-sexp-continue-over-comment-marker): New tests.
