A few bits of the code were relying on the fact that XFIXNUM,
XFIXNAT, and XUFIXNUM do something even with arguments that
are not fixnums/fixnats. Separate these rare uses out into
XFIXNUM_RAW and XUFIXNUM_RAW.
Problem and original patch reported by Pip Cet (Bug#36370).
* src/ccl.c (Fccl_execute_on_string):
* src/fileio.c (Finsert_file_contents, a_write)
(Fdo_auto_save):
* src/process.c (conv_lisp_to_sockaddr):
* src/textprop.c (Fnext_single_char_property_change)
(Fprevious_single_char_property_change)
(Fnext_property_change, Fnext_single_property_change)
(Fprevious_property_change)
(Fprevious_single_property_change):
Don’t assume fixnums are nonnegative.
* src/ccl.c (Fccl_execute_on_string):
Fix range-checking bug if AREF (status, i) is out of int range.
* src/data.c (arith_driver): Use XFIXNUM_RAW as we want
efficient garbage if the value is not a fixnum.
* src/dosfns.c (Fint86, Fdos_memput):
Check that args are nonnegative.
* src/image.c (lookup_image): Check that args are in range.
* src/lisp.h (lisp_h_XHASH): Use XUFIXNUM_RAW, since this
is for hashing.
(lisp_h_XFIXNAT, XFIXNAT) [USE_LSB_TAG]: Remove macros.
(lisp_h_XFIXNUM_RAW, XFIXNUM_RAW) [USE_LSB_TAG]: New macros, with
the semantics of the old macros without _RAW.
(XFIXNUM_RAW, XUFIXNUM_RAW): New inline functions, with the
semantics of the old functions without _RAW.
(FIXNUMP): Move definition up to avoid forward use.
(XFIXNUM, XFIXNAT, XUFIXNUM): Use eassume to add a runtime
check (when debugging) that the argument has the proper form.
(XFIXNUM, XFIXNAT): Now inline functions only, since they
refer to their arguments more than once now that they use eassume.
* src/textprop.c (Fprevious_single_char_property_change):
Avoid fixnum overflow with invalid input.
(set_text_properties): Fix unlikely failure
to validate arguments, by using EQ instead of XFIXNAT.
* src/w32term.c (w32_draw_glyph_string):
* src/xterm.c (x_draw_glyph_string):
Treat negative minimums as 0 rather than as garbage patterns.
This is a minor patch to remove some fixnum restrictions.
Many more such patches are needed, but one thing at a time.
* doc/emacs/custom.texi (Examining): Update fill-column example.
* src/buffer.c (fill-column, left-margin, tab-width)
(buffer-saved-size, left-margin-width, right-margin-width)
(left-fringe-width, right-fringe-width, scroll-bar-width)
(scroll-bar-height, buffer-display-count):
Allow any integer; do not restrict to fixnums.
* src/character.h (SANE_TAB_WIDTH): Do not assume tab_width
is a nonnegative fixnum.
(sanitize_tab_width): Take a Lisp_Object integer, not an
EMACS_INT. Only use changed.
* src/data.c (store_symval_forwarding): Remove unnecessary
SYMBOLP since the predicate (e.g., Qintegerp) is always a
symbol (leave the test in as an eassert). Avoid assignments
inside if-conditions.
* src/fileio.c (Fdo_auto_save): Do not assume
buffer-saved-size is a fixnum. Avoid undefined behavior
on EMACS_INT overflow by multiplying a fixnum by at most 4,
not by at most 13.
* src/window.c (set_window_buffer): When buffer-display-count
is too large for a fixnum, make it a bignum.
* src/xdisp.c (FILL_COLUMN_INDICATOR_NEEDED): Remove macro, ...
(fill_column_indicator_column): ... replacing with this new function.
All uses changed. The function is a bit pickier, to prevent
problems with non-character fixnums and columns out of range
for int, and to remove the assumption that integers are in
fixnum range.
(append_space_for_newline, extend_face_to_end_of_line):
Avoid undefined behavior with signed integer overflow.
Simplify.
* src/data.c (set_internal, set_default_internal):
Replace obsolescent /* FALLTHROUGH! */ comments with ‘break;’,
as ‘FALLTHROUGH;’ is not needed here.
Since DEFSYM doesn't by itself do anything and make-docfile ignores
preprocessor statements, conditional compilation of DEFSYMs is
ineffective.
* src/data.c (syms_of_data): Remove ineffective #ifdefs.
Even if Emacs is compiled without module support, we don't have to
comment out every bit of user pointer support. Defining the basic
structures and functions and detecting user pointers in switch
statements is harmless, and we're already doing the same for module
functions. Removing these #ifdefs makes the code a bit easier to
read.
* src/lisp.h (PVEC_USER_PTR, struct Lisp_User_Ptr, USER_PTRP)
(XUSER_PTR): Define unconditionally.
* src/data.c (Ftype_of):
* src/alloc.c (cleanup_vector):
* src/print.c (print_vectorlike):
* src/pdumper.c (dump_vectorlike): Remove #ifdef for user pointers.
Initialize these objects statically, and make them constants.
This is a bit safer and more efficient.
* src/data.c (XBOOLFWD, XKBOARD_OBJFWD, XFIXNUMFWD, XOBJFWD):
* src/lisp.h (XBUFFER_OBJFWD):
Return a pointer-to-const instead of an unrestricted pointer.
(lispfwd): fwdptr is now a pointer-to-const instead of an
unrestricted pointer. All uses changed.
(SET_SYMBOL_FWD): Accept pointer-to-const instead of an
unrestricted pointer.
(DEFVAR_LISP, DEFVAR_LISP_NOPRO, DEFVAR_BOOL, DEFVAR_INT)
(DEFVAR_KBOARD): Initialize static structures statically
instead of dynamically, and make them const.
* src/lread.c (defvar_int, defvar_bool, defvar_lisp_nopro)
(defvar_lisp, defvar_kboard): Accept pointer-to-const instead
of an unrestricted pointer; it’s now the caller’s
responsibility to initialize the pointed-to storage. No need
for a separate address argument any more. All callers
changed.
It's not portable to cast (e.g.) struct Lisp_Objfwd * to union
Lisp_Fwd * and then back again, because the compiler can then assume
that the pointer is aligned for union Lisp_Fwd * when accessing
the struct Lisp_Objfwd * components, and this assumption might
be incorrect becase we don't force that alignment.
* src/lisp.h (lispfwd): New type, replacing ...
(union Lisp_Fwd): ... this type, which was removed.
All uses changed.
(SET_SYMBOL_FWD): 2nd arg is now void *, not lispfwd.
All uses changed (casts no longer needed; they were
not portable anyway).
* lisp/emacs-lisp/bytecomp.el (byte-compile-setq-default): Delete.
(byte-compile-set-default): Inline the part that it used.
* lisp/emacs-lisp/edebug.el (setq-default): Remove the debug spec.
* src/data.c (Fsetq_default): Delete.
(syms_of_data): Don't register.
Formerly they were fixnums, which led to problems when dealing
with values that might not fit on 32-bit platforms, such as
string-chars-consed or floats_consed. 64-bit counters should
be good enough for these (for a while, anyway...).
While we’re at it, fix some unlikely integer overflow bugs
that have been in the code for a while.
* lib-src/make-docfile.c (write_globals):
* src/data.c (do_symval_forwarding, store_symval_forwarding):
* src/eval.c (restore_stack_limits, call_debugger):
* src/frame.h (struct frame.cost_calculation_baud_rate):
* src/keyboard.c (last_auto_save, bind_polling_period, read_char):
* src/lisp.h (struct Lisp_Intfwd.intvar):
* src/lread.c (defvar_int):
* src/pdumper.c (dump_fwd_int):
* src/thread.h (struct thread_state.m_lisp_eval_depth):
* src/undo.c (truncate_undo_list):
* src/xselect.c (wait_for_property_change)
(x_get_foreign_selection):
* src/xterm.c (x_emacs_to_x_modifiers):
DEFVAR_INT variables now have the C type intmax_t, not EMACS_INT.
* src/data.c (store_symval_forwarding):
* src/gnutls.c (Fgnutls_boot):
* src/keyboard.c (bind_polling_period):
* src/macros.c (pop_kbd_macro, Fexecute_kbd_macro):
* src/undo.c (truncate_undo_list):
Allow any integer that fits into intmax_t, instead of
requiring it to be a Lisp fixnum.
* src/dispnew.c (update_window):
* src/frame.c (x_figure_window_size):
* src/gnutls.c (init_gnutls_functions)
(emacs_gnutls_handle_error):
* src/keyboard.c (make_lisp_event):
* src/nsterm.m (ns_dumpglyphs_image):
* src/profiler.c (make_log):
* src/scroll.c (calculate_scrolling)
(calculate_direct_scrolling):
* src/termcap.c (tputs):
* src/xterm.c (x_draw_image_relief):
Avoid implementation-defined behavior on conversion of
out-of-range integers.
* src/eval.c (when_entered_debugger): Now intmax_t.
(max_ensure_room): New function, that avoids signed integer overflow.
(call_debugger, signal_or_quit): Use it.
* src/fileio.c (Fdo_auto_save):
* src/keyboard.c (make_lisp_event):
* src/term.c (calculate_costs):
* src/xdisp.c (build_desired_tool_bar_string)
(hscroll_window_tree, try_scrolling, decode_mode_spec)
(x_produce_glyphs):
Avoid signed integer overflow.
* src/lisp.h (clip_to_bounds): Generalize to intmax_t.
* src/pdumper.c (dump_emacs_reloc_immediate_emacs_int): Remove, ...
(dump_emacs_reloc_immediate_intmax_t): ... replacing with this
function. All uses changed.
* src/profiler.c (make_log): Omit args. All callers changed.
* src/termcap.c: Include stdlib.h, for atoi.
Include intprops.h.
* src/window.c (sanitize_next_screen_context_lines): New function.
(window_scroll_pixel_based, window_scroll_line_based):
Use it to avoid signed integer overflow.
Add a new portable dumper as an alternative to unexec. Use it by default.
* src/dmpstruct.awk: New file.
* src/doc.c (get_doc_string): use will_dump_p().
* src/editfns.c (styled_format): silence compiler warning
with UNINIT.
* src/emacs-module.c (syms_of_module): staticpro ltv_mark.
* src/emacs.c (gflags): new variable.
(init_cmdargs): unwrap
(string_starts_with_p, find_argument, dump_error_to_string)
(load_pdump): new functions.
(main): detect pdumper and --temacs invocation; actually load
portable dump when detected; set gflags as appropriate; changes to
init functions throughout to avoid passing explicit
'initialized' argument.
* src/eval.c (inhibit_lisp_code): remove unused variable.
(init_eval_once_for_pdumper): new function.
(init_eval_once): call it.
* src/filelock.c: CANNOT_DUMP -> will_dump_p()
* src/fingerprint-dummy.c: new file
* src/fingerprint.h: new file
* src/fns.c: CANNOT_DUMP -> will_dump_p(), etc.
(weak_hash_tables): remove
(hashfn_equal, hashfn_eql): un-staticify
(make_hash_table): set new 'next_weak' hash table field; drop
global weak_hash_tables logic.
(copy_hash_table): drop global weak_hash_tables logic.
(hash_table_rehash): new function.
(hash_lookup, hash_put, hash_remove_from_table, hash_clear):
rehash if needed.
(sweep_weak_table): un-staticify; explain logic; bool-ify.
(sweep_weak_hash_tables): remove function.
* src/font.c (syms_of_font): remember pdumper stuff.
* src/fontset.c (syms_of_fontset): remember pdumper stuff.
* src/frame.c (make_initial_frame): don't reset Vframe_list.
(init_frame_once_for_pdumper, init_frame_once): new functions.
(syms_of_frame): remove redundant staticpro.
* src/fringe.c (init_fringe_once_for_pdumper): new functin.
(init_fringe_once): call it.
* src/ftcrfont.c (syms_of_ftcrfont_for_pdumper): new function.
(syms_of_ftcrfont): call it.
* src/ftfont.c (syms_of_ftfont_for_pdumper): new function.
(syms_of_ftfont): call it.
* src/ftxont.c (syms_of_ftxfont_for_pdumper): new function.
(syms_of_ftxfont): call it.
* src/gmalloc.c: adjust for pdumper througout
(DUMPED): remove weird custom dumped indicator.
* src/gnutls.c (syms_of_gnutls): pdumper note for
gnutls_global_initialized.
* src/image.c (syms_of_image): add pdumper comment,
initializer note.
* src/insdel.c (prepare_to_modify_buffer_1): account
for buffer contents possibly being in dump image.
* src/keyboard.c (syms_of_keyboard_for_pdumper): new function.
(syms_of_keyboard): staticpro more; call pdumper syms function.
* src/lisp.h: add comments throughout
(gflags): declare.
(will_dump_p, will_bootstrap_p, will_dump_with_pdumper_p)
(dumped_with_pdumper_p, will_dump_with_unexec_p)
(dumped_with_unexec_p, definitely_will_not_unexec_p): new
functions.
(POWER_OF_2, ROUNDUP): move macros.
(PSEUDOVECTOR_TYPE, PSEUDOVECTOR_TYPEP): take vectorlike header
pointer instead of vector; constify.
(Lisp_Hash_Table): add comment about need to rehash on access; add
comment for next_weak.
(HASH_KEY, HASH_VALUE, HASH_HASH, HASH_TABLE_SIZE): const-ify.
(hash_table_rehash): declare.
(hash_rehash_needed_p, hash_rehash_if_needed): new functions.
(finalizers, doomed_finalizers): declare extern.
(SUBR_SECTION_ATTRIBUTE): new macro.
(staticvec, staticidx): un-static-ify.
(sweep_weak_hash_tables): remove declaration.
(sweep_weak_table): declare.
(hashfn_eql, hashfn_equal): declare.
(number_finalizers_run): new variable.
(Vdead): externify when ENABLE_CHECKING.
(gc_root_type): new enumeration.
(gc_root_visitor): new struct.
(visit_static_gc_roots): declare.
(vectorlike_nbytes): declare.
(vector_nbytes): define as trivial inline function wrapper for
vectorlike_nbytes.
(init_obarray_once): change signature.
(primary_thread): extern-ify.
(init_buffer): change signature.
(init_frame_once): declare.
* src/lread.c (readevalloop): adjust for new dumped predicates.
(init_obarray_once): new function.
(ndefsubr): new variable.
(defsubr): increment it.
(load_path_check): adjust for pdumper.
(load_path_default): use pdumper functions; adjust for
dump search.
* src/macfont.m (macfont_init_font_change_handler): avoid
shadowing global.
(syms_of_macfont_for_pdumper): new function.
(syms_of_macfont): call it.
* src/menu.c (syms_of_menu): staticpro more stuff.
* src/minibuf.c (Ftry_completion): rehash if needed.
(init_minibuf_once_for_pdumper): new function.
(init_minibuf_once): call it.
* src/nsfont.m (syms_of_nsfns): staticpro more.
* src/nsfont.m (syms_of_nsfont_for_pdumper): new function.
(syms_of_nsfont): call it.
* src/nsterm.m (syms_of_nsfont): remember pdumper stuff.
* src/pdumper.c: new file.
* src/pdumper.h: new file.
* src/process.c (init_process_emacs): use new pdumper functions
instead of CANNOT_DUMP.
* src/profiler.c (syms_of_profiler_for_pdumper): new function.
(syms_of_profiler_for_pdumper): call it.
* src/search.c (syms_of_search_for_pdumper): new function.
(syms_of_search_for_pdumper): call it.
* src/sheap.c (bss_sbrk_did_unexec): remove.
* src/sheap.h (bss_sbrk_did_unexec): remove.
* src/syntax.c (syms_of_syntax): don't redundantly staticpro
re_match_object.
* src/sysdep.c: use will_dump_with_unexec_p() instead of bss
hack thing.
* src/syssignals.h (init_sigsegv): declare.
* src/systime.h (init_timefns): remove bool from signature.
* src/textprop.c (syms_of_textprop): move staticpro.
* src/thread.c (main_thread_p): constify.
* src/thread.h (main_thread_p): constify.
* src/timefns.c (init_timefns): remove bool from signature.
(syms_of_timefns_for_pdumper): new function.
(syms_of_timefns): call it.
* src/w32.c: rearrange code.
* src/w32.h (w32_relocate): declare.
* src/w32fns.c (syms_of_w32fns): add pdumper note.
* src/w32font.c (syms_of_w32font_for_pdumper): new function.
(syms_of_w32font): call it.
* src/w32heap.c (using_dynamic_heap): new variable.
(init_heap): use it.
* src/w32menu.c (syms_of_w32menu): add pdumper note.
* src/w32proc.c
(ctrl_c_handler, mainCRTStartup, _start, open_input_file)
(rva_to_section, close_file_data): move here.
* src/w32uniscribe.c (syms_of_w32uniscribe_for_pdumper):
new function.
(syms_of_w32uniscribe): call it.
* src/window.c (init_window_once_for_pdumper): new function.
(init_window_once): call it; staticpro more stuff.
* src/xfont.c (syms_of_xfont_for_pdumper): new function.
(syms_of_xfont): call it.
* src/xftfont.c (syms_of_xftfont_for_pdumper): new function.
(syms_of_xftfont): call it.
* src/xmenu.c (syms_of_xmenu_for_pdumper): new function.
(syms_of_xmenu): call it.
* src/xselect.c (syms_of_xselect_for_pdumper): new function.
(syms_of_xselect): call it.
* src/xsettings.c (syms_of_xsettings): add more pdumper notes.
* src/term.c (syms_of_xterm): add pdumper note.
* src/dispnew.c (init_faces_initial): new function.
(init_display_interactive): rename from init_display; use
will_dump_p instead of !initialized. Initialize faces early for
pdumper if needed.
(init_display): new function.
(syms_of_display_for_pdumper): new function.
(syms_of_display): call it.
* src/dbusbind.c (syms_of_dbusbind): Add TODO for bus reset
on pdumper load.
* src/data.c (Fdefalias): Use will_dump_p
instead of Vpurify_flag.
(Fmake_variable_buffer_local): silence compiler warning with -Og
by making valcontents UNINIT.
(arith_driver): silence compiler warning with UNINIT.
* src/conf_post.h (ATTRIBUTE_SECTION): new macro.
* src/composite.c (composition_gstring_put_cache): rehash hash
table if needed.
* src/coding.c (init_coding_once, syms_of_coding): remember
pdumper stuff.
* src/charset.h (charset_table_size, charset_table_user): declare.
* src/charset.c (charset_table_used, charset_table_size): un-static.
(init_charset_oncem, syms_of_charset): remember pdumper stuff.
* src/category.c (category_table_version): remove obsolete
variable.
* src/callint.c (syms_of_callint): staticpro 'preserved_fns'
(init_callproc): use will_dump_p instead of !CANNOT_DUMP.
* src/bytecode.c (exec_byte_code): rehash table tables if needed
* src/buffer.c (alloc_buffer_text, free_buffer_text): account for
pdumper
(init_buffer_once): add TODO; remember stuff for pdumper.
(init_buffer): don't take initialized argument; adjust
for pdumper.
* src/atimer.c (init_atimer): initialize subr only if
!initialized.
* src/alloc.c: (vector_marked_p, set_vector_marked)
(vectorlike_marked_p, set_vectorlike_marked, cons_marked_p)
(set_cons_marked, string_marked_p, set_string_marked)
(symbol_marked_p, set_symbol_marked, interval_marked_p)
(set_interval_marked): new accessor routines. Use them
instead of raw GC access throughout.
(Vdead): make non-static when ENABLE_CHECKING.
(vectorlike_nbytes): rename of 'vector_nbytes'; take a vectorlike
header as input instead of a vector.
(number_finalizers_run): new internal C variable.
(mark_maybe_object): check for pdumper objects.
(valid_pointer_p): don't be gratuitously inefficient under rr(1).
(make_pure_c_string): add support for size_byte = -2 mode
indicating that string data points into Emacs image rodata.
(visit_vectorlike_root): visits GC roots embedded in
vectorlike objects.
(visit_buffer_root): visits GC roots embedded in
our totally-not-a-buffer buffer global objects.
(visit_static_gc_roots): visit GC roots in the Emacs data section.
(mark_object_root_visitor): root callback used for conventional GC
marking
(weak_hash_tables): new internal variable for tracking found weak
hash tables during GC.
(mark_and_sweep_weak_table_contents): new weak hash table marking.
(garbage_collect_1): use new GC root visitor machinery.
(mark_vectorlike): accept a vectorlike_header instead of a
Lisp_Vector.
(mark_frame, mark_window, mark_hash_table): new functions.
(mark_object): initialize 'm'; check for pdumper objects and use
new mark-bit accessors throughout. Remove some object-specific
marking code and move to helper functions above.
(survives_gc_p): check for pdumper objects.
(gc-sweep): clear pdumper mark bits.
(init_alloc_once_for_pdumper): new helper function for early init
called both during normal init and pdumper load.
(init_alloc_once): pdumper integration.
* src/Makefile.in: Rewrite dumping for pdumper; add pdumper.o;
invoke temacs with --temacs command line option; build dmpstruct.h
from dmpstruct.awk; stop relying on CANNOT_DUMP; clean up pdumper
intermediate files during build.
* nextstep/Makefile.in: build emacs.pdmp into NS packages
* lisp/startup.el: account for new '--temacs' and '--dump-file'
command line option.
* lisp/loadup.el: rewrite early init to account for pdumper; use
injected 'dump-mode' variable (set via the new '--temacs' option)
instead of parsing command line.
* lisp/cus-start.el: Check 'dump-mode' instead of 'purify-flag',
since the new 'dump-mode'
* lib-src/make-fingerprint.c: new program
* lib-src/Makefile.in: built make-fingerprint utility program
* configure.ac: Add --with-pdumper toggle to control pdumper
support; add --with-unexec toggle to control unexec support.
Add --with-dumping option to control which dumping strategy we use
by default. Adjust for pdumper throughout. Check for
posix_madvise.
* Makefile.in: Add @DUMPING@ substitution; add pdumper mode.
* .gitignore: Add make-fingerprint, temacs.in, fingerprint.c,
dmpstruct.h, and pdumper dump files.
When calculating the length of a Lisp object whose type is
known, use a specialized length operation on it to save a bit
of runtime overhead.
* src/callint.c (Fcall_interactively):
* src/minibuf.c (read_minibuf_unwind):
Use ASIZE rather than Flength on values that must be vectors.
* src/charset.c (Fsort_charsets):
* src/coding.c (detect_coding_sjis):
(Fdefine_coding_system_internal):
* src/data.c (wrong_choice):
* src/eval.c (Flet, eval_sub, Fapply, apply_lambda):
* src/fns.c (sort_list):
* src/font.c (font_vconcat_entity_vectors)
(font_find_for_lface):
* src/frame.c (Fmodify_frame_parameters):
* src/fringe.c (get_logical_fringe_bitmap):
* src/ftfont.c (ftfont_get_open_type_spec):
* src/gtkutil.c (xg_print_frames_dialog):
* src/lread.c (read1, read_vector):
* src/keymap.c (Fkey_description):
* src/kqueue.c (Fkqueue_add_watch):
* src/macfont.m (macfont_get_open_type_spec):
* src/menu.c (parse_single_submenu, x_popup_menu_1):
* src/minibuf.c (Finternal_complete_buffer):
* src/nsfont.m (ns_findfonts, nsfont_list_family):
* src/process.c (Fmake_process):
* src/search.c (Fset_match_data):
* src/xfaces.c (Fx_family_fonts):
Use list_length rather than Flength on values that must be lists.
* src/fns.c (list_length): New function.
(Flength): Use it.
* src/nsfont.m (ns_findfonts):
Use !NILP (x) rather than XFIXNUM (Flength (x)) != 0.
* src/xdisp.c (store_mode_line_string):
Use SCHARS rather than Flength on values that must be strings.
These bugs were introduced after bignums were added.
* src/data.c (cons_to_unsigned, cons_to_signed):
* src/xdisp.c (calc_line_height_property):
Invoke integer_to_intmax and integer_to_uintmax only on integers.
* src/data.c (emacs_mpz_mul_2exp): 2nd arg is now a
nonnegative EMACS_INT not mp_bitcnt_t, to simplify checking.
(Fash): Support COUNT values that are bignums or that
exceed mp_bitcnt_t range.
* test/src/data-tests.el (data-tests-ash-lsh): Test this.
* src/bignum.c (make_neg_biguint): New function.
* src/lread.c (read1): Do not mishandle an unquoted symbol
with name equal to something like "1\0x", i.e., a string
of numeric form followed by a NUL byte.
Formerly these symbols were misread as numbers.
(string_to_number): Change last argument from an integer flag
to a pointer to the length. This lets the caller figure out
how much of the prefix was used. All callers changed.
Add a fast path if the integer (sans sign) fits in uintmax_t.
Update comments and simplify now that bignums are present.
* src/print.c (print_object): Fix quoting of symbols that look
like numbers, by relying on string_to_number for the tricky
cases rather than trying to redo its logic, incorrectly. For
example, (read (prin1-to-string '\1e+NaN)) formerly returned
"1e+NaN", which was wrong: a backslash is needed in the output
to prevent it from being read as a NaN. Escape NO_BREAK_SPACE
too, since lread.c treats it like SPACE.
* test/src/print-tests.el (print-read-roundtrip):
Add tests illustrating the abovementioned bugs.
* src/bignum.c (mpz_bufsize): New function.
(bignum_bufsize): Use it.
(mpz_get_d_rounded): New function.
(bignum_to_double): Use it.
* src/bignum.c (bignum_to_double):
* src/data.c (bignum_arith_driver):
When converting bignums to double, round instead of
truncating, to be consistent with what happens with fixnums.
* test/src/floatfns-tests.el (bignum-to-float): Test rounding.
This better corresponds to what emacs-26 did in the
rare cases where it checked for integer overflow.
* src/alloc.c (range_error): Remove.
All uses changed to overflow_error.
* src/eval.c (overflow_error): New function.
Improve arithmetic performance by avoiding bignums until needed.
Also, simplify bignum memory management, fixing some unlikely leaks.
This patch improved the performance of (+ 2 2) by a factor of ten
on a simple microbenchmark computing (+ x 2), byte-compiled,
with x a local variable initialized to 2 via means the byte
compiler could not predict: performance improved from 135 to 13 ns.
The platform was Fedora 28 x86-64, AMD Phenom II X4 910e.
Performance also improved 0.6% on ‘make compile-always’.
* src/bignum.c (init_bignum_once): New function.
* src/emacs.c (main): Use it.
* src/bignum.c (mpz): New global var.
(make_integer_mpz): Rename from make_integer. All uses changed.
* src/bignum.c (double_to_bignum, make_bignum_bits)
(make_bignum, make_bigint, make_biguint, make_integer_mpz):
* src/data.c (bignum_arith_driver, Frem, Flogcount, Fash)
(expt_integer, Fadd1, Fsub1, Flognot):
* src/floatfns.c (Fabs, rounding_driver, rounddiv_q):
* src/fns.c (Fnthcdr):
Use mpz rather than mpz_initting and mpz_clearing private
temporaries.
* src/bignum.h (bignum_integer): New function.
* src/data.c (Frem, Fmod, Fash, expt_integer):
* src/floatfns.c (rounding_driver):
Use it to simplify code.
* src/data.c (FIXNUMS_FIT_IN_LONG, free_mpz_value):
Remove. All uses removed.
(floating_point_op): New function.
(floatop_arith_driver): New function, with much of the guts
of the old float_arith_driver.
(float_arith_driver): Use it.
(floatop_arith_driver, arith_driver):
Simplify by assuming NARGS is at least 2.
All callers changed.
(float_arith_driver):
New arg, containing the partly converted value of the next arg.
Reorder args for consistency. All uses changed.
(bignum_arith_driver): New function.
(arith_driver): Use it. Do fixnum-only integer calculations
in intmax_t instead of mpz_t, when they fit.
Break out mpz_t calculations into bignum_arith_driver.
(Fquo): Use floatop_arith_driver instead of float_arith_driver,
since the op is known to be valid.
(Flogcount, Fash): Simplify by coalescing bignum and fixnum code.
(Fadd1, Fsub1): Simplify by using make_int.
* src/bignum.c, src/bignum.h: New files. Only modules that
need to know how bignums are implemented should include
bignum.h. Currently these are alloc.c, bignum.c (of course),
data.c, emacs.c, emacs-module.c, floatfns.c, fns.c, print.c.
* src/Makefile.in (base_obj): Add bignum.o.
* src/alloc.c (make_bignum_str): Move to bignum.c.
(make_number): Remove; replaced by bignum.c’s make_integer.
All callers changed.
* src/conf_post.h (ARG_NONNULL): New macro.
* src/json.c (json_to_lisp): Use it.
* src/data.c (Fnatnump):
Move NATNUMP’s implementation here from lisp.h.
* src/data.c (Fnumber_to_string):
* src/editfns.c (styled_format):
Move conversion of string to bignum to bignum_to_string, and
call it here.
* src/emacs-module.c (module_make_integer):
* src/floatfns.c (Fabs):
Simplify by using make_int.
* src/emacs.c: Include bignum.h, to expand its inline fns.
* src/floatfns.c (Ffloat): Simplify by using XFLOATINT.
(rounding_driver): Simplify by using double_to_bignum.
(rounddiv_q): Clarify use of temporaries.
* src/lisp.h: Move decls that need to know bignum internals to
bignum.h. Do not include gmp.h or mini-gmp.h; that is now
bignum.h’s job.
(GMP_NUM_BITS, struct Lisp_Bignum, XBIGNUM, mpz_set_intmax):
Move to bignum.h.
(make_int): New function.
(NATNUMP): Remove; all callers changed to use Fnatnump.
(XFLOATINT): If arg is a bignum, use bignum_to_double, so that
bignum internals are not exposed here.
* src/print.c (print_vectorlike): Use SAFE_ALLOCA to avoid the
need for a record_unwind_protect_ptr.
Also, since Emacs historically reported a range error when
rounding operations overflowed, do that consistently for all
bignum overflows.
* doc/lispref/errors.texi (Standard Errors):
* doc/lispref/numbers.texi (Integer Basics): Document range errors.
* src/alloc.c (range_error): Rename from integer_overflow.
All uses changed.
* src/floatfns.c (rounding_driver): When the result of a floating
point rounding operation does not fit into a fixnum, put it
into a bignum instead of always signaling an range error.
* test/src/floatfns-tests.el (divide-extreme-sign):
These tests now return the mathematically-correct answer
instead of signaling an error.
(bignum-round): Check that integers round to themselves.
* doc/lispref/objects.texi (Integer Type): Mention
most-negative-fixnum and most-positive-fixnum as alternatives
to fixnump and bignump.
* lisp/subr.el (fixnump, bignump): Now written in Lisp.
* src/data.c (Ffixnump, Fbignump): No longer written in C,
as these new functions are not crucial for performance.
libgmp calls ‘abort’ when given numbers too big for its
internal data structures. The numeric limit is large and
platform-dependent; with 64-bit GMP 6.1.2 it is around
2**2**37. Work around the problem by refusing to call libgmp
functions with arguments that would cause an abort. With luck
libgmp will have a better way to do this in the future.
Also, introduce a variable integer-width that lets the user
control how large bignums can be. This currently defaults
to 2**16, i.e., it allows bignums up to 2**2**16. This
should be enough for ordinary computation, and should
help Emacs to avoid thrashing or hanging.
Problem noted by Pip Cet (Bug#32463#71).
* doc/lispref/numbers.texi, etc/NEWS:
Document recent bignum changes, including this one.
Improve documentation for bitwise operations, in the light
of bignums.
* src/alloc.c (make_number): Enforce integer-width.
(integer_overflow): New function.
(xrealloc_for_gmp, xfree_for_gmp):
Move here from emacs.c, as it's memory allocation.
(init_alloc): Initialize GMP here, rather than in emacs.c.
(integer_width): New var.
* src/data.c (GMP_NLIMBS_MAX, NLIMBS_LIMIT): New constants.
(emacs_mpz_size, emacs_mpz_mul)
(emacs_mpz_mul_2exp, emacs_mpz_pow_ui): New functions.
(arith_driver, Fash, expt_integer): Use them.
(expt_integer): New function, containing integer code
that was out of place in floatfns.c.
(check_bignum_size, xmalloc_for_gmp): Remove.
* src/emacs.c (main): Do not initialize GMP here.
* src/floatfns.c (Fexpt): Use expt_integer, which
now contains integer code moved from here.
* src/lisp.h (GMP_NUMB_BITS): Define if gmp.h doesn’t.
* src/alloc.c (mpz_set_intmax_slow): Tighten assertion.
Work even in the unlikely case where libgmp uses nails.
* src/data.c (FIXNUMS_FIT_IN_LONG): New constant.
(arith_driver): Use it to tighten compile-time checks.
* src/lisp.h (mpz_set_intmax): Do not assume that converting
an out-of-range value to ‘long’ is harmless, as it might raise
a signal. Use simpler expression; compiler can optimize.
* src/data.c (isnan): Remove, as we can assume C99.
(bignumcompare): Remove, folding its functionality
into arithcompare.
(arithcompare): Compare bignums directly here.
Fix bugs when comparing NaNs to bignums.
When comparing a bignum to a fixnum, just look at the
bignum’s sign, as that’s all that is needed.
Decrease scope of locals when this is easy.
* test/src/data-tests.el (data-tests-bignum): Test bignum vs NaN.
* doc/lispref/numbers.texi (Bitwise Operations): Document that
the traditional (lsh A B) behavior is for fixnums, and that it
is an error if A and B are both negative and A is a bignum.
See Bug#32463.
* lisp/subr.el (lsh): New function, moved here from src/data.c.
* src/data.c (ash_lsh_impl): Remove, moving body into Fash
since it’s the only caller now.
(Fash): Check for out-of-range counts. If COUNT is zero,
return first argument instead of going through libgmp. Omit
lsh code since lsh is now done in Lisp. Add code for shifting
fixnums right, to avoid a round trip through libgmp.
(Flsh): Remove; moved to lisp/subr.el.
* test/lisp/international/ccl-tests.el (shift):
Test for traditional lsh behavior, instead of assuming
lsh is like ash when bignums are present.
* test/src/data-tests.el (data-tests-logand)
(data-tests-logior, data-tests-logxor, data-tests-ash-lsh):
New tests.
These operations incorrectly treated negative fixnums as
bignums greater than most-positive-fixnum.
* src/alloc.c (mpz_set_intmax_slow): Avoid undefined
behavior if signed unary negation overflows, while
we’re in the neighborhood.
(mpz_set_uintmax_slow): Remove. All uses removed.
* src/data.c (arith_driver): Treat fixnums as signed, not
unsigned, even for logical operations.
* src/lisp.h (mpz_set_uintmax): Remove. All uses removed.
* test/src/data-tests.el (data-tests-logand)
(data-tests-logior, data-tests-logxor): New tests.
Problem reported by Eli Zaretskii in:
https://lists.gnu.org/r/emacs-devel/2018-08/msg00380.html
* src/data.c (arithcompare): Work around incompatibility
between gcc -fexcess-precision=fast and the C standard on x86,
by capturing the results of floating-point comparisons before
the excess precision spontaneously decays. Although this fix
might not work in general, it does work here and is probably
good enough for the platforms we care about.
(cherry picked from commit a84cef9095)
Problem reported by Eli Zaretskii in:
https://lists.gnu.org/r/emacs-devel/2018-08/msg00380.html
* src/data.c (arithcompare): Work around incompatibility
between gcc -fexcess-precision=fast and the C standard on x86,
by capturing the results of floating-point comparisons before
the excess precision spontaneously decays. Although this fix
might not work in general, it does work here and is probably
good enough for the platforms we care about.
Eliminate the category of miscellaneous objects, and turn all
such objects into pseudovectors. The immediate motivation
for this change is to free up an enum Lisp_Type tag value, a
scarce resource that can be better used elsewhere. However,
this change is worthwhile in its own right, as it improves
performance slightly on my platform, 0.3% faster for 'make
compile-always' on Fedora 28, and it simplifies the garbage
collector and interpreter (Bug#32405).
* doc/lispref/internals.texi (Garbage Collection):
* etc/NEWS:
Document change to garbage-collect return value.
* src/alloc.c (total_markers, total_free_markers):
(union aligned_Lisp_Misc, MARKER_BLOCK_SIZE)
(struct marker_block, marker_block, marker_block_index)
(misc_free_list, allocate_misc, live_misc_holding)
(live_misc_p, sweep_misc):
* src/lisp.h (lisp_h_MARKERP, lisp_h_MISCP, MARKERP, MISCP)
(Lisp_Misc, enum Lisp_Misc_Type, Lisp_Misc_Free)
(Lisp_Misc_Marker, Lisp_Misc_Overlay, Lisp_Misc_Finalizer)
(Lisp_Misc_Ptr, Lisp_Misc_User_Ptr, Lisp_Misc_Limit)
(Lisp_Misc_Bignum)
(XSETMISC, struct Lisp_Misc_Any, XMISCANY, XMISCTYPE)
(struct Lisp_Free, union Lisp_Misc, XMISC):
Remove. All uses removed.
(cleanup_vector): Clean up objects that were formerly misc
and are now pseudovectors.
(make_misc_ptr, build_overlay, Fmake_marker, build_marker)
(make_bignum_str, make_number, make_pure_bignum)
(make_user_ptr, Fmake_finalizer):
Build as pseudovectors, not as misc objects.
(mark_finalizer_list, queue_doomed_finalizers)
(compact_undo_list, mark_overlay, mark_object)
(unchain_dead_markers):
Mark as vector-like objects, not as misc objects.
(mark_maybe_object, mark_maybe_pointer, valid_lisp_object_p)
(total_bytes_of_live_objects, survives_gc_p):
* src/fns.c (sxhash):
No need to worry about misc objects.
(garbage_collect_1): Do not generate a 'misc' component.
(syms_of_alloc): No need for 'misc' symbol.
* src/buffer.c (overlays_at, overlays_in, overlay_touches_p)
(overlay_strings, recenter_overlay_lists)
(fix_start_end_in_overlays, fix_overlays_before)
(Foverlay_lists, report_overlay_modification)
(evaporate_overlays):
* src/editfns.c (overlays_around):
* src/data.c (Ftype_of):
* src/fns.c (internal_equal):
* src/lisp.h (mint_ptrp, xmint_pointer, FINALIZERP)
(XFINALIZER, MARKERP, XMARKER, OVERLAYP, XOVERLAY, USER_PTRP)
(XUSER_PTR, BIGNUMP, XBIGNUM):
* src/print.c (print_vectorlike, print_object):
* src/undo.c (record_marker_adjustments):
* src/xdisp.c (load_overlay_strings):
Formerly misc objects are now pseudovectors.
* src/lisp.h (PVEC_MARKER, PVEC_OVERLAY, PVEC_FINALIZER)
(PVEC_BIGNUM, PVEC_MISC_PTR, PVEC_USER_PTR):
New constants, replacing their misc versions. All uses changed.
(struct Lisp_Marker, struct Lisp_Overlay, struct Lisp_Misc_Ptr)
(struct Lisp_Bignum, struct Lisp_User_Ptr, struct Lisp_Finalizer):
Make usable as a pseudovector by using a pseudovector header,
replacing any DIY components, and putting Lisp_Object members
first. All uses changed.
* lisp/international/ccl.el (ccl-fixnum): New function.
(ccl-embed-data, ccl-embed-current-address, ccl-dump): Use it.
* src/alloc.c (make_number): Handle case where EMACS_INT is
larger than long.
* src/data.c (bignumcompare): Handle case where EMACS_INT is
larger than long.
(arith_driver): Likewise. Coerce markers.
(float_arith_driver): Coerce markers.
(Flogcount): Use mpz_sgn.
(ash_lsh_impl): Fix bugs.
(Fsub1): Fix underflow check.
* src/lisp.h (NUMBERP): Don't check BIGNUMP.
(CHECK_FIXNUM_OR_FLOAT_COERCE_MARKER): Fix indentation.
* test/lisp/international/ccl-tests.el: New file.
