Do not impose an arbitrary 32-bit limit on malloc sizes when debugging.
(__malloc_size_t): Default to size_t, not to int.
(pure_size, pure_bytes_used_before_overflow, stack_copy_size)
(Fgarbage_collect, mark_object_loop_halt, mark_object):
Prefer ptrdiff_t to size_t when either would do, as we prefer
signed integers.
(XMALLOC_OVERRUN_CHECK_OVERHEAD): New macro.
(xmalloc_overrun_check_header, xmalloc_overrun_check_trailer):
Now const. Initialize with values that are in range even if char
is signed.
(XMALLOC_PUT_SIZE, XMALLOC_GET_SIZE): Remove, replacing with ...
(xmalloc_put_size, xmalloc_get_size): New functions. All uses changed.
These functions do the right thing with sizes > 2**32.
(check_depth): Now ptrdiff_t, not int.
(overrun_check_malloc, overrun_check_realloc, overrun_check_free):
Adjust to new way of storing sizes. Check for size overflow bugs
in rest of code.
(STRING_BYTES_MAX): Adjust to new overheads. The old code was
slightly wrong anyway, as it missed one instance of
XMALLOC_OVERRUN_CHECK_OVERHEAD.
(refill_memory_reserve): Omit needless cast to size_t.
(mark_object_loop_halt): Mark as externally visible.
(header_size, word_size): New constants.
(allocate_vectorlike): Don't check size overflow here.
(allocate_vector): Check it here instead, since this is the only
caller of allocate_vectorlike that could cause overflow.
Check that the new vector's length is representable as a fixnum.
This partly undoes my 2011-03-30 change, which replaced int with size_t.
Back then I didn't know that the Emacs coding style prefers signed int.
Also, in the meantime I found a few more instances where arguments
were being counted with int, which may truncate counts on 64-bit
machines, or EMACS_INT, which may be unnecessarily wide.
* lisp.h (struct Lisp_Subr.function.aMANY)
(DEFUN_ARGS_MANY, internal_condition_case_n, safe_call):
Arg counts are now ptrdiff_t, not size_t.
All variadic functions and their callers changed accordingly.
(struct gcpro.nvars): Now size_t, not size_t. All uses changed.
* bytecode.c (exec_byte_code): Check maxdepth for overflow,
to avoid potential buffer overrun. Don't assume arg counts fit in 'int'.
* callint.c (Fcall_interactively): Check arg count for overflow,
to avoid potential buffer overrun. Use signed char, not 'int',
for 'varies' array, so that we needn't bother to check its size
calculation for overflow.
* editfns.c (Fformat): Use ptrdiff_t, not EMACS_INT, to count args.
* eval.c (apply_lambda):
* fns.c (Fmapconcat): Use XFASTINT, not XINT, to get args length.
(struct textprop_rec.argnum): Now ptrdiff_t, not int. All uses changed.
(mapconcat): Use ptrdiff_t, not int and EMACS_INT, to count args.
(struct Lisp_Save_Value): Use ptrdiff_t, not int, for 'integer' member.
* alloc.c (make_save_value): Integer argument is now of type
ptrdiff_t, not int.
(mark_object): Use ptrdiff_t, not int.
* lisp.h (pD): New macro.
* print.c (print_object): Use it.
(n_vectors, n_symbol_blocks, n_marker_blocks): Remove.
These were 'int' variables that could overflow on 64-bit hosts;
they were never used, so remove them instead of repairing them.
(total_conses, total_markers, total_symbols, total_vector_size)
(total_free_conses, total_free_markers, total_free_symbols)
(total_free_floats, total_floats, total_free_intervals, total_intervals)
(total_strings, total_free_strings):
Now EMACS_INT, not int. All uses changed.
(Fgarbage_collect): Compute overall total using a double, so that
integer overflow is less likely to be a problem. Check for overflow
when converting back to an integer.
(XMALLOC_OVERRUN_CHECK_SIZE) [!XMALLOC_OVERRUN_CHECK]: Define to 0,
for convenience in STRING_BYTES_MAX.
(STRING_BYTES_MAX): New macro, superseding the old one in lisp.h.
The definition here is exact; the one in lisp.h was approximate.
(allocate_string_data): Check for string overflow. This catches
some instances we weren't catching before. Also, it catches
size_t overflow on (unusual) hosts where SIZE_MAX <= min
(PTRDIFF_MAX, MOST_POSITIVE_FIXNUM), e.g., when size_t is 32 bits
and ptrdiff_t and EMACS_INT are both 64 bits.
* character.c, coding.c, doprnt.c, editfns.c, eval.c:
All uses of STRING_BYTES_MAX replaced by STRING_BYTES_BOUND.
* lisp.h (STRING_BYTES_BOUND): Renamed from STRING_BYTES_MAX.
* buffer.h (BUF_BYTES_MAX): New macro.
* lisp.h (STRING_BYTES_MAX): New macro.
* alloc.c (Fmake_string):
* character.c (string_escape_byte8):
* coding.c (coding_alloc_by_realloc):
* doprnt.c (doprnt):
* editfns.c (Fformat):
* eval.c (verror):
Use STRING_BYTES_MAX, not MOST_POSITIVE_FIXNUM,
since they may not be the same number.
* editfns.c (Finsert_char):
* fileio.c (Finsert_file_contents):
Likewise for BUF_BYTES_MAX.
* category.c (hash_get_category_set): Use 'EMACS_UINT' and 'EMACS_INT'
for hashes and hash indexes, instead of 'unsigned' and 'int'.
* ccl.c (ccl_driver): Likewise.
* charset.c (Fdefine_charset_internal): Likewise.
* charset.h (struct charset.hash_index): Likewise.
* composite.c (get_composition_id, gstring_lookup_cache):
(composition_gstring_put_cache): Likewise.
* composite.h (struct composition.hash_index): Likewise.
* dispextern.h (struct image.hash): Likewise.
* fns.c (next_almost_prime, larger_vector, cmpfn_eql):
(cmpfn_equal, cmpfn_user_defined, hashfn_eq, hashfn_eql):
(hashfn_equal, hashfn_user_defined, make_hash_table):
(maybe_resize_hash_table, hash_lookup, hash_put):
(hash_remove_from_table, hash_clear, sweep_weak_table, SXHASH_COMBINE):
(sxhash_string, sxhash_list, sxhash_vector, sxhash_bool_vector):
(Fsxhash, Fgethash, Fputhash, Fmaphash): Likewise.
* image.c (make_image, search_image_cache, lookup_image):
(xpm_put_color_table_h): Likewise.
* lisp.h (struct Lisp_Hash_Table): Likewise, for 'count', 'cmpfn',
and 'hashfn' members.
* minibuf.c (Ftry_completion, Fall_completions, Ftest_completion):
Likewise.
* print.c (print): Likewise.
* alloc.c (allocate_vectorlike): Check for overflow in vector size
calculations.
* ccl.c (ccl_driver): Check for overflow when converting EMACS_INT
to int.
* fns.c, image.c: Remove unnecessary static decls that would otherwise
need to be updated by these changes.
* fns.c (make_hash_table, maybe_resize_hash_table): Check for integer
overflow with large hash tables.
(make_hash_table, maybe_resize_hash_table, Fmake_hash_table):
Prefer the faster XFLOAT_DATA to XFLOATINT where either will do.
(SXHASH_REDUCE): New macro.
(sxhash_string, sxhash_list, sxhash_vector, sxhash_bool_vector):
Use it instead of discarding useful hash info with large hash values.
(sxhash_float): New function.
(sxhash): Use it. No more need for "& INTMASK" due to above changes.
* lisp.h (FIXNUM_BITS): New macro, useful for SXHASH_REDUCE etc.
(MOST_NEGATIVE_FIXNUM, MOST_POSITIVE_FIXNUM, INTMASK): Rewrite
to use FIXNUM_BITS, as this simplifies things.
(next_almost_prime, larger_vector, sxhash, hash_lookup, hash_put):
Adjust signatures to match updated version of code.
(consing_since_gc): Now EMACS_INT, since a single hash table can
use more than INT_MAX bytes.
Malloc failure behavior now depends on size of allocation.
* lib/allocator.h (struct allocator.die): New size arg.
* lib/careadlinkat.c (careadlinkat): Pass size to 'die' function.
If the actual problem is an ssize_t limitation, not a size_t or
malloc failure, fail with errno == ENAMETOOLONG instead of calling 'die'.
[src/ChangeLog]
Malloc failure behavior now depends on size of allocation.
* alloc.c (buffer_memory_full, memory_full): New arg NBYTES.
* lisp.h: Change signatures accordingly.
* alloc.c, buffer.c, editfns.c, menu.c, minibuf.c, xterm.c:
All callers changed.
(Fmake_string): Use it. This doesn't change behavior, but saves
a few bytes and will simplify future changes.
* character.c (string_escape_byte8): Likewise.
* lisp.h (string_overflow): New decl.
This removes an assumption that EMACS_INT and long are the same
width as pointers. The assumption is true for Emacs porting targets
now, but we want to make other targets possible.
* lisp.h: Include <inttypes.h>, for INTPTR_MAX, UINTPTR_MAX.
(EMACS_INTPTR, EMACS_UINTPTR): New macros.
In the rest of the code, change types of integers that hold casted
pointers to EMACS_INTPTR and EMACS_UINTPTR, systematically
replacing EMACS_INT, long, EMACS_UINT, and unsigned long.
(XTYPE): Don't cast arg to EMACS_UINT; normally is not needed.
(XSET): Cast type of XTYPE arg to EMACS_INTPTR; it is needed here.
No need to cast type when ORing.
(XPNTR): Return a value of type EMACS_INTPTR or EMACS_UINTPTR.
* alloc.c (lisp_align_malloc): Remove a no-longer-needed cast.
* doc.c (store_function_docstring): Use EMACS_INTPTR, so as not to
assume EMACS_INT is the same width as char *.
* gtkutil.c (xg_gtk_scroll_destroy, xg_tool_bar_button_cb):
(xg_tool_bar_callback, xg_tool_bar_help_callback, xg_make_tool_item):
Remove no-longer-needed casts.
(xg_create_scroll_bar, xg_tool_bar_button_cb, xg_tool_bar_callback):
(xg_tool_bar_help_callback, xg_make_tool_item):
Use EMACS_INTPTR to hold an integer
that will be cast to void *; this can avoid a GCC warning
if EMACS_INT is not the same width as void *.
* menu.c (find_and_call_menu_selection): Remove no-longer-needed cast.
* xdisp.c (display_echo_area_1, resize_mini_window_1):
(current_message_1, set_message_1):
Use a local to convert to proper width without a cast.
* xmenu.c (dialog_selection_callback): Likewise.
GCC 4.6.0 optimizes based on type-based alias analysis. For
example, if b is of type struct buffer * and v of type struct
Lisp_Vector *, then gcc -O2 was incorrectly assuming that &b->size
!= &v->size, and therefore "v->size = 1; b->size = 2; return
v->size;" must therefore return 1. This assumption is incorrect
for Emacs, since it type-puns struct Lisp_Vector * with many other
types. To fix this problem, this patch adds a new type struct
vector_header that documents the constraints on layout of vectors
and pseudovectors, and helps optimizing compilers not get fooled
by Emacs's type punning. It also adds the macros XSETTYPED_PVECTYPE
XSETTYPED_PSEUDOVECTOR, TYPED_PSEUDOVECTORP, for similar reasons.
* lisp.h (XVECTOR_SIZE): New convenience macro. All previous uses of
XVECTOR (foo)->size replaced to use this macro, to avoid the hassle
of writing XVECTOR (foo)->header.size.
(XVECTOR_HEADER_SIZE): New macro, for use in XSETPSEUDOVECTOR.
(XSETTYPED_PVECTYPE): New macro, specifying the name of the size
member.
(XSETPVECTYPE): Rewrite in terms of new macro.
(XSETPVECTYPESIZE): New macro, specifying both type and size.
This is a bit clearer, and further avoids the possibility of
undesirable aliasing.
(XSETTYPED_PSEUDOVECTOR): New macro, specifying the size.
(XSETPSEUDOVECTOR): Rewrite in terms of XSETTYPED_PSEUDOVECTOR
and XVECTOR_HEADER_SIZE.
(XSETSUBR): Rewrite in terms of XSETTYPED_PSEUDOVECTOR and XSIZE,
since Lisp_Subr is a special case (no "next" field).
(ASIZE): Rewrite in terms of XVECTOR_SIZE.
(struct vector_header): New type.
(TYPED_PSEUDOVECTORP): New macro, also specifying the C type of the
object, to help avoid aliasing.
(PSEUDOVECTORP): Rewrite in terms of TYPED_PSEUDOVECTORP.
(SUBRP): Likewise, since Lisp_Subr is a special case.
* lisp.h (struct Lisp_Vector, struct Lisp_Char_Table):
(struct Lisp_Sub_Char_Table, struct Lisp_Bool_Vector):
(struct Lisp_Hash_Table): Combine first two members into a single
struct vector_header member. All uses of "size" and "next" members
changed to be "header.size" and "header.next".
* buffer.h (struct buffer): Likewise.
* font.h (struct font_spec, struct font_entity, struct font): Likewise.
* frame.h (struct frame): Likewise.
* process.h (struct Lisp_Process): Likewise.
* termhooks.h (struct terminal): Likewise.
* window.c (struct save_window_data, struct saved_window): Likewise.
* window.h (struct window): Likewise.
* alloc.c (allocate_buffer, Fmake_bool_vector, allocate_pseudovector):
Use XSETPVECTYPESIZE, not XSETPVECTYPE, to avoid aliasing problems.
* buffer.c (init_buffer_once): Likewise.
* lread.c (defsubr): Use XSETTYPED_PVECTYPE, since Lisp_Subr is a
special case.
* process.c (Fformat_network_address): Use local var for size,
for brevity.
