GCC 4.6.0 optimizes based on type-based alias analysis. For
example, if b is of type struct buffer * and v of type struct
Lisp_Vector *, then gcc -O2 was incorrectly assuming that &b->size
!= &v->size, and therefore "v->size = 1; b->size = 2; return
v->size;" must therefore return 1. This assumption is incorrect
for Emacs, since it type-puns struct Lisp_Vector * with many other
types. To fix this problem, this patch adds a new type struct
vector_header that documents the constraints on layout of vectors
and pseudovectors, and helps optimizing compilers not get fooled
by Emacs's type punning. It also adds the macros XSETTYPED_PVECTYPE
XSETTYPED_PSEUDOVECTOR, TYPED_PSEUDOVECTORP, for similar reasons.
* lisp.h (XVECTOR_SIZE): New convenience macro. All previous uses of
XVECTOR (foo)->size replaced to use this macro, to avoid the hassle
of writing XVECTOR (foo)->header.size.
(XVECTOR_HEADER_SIZE): New macro, for use in XSETPSEUDOVECTOR.
(XSETTYPED_PVECTYPE): New macro, specifying the name of the size
member.
(XSETPVECTYPE): Rewrite in terms of new macro.
(XSETPVECTYPESIZE): New macro, specifying both type and size.
This is a bit clearer, and further avoids the possibility of
undesirable aliasing.
(XSETTYPED_PSEUDOVECTOR): New macro, specifying the size.
(XSETPSEUDOVECTOR): Rewrite in terms of XSETTYPED_PSEUDOVECTOR
and XVECTOR_HEADER_SIZE.
(XSETSUBR): Rewrite in terms of XSETTYPED_PSEUDOVECTOR and XSIZE,
since Lisp_Subr is a special case (no "next" field).
(ASIZE): Rewrite in terms of XVECTOR_SIZE.
(struct vector_header): New type.
(TYPED_PSEUDOVECTORP): New macro, also specifying the C type of the
object, to help avoid aliasing.
(PSEUDOVECTORP): Rewrite in terms of TYPED_PSEUDOVECTORP.
(SUBRP): Likewise, since Lisp_Subr is a special case.
* lisp.h (struct Lisp_Vector, struct Lisp_Char_Table):
(struct Lisp_Sub_Char_Table, struct Lisp_Bool_Vector):
(struct Lisp_Hash_Table): Combine first two members into a single
struct vector_header member. All uses of "size" and "next" members
changed to be "header.size" and "header.next".
* buffer.h (struct buffer): Likewise.
* font.h (struct font_spec, struct font_entity, struct font): Likewise.
* frame.h (struct frame): Likewise.
* process.h (struct Lisp_Process): Likewise.
* termhooks.h (struct terminal): Likewise.
* window.c (struct save_window_data, struct saved_window): Likewise.
* window.h (struct window): Likewise.
* alloc.c (allocate_buffer, Fmake_bool_vector, allocate_pseudovector):
Use XSETPVECTYPESIZE, not XSETPVECTYPE, to avoid aliasing problems.
* buffer.c (init_buffer_once): Likewise.
* lread.c (defsubr): Use XSETTYPED_PVECTYPE, since Lisp_Subr is a
special case.
* process.c (Fformat_network_address): Use local var for size,
for brevity.
This makes it easier for human readers (and static analyzers)
to see whether these variables are used from other modules.
* alloc.c, buffer.c, bytecode.c, callint.c, casetab.c, category.c:
* ccl.c, character.c, charset.c, cmds.c, coding.c, composite.c:
* data.c, dbusbind.c, dired.c, editfns.c, eval.c, fileio.c, fns.c:
* font.c, frame.c, fringe.c, ftfont.c, image.c, keyboard.c, keymap.c:
* lread.c, macros.c, minibuf.c, print.c, process.c, search.c:
* sound.c, syntax.c, textprop.c, window.c, xdisp.c, xfaces.c, xfns.c:
* xmenu.c, xselect.c:
Declare Q* vars static if they are not used in other modules.
* ccl.h, character.h, charset.h, coding.h, composite.h, font.h:
* frame.h, intervals.h, keyboard.h, lisp.h, process.h, syntax.h:
Remove decls of unexported vars.
* keyboard.h (EVENT_HEAD_UNMODIFIED): Remove now-unused macro.
This makes it easier for human readers (and static analyzers)
to see whether these functions can be called from other modules.
DEFUN now defines a static function. To make the function external
so that it can be used in other C modules, use the new macro DEFUE.
* lisp.h (DEFINE_FUNC): New macro, with the old contents of DEFUN.
(DEFUN): Rewrite in terms of DEFINE_FUNC. It now generates a
static function definition. Use DEFUE if you want an extern one.
(DEFUE, INFUN): New macros.
(Funibyte_char_to_multibyte, Fsyntax_table_p, Finit_image_library):
(Feval_region, Fbacktrace, Ffetch_bytecode, Fswitch_to_buffer):
(Ffile_executable_p, Fmake_symbolic_link, Fcommand_execute):
(Fget_process, Fdocumentation_property, Fbyte_code, Ffile_attributes):
Remove decls, since these functions are now static.
(Funintern, Fget_internal_run_time): New decls, since these functions
were already external.
* alloc.c, buffer.c, callint.c, callproc.c, casefiddle.c, casetab.c:
* ccl.c, character.c, chartab.c, cmds.c, coding.c, data.c, dispnew.c:
* doc.c, editfns.c, emacs.c, eval.c, fileio.c, filelock.c, floatfns.c:
* fns.c, font.c, fontset.c, frame.c, image.c, indent.c:
* keyboard.c, keymap.c, lread.c:
* macros.c, marker.c, menu.c, minibuf.c, print.c, process.c, search.c:
* syntax.c, term.c, terminal.c, textprop.c, undo.c:
* window.c, xdisp.c, xfaces.c, xfns.c, xmenu.c, xsettings.c:
Mark functions with DEFUE instead of DEFUN,
if they are used in other modules.
* buffer.c (Fset_buffer_major_mode, Fdelete_overlay): New forward
decls for now-static functions.
* buffer.h (Fdelete_overlay): Remove decl.
* callproc.c (Fgetenv_internal): Mark as internal.
* composite.c (Fremove_list_of_text_properties): Remove decl.
(Fcomposition_get_gstring): New forward static decl.
* composite.h (Fcomposite_get_gstring): Remove decl.
* dired.c (Ffile_attributes): New forward static decl.
* doc.c (Fdocumntation_property): New forward static decl.
* eval.c (Ffetch_bytecode): New forward static decl.
(Funintern): Remove extern decl; now in .h file where it belongs.
* fileio.c (Fmake_symbolic_link): New forward static decl.
* image.c (Finit_image_library): New forward static decl.
* insdel.c (Fcombine_after_change_execute): Make forward decl static.
* intervals.h (Fprevious_property_change):
(Fremove_list_of_text_properties): Remove decls.
* keyboard.c (Fthis_command_keys): Remove decl.
(Fcommand_execute): New forward static decl.
* keymap.c (Flookup_key): New forward static decl.
(Fcopy_keymap): Now static.
* keymap.h (Flookup_key): Remove decl.
* process.c (Fget_process): New forward static decl.
(Fprocess_datagram_address): Mark as internal.
* syntax.c (Fsyntax_table_p): New forward static decl.
(skip_chars): Remove duplicate decl.
* textprop.c (Fprevious_property_change): New forward static decl.
* window.c (Fset_window_fringes, Fset_window_scroll_bars):
Now internal.
(Fset_window_margins, Fset_window_vscroll): New forward static decls.
* window.h (Fset_window_vscroll, Fset_window_margins): Remove decls.
* lisp/help.el (help-form-show): New function, to be called from C.
Put help-form output in a buffer named differently than *Help*.
* src/keyboard.c (read_char): Call Lisp function help-form-show,
instead of using internal_with_output_to_temp_buffer.
(Qhelp_form_show): New var.
* src/lisp.h (internal_with_output_to_temp_buffer): Remove prototype.
* src/print.c (internal_with_output_to_temp_buffer): Function deleted.
Without this change, on typical 64-bit hosts error ("...%d...", N)
was used to print both 32- and 64-bit integers N, which relied on
undefined behavior.
* lisp.h, src/m/amdx86-64.h, src/m/ia64.h, src/m/ibms390x.h (pEd):
New macro.
* lisp.h (error, verror): Mark as printf-like functions.
* eval.c (verror): Use vsnprintf, not doprnt, to do the real work.
Report overflow in size calculations when allocating printf buffer.
Do not truncate output string at its first null byte.
* xdisp.c (vmessage): Use vsnprintf, not doprnt, to do the real work.
Truncate the output at a character boundary, since vsnprintf does not
do that.
* charset.c (check_iso_charset_parameter): Convert internal
character to string before calling 'error', since %c now has the
printf meaning.
* coding.c (Fdecode_sjis_char, Fdecode_big5_char): Avoid int
overflow when computing char to be passed to 'error'. Do not
pass Lisp_Object to 'error'; pass the integer instead.
* nsfns.m (Fns_do_applescript): Use int, not long, since it's
formatted with plain %d.
* doc/lispref/eval.texi (Eval): Discourage the use of `eval'.
Document its new `lexical' argument.
* doc/lispref/variables.texi (Defining Variables): Mention the new meaning of `defvar'.
(Lexical Binding): New sub-section.
* lisp/Makefile.in (BIG_STACK_DEPTH, BIG_STACK_OPTS, BYTE_COMPILE_FLAGS):
New variables.
(compile-onefile, .el.elc, compile-calc, recompile): Use them.
(COMPILE_FIRST): Add macroexp and cconv.
* lisp/makefile.w32-in: Mirror changes in Makefile.in.
* lisp/vc/cvs-status.el:
* lisp/vc/diff-mode.el:
* lisp/vc/log-edit.el:
* lisp/vc/log-view.el:
* lisp/vc/smerge-mode.el:
* lisp/textmodes/bibtex-style.el:
* textmodes/css.el:
* lisp/startup.el:
* lisp/uniquify.el:
* lisp/minibuffer.el:
* lisp/newcomment.el:
* lisp/reveal.el:
* lisp/server.el:
* lisp/mpc.el:
* lisp/emacs-lisp/smie.el:
* lisp/doc-view.el:
* lisp/dired.el:
* lisp/abbrev.el: Use lexical binding.
* lisp/custom.el (custom-initialize-default, custom-declare-variable):
Use `defvar'.
* lisp/files.el (lexical-binding): Declare safe.
* lisp/help-fns.el (help-split-fundoc): Return nil if there's nothing else
than the arglist.
(help-add-fundoc-usage): Don't add `Not documented'.
(help-function-arglist): Handle closures, subroutines, and new
byte-code-functions.
(help-make-usage): Remove leading underscores.
(describe-function-1): Handle closures.
(describe-variable): Use special-variable-p for completion.
* lisp/simple.el (with-wrapper-hook, apply-partially): Move to subr.el.
* lisp/subr.el (apply-partially): Use new closures rather than CL.
(--dolist-tail--, --dotimes-limit--): Don't declare dynamic.
(dolist, dotimes): Use slightly different expansion for lexical code.
(functionp): Move to C.
(letrec): New macro.
(with-wrapper-hook): Use it and apply-partially instead of CL.
(eval-after-load): Preserve lexical-binding.
(save-window-excursion, with-output-to-temp-buffer): Turn them
into macros.
* lisp/emacs-lisp/advice.el (ad-arglist): Use help-function-arglist.
* lisp/emacs-lisp/autoload.el (make-autoload): Don't burp on trivial macros.
* lisp/emacs-lisp/byte-opt.el: Use lexical binding.
(byte-inline-lapcode): Remove (to bytecomp).
(byte-compile-inline-expand): Pay attention to inlining to/from
lexically bound code.
(byte-compile-unfold-lambda): Don't handle byte-code-functions
any more.
(byte-optimize-form-code-walker): Don't handle save-window-excursion
any more and don't call compiler-macros.
(byte-compile-splice-in-already-compiled-code): Remove.
(byte-code): Don't inline any more.
(disassemble-offset): Receive `bytes' as argument rather than via
dynamic scoping.
(byte-compile-tag-number): Declare before first use.
(byte-decompile-bytecode-1): Handle new byte-codes, don't change
`return' even if make-spliceable.
(byte-compile-side-effect-and-error-free-ops): Add stack-ref, remove
obsolete interactive-p.
(byte-optimize-lapcode): Optimize new lap-codes.
Don't trip up on new form of `byte-constant' lap code.
* lisp/emacs-lisp/byte-run.el (make-obsolete): Don't set the `byte-compile'
handler any more.
* lisp/emacs-lisp/bytecomp.el: Use lexical binding instead of
a "bytecomp-" prefix. Macroexpand everything as a separate phase.
(byte-compile-initial-macro-environment):
Handle declare-function here.
(byte-compile--lexical-environment): New var.
(byte-stack-ref, byte-stack-set, byte-discardN)
(byte-discardN-preserve-tos): New lap codes.
(byte-interactive-p): Don't use any more.
(byte-compile-push-bytecodes, byte-compile-push-bytecode-const2):
New macros.
(byte-compile-lapcode): Use them and handle new lap codes.
(byte-compile-obsolete): Remove.
(byte-compile-arglist-signature): Handle new byte-code arg"lists".
(byte-compile-arglist-warn): Check late def of inlinable funs.
(byte-compile-cl-warn): Don't silence warnings for compiler-macros
since they should have been expanded by now.
(byte-compile--outbuffer): Rename from bytecomp-outbuffer.
(byte-compile-from-buffer): Remove unused second arg.
(byte-compile-preprocess): New function.
(byte-compile-toplevel-file-form): New function to distinguish
file-form calls from outside from file-form calls from hunk-handlers.
(byte-compile-file-form): Simplify.
(byte-compile-file-form-defsubst): Remove.
(byte-compile-file-form-defmumble): Simplify now that
byte-compile-lambda always returns a byte-code-function.
(byte-compile): Preprocess.
(byte-compile-byte-code-maker, byte-compile-byte-code-unmake):
Remove, not used any more.
(byte-compile-arglist-vars, byte-compile-make-lambda-lexenv)
(byte-compile-make-args-desc): New funs.
(byte-compile-lambda): Handle lexical functions. Always return
a byte-code-function.
(byte-compile-reserved-constants): New var, to make up room for
closed-over variables.
(byte-compile-constants-vector): Obey it.
(byte-compile-top-level): New args `lexenv' and `reserved-csts'.
(byte-compile-macroexpand-declare-function): New function.
(byte-compile-form): Call byte-compile-unfold-bcf to inline immediate
byte-code-functions.
(byte-compile-form): Check obsolescence here.
(byte-compile-inline-lapcode, byte-compile-unfold-bcf): New functions.
(byte-compile-variable-ref): Remove.
(byte-compile-dynamic-variable-op): New fun.
(byte-compile-dynamic-variable-bind, byte-compile-variable-ref)
(byte-compile-variable-set): New funs.
(byte-compile-discard): Add 2 args.
(byte-compile-stack-ref, byte-compile-stack-set)
(byte-compile-make-closure, byte-compile-get-closed-var): New funs.
(byte-compile-funarg, byte-compile-funarg-2): Remove, handled in
macroexpand-all instead.
(byte-compile-quote-form): Remove.
(byte-compile-push-binding-init, byte-compile-not-lexical-var-p)
(byte-compile-bind, byte-compile-unbind): New funs.
(byte-compile-let): Handle let* and lexical binding.
(byte-compile-let*): Remove.
(byte-compile-catch, byte-compile-unwind-protect)
(byte-compile-track-mouse, byte-compile-condition-case):
Handle a new :fun-body form, used for lexical scoping.
(byte-compile-save-window-excursion)
(byte-compile-with-output-to-temp-buffer): Remove.
(byte-compile-defun): Simplify.
(byte-compile-stack-adjustment): New fun.
(byte-compile-out): Use it.
(byte-compile-refresh-preloaded): Don't reload byte-compiler files.
* lisp/emacs-lisp/cconv.el: New file.
* lisp/emacs-lisp/cl-extra.el (cl-macroexpand-all): Properly quote CL
closures.
* lisp/emacs-lisp/cl-macs.el (cl-byte-compile-block)
(cl-byte-compile-throw): Remove.
(cl-block-wrapper, cl-block-throw): Use compiler-macros instead.
* lisp/emacs-lisp/cl.el (pushnew): Silence warning.
* lisp/emacs-lisp/disass.el (disassemble-internal): Handle new
`closure' objects.
(disassemble-1): Handle new byte codes.
* lisp/emacs-lisp/edebug.el (edebug-eval-defun)
(edebug-eval-top-level-form): Use eval-sexp-add-defvars.
(edebug-toggle): Avoid `eval'.
* lisp/emacs-lisp/eieio-comp.el: Remove.
* lisp/emacs-lisp/eieio.el (byte-compile-file-form-defmethod):
Don't autoload.
(eieio-defgeneric-form-primary-only-one): Use `byte-compile' rather
than the internal `byte-compile-lambda'.
(defmethod): Don't hide code under quotes.
(eieio-defmethod): New `code' argument.
* lisp/emacs-lisp/float-sup.el (pi): Don't declare as dynamically bound.
* lisp/emacs-lisp/lisp-mode.el (eval-last-sexp-1):
Use eval-sexp-add-defvars.
(eval-sexp-add-defvars): New fun.
* lisp/emacs-lisp/macroexp.el: Use lexical binding.
(macroexpand-all-1): Check obsolete macros. Expand compiler-macros.
Don't convert ' to #' without checking that it's indeed quoting
a lambda.
* lisp/emacs-lisp/pcase.el: Don't use destructuring-bind.
(pcase--memoize): Rename from pcase-memoize. Change weakness.
(pcase): Add `let' pattern.
Change memoization so it actually works.
(pcase-mutually-exclusive-predicates): Add byte-code-function-p.
(pcase--u1) <guard, pred>: Fix possible shadowing problem.
<let>: New case.
* src/alloc.c (Fmake_symbol): Init new `declared_special' field.
* src/buffer.c (defvar_per_buffer): Set new `declared_special' field.
* src/bytecode.c (Bstack_ref, Bstack_set, Bstack_set2, BdiscardN):
New byte-codes.
(exec_byte_code): New function extracted from Fbyte_code to handle new
calling convention for byte-code-functions. Add new byte-codes.
* src/callint.c (Fcall_interactively): Preserve lexical-binding mode for
interactive spec.
* src/doc.c (Fdocumentation, store_function_docstring):
* src/data.c (Finteractive_form): Handle closures.
* src/eval.c (Fsetq): Handle lexical vars.
(Fdefun, Fdefmacro, Ffunction): Make closures when needed.
(Fdefconst, Fdefvaralias, Fdefvar): Mark as dynamic.
(FletX, Flet): Obey lexical binding.
(Fcommandp): Handle closures.
(Feval): New `lexical' arg.
(eval_sub): New function extracted from Feval. Use it almost
everywhere where Feval was used. Look up vars in lexical env.
Handle closures.
(Ffunctionp): Move from subr.el.
(Ffuncall): Handle closures.
(apply_lambda): Remove `eval_flags'.
(funcall_lambda): Handle closures and new byte-code-functions.
(Fspecial_variable_p): New function.
(syms_of_eval): Initialize the Vinternal_interpreter_environment var,
but without exporting it to Lisp.
* src/fns.c (concat, mapcar1): Accept byte-code-functions.
* src/image.c (parse_image_spec): Use Ffunctionp.
* src/keyboard.c (eval_dyn): New fun.
(menu_item_eval_property): Use it.
* src/lisp.h (struct Lisp_Symbol): New field `declared_special'.
* src/lread.c (lisp_file_lexically_bound_p): New function.
(Fload): Bind Qlexical_binding.
(readevalloop): Remove `evalfun' arg.
Bind Qinternal_interpreter_environment.
(Feval_buffer): Bind Qlexical_binding.
(defvar_int, defvar_bool, defvar_lisp_nopro, defvar_kboard):
Mark as dynamic.
(syms_of_lread): Declare `lexical-binding'.
* src/window.c (Ftemp_output_buffer_show): New fun.
(Fsave_window_excursion):
* src/print.c (Fwith_output_to_temp_buffer): Move to subr.el.
* lisp/emacs-lisp/byte-opt.el (byte-compile-side-effect-and-error-free-ops):
Remove interactive-p.
(byte-optimize-lapcode): Update optimizations now that stack-refs are
relative to the top rather than to the bottom.
* lisp/emacs-lisp/bytecomp.el (byte-compile-lapcode):
Turn stack-ref-0 into dup.
(byte-compile-form): Don't indirect-function since it can signal
errors.
(byte-compile-stack-ref, byte-compile-stack-set): Adjust to stack-refs
being relative to top rather than to bottom in the byte-code.
(with-output-to-temp-buffer): Remove.
(byte-compile-with-output-to-temp-buffer): Remove.
* lisp/emacs-lisp/cconv.el: Use lexical-binding.
(cconv--lookup-let): Rename from cconv-lookup-let.
(cconv-closure-convert-rec): Fix handling of captured+mutated
arguments in defun/defmacro.
* lisp/emacs-lisp/eieio-comp.el (eieio-byte-compile-file-form-defmethod):
Rename from byte-compile-file-form-defmethod.
Don't byte-compile-lambda.
(eieio-byte-compile-defmethod-param-convert): Rename from
byte-compile-defmethod-param-convert.
* lisp/emacs-lisp/eieio.el (eieio-defgeneric-form-primary-only-one):
Call byte-compile rather than byte-compile-lambda.
* src/alloc.c (Fgarbage_collect): Don't mark the byte-stack redundantly.
* src/bytecode.c (exec_byte_code): Change stack_ref and stack_set to use
offsets relative to top rather than to bottom.
* lisp/subr.el (with-output-to-temp-buffer): New macro.
* lisp/simple.el (count-words-region): Don't use interactive-p.
* lisp/emacs-lisp/lisp-mode.el (save-window-excursion): Don't touch.
* lisp/emacs-lisp/cconv.el (cconv-closure-convert-rec, cconv-analyse-form):
Don't handle save-window-excursion any more.
* lisp/emacs-lisp/bytecomp.el (interactive-p, save-window-excursion):
Don't use the byte-code any more.
(byte-compile-form): Check macro expansion was done.
(byte-compile-save-window-excursion): Remove.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form-code-walker):
Ignore save-window-excursion. Don't macroepand any more.
* src/window.c (Fsave_window_excursion): Remove. Moved to Lisp.
(syms_of_window): Don't defsubr it.
* src/window.h (Fsave_window_excursion): Don't declare it.
* src/bytecode.c (exec_byte_code): Inline Fsave_window_excursion.
* window.c (inhibit_point_swap): New variable.
(Fselect_window): If inhibit_point_swap is nonzero, avoid swapping
point this time.
(Fset_window_configuration): Set inhibit_point_swap to 1 instead
of setting selected_window to nil (Bug#7728).
