When loading a file if in this session there was ever a file loaded
with that name rename it before loading it to make sure we always get
a new handle from the standard library.
When recompiling remove the corresponding stale elns found in the
`comp-eln-load-path'.
When removing a package remove the corresponding elns too.
On Windows both of these are performed only when possible, when it's
not the file is renamed as .eln.old and a last attempt to remove this
is performed closing the Emacs session. When a file being deleted was
loaded by multiple Emacs sessions the last one being closed should
delete it.
* lisp/emacs-lisp/comp.el (comp-clean-up-stale-eln): New function.
(comp-delete-or-replace-file): Rename from
`comp--replace-output-file' and update so it can be used for
replacing or deleting shared libs safetly.
* lisp/emacs-lisp/package.el (package--delete-directory): When
native compiled just call `comp-clean-up-stale-eln' for each
eln file we want to clean-up.
* src/alloc.c (cleanup_vector): Call directly the dynlib_close.
* src/comp.c (syms_of_comp): Update for comp_u->cfile removal.
Make 'all_loaded_comp_units_h' key-value weak as now the key will
be the filename.
(load_comp_unit): Register the compilation unit only when the load
is fully completed.
(register_native_comp_unit): Make the key of
all_loaded_comp_units_h the load filename.
(eln_load_path_final_clean_up): New function.
(dispose_comp_unit)
(finish_delayed_disposal_of_comp_units)
(dispose_all_remaining_comp_units)
(clean_package_user_dir_of_old_comp_units): Remove.
(Fcomp__compile_ctxt_to_file): Update for
`comp--replace-output-file' -> `comp-delete-or-replace-file'
rename.
* src/comp.h (dispose_comp_unit)
(finish_delayed_disposal_of_comp_units)
(dispose_all_remaining_comp_units)
(clean_package_user_dir_of_old_comp_units): Remove.
(eln_load_path_final_clean_up): Add.
(struct Lisp_Native_Comp_Unit): Remove cfile field.
* src/emacs.c (Fkill_emacs): Call 'eln_load_path_final_clean_up'.
* src/pdumper.c (dump_do_dump_relocation): Do not set comp_u->cfile.
Rename eln sys directory into 'native-lisp' and move it from
"$(DESTDIR)${libexecdir}/emacs/${version}/${configuration}/" to
"$(DESTDIR)${libdir}/emacs/".
Add to the directory name used to disambiguate the eln compatibility
the Emacs version to have it more user friendly.
* Makefile.in (clean, install-eln): Rename eln-cache into
native-lisp.
(ELN_DESTDIR): Move from
'$(DESTDIR)${libexecdir}/emacs/${version}/${configuration}/' to
'$(DESTDIR)${libdir}/emacs/'.
* lisp/loadup.el: Update for comp-native-path-postfix ->
comp-native-version-dir rename.
* src/comp.c (syms_of_comp): Rename eln-cache -> native-lisp.
(syms_of_comp, Fcomp_el_to_eln_filename): Rename
comp-native-path-postfix -> comp-native-version-dir.
(hash_native_abi): Rework and add emacs-version to
comp-native-version-dir.
* Instead of conditionalizing on the wrong preprocessor flag, now use
the right one: LIBGCCJIT_HAVE_gcc_jit_context_add_driver_option
* Also perform the driver-option-adding step on win NT, but only if
the function is non-NULL.
* Make the function declaration for add_driver_options non-old-style.
Generate eln filename hashing also the source file content in the form:
/absolute/path/filename.el + content ->
eln-cache/filename-path_hash-content_hash.eln
* src/lread.c (maybe_swap_for_eln): Always call
Fcomp_el_to_eln_filename on an existing source file.
* src/comp.c (md5.h, sysstdio.h, zlib.h): New include.
(comp_hash_string): Use md5 instead of sha512.
(MD5_BLOCKSIZE): New macro.
(accumulate_and_process_md5, final_process_md5, md5_gz_stream)
(comp_hash_source_file): New functions.
(Fcomp_el_to_eln_filename): Rework for hasing using also source
file content.
* src/lread.c (maybe_swap_for_eln): Rename el_name -> src_name as
this can be also a have .el.gz extention.
Add a customizable variable for driver options (such as linker flags)
to pass to libgccjit (Bug #42761).
* lisp/emacs-lisp/comp.el (comp-native-driver-options): New
customization variable.
* src/comp.c: Use comp-native-driver-options to set libgccjit's driver
options, if supported on the library's ABI version.
'dlopen' can return the same handle if two shared with the same
filename are loaded in two different times (even if the first was
deleted!). To prevent this scenario the last modification time of the
source file is included in the hashing algorithm.
* src/comp.c (Fcomp_el_to_eln_filename): Update hashing algo to
include the source last modification date.
* src/lread.c (maybe_swap_for_eln): Do not check for eln newer
then elc as this is now unnecessary.
Make eln filename hashing logic insensitive to the installation
process.
* src/comp.c (epaths.h): New include to have PATH_DUMPLOADSEARCH,
PATH_LOADSEARCH definitions.
(loadsearch_re_list): New static var.
(Fcomp_el_to_eln_filename): Update logic to have the eln hashing
insensitive to the installation process.
(syms_of_comp): GC protect 'loadsearch_re_list'.
This is needed for installed instances compiled with NATIVE_FAST_BOOT
* src/comp.c (maybe_defer_native_compilation): Search for .el.gz
too as a source if the .el is not found.
(Fcomp_el_to_eln_filename): Remove the .gz in case to
generate the hash.
* lisp/emacs-lisp/comp.el (comp-valid-source-re): New defconst.
(comp-run-async-workers, native-compile-async): Make use of
`comp-valid-source-re'.
* src/comp.h (fixup_eln_load_path): New extern.
* src/comp.c (fixup_eln_load_path): New function.
* src/pdumper.c (dump_do_dump_relocation): Update to make use of
'fixup_eln_load_path'.
* lisp/loadup.el: Update to store in the compilation unit the
correct eln-cache installed path. Rename --lisp-dest -> --eln-dest
and.
* Makefile.in: Pass the eln destination directory to
src/Makefile. Rename LISP_DESTDIR -> ELN_DESTDIR.
(ELN_DESTDIR): Define.
(install-eln): New target.
(install): Add install-eln as prerequisite.
* src/Makefile.in: Rename --lisp-dest -> --eln-dest and
LISP_DESTDIR -> ELN_DESTDIR.
* src/comp.c (comp_deferred_compilation): Doc update and set it to
true by default.
* lisp/emacs-lisp/comp.el (comp-deferred-compilation): Remove
customize.
When loading a elc file search for a corresponding eln one into
`comp-eln-load-path' directories and load it if available.
`comp-eln-load-path' contains by default two directory (user and
system one).
* src/pdumper.c (dump_do_dump_relocation): While resurrecting from
load set eln cache sys dir in `Vcomp_eln_load_path'.
* src/lread.c (maybe_swap_for_eln): New function.
(Fload): Clean-up some now unnecessary code going
back to the master one.
(Fload): Make use of Vcomp_eln_to_el_h for the reverse file
look-up.
(openp_add_middle_dir_to_suffixes)
(openp_max_middledir_and_suffix_len, openp_fill_filename_buffer):
Remove functions.
(openp): As for Fload revert code modifications.
(openp): When a .elc file is being loaded check if a corresponding
eln can be loaded in place.
* src/comp.c (ELN_FILENAME_HASH_LEN): New macro.
(comp_hash_string): New function.
(hash_native_abi): Make use of 'comp_hash_string'.
(hash_native_abi): Change `comp-native-path-postfix' format.
(Fcomp_el_to_eln_filename): New function.
(Fcomp__compile_ctxt_to_file): Have file_name as a input.
(Vcomp_eln_to_el_h, Vcomp_eln_load_path): New global varaibles.
* lisp/startup.el (normal-top-level): Add user eln cache directory
in `comp-eln-load-path'.
* lisp/help-fns.el (find-lisp-object-file-name): Reverse look-up
files using `comp-eln-to-el-h'.
* lisp/files.el (locate-file): Likewise.
* lisp/emacs-lisp/find-func.el (find-library-name): Likewise.
* lisp/emacs-lisp/comp.el (comp-output-directory)
(comp-output-base-filename, comp-output-filename): Remove function.
(comp-compile-ctxt-to-file): Create parent directories if
necessary.
(comp-run-async-workers, native-compile, native-compile-async):
Make use `comp-el-to-eln-filename'.
* src/comp.c (compile_function): Allocate function frame as array
if non local exits are present to retain correct Elisp semantic.
(emit_limple_call_ref): Directly use the frame array for ref calls
to have GCC spills into it before calling.
All frame slots are now simple automatic variables given the array
allocation and fill is done in 'emit_limple_call_ref'.
* src/comp.c (comp_t): Remove 'f_frame' 'arrays' slots, add
'frame'.
(emit_mvar_lval): Simplify to make use of 'comp.frame'.
(compile_function): Clean-up and add comp.frame initialization.
* src/comp.c (comp_t): Add 'zero' field.
(emit_limple_call_ref): Allocate an array to host the parametes
and generate the code moving values into.
(Fcomp__init_ctxt): Initialize comp.zero.
* src/comp.c (emit_static_object): Bind a bunch of special
variables to setup `prin1-to-string' as
`byte-compile-output-file-form' does. This to preserve
uninterned symbols.
* src/comp.c (COMP_SPEED): Rename.
(comp_t): Add 'func_speed' field.
(emit_mvar_lval, compile_function): Update for per function speed.
(Fcomp__compile_ctxt_to_file): COMP_SPEED renamed.
* lisp/emacs-lisp/comp.el (comp-speed): Doc update.
(comp-func): New 'speed' slot.
(comp-spill-speed): New function.
(comp-spill-lap-function, comp-intern-func-in-ctxt): Fill 'speed'
slot.
(comp-spill-lap-function): Gate -1 speed functions for native
compilation and emit bytecode instead.
(comp-spill-lap): Close over `byte-to-native-plist-environment'.
(comp-latch-make-fill): Update for per function speed.
(comp-limplify-top-level): Fill speed.
(comp-propagate1, comp-call-optim-form-call, comp-call-optim)
(comp-dead-code, comp-tco, comp-remove-type-hints): Update for per
function speed.
Add an initial implementation to support dynamic scope. Arg
parsing/binding it's done using the existing code in use for
bytecode (no ad-hoc code is synthetized for that).
* src/lisp.h (struct Lisp_Subr): Add lambda_list field.
(SUBR_NATIVE_COMPILED_DYNP): New inliner.
* src/alloc.c (mark_object): Update for Add lambda_list field.
* src/eval.c (eval_sub, Ffuncall, funcall_lambda): Handle native
compiled dynamic scope
* src/comp.c (declare_lex_function): Rename from declare_function
and rework.
(declare_function): New function.
(make_subr): Handle daynamic scope
* src/pdumper.c (dump_subr): Update for lambda_list field.
* lisp/emacs-lisp/comp.el (comp-func): Remove args slot.
(comp-func-l, comp-func-d): New classes deriving from `comp-func'.
(comp-spill-lap-function): Rework.
(comp-prepare-args-for-top-level): New function.
(comp-emit-for-top-level, comp-emit-lambda-for-top-level): Make
use of `comp-prepare-args-for-top-level'.
(comp-limplify-top-level): Use `comp-func-l'.
(comp-limplify-function): Emit arg prologue only for dynamic
scoped functions.
(comp-call-optim-form-call): Use `comp-func-l'.
(comp-call-optim, comp-tco): Do not optimize dynamic scoped code.
Implement the backend side of 'maybe_gc_or_quit' so that every time a
call to it is emitted we render it accordingly. This allow GC to
kicks in during long loops in Lisp code.
* src/comp.c (comp_t): Add 'maybe_gc_or_quit' field.
(helper_link_table): Add 'maybe_gc', 'maybe_quit'.
(emit_maybe_gc_or_quit): New function.
(declare_runtime_imported_funcs): Import 'maybe_gc', 'maybe_quit'
functions.
(define_maybe_gc_or_quit): New function.
(Fcomp__init_ctxt): Register emitter.
(Fcomp__compile_ctxt_to_file): Call 'define_maybe_gc_or_quit'.
(syms_of_comp): Define Qcomp_maybe_gc_or_quit.
Removing `Vcomp_sym_subr_c_name_h' all c_name functions are GC
markable only through 'data_ephemeral_vec'. A recursive load must not
overide its content otherwise a previously activated load will have
the original content collected before it's used.
* src/comp.h (struct Lisp_Native_Comp_Unit): Add 'load_ongoing'
field.
* src/comp.c (unset_cu_load_ongoing): New function.
(load_comp_unit): Update logic to detect and handle recursive
loads.
Given there's no more unique relation symbol-name -> c-name remove
`Vcomp_sym_subr_c_name_h' and store the c_name directly in struct
Lisp_Subr. The old approach would have failed dumping two functions
with the same symbol-name.
* src/lisp.h (struct Lisp_Subr): Add 'native_c_name' field.
* src/pdumper.c (dump_subr): Update hash + dump 'native_c_name'.
(dump_cold_native_subr): dump 'native_c_name'.
(dump_do_dump_relocation): Update logic for reviving using
'native_c_name'.
* src/comp.c (make_subr): Update for 'native_c_name' field.
(Fcomp__register_lambda, Fcomp__register_subr): Clean-up code for
'Vcomp_sym_subr_c_name_h' removal.
(syms_of_comp): Remove 'Vcomp_sym_subr_c_name_h'.
* src/comp.c (load_comp_unit): When swapping the compilation unit
abandoning the new one for the original do not forget to set its
loaded_once field to true because is in use by
`comp--register-lambda'.
(Fcomp__register_lambda): Add sanity a check to spot
early if we are trying to load the same lambda twice.
Is cleaner to have the function c-name as first argument of
'direct-call' 'direct-callref'. This is preparatory to anonymous
lambdas optimization.
* lisp/emacs-lisp/comp.el (comp-propagate-insn): Use c-name when
gathering the comp-func definition for direct calls.
(comp-call-optim-form-call): Add put c-name as first argument of
direct-call direct-callref when optimizing.
* src/comp.c (emit_call): Update logic for having c-name as
first arg of direct calls.
(emit_call_ref): Rename 'subr_sym' into 'func'.
* src/comp.c (init_gccjit_functions): Use LOAD_DLL_FN_OPT macro to
load gcc_jit_version_major, gcc_jit_version_major and
gcc_jit_version_patchlevel.
* src/w32common.h (LOAD_DLL_FN_OPT): Add macro optionally load a
function from a DLL.
* src/comp.c (cast_kind_of_type): Enum that specifies the kind of type
in the cast enum (unsigned, signed, pointer).
(emit_coerce): Throw an ICE when asked to emit a cast with sign
extension.
(define_cast_from_to): Return NULL for casts involving sign extension.
(define_cast_functions): Specify the kind of each type in the cast
union.
This is to dump prettier C files.
This does not affect compilation times in my tests.
* src/comp.c: Define a 15x15 cast matrix. Use it in emit_coerce().
* src/comp.c (emit_static_object): Use a chunck size of 200 bytes
on bugged GCCs and a longer one (1024) in sane ones. Rename
str in buff to disambiguate and prefer xmalloc to a VLA given
the buffer is not that small.
This change drastically reduce compile time. Apparently GCC optimizer
does not scale up well at all for long sequences of assignments into a
single array.
Nicolás Bértolo <nicolasbertolo@gmail.com>
Andrea Corallo <akrl@sdf.org>
* src/comp.c (gcc_jit_context_new_string_literal)
(gcc_jit_block_add_assignment_op): New imports.
(comp_t): New 'size_t_type' 'memcpy' fields.
(emit_static_object): Define static objects using string literals
and memcpy.
(define_memcpy): New function.
(Fcomp__init_ctxt): Define 'size_t_type' and 'memcpy'.
