Commit graph

9049 commits

Author SHA1 Message Date
Marius Gerbershagen
a01603a04a Merge branch 'bytevm-locals-on-stack' into 'develop'
ByteVM locals are stored on the lisp stack

See merge request embeddable-common-lisp/ecl!346
2025-05-10 17:31:24 +00:00
Daniel Kochmański
1b5a03ffca tests: disable a test on bytecmp that uses compiler-cc 2025-05-09 13:55:59 +02:00
Daniel Kochmański
db9f22c190 lisp stack: deallocate the old stack after resizing
Just like with other stacks we call ecl_dealloc on the old stack. Previously
we had stack frames that could have referenced it, but we can treat it like
other stacks, because there are not lingering references.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
2f9ce70e8f stack frames: dereference directly to env->stack
Previously we've cached the stack base and dereferenced from there, but when the
stack is resized, this reference is invalidated and there is no good fix it in
all frames (we don't store back references).

This commit replaces pointers with indexes, so the stack frame is always
displaced onto the current lisp stack.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
293f93b425 stack frames: don't displace onto env->values vector
ad6f1f7f10 (2008-05-12) allowed for displacing
stack frames onto the values vector. This commit seems to be before we've stored
them on lisp stack.

As for the rationale why we revert that change:

1. There may be a slight performance improvement because the standard gf
dispatch won't be forced to copy the stack frame

2. Stack frame operators assume that they are displaced onto env->stack*, for
example ecl_stack_frame_push increments env->stack_top

3. If we fully buy into assumption that the stack frame is displaced onto
env->stack* then we may replace pointers with indexes and protect the user from
accessing invalidated memory when the stack grows

The last point will be implemented in the next commit. It is worth noting, that
frames auto-heal themselves if their own operation is the cause of the stack
growth, but all other stack frames are left invalid after the operation.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
aefcd77e02 tests: add a regression test for an encountered issue
This issue was never manifestable in develop branch.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
fdf7c77909 bytevm: faster closure creation and less consing
OP_FLET previously had to first create functions referencing the env "before"
and then add these functions to the env "after". This is because we were
addressing from the stack top:

  env: bottom[0:var]top
  fn1: ref(top-1)
  fn2: ref(top-1)
  env: bottom[0:var, 1:fn1, 2:f2n]top

Otherwise fn2 referencing top-1 would point to fn1 instead of var.

Now that locals are addressed from the stack bottom we can add functions eagerly
to locals without consing intermediate sequence of functions:

  env: bottom[0:var]top
  fn1: ref(bottom+0)
  env: bottom[0:var, 1:fn1]top
  fn2: ref(bottom+0)
  env: bottom[0:var, 1:fn1, 2:f2n]top

That saves us one loop (nfun iterations).

A similar observation applies to LABELS, although we still need the closure
fixup because earlier function may reference a function defined later, so we
still need a second loop. That said we don't have to cons a separate sequence of
functions, because we may iterate over locals vector instead.

Both changes save us from an operator that adds a collection to the lcl_env, so
we remove tack_lcl macro and its expansion function foot_lcl.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
0bf83ad30c bytevm: allocate locals on the stack
This should speed things up as well as reduce consing. To be verified.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
2382fa41ab bytecmp: track the maximal size of the locals environment
After the function is compiled we assign the width to nlcl. This will be used to
determine the necessary size of the stack frame for locals.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
512a31301d bytecmp: [unbug] remove assertions that are now not necessary 2025-05-09 13:55:59 +02:00
Daniel Kochmański
16996f8d38 bytevm: treat blocks and tags as normal local bindings
Previously when unbinding we've skipped blocks and tags and let the frame exit
to collect the frame variable. This commit simplifies managing the environment
in the compiler.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
878fc3441c bytecmp: compute the locals env size correctly
Previously our calculations were off because:

- we've counted special variables
- we didn't decrement the env size when unbound

One could argue that we need only to increment it, but then we'd end up with the
size too big. Consider:

(progn
 (let ((a 1)) (foobar))
 (let ((b 1)) (foobar)))

In this snippet we have two local variables, but we never look at locals beyond
the first element.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
6babf49b1c bytevm: reshape OP_FRAME to resemble OP_TAGBODY and OP_PROTECT
This is a cosmetic change that makes unwinding look similar across operators.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
8bc957d88f bytecmp: [debug] add assertions for the current environment size
env_size is not computed correctly (neither max size nor current size), these
assertions are meant to help with mending the issue in order to correctly
determine the size of the locals environment.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
600901da05 bytecodes: add a new slot nlcl denoting the maximal number of locals
Currently it is initialized to NIL, the next commit will assign to it a correct
value.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
9addb2cc4a bytevm: access interpreter locals in fifo order
This is a preliminary step before replacing a structure consed on a heap with
the lisp stack.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
34f546027f compiler: factor the local entry index into a function
c_lcl_idx computes the location of the entry in the locals env. It does traverse
the list again, so it adds a little overhead, but this will allow us to
customize it later to compute a location from the bottom.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
c2d1a0c2d2 ecl_interpreter: abstract away all lcl accessors
We are going to change the representation of the local environment, but first we
make identify accessors and put them behind macros.

While doing so the OP_LABELS has been changed to look similar to OP_FLET. Among
other things we cons separately functions into fun_env, but this inefficiency
will be removed later when we address local entries from the frame.base.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
a44a74a3f4 internal: ignore array-bounds errors on GCC >= 12
As noted in the comment, GCC 12.0 onwards generates an invalid warning about
array bounds when we cast a stack allocated object to cl_object and use it

    warning: array subscript ‘union cl_lispunion[0]’ is partly outside array
    bounds of ‘struct ecl_stack_frame[1]’ [-Warray-bounds=]

The code is conforming, but apparently GCC IR has generated a dead branch that
accesses the array out of bounds. For time being we disable this warning.

See also:
https://gitlab.com/libeigen/eigen/-/issues/2506
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106274
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105523

and generally explroe gcc bugzilla for this kind of false positive.
2025-05-09 13:55:59 +02:00
Daniel Kochmański
2948a57174 interpreter: fix a braino
During recent refactor we've dereferenced a frame instaed of bytecodes.
2025-05-09 12:12:05 +02:00
Daniel Kochmański
885e53a0b5 serialize: fix a braino
ecl_frame was used instead of ecl_stack_frame to compute the size of t_frame.
2025-05-09 12:12:05 +02:00
Daniel Kochmański
895f9571c3 symbols: initialize .macfun field
Apparently this one was missing.
2025-05-05 10:26:51 +02:00
Marius Gerbershagen
f69ded9777 remove bogus #undef bool statement
Not sure why that was ever there.
2025-05-04 18:02:59 +02:00
Marius Gerbershagen
b4e22b1854 fix typo 2025-05-04 17:52:50 +02:00
Daniel Kochmański
8099af9350 Merge branch 'c23-port' into 'develop'
port to C23 standard

Closes #775

See merge request embeddable-common-lisp/ecl!345
2025-05-03 18:49:37 +00:00
Daniel Kochmański
e5e6a97676 core: simplify the signature of si:interpreter-stack
Currently it returns NIL because runtime_stack is not a vector nor a list (or
other lisp structure). Technically we could return a foreign data pointer.
2025-05-02 14:58:26 +02:00
Marius Gerbershagen
1aec8f741f port to C23 standard
There are two things to fix:

- `bool` is a keyword in C23, so `typedef int bool` is invalid. We
  already require C99, so just include stdbool.h instead. This also
  means that `bool` and `int` are no longer synonymous, so we have to
  be more careful in defining return types for some functions.
- Function definitions and function pointers with unspecified
  arguments are no longer valid. Fix the definitions to include
  arguments and add casts for the function pointers.
2025-04-26 18:13:40 +02:00
Daniel Kochmański
b08f1dc986 Merge branch 'fix-750' into 'develop'
deftype: implement &environment parameters

Closes #750

See merge request embeddable-common-lisp/ecl!344
2025-04-22 07:02:58 +00:00
Daniel Kochmański
94a2725ddb Merge branch 'fix-774' into 'develop'
Fix issue #774

Closes #774

See merge request embeddable-common-lisp/ecl!343
2025-04-22 06:57:28 +00:00
Marius Gerbershagen
fba016b093 fix mingw build 2025-04-13 16:54:19 +02:00
Marius Gerbershagen
deefc21c9d deftype: implement &environment parameters
The deftype expansion functions now take two parameters, the type
argument and an environment. More precisely, for an atomic type 'x the
type argument for the expansion function is given by '(x) while for a
non-atomic type '(x y z) it is given by '(x y z). This also fixes the
value of &whole parameters in deftype lambda lists. The new behaviour
is consistent with SBCL and CCL.

Fixes #750
2025-04-08 21:57:53 +02:00
Marius Gerbershagen
72c83ac9b5 cmp: fix bug for calls to local functions with more arguments than c-arguments-limits
Closes #774.
2025-04-05 22:14:17 +02:00
Marius Gerbershagen
7b546e7fd2 Merge branch 'first-env' into 'develop'
refactor boot order

See merge request embeddable-common-lisp/ecl!341
2025-04-05 16:50:13 +00:00
Daniel Kochmański
868c3e4d31 core: don't use gc allocator to allocate a runtime environment
When we don't use mprotect (nor guard page), we allocate the memory manually.
This simplifies some code and makes the booting process less intervened with GC.
2025-03-31 20:44:46 +02:00
Daniel Kochmański
72af2d38b0 core: move the first environment initialization before symbols
Also split init_alloc in two passes like init_unixint, so we don't invoke
directly GC_enable. The first pass leaves GC explicitly disabled.
2025-03-31 20:44:46 +02:00
Daniel Kochmański
0f4e9b1e17 core: factor out process managament from thread managament
This is a preliminary step towards coroutines and for switching global env in
single-threaded builds.
2025-03-31 20:44:46 +02:00
Daniel Kochmański
f49cc7f3fd core: the first environment is statically allocated
This is to avoid circularity between the first environment (and interrupts) and
the garbage collector.
2025-03-31 20:44:46 +02:00
Daniel Kochmański
09ebfb47dd core: add wrapper for manual memory managament
We will use this operators to allocate the first environment and stacks so that
there is no circularity between ecl_boot and starting the garbage collector.
2025-03-31 20:44:46 +02:00
Daniel Kochmański
8f8ffc1772 rename the operator ecl_make_stack -> ecl_make_vector 2025-03-31 20:42:17 +02:00
Daniel Kochmański
d3ae7bd532 Merge branch 'bytevm-flat-closures' into 'develop'
Implement flat closures for bytecodes virtual machine

See merge request embeddable-common-lisp/ecl!338
2025-03-31 18:37:31 +00:00
Daniel Kochmański
63a106b4cb bytevm: add a new low-level operator ecl_make_stack
The stack is represented as an actually adjustable vector with a fill
pointer. The main difference from other vector constructors is that it does not
modify the process env -- most notably VALUES vector -- and can be safely used
in the interpreter.
2025-03-27 22:45:34 +01:00
Daniel Kochmański
eb6a64def9 lexenv: remove unused function from the compiler 2025-03-27 22:45:34 +01:00
Daniel Kochmański
75dcfaf705 lexenv: don't bypass referencing the cl_object instance
This is a preliminary step towards consistent access between LCL and LEX envs.
2025-03-27 22:45:34 +01:00
Daniel Kochmański
bd5e72def6 lexenv: don't use adjustable vector -- lexenv size is known 2025-03-27 22:45:34 +01:00
Daniel Kochmański
99b37c8ab5 boot: initialize macfun of ECL_T and ECL_NIL_SYMBOL to ECL_NIL
Prevoiusly this slot was left uninitialized and that lead to a segmentation
fault under certain conditions when we did (copy-symbol t t).
2025-03-27 22:45:34 +01:00
Daniel Kochmański
479400f6fb disassembler: add newly added opcodes to bytecodes disassembler 2025-03-27 22:45:34 +01:00
Daniel Kochmański
90e7a8e560 bytevm: change error names to be more descriptive
VEbad_lambda_arg_excd -> VEbad_lambda_too_many_args
VEbad_lambda_unk_keyw -> VEbad_lambda_unknown_keyword
2025-03-27 22:45:34 +01:00
Daniel Kochmański
77fe30efdd bytecmp: ensure that we close around referenced local macros
This is necessary to recompile bytecodes when they contain macro expansions.
2025-03-27 22:45:34 +01:00
Daniel Kochmański
c9c5323578 tests: add a test for an important corner case
In case that someone wants to store the definition when compiling the file, we
need to make sure that the compiler does not error if it has unreadable objects.
2025-03-27 22:45:34 +01:00
Daniel Kochmański
f56b2b9192 tests: add tests for local macros closing over each other 2025-03-27 22:45:34 +01:00