Commit graph

9082 commits

Author SHA1 Message Date
Daniel Kochmański
ea11e2c433 streams: rename common sequence stream accessors
SEQ_{INPUT,OUTPUT}* -> SEQ_STREAM*

Don't use IO_STREAM_ELT_TYPE in sequences and define SEQ_STREAM_ELT_TYPE
instread to avoid ambiguity.

This is a cleanup that signifies similarities between both objects.
2025-08-11 10:01:40 +02:00
Daniel Kochmański
37c3955180 manual: add documentation for new binary stream interfaces 2025-08-11 10:01:38 +02:00
Daniel Kochmański
7ee0977a50 tests: add test for binary and bivalent streams with extensions
Tests reading, peeking and unreading both characters and bytes.
2025-08-11 10:01:37 +02:00
Daniel Kochmański
a8e57c60a5 streams: implement new interfaces for unreading and peeking bytes
ecl_file_ops has two new members:

  void (*unread_byte)(cl_object strm, cl_object byte);
  cl_object (*peek_byte)(cl_object strm);

C API additions:

  void ecl_unread_byte (cl_object byte, cl_object strm)
  cl_object ecl_peek_byte (cl_object strm)

  si_unread_byte(cl_object strm, cl_object byte)    [1]
  si_peek_byte(cl_object strm, cl_object byte)      [2]

Lisp API additions:

  (ext:unread-byte stream byte) :: integer          [1]
  (ext:peek-byte   stream byte) :: (or integer nil) [2]

  (gray:stream-unread-byte stream byte) :: null
  (gray:stream-peek-byte stream) :: (or integer :eof)

We implement a "generic" version of unread-byte by storing it in a new slot
last_byte.
2025-08-11 10:01:37 +02:00
Daniel Kochmański
a887d040a2 tests: add a new test suite "stream"
Currently it contains only a check for a recently fixed bug in:
streams: fix a braino in str_in_unread_char
2025-08-11 10:01:37 +02:00
Daniel Kochmański
a916a5ccff tests: make finishes return the values from the executed form 2025-08-11 10:01:37 +02:00
Daniel Kochmański
e98e36dfca streams: fix a braino in str_in_unread_char
We've tested a wrong variable so this function allowed us to plum into negative
indexes.
2025-08-11 10:01:37 +02:00
Daniel Kochmański
407fe456fe streams: make ecl_read_byte return OBJNULL on EOF
This is to allow for sequence streams to return arbitrary objects (when
appropriately constructed) without many changes.
2025-08-11 10:01:37 +02:00
Daniel Kochmański
431132e4d1 streams: ecl_file_ops cleanup and some minor fixes
1. ecl_peek_char had outdated comment presumbly from before we've introduced
   stream dispatch tables - that comment has been removed.

2. fix erroneous specializations

   - of STREAM-UNREAD-CHAR

   By mistake we had two methods specialized to ANSI-STREAM, while one were
   clearly meant to specialize to T (in order to call BUG-OR-ERROR).

   - of winsock winsock_stream_output_ops

     stream peek char was set to ecl_generic_peek_char instead of
     ecl_not_input_read_char

3. change struct ecl_file_ops definition

a) ecl_file_ops structure change order of some operations to always feature READ
   before WRITE (for consistency)

b) we are more precise in dispatch function declaration and specify the return
   type to be ecl_character where applicable
2025-08-11 10:01:37 +02:00
Daniel Kochmański
fad7073f10 ffi: convert-to-foreign-string: ensure a cstring
The function operates on base_string while if it was supplied with an extended
string then ecl_base_char array became ecl_character, and that lead to bad
copies. To fix it we ensure that the passes string is first coerced to cstring.
2025-08-11 09:16:00 +02:00
Daniel Kochmański
77ceb401f9 doc: man: stack sizes are specified in bytes not kilobytes
Fixes #788.
2025-08-10 22:29:16 +02:00
Daniel Kochmański
bebb43d558 INSTALL: update the version of te tested emsdk version to 4.0.12
That's the current latest and it works OK when we compile host and wasm target
according to the most up-to-date instructions.
2025-08-04 10:06:31 +02:00
Marius Gerbershagen
a7126313d9 Merge branch 'refactor-streams' into 'develop'
Split file.d into various stream implementations

See merge request embeddable-common-lisp/ecl!351
2025-07-29 14:35:55 +00:00
Daniel Kochmański
3f2ad992e6 Merge branch 'openbsd-file_cnt' into 'develop'
openbsd: implement FILE_CNT() on opaque FILE

See merge request embeddable-common-lisp/ecl!354
2025-07-29 11:23:21 +00:00
Daniel Kochmański
68a78e252f msvc: update makefile 2025-07-26 16:59:42 +02:00
Daniel Kochmański
6ce9c22dda stream: split file.d into different stream types
This commit splits one garguntulum file into numerous orthogonal stream types:

- strm_os -- c99/posix/windows streams
- strm_clos -- gray streams
- strm_string -- string streams
- strm_composite -- composite streams (echo, broadcast, synonym ...)
- strm_common -- common errors, byte manipulation routines etc
- strm_sequence -- si_write_sequence and si_read_sequence (fast I/O)
- strm_eformat -- external format processing routines (unicode etc)

After this split file.d contains only open/close. This will be the place to
dispatch to a virtual filesystem.
2025-07-26 16:59:42 +02:00
Daniel Kochmański
6fcb977052 stream: factor out general stream interface from file.d 2025-07-26 16:59:42 +02:00
Daniel Kochmański
4542318b80 stream: add missing lisp interfaces
The api for these is cleaned up compared to Common Lisp counterparts.
2025-07-26 16:59:42 +02:00
Daniel Kochmański
a5c922b849 stream: unread_error and unread_twice are now not continuable
At least in two from four cases continuing from the error lead to an error:

- for concatenated stream we've tried to dispatch on CAR that was NIL (segfault)
- for string stream we've decremented the position below 0

Also change these functions to defined macros ecl_unread_* that expand to
FEerror (in internal.h). This is in anticipation of splitting file.d.
2025-07-26 16:59:42 +02:00
Daniel Kochmański
b8e8899280 stream: change order of arguments in internal write_byte methods
That said this is inconsistent with how we implement ECL-WRITE-CHAR, and with
STREAM-WRITE-BYTE, so for convenience we swap the argument order.
2025-07-26 16:59:42 +02:00
Daniel Kochmański
c828e7b64d core: move atomics from threads directory to core
Atomics are needed by stacks.

Replace ecl_atomic_push -> ecl_atomic_psh that takes as an argument a
preallocated cons. ecl_atomic_push is replaced with a macro.
2025-07-26 16:59:42 +02:00
Daniel Kochmański
0dad009527 c/makefile: group files in rough categories 2025-07-26 16:59:42 +02:00
Daniel Kochmański
ca8eea81a1 Update gitlab-ci to run pipeline less frequently (2) 2025-07-26 16:59:24 +02:00
Daniel Kochmański
31a82c296c Update gitlab-ci to run pipeline less frequently
We've hit qutie fast 3/4 of the limit, so this pull request limits pipelines to
be run only when we commit to the branch develop and on merge requests.
2025-07-26 08:31:28 +02:00
Sebastien Marie
b427defc39 openbsd: implement FILE_CNT() on opaque FILE
On OpenBSD, FILE is opaque (starting from upcoming OpenBSD 7.8).
FILE_CNT() macro is implementable using `size_t __freadahead(FILE *stream)` function (provided for gnulib compat).
2025-07-23 08:51:20 +00:00
Daniel Kochmański
84e9b0b450 Merge branch 'cross-compilation-core' into 'develop'
Cross compilation improvements

Closes #741 and #747

See merge request embeddable-common-lisp/ecl!352
2025-07-21 08:04:44 +00:00
Daniel Kochmański
3e3da7dc27 gitlab-ci.yml: specify artifacts and make test jobs work
We don't fail when some tests don't pass -- we run them instead and the
submitter/reviewer now can check results before merging.
2025-07-21 10:02:08 +02:00
Marius Gerbershagen
b77a0bbf06 update compilation instructions for iOS and emscripten
We don't need a separate host compiler anymore.
2025-07-19 16:33:22 +02:00
Marius Gerbershagen
f0ef267ce7 update compilation instructions for android
With recent versions of the android NDK, there is no need to create a
separate toolchain anymore. Moreover, the LDFLAGS and CPPFLAGS are
obsolete. Also, we can't call the C compiler with a `-Dandroid`
argument as we used to since that conflicts with uses of the `android`
identifier within the header files of the android C standard library.
Therefore, we change from `thehost=android` to `thehost=ANDROID` in
uppercase. Finally, make the test script run more reliably by starting
an adb server before we change TMPDIR.
2025-07-19 16:33:22 +02:00
Marius Gerbershagen
6a3605b768 clos: builtin classes have the same index independent of configure options
This is needed to allow for cross compiling from a compiler with a
different set of configure options (e.g. compiling for a target which
doesn't support complex floats from a host which does).
2025-07-19 16:33:22 +02:00
Marius Gerbershagen
b194811ee3 src/util: add script to help in testing cross compilation 2025-07-19 16:33:22 +02:00
Marius Gerbershagen
d51ce511f6 handle *features* entirely in the configure script
Previously, there were a lot of places that turned on or off C macros
which finally determined whether some particular entry was found
in *features* or not. Now, we do this all in the configure script.
This is necessary for cross compilation from a non-compatible host
which may have different features.
2025-07-19 16:33:22 +02:00
Marius Gerbershagen
cd90ecfd55 cmp: signal internal error in wt-vv-value if we detect a programming error 2025-07-19 16:33:22 +02:00
Marius Gerbershagen
887115fe95 cmp: implement cross compilation from host compiler with different word size 2025-07-19 16:31:55 +02:00
Marius Gerbershagen
a1b1eee8b5 cmp: allow for storing type definitions in the compiler environment
Currently unused; will be used in the implementation of
cross-compilation.
2025-07-19 16:31:55 +02:00
Daniel Kochmański
c7b5b9f70e ecl_init_module: rebind *readtable* and *package* to protect the env
Fixes #785.
2025-07-17 16:02:18 +02:00
Marius Gerbershagen
af89c4a972 don't forget to initialize C stack limits if threads are disabled
This was broken after the recent refactor of the initialization code.
2025-07-06 14:40:21 +02:00
Marius Gerbershagen
9457813343 fix initialization of the main thread environment
The default sigmask was NULL previously (bug introduced in the
refactor in commit 0f4e9b1e1).
2025-07-05 22:07:28 +02:00
Marius Gerbershagen
a213b4eaa1 fix typo preventing non-threaded builds from working
Introduced in the refactor in commit
ecaa73155b
2025-06-28 20:08:16 +02:00
Marius Gerbershagen
b0f7711bd1 tests: fix process-environ
We were trying to check if the output of the program is empty but we
were passing `:output nil` to ext:run-program. Thus, the stream from
which we trying to read was nil which is the stream designator for the
standard input. The empty read succeeded only by chance if the
standard input had no available characters to read.
2025-06-28 19:45:50 +02:00
Daniel Kochmański
7f18d20ad3 Merge branch 'fix-78x' into develop 2025-06-26 14:31:37 +02:00
Daniel Kochmański
0857a438f6 logical pathnames: add a regression test and update the changelog
I've tested it against ansi-test and our own test suite and there are no
regressions.

Related to #784.
2025-06-26 14:31:07 +02:00
Daniel Kochmański
5818808f7c logical pathnames: fix logical pathname translation for multiple *
After this commit

    (setf (logical-pathname-translations "x")
          (list ("X:a;*;b;*;*.*" "/hello/*/hi/*/what/*.*")))
    (translate-logical-pathname #p"x:a;bonjour;b;barev;greetings.me")

will return a correct result

    #P"/hello/bonjour/hi/barev/what/greetings.me"

Related to #784.
2025-06-26 14:06:17 +02:00
Daniel Kochmański
03e61206ba logical pathnames: fix find_list_wilds to not produce self-refs
The function find_list_wilds incorrectly collected results from calling
find_wilds, and the latter returned incorrect results when the component fully
matched. It seems that this was a single function that was later split with
errors.

This change saves us from a segfault when we try to match multiple wild
components. Previously find_list_wilds for

(setf (logical-pathname-translations "x")
      `(("X:a;*;b;*;*.*" "/hello/*/hi/*/what/*.*")))
;; ((#P"X:A;*;B;*;*.*" #P"/hello/*/hi/*/what/*.*"))
(translate-logical-pathname #p"x:a;bonjour;b;barev;greetings.me")

returned #1=((BONJUR) #1# BAREV).

Moreover fix an issue where we've compared incorrect character in find_wilds to
determine the end of the glob (we've compared the source character with #\*
instead of the character after).

Fixes #784.
2025-06-26 13:28:42 +02:00
Marius Gerbershagen
ea5a10cbc1 copy-symbol: work around a bug in gcc
On x86_64, gcc 12.2.0 from debian 12 miscompiles the code we had
previously. It generates code that copies symbol.macfun and
symbol.sfdef using an aligned move SSE instruction which requires 16
byte alignment of its operand. This alignment is not present in
general, leading to a segfault for instance when calling (copy-symbol
t t). By reordering the assignments, we avoid triggering the bug.
2025-06-22 10:45:53 +02:00
Daniel Kochmański
d880e6f276 stacks: fix issues with ecl_make_stack
these functions are not used by us yet, so these issues were not noticed. Most
notably we did not assign the vector type and did not release the object.
2025-06-06 15:19:06 +02:00
Daniel Kochmański
9a894b7dab stacks: fix a type in an error message 2025-06-03 21:16:33 +02:00
Daniel Kochmański
0f24472237 bytevm: allow for bytecodes data segment to be NIL
We don't eagerly dereference bytecodes.data->vector.self in case when data is
assigned ECL_NIL. IN that case the *data vector is NULL. That saves us from the
necessity to allocate a vector when we don't use the data segment at all.
2025-06-02 12:05:50 +02:00
Marius Gerbershagen
a158731cc9 Merge branch 'bytecmp-fix-eval-with-env' into 'develop'
Fix a recent regression in bytecodes compiler

See merge request embeddable-common-lisp/ecl!350
2025-05-31 14:54:49 +00:00
Marius Gerbershagen
cb9b63a9a7 Merge branch 'refactor-stacks' into 'develop'
refactor stacks to reduce dependencies

See merge request embeddable-common-lisp/ecl!348
2025-05-31 14:39:34 +00:00