* doc/emacs/mule.texi (International): Refer to X Coding as
well.
(Communication Coding): Document that locale-coding-system is
not always used on X to decode keyboard input.
(X Coding): New node.
* etc/NEWS: Announce change to input method coding resolution.
* lisp/term/x-win.el (x-get-input-coding-system): New function.
* src/coding.c (syms_of_coding): Update doc string of
locale-coding-system.
* src/xfns.c (struct x_xim_text_conversion_data)
(x_xim_text_to_utf8_unix_1, x_xim_text_to_utf8_unix_2)
(x_xim_text_to_utf8_unix): Accept dpyinfo. Use the coding
system specified inside if possible.
(xic_preedit_draw_callback): Pass dpyinfo.
* src/xterm.c (handle_one_xevent): Use XIM coding system for IM
input.
(xim_open_dpy): Try to determine the input method coding system.
(mark_xterm): Mark `xim_coding'.
(syms_of_xterm): New variable `x-input-coding-system'.
* src/xterm.h (struct x_display_info): New field `xim_coding'.
(FRAME_X_XIM_CODING): New macro.
* src/coding.c (encode_coding_object): Mark saved_pt_byte with UNINIT
to silence a false alarm from -Wmaybe-uninitialized. This was already
done in decode_coding_object. The fix is correct because saved_pt is
used as a sentinel before saved_pt_byte is used (bug#57219).
* src/coding.c (detect_coding): Always initialize all
components of detect_info, so that detect_coding_utf_8 etc.
do not have undefined behavior when they read detect_info.checked.
This bug is not likely to cause problems on real systems.
Problem found by GCC 12 -fanalyzer.
(detect_coding_system): Use consistent style with detect_coding
initialization.
* src/coding.c (Fdecode_coding_region): Doc fix.
* doc/lispref/nonascii.texi (Coding System Basics)
(Explicit Encoding): Explain the significance of using 'undecided'
in 'decode-coding-*' functions.
* src/fileio.c (expand-file-name): Check for null bytes for both
NAME and DEFAULT-DIRECTORY arguments. Also check for null bytes
in buffer-local default-directory, assuming it is used.
* src/coding.c (encode_file_name): Use CHECK_STRING_NULL_BYTES.
* src/lisp.h (CHECK_STRING_NULL_BYTES): Add function for checking
for null bytes in Lisp strings.
* test/src/fileio-tests.el (fileio-test--expand-file-name-null-bytes):
Add test for new changes to expand-file-name.
* etc/NEWS: Announce changes.
* src/coding.c (encode_coding_object): Don't assume that
src_object == dst_object means src_object is the current buffer.
Add the missing commentary that explains the arguments.
(Bug#49127)
* src/coding.c (string_ascii_p): Make it non-static.
* src/fileio.c (Fdirectory_append): Fix check for whether we need
to convert to multibyte.
* src/fns.c (string_ascii_p): Remove copy.
* src/lisp.h: Declare string_ascii_p.
Such filenames are dangerous, as Emacs would silently only use the
part up to the first NUL byte. Reject them explicitly instead.
* src/coding.c (encode_file_name_1): New helper function.
(encode_file_name): Check that encoded filename doesn't contain a
NUL byte.
(syms_of_coding): Define 'filenamep' symbol.
* test/src/fileio-tests.el (fileio-tests/null-character): New unit
test.
* etc/NEWS: Document change.
Give get-buffer-create an optional argument to inhibit buffer hooks
in internal or temporary buffers for efficiency (bug#34765).
* etc/NEWS: Announce new parameter of get-buffer-create and
generate-new-buffer, and that with-temp-buffer and with-temp-file
now inhibit buffer hooks.
* doc/lispref/buffers.texi (Buffer Names): Fix typo.
(Creating Buffers): Document new parameter of get-buffer-create and
generate-new-buffer.
(Buffer List, Killing Buffers): Document when buffer hooks are
inhibited.
(Current Buffer):
* doc/lispref/files.texi (Writing to Files): Document that
with-temp-buffer and with-temp-file inhibit buffer hooks.
* doc/lispref/internals.texi (Buffer Internals): Document
inhibit_buffer_hooks flag. Remove stale comment.
* doc/misc/gnus-faq.texi (FAQ 5-8):
* lisp/simple.el (shell-command-on-region): Fix indentation.
* lisp/files.el (kill-buffer-hook): Document when hook is inhibited.
(create-file-buffer):
* lisp/gnus/gnus-uu.el (gnus-uu-unshar-article):
* lisp/international/mule.el (load-with-code-conversion):
* lisp/mh-e/mh-xface.el (mh-x-image-url-fetch-image):
* lisp/net/imap.el (imap-open):
* lisp/net/mailcap.el (mailcap-maybe-eval):
* lisp/progmodes/flymake-proc.el
(flymake-proc--read-file-to-temp-buffer)
(flymake-proc--copy-buffer-to-temp-buffer): Simplify.
* lisp/subr.el (generate-new-buffer): Forward new optional argument
to inhibit buffer hooks to get-buffer-create.
(with-temp-file, with-temp-buffer, with-output-to-string):
* lisp/json.el (json-encode-string): Inhibit buffer hooks in buffer
used.
* src/buffer.c (run_buffer_list_update_hook): New helper function.
(Fget_buffer_create): Use it. Add optional argument to set
inhibit_buffer_hooks flag instead of comparing the buffer name to
Vcode_conversion_workbuf_name. All callers changed.
(Fmake_indirect_buffer, Frename_buffer, Fbury_buffer_internal)
(record_buffer): Use run_buffer_list_update_hook.
(Fkill_buffer): Document when buffer hooks are inhibited. Use
run_buffer_list_update_hook.
(init_buffer_once): Inhibit buffer hooks in Vprin1_to_string_buffer.
(Vkill_buffer_query_functions, Vbuffer_list_update_hook): Document
when hooks are inhibited.
* src/buffer.h (struct buffer): Update inhibit_buffer_hooks
commentary.
* src/coding.h (Vcode_conversion_workbuf_name):
* src/coding.c (Vcode_conversion_workbuf_name): Make static again
since it is no longer needed in src/buffer.c.
(code_conversion_restore, code_conversion_save, syms_of_coding):
Prefer boolean over integer constants.
* src/fileio.c (Finsert_file_contents): Inhibit buffer hooks in
" *code-converting-work*" buffer.
* src/window.c (Fselect_window): Fix grammar. Mention
window-selection-change-functions alongside buffer-list-update-hook.
* test/src/buffer-tests.el: Fix requires.
(buffer-tests-inhibit-buffer-hooks): New test.
This fixes assertion violations when the mnemonic is
given as a string, and allows non-ASCII characters be
used as mode-line mnemonic of a coding-system.
* src/xdisp.c (decode_mode_spec_coding): Handle multibyte
characters as coding-system's mnemonic.
(display_mode_element): If decode_mode_spec returns a multibyte
string, display it as multibyte.
* src/coding.c (Fdefine_coding_system_internal)
(Fcoding_system_put): If :mnemonic is a string, use its first
character. This avoids assertion violations if someone uses a
string as the mnemonic of a coding-system.
Avoid problems if GC occurs while initializing a vector.
Problem with Fdelete reported by Pip Cet in:
https://lists.gnu.org/r/emacs-devel/2020-08/msg00313.html
I looked for similar problems elsewhere and found quite a few.
* src/coding.c (make_subsidiaries):
* src/composite.c (syms_of_composite):
* src/font.c (build_style_table, Ffont_get_glyphs):
* src/nsselect.m (clean_local_selection_data):
* src/nsxwidget.m (js_to_lisp):
* src/syntax.c (init_syntax_once):
* src/window.c (Fcurrent_window_configuration):
* src/xselect.c (selection_data_to_lisp_data)
(clean_local_selection_data):
Use make_nil_vector instead of make_uninit_vector.
* src/fns.c (Fdelete):
* src/xwidget.c (webkit_js_to_lisp):
Use allocate_nil_vector instead of allocate_vector.
* src/search.c (Fnewline_cache_check):
Use make_vector instead of make_uninit_vector.
* lisp/international/mule.el (define-coding-system):
Require :pre-write-conversion and :post-read-conversion functions
to leave the match data untouched.
* src/coding.c (Fdecode_coding_string, Fencode_coding_string):
Document functions as match-data-preserving.
Suggested by Stefan Monnier (see bug#41445).
* src/character.h (MULTIBYTE_LENGTH, MULTIBYTE_LENGTH_NO_CHECK):
Remove, replacing with ...
(multibyte_length): ... this new function. All callers changed.
The new function rejects overlong multibyte forms.
* test/src/buffer-tests.el (buffer-multibyte-overlong-sequences):
New test.
* src/buffer.h (fetch_char_advance, fetch_char_advance_no_check)
(buf_next_char_len, next_char_len, buf_prev_char_len)
(prev_char_len, inc_both, dec_both): New inline functions,
replacing the old character.h macros FETCH_CHAR_ADVANCE,
FETCH_CHAR_ADVANCE_NO_CHECK, BUF_INC_POS, INC_POS, BUF_DEC_POS,
DEC_POS, INC_BOTH, DEC_BOTH respectively. All callers changed.
These new functions all assume buffer primitives and so need
to be here rather than in character.h.
* src/casefiddle.c (make_char_unibyte): New static function,
replacing the old MAKE_CHAR_UNIBYTE macro. All callers changed.
(do_casify_unibyte_string): Use SINGLE_BYTE_CHAR_P instead
of open-coding it.
* src/ccl.c (GET_TRANSLATION_TABLE): New static function,
replacing the old macro of the same name.
* src/character.c (string_char): Omit 2nd arg. 3rd arg can no
longer be NULL. All callers changed.
* src/character.h (SINGLE_BYTE_CHAR_P): Move up.
(MAKE_CHAR_UNIBYTE, MAKE_CHAR_MULTIBYTE, PREV_CHAR_BOUNDARY)
(STRING_CHAR_AND_LENGTH, STRING_CHAR_ADVANCE)
(FETCH_STRING_CHAR_ADVANCE)
(FETCH_STRING_CHAR_AS_MULTIBYTE_ADVANCE)
(FETCH_STRING_CHAR_ADVANCE_NO_CHECK, FETCH_CHAR_ADVANCE)
(FETCH_CHAR_ADVANCE_NO_CHECK, INC_POS, DEC_POS, INC_BOTH)
(DEC_BOTH, BUF_INC_POS, BUF_DEC_POS): Remove.
(make_char_multibyte): New static function, replacing
the old macro MAKE_CHAR_MULTIBYTE. All callers changed.
(CHAR_STRING_ADVANCE): Remove; all callers changed to use
CHAR_STRING.
(NEXT_CHAR_BOUNDARY): Remove; it was unused.
(raw_prev_char_len): New inline function, replacing the
old PREV_CHAR_BOUNDARY macro. All callers changed.
(string_char_and_length): New inline function, replacing the
old STRING_CHAR_AND_LENGTH macro. All callers changed.
(STRING_CHAR): Rewrite in terms of string_char_and_length.
(string_char_advance): New inline function, replacing the old
STRING_CHAR_ADVANCE macro. All callers changed.
(fetch_string_char_advance): New inline function, replacing the
old FETCH_STRING_CHAR_ADVANCE macro. All callers changed.
(fetch_string_char_as_multibyte_advance): New inline function,
replacing the old FETCH_STRING_CHAR_AS_MULTIBYTE_ADVANCE macro.
All callers changed.
(fetch_string_char_advance_no_check): New inline function,
replacing the old FETCH_STRING_CHAR_ADVANCE_NO_CHECK macro. All
callers changed.
* src/regex-emacs.c (HEAD_ADDR_VSTRING): Remove; no longer used.
* src/syntax.c (scan_lists): Use dec_bytepos instead of
open-coding it.
* src/xdisp.c (string_char_and_length): Rename from
string_char_and_length to avoid name conflict with new function in
character.h. All callers changed.
They already did return their argument under some circumstances;
this change broadens it to further reduce allocation in common cases
(bug#40407).
* src/coding.c (convert_string_nocopy): New function.
(decode_file_name, encode_file_name): Use convert_string_nocopy.
* src/coding.h (ENCODE_FILE, DECODE_FILE): Note the nocopy semantics.
Spotted by Kazuhiro Ito.
* src/coding.c (code_convert_string):
Set Vlast_coding_system if appropriate.
* test/src/coding-tests.el (coding-nocopy-ascii): Add test.
To sidestep conversion altogether when EOL conversion applies, we must
either be encoding a string without NL, or decoding without CR.
* src/coding.c (string_ascii_p): Revert to a pure predicate.
(code_convert_string): Fix logic. Don't use uninitialised
ascii_p (removed). Use memchr to detect CR or LF in string when needed.
* test/src/coding-tests.el (coding-nocopy-ascii):
Update tests to include encodings with explicit EOL conversions.
* src/coding.c (Fcheck_coding_systems_region): Don't crash if
the third arg contains something that isn't a coding system.
* test/src/coding-tests.el (coding-check-coding-systems-region):
New test.
* src/coding.c (string_ascii_p): Return a negative value if an
all-ASCII string STR includes the CR character, otherwise a
positive value.
(code_convert_string): If the string is ASCII, but includes CR
characters, use the fast path only if EOL doesn't need to be
decoded. (Bug#40519)
* test/src/coding-tests.el (coding-nocopy-ascii): Add more tests
for bug#40519.
* src/coding.c (string_ascii_p): Return a negative value if an
all-ASCII string STR includes the CR character, otherwise a
positive value.
(code_convert_string): If the string is ASCII, but includes CR
characters, use the fast path only if EOL doesn't need to be
decoded. (Bug#40519)
* test/src/coding-tests.el (coding-nocopy-ascii): Add tests for
bug#40519.
Optimise for the common case of encoding or decoding an ASCII-only
string using an ASCII-compatible coding, for file names in particular.
* src/coding.c (string_ascii_p): New function.
(code_convert_string): Return the input string for ASCII-only inputs
and ASCII-compatible codings.
* test/src/coding-tests.el (coding-nocopy-ascii): New test.
In {encode,decode}-coding-string, the NOCOPY parameter had the
opposite effect to what was intended and documented. This 18 year old
bug (introduced in 4031e2bf0a) only affected calls with CODING-SYSTEM
being nil.
* src/coding.c (code_convert_string): Correct use of NOCOPY.
* test/src/coding-tests.el (coding-nocopy-trivial): New test.
* src/bignum.c (check_integer_range, check_uinteger_max)
(check_int_nonnegative): New functions.
* src/frame.c (check_frame_pixels): New function.
(Fset_frame_height, Fset_frame_width, Fset_frame_size): Use it.
* src/lisp.h (CHECK_RANGED_INTEGER, CHECK_TYPE_RANGED_INTEGER):
Remove these macros. Unless otherwise specified, all callers
replaced by calls to check_integer_range, check_uinteger_range,
check_int_nonnegative.
* src/frame.c (gui_set_right_divider_width)
(gui_set_bottom_divider_width):
* src/nsfns.m (ns_set_internal_border_width):
* src/xfns.c (x_set_internal_border_width):
Using check_int_nonnegative means these functions no longer
incorrectly reject negative bignums; they treat them as 0,
just like negative fixnums.
If a position argument to get-byte etc. is an out-of-range integer,
treat it the same regardless of whether it is a fixnum or a bignum.
* src/buffer.c (fix_position): New function.
* src/buffer.c (validate_region):
* src/character.c (Fget_byte):
* src/coding.c (Ffind_coding_systems_region_internal)
(Fcheck_coding_systems_region):
* src/composite.c (Ffind_composition_internal):
* src/editfns.c (Fposition_bytes, Fchar_after, Fchar_before)
(Finsert_buffer_substring, Fcompare_buffer_substrings)
(Fnarrow_to_region):
* src/fns.c (Fsecure_hash_algorithms):
* src/font.c (Finternal_char_font, Ffont_at):
* src/fringe.c (Ffringe_bitmaps_at_pos):
* src/search.c (search_command):
* src/textprop.c (get_char_property_and_overlay):
* src/window.c (Fpos_visible_in_window_p):
* src/xdisp.c (Fwindow_text_pixel_size):
Use it instead of CHECK_FIXNUM_COERCE_MARKER, so that
the code is simpler and treats bignums consistently with fixnums.
* src/buffer.h (CHECK_FIXNUM_COERCE_MARKER): Define here
rather than in lisp.h, and reimplement in terms of fix_position
so that it treats bignums consistently with fixnums.
* src/lisp.h (CHECK_FIXNUM_COERCE_MARKER): Move to buffer.h.
* src/textprop.c (validate_interval_range): Signal with original
bounds rather than modified ones.
Now that decode_string_utf_8 is available, we can use it to signal
errors on invalid input.
* src/coding.c (syms_of_coding): Move Qutf_8_string_p from json.c
since it’s now used outside json.c.
* src/emacs-module.c (module_decode_utf_8): New helper function.
(module_make_function, module_copy_string_contents): Use it.
* src/coding.c (Fencode_coding_region): Clarify what this does.
(Fdecode_coding_region): Ditto.
* src/fns.c (Fbase64_decode_region): Clarify that this function
returns bytes, not text (bug#38587).
(Fbase64_encode_region): Clarify that this function takes bytes,
not text.
* src/coding.c (get_char_bytes, encode_string_utf_8)
(decode_string_utf_8): Fix commentary.
(encode_string_utf_8): Return the original ASCII string only
if NOCOPY is non-zero.
(decode_string_utf_8): Accept 2 additional arguments STR and
STR_LEN, which allow to pass the input text as a C string.
(make_string_from_utf8): Delegate the job to decode_string_utf_8.
* src/coding.h: Update the prototype of decode_string_utf_8.
* src/json.c (json_encode): Call encode_string_utf_8.
This mostly changes http: to https: in URLs. It also updates
some URLs that have moved, removes some URLs that no longer
work, recommends against using procmail (procmail.org no
longer works), and removes some mentions of the
no-longer-existing Gmane, LPF and VTW.
It doesn't update all URLs, just the ones I had time for.
* GNUmakefile (help):
* admin/admin.el (manual-doctype-string):
* admin/charsets/Makefile.in (${charsetdir}/ALTERNATIVNYJ.map):
* admin/charsets/mapconv:
* lisp/net/soap-client.el (soap-create-envelope):
* lisp/org/org.el (org-doi-server-url):
* lisp/textmodes/bibtex.el (bibtex-generate-url-list):
Prefer https: to http: un URLs.
This partially reverts the changes in "extern function cleanup".
* src/coding.c (encode_string_utf_8, decode_string_utf_8): Now
extern again. They should NOT be static, as they are intended
to be used by the likes of json.c, where we need
highly-optimized code for processing UTF-8 strings. E.g.,
decode_string_utf_8 beats make_string_from_utf8 by a factor of
2 to 5 in a large number of scenarios.
Most of these functions can be static. A few are unused.
* src/coding.c (encode_string_utf_8, decode_string_utf_8):
Define only if ENABLE_UTF_8_CONVERTER_TEST, as they're
not needed otherwise.
* src/coding.c (encode_string_utf_8, decode_string_utf_8):
* src/data.c (integer_mod):
* src/fns.c (base64_encode_region_1, base64_encode_string_1):
* src/ftfont.c (ftfont_get_fc_charset):
Now static.
* src/sysdep.c (verrprintf): Remove; unused.
* src/inotify.c (syms_of_inotify) <Qignored>: Don't define here...
* src/coding.c (syms_of_coding) <Qignored>: ...define it here,
because it is needed also on platforms that don't compile
inotify.c.
