<feed xmlns='http://www.w3.org/2005/Atom'>
<title>sciteco/src/parser.h, branch tail-calls</title>
<subtitle>Scintilla-based Text Editor and COrrector</subtitle>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/'/>
<entry>
<title>functions that can tail-call must be tail-called as well</title>
<updated>2026-08-23T19:51:48+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-08-23T19:51:48+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=3f3db5422397f1bf8d5af1ade256638d89b1081c'/>
<id>3f3db5422397f1bf8d5af1ade256638d89b1081c</id>
<content type='text'>
Otherwise we could "leak" stack frames, which causes crashes on platforms with small stack sizes.
This still did not result in a reproducable speedup with GCC 16.

fixup 2e097cec409182c3cb39b74489387480bf8a278f
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Otherwise we could "leak" stack frames, which causes crashes on platforms with small stack sizes.
This still did not result in a reproducable speedup with GCC 16.

fixup 2e097cec409182c3cb39b74489387480bf8a278f
</pre>
</div>
</content>
</entry>
<entry>
<title>optimize main state machine transitions with tail calls</title>
<updated>2026-08-22T20:27:33+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-08-22T20:27:33+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=2e097cec409182c3cb39b74489387480bf8a278f'/>
<id>2e097cec409182c3cb39b74489387480bf8a278f</id>
<content type='text'>
* On newer GCC (&gt;= 15) and Clang (&gt;= 13) versions we can tail call
  at the end of input_cb() implementations to the next state's input_cb(),
  which will be optimized to jumps (often direct jumps).
  That is, the compilers always optimized tail calls, but we can
  guarantee tail calls with the __attribute__((musttail)) statement
  attribute.
* Every `return &amp;teco_state_xxx` has to be replaced with
  `TECO_RETURN(ctx, &amp;teco_state_xxx, error)`.
* This speeds up `-O2 -flto` builds by 18%
  (e.g. tested on grosciteco for sciteco(7)).
  Part of the speed up could also be because of inlining through
  TECO_RETURN().
* The other state machines (stringbuilding and q-reg spec) cannot
  currently be optimized the same way since they get their characters
  passed in from the "main" state machine.
* All loops around callbacks could be optimized the same way.
  E.g. the undo token runner could also tail call into the next runner,
  but it's probably not important to optimize undo token executions.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* On newer GCC (&gt;= 15) and Clang (&gt;= 13) versions we can tail call
  at the end of input_cb() implementations to the next state's input_cb(),
  which will be optimized to jumps (often direct jumps).
  That is, the compilers always optimized tail calls, but we can
  guarantee tail calls with the __attribute__((musttail)) statement
  attribute.
* Every `return &amp;teco_state_xxx` has to be replaced with
  `TECO_RETURN(ctx, &amp;teco_state_xxx, error)`.
* This speeds up `-O2 -flto` builds by 18%
  (e.g. tested on grosciteco for sciteco(7)).
  Part of the speed up could also be because of inlining through
  TECO_RETURN().
* The other state machines (stringbuilding and q-reg spec) cannot
  currently be optimized the same way since they get their characters
  passed in from the "main" state machine.
* All loops around callbacks could be optimized the same way.
  E.g. the undo token runner could also tail call into the next runner,
  but it's probably not important to optimize undo token executions.
</pre>
</div>
</content>
</entry>
<entry>
<title>updated copyright to 2026</title>
<updated>2026-01-01T06:59:49+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-01-01T06:59:49+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=c2feb2a6f71fc9adb20226fb3c2260c236e974e0'/>
<id>c2feb2a6f71fc9adb20226fb3c2260c236e974e0</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>teco_string_t is now passed by value like a scalar if the callee isn't expected to modify it</title>
<updated>2025-12-28T19:57:31+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2025-12-28T15:23:22+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=ea0a23645f03a42252ab1ce8df45ae4076ebae75'/>
<id>ea0a23645f03a42252ab1ce8df45ae4076ebae75</id>
<content type='text'>
* When passing a struct that should not be modified, I usually use a const pointer.
* Strings however are small 2-word objects and they are often now already passed via separate
  `gchar*` and gsize parameters. So it is consistent to pass teco_string_t by value as well.
  A teco_string_t will usually fit into registers just like a pointer.
* It's now obvious which function just _uses_ and which function _modifies_ a string.
  There is also no chance to pass a NULL pointer to those functions.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* When passing a struct that should not be modified, I usually use a const pointer.
* Strings however are small 2-word objects and they are often now already passed via separate
  `gchar*` and gsize parameters. So it is consistent to pass teco_string_t by value as well.
  A teco_string_t will usually fit into registers just like a pointer.
* It's now obvious which function just _uses_ and which function _modifies_ a string.
  There is also no chance to pass a NULL pointer to those functions.
</pre>
</div>
</content>
</entry>
<entry>
<title>TECO_DEFINE_STATE() no longer constructs callback names for mandatory callbacks, but tries to use static assertions</title>
<updated>2025-12-26T17:10:42+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2025-12-26T17:10:42+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=c2114fa0af73b42bc1ef302f7511ef87690cc0b1'/>
<id>c2114fa0af73b42bc1ef302f7511ef87690cc0b1</id>
<content type='text'>
* Requiring state callbacks by generating their names (e.g. NAME##_input) has several disadvantages:
  * The callback is not explicitly referenced when the state is defined.
    So an unintroduced reader will see some static function, which is nowhere referenced and still
    doesn't cause "unused" warnings.
  * You cannot choose the name of function that implements the callback freely.
  * In "substates" you need to generate a callback function if you want to provide a default.
    You also need to provide dummy wrapper functions whenever you want to reuse some existing
    function as the implementation.
* Instead, we are now using static assertions to check whether certain callbacks have been
  implemented.
  Unfortunately, this does not work on all compilers. In particular GCC won't consider
  references to state objects fully constant (even though they are) and does not allow
  them in _Static_assert (G_STATIC_ASSERT). This could only be made to work in newer GCC
  with -std=c2x or -std=gnu23 in combination with constexpr.
  It does work on Clang, though.
  So I introduced TECO_ASSERT_SAFE() which also passes if the expression is *not* constant.
  These static assertions are not crucial - they do not check anything that can differ between
  systems. So we can always rely on the checks performed by FreeBSD CI for instance.
  Also, you will of course quickly notice missing callbacks at runtime - with and without
  additional runtime assertions.
* All mandatory callbacks must still be explicitly initialized in the TECO_DEFINE_STATE calls.
* After getting rid of generated callback implementations, the TECO_DEFINE_STATE macros
  can finally be qualified with `static`.
* The TECO_DECLARE_STATE() macro has been removed. It no longer abstracts anything
  and cannot be used to declare static teco_state_t anyway.
  Also TECO_DEFINE_UNDO_CALL() also doesn't have a DECLARE counterpart.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Requiring state callbacks by generating their names (e.g. NAME##_input) has several disadvantages:
  * The callback is not explicitly referenced when the state is defined.
    So an unintroduced reader will see some static function, which is nowhere referenced and still
    doesn't cause "unused" warnings.
  * You cannot choose the name of function that implements the callback freely.
  * In "substates" you need to generate a callback function if you want to provide a default.
    You also need to provide dummy wrapper functions whenever you want to reuse some existing
    function as the implementation.
* Instead, we are now using static assertions to check whether certain callbacks have been
  implemented.
  Unfortunately, this does not work on all compilers. In particular GCC won't consider
  references to state objects fully constant (even though they are) and does not allow
  them in _Static_assert (G_STATIC_ASSERT). This could only be made to work in newer GCC
  with -std=c2x or -std=gnu23 in combination with constexpr.
  It does work on Clang, though.
  So I introduced TECO_ASSERT_SAFE() which also passes if the expression is *not* constant.
  These static assertions are not crucial - they do not check anything that can differ between
  systems. So we can always rely on the checks performed by FreeBSD CI for instance.
  Also, you will of course quickly notice missing callbacks at runtime - with and without
  additional runtime assertions.
* All mandatory callbacks must still be explicitly initialized in the TECO_DEFINE_STATE calls.
* After getting rid of generated callback implementations, the TECO_DEFINE_STATE macros
  can finally be qualified with `static`.
* The TECO_DECLARE_STATE() macro has been removed. It no longer abstracts anything
  and cannot be used to declare static teco_state_t anyway.
  Also TECO_DEFINE_UNDO_CALL() also doesn't have a DECLARE counterpart.
</pre>
</div>
</content>
</entry>
<entry>
<title>throw an error immediately after nEB if n != 0</title>
<updated>2025-10-06T21:58:11+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2025-10-06T21:58:11+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=a62323baa454b4a6ac698067043c8d4cf50c6561'/>
<id>a62323baa454b4a6ac698067043c8d4cf50c6561</id>
<content type='text'>
* When typing nEBfilename$ (n != 0) you would find out that the construct is invalid
  only after typing out the entire command.
  We now throw an error immediately, ie. only Escape or string termination will be expected
  in interactive mode.
* In batch mode, nothing should have changed.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* When typing nEBfilename$ (n != 0) you would find out that the construct is invalid
  only after typing out the entire command.
  We now throw an error immediately, ie. only Escape or string termination will be expected
  in interactive mode.
* In batch mode, nothing should have changed.
</pre>
</div>
</content>
</entry>
<entry>
<title>fixed serious bug with certain alternative string termination chars in commands with multiple string arguments</title>
<updated>2025-08-02T10:16:16+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-08-02T10:16:16+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=e46352bc614cf9777ca76deb47330fb408bc1a23'/>
<id>e46352bc614cf9777ca76deb47330fb408bc1a23</id>
<content type='text'>
* When `@`-modifying a command with several string arguments and choosing `{` as the alternative
  string termination character, the parser would get totally confused.
  Any sequence of `{` would be ignored and only the first non-`{` would become the termination character.
  Consequently you also couldn't choose a new terminator after the closing `}`.
  So even a documented code example from sciteco(7) wouldn't work.
  The same was true when using $ (escape) or ^A as the alternative termination character.
* We can now correctly parse e.g. `@FR{foo}{bar}` or `@FR$foo$bar$` (even though the
  latter one is quite pointless).
* has probably been broken forever (has been broken even before v2.0).
* Whitespace is now ignored in front of alternative termination characters as in TECO-64, so
  we can also write `@S /foo/` or even
  ```
  @^Um
  {
    !* blabla *!
  }
  ```
  I wanted to disallow whitespace termination characters, so the alternative would have been
  to throw an error.
  The new implementation at least adds some functionality.
  * Avoid redundancies when parsing no-op characters via teco_is_noop().
    I assume that this is inlined and drawn into any jump-table what would be
    generated for the switch-statement in teco_state_start_input().
 * Alternative termination characters are still case-folded, even if they are Unicode glyphs,
   so `@IЖfooж` would work and insert `foo`.
   This should perhaps be restricted to ANSI characters?
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* When `@`-modifying a command with several string arguments and choosing `{` as the alternative
  string termination character, the parser would get totally confused.
  Any sequence of `{` would be ignored and only the first non-`{` would become the termination character.
  Consequently you also couldn't choose a new terminator after the closing `}`.
  So even a documented code example from sciteco(7) wouldn't work.
  The same was true when using $ (escape) or ^A as the alternative termination character.
* We can now correctly parse e.g. `@FR{foo}{bar}` or `@FR$foo$bar$` (even though the
  latter one is quite pointless).
* has probably been broken forever (has been broken even before v2.0).
* Whitespace is now ignored in front of alternative termination characters as in TECO-64, so
  we can also write `@S /foo/` or even
  ```
  @^Um
  {
    !* blabla *!
  }
  ```
  I wanted to disallow whitespace termination characters, so the alternative would have been
  to throw an error.
  The new implementation at least adds some functionality.
  * Avoid redundancies when parsing no-op characters via teco_is_noop().
    I assume that this is inlined and drawn into any jump-table what would be
    generated for the switch-statement in teco_state_start_input().
 * Alternative termination characters are still case-folded, even if they are Unicode glyphs,
   so `@IЖfooж` would work and insert `foo`.
   This should perhaps be restricted to ANSI characters?
</pre>
</div>
</content>
</entry>
<entry>
<title>fixed minor memory leaks of per-state data in teco_machine_main_t</title>
<updated>2025-07-17T21:34:56+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-07-17T21:34:56+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=3a2583e918bcc805fe860252f8a520fc2f9b26ce'/>
<id>3a2583e918bcc805fe860252f8a520fc2f9b26ce</id>
<content type='text'>
* These were leaked e.g. in case of end-of-macro errors,
  but also in case of syntax highlighting (teco_lexer_style()).
  I considered to solve this by overwriting more of the end_of_macro_cb,
  but it didn't turn out to be trivial always.
* Considering that the union in teco_machine_main_t saved only 3 machine words
  of memory, I decided to sacrifice those for more robust memory management.
* teco_machine_qregspec_t cannot be directly embedded into teco_machine_main_t
  due to recursive dependencies with teco_machine_stringbuilding_t.
  It could now and should perhaps be allocated only once in teco_machine_main_init(),
  but it would require more refactoring.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* These were leaked e.g. in case of end-of-macro errors,
  but also in case of syntax highlighting (teco_lexer_style()).
  I considered to solve this by overwriting more of the end_of_macro_cb,
  but it didn't turn out to be trivial always.
* Considering that the union in teco_machine_main_t saved only 3 machine words
  of memory, I decided to sacrifice those for more robust memory management.
* teco_machine_qregspec_t cannot be directly embedded into teco_machine_main_t
  due to recursive dependencies with teco_machine_stringbuilding_t.
  It could now and should perhaps be allocated only once in teco_machine_main_init(),
  but it would require more refactoring.
</pre>
</div>
</content>
</entry>
<entry>
<title>implemented ^E&lt;code&gt; string building constructs for embedding bytes and codepoints in a strtoul()-like manner</title>
<updated>2025-07-03T14:04:01+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-07-03T13:21:37+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=7bc7662f3cd1ceaf55e00f3d5f84e9772574afc8'/>
<id>7bc7662f3cd1ceaf55e00f3d5f84e9772574afc8</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>new string building construct ^P disables all further string building magic</title>
<updated>2025-05-23T22:31:22+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-05-23T22:31:22+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=4b266c9616f4eb359be71c44b9b2fa3373265bb0'/>
<id>4b266c9616f4eb359be71c44b9b2fa3373265bb0</id>
<content type='text'>
* Now, `I^P` can replace `EI`.
  EI is therefore now free to be repurposed as the new "mung file" command for improved TECO-11 compatibility.
* On the downside when inserting large blocks of TECO code, you will have to write something like
  `@I{^P !...! }`
* The construct is also useful when searching for carets as in `S^P^Q^`.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Now, `I^P` can replace `EI`.
  EI is therefore now free to be repurposed as the new "mung file" command for improved TECO-11 compatibility.
* On the downside when inserting large blocks of TECO code, you will have to write something like
  `@I{^P !...! }`
* The construct is also useful when searching for carets as in `S^P^Q^`.
</pre>
</div>
</content>
</entry>
</feed>
