<feed xmlns='http://www.w3.org/2005/Atom'>
<title>sciteco/src, branch master</title>
<subtitle>Scintilla-based Text Editor and COrrector</subtitle>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/'/>
<entry>
<title>terex is built with assertions now on --enable-debug</title>
<updated>2026-07-05T10:11:23+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-07-05T10:11:23+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=01d77dafb8fc4d40f8603e0d776b9b4b9283918d'/>
<id>01d77dafb8fc4d40f8603e0d776b9b4b9283918d</id>
<content type='text'>
* terex disables assertions by default unless you add `-DREG_DEBUG`.
  Since we heavily modified the original by Henry Spencer it makes sense
  to enable assertions.
* dlmalloc will still be built without assertions even if --enable-debug
  is given since that has a significant speed impact and I consider
  dlmalloc to be rock solid. It would need `-DDEBUG=1` to enable assertions
  (among other things).
  We only disable additional checks in dlmalloc if --disable-debug.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* terex disables assertions by default unless you add `-DREG_DEBUG`.
  Since we heavily modified the original by Henry Spencer it makes sense
  to enable assertions.
* dlmalloc will still be built without assertions even if --enable-debug
  is given since that has a significant speed impact and I consider
  dlmalloc to be rock solid. It would need `-DDEBUG=1` to enable assertions
  (among other things).
  We only disable additional checks in dlmalloc if --disable-debug.
</pre>
</div>
</content>
</entry>
<entry>
<title>FreeBSD: fixed crashes with `EC` or `EQq`</title>
<updated>2026-07-03T22:59:05+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-07-03T22:59:05+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=ecd1b0d916bbbe002af9624f152c062a891e4633'/>
<id>ecd1b0d916bbbe002af9624f152c062a891e4633</id>
<content type='text'>
* GSpawn ends up calling posix_spawnp() which passes down a small
  4kb stack to the child process until it exec()s.
  This stack could be overflowed easily on code paths where the
  path is not already absolute and when many shared libraries
  are involved.
* The crashes could therefore only be observed on Gtk builds and
  in UNIX shell emulation mode (0,128ED). Sample test case:
  gsciteco -e '0,128ED @EC"ls"'
  Theoretically a relative $SHELL variable could have also triggered
  it.
* I assume that the bug will be fixed in libc at least by the time of
  FreeBSD 16.
* As a workaround we resolve relative program paths before passing
  them to g_spawn_async_with_pipes().
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* GSpawn ends up calling posix_spawnp() which passes down a small
  4kb stack to the child process until it exec()s.
  This stack could be overflowed easily on code paths where the
  path is not already absolute and when many shared libraries
  are involved.
* The crashes could therefore only be observed on Gtk builds and
  in UNIX shell emulation mode (0,128ED). Sample test case:
  gsciteco -e '0,128ED @EC"ls"'
  Theoretically a relative $SHELL variable could have also triggered
  it.
* I assume that the bug will be fixed in libc at least by the time of
  FreeBSD 16.
* As a workaround we resolve relative program paths before passing
  them to g_spawn_async_with_pipes().
</pre>
</div>
</content>
</entry>
<entry>
<title>regular expression matching can now be reliably interrupted using CTRL+C</title>
<updated>2026-06-29T17:20:55+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-29T17:20:55+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=49587788101dce76111a8b16dfed0889c26b0e61'/>
<id>49587788101dce76111a8b16dfed0889c26b0e61</id>
<content type='text'>
* The previous checks for interruptions only helped in a few corner cases
  like for very high search-repeat counts or during backwards searches across
  the entire buffer.
* But even with terex' more predictable runtime properties
  a single regex execution can hang quite a long time.
  E.g. `S^EM^X$` on a huge buffer or even more so with backreferences as in
  `S^~(.*)\1$`.
* We now use the new tere_set_is_interrupted_cb() to register
  teco_interface_is_interrupted(). Types should be compatible as long
  as gboolean resolves to int.
* It's no longer necessary to manually check for teco_interface_is_interrupted()
  since tere_exec() now returns REG_EINTR in case the callback returned TRUE
  in which case it's handled by teco_error_regex_set().
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* The previous checks for interruptions only helped in a few corner cases
  like for very high search-repeat counts or during backwards searches across
  the entire buffer.
* But even with terex' more predictable runtime properties
  a single regex execution can hang quite a long time.
  E.g. `S^EM^X$` on a huge buffer or even more so with backreferences as in
  `S^~(.*)\1$`.
* We now use the new tere_set_is_interrupted_cb() to register
  teco_interface_is_interrupted(). Types should be compatible as long
  as gboolean resolves to int.
* It's no longer necessary to manually check for teco_interface_is_interrupted()
  since tere_exec() now returns REG_EINTR in case the callback returned TRUE
  in which case it's handled by teco_error_regex_set().
</pre>
</div>
</content>
</entry>
<entry>
<title>fixed searching for `)`</title>
<updated>2026-06-29T15:31:25+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-29T15:31:25+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=8601ccfb905a5cc6fe10f7d81a491155f9236f35'/>
<id>8601ccfb905a5cc6fe10f7d81a491155f9236f35</id>
<content type='text'>
It for some strange reason had to be escaped for AREs
even though a single freestanding `)` cannot mean anything.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
It for some strange reason had to be escaped for AREs
even though a single freestanding `)` cannot mean anything.
</pre>
</div>
</content>
</entry>
<entry>
<title>implemented the ^~ pattern match construct: the rest of the pattern will be an Advanced Regular Expression</title>
<updated>2026-06-28T22:32:13+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-28T22:15:46+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=3522966d9584ec16e2f469acd0fe8727857a9d25'/>
<id>3522966d9584ec16e2f469acd0fe8727857a9d25</id>
<content type='text'>
* Allows searching by regular expressions.
  We will never support all ARE constructs in TECO patterns, so this is useful to have available.
* Can only be typed upcaret.
  This leaves ^E~q available as an escape-regexp string building construct.
* Once we replace the pattern2regexp converter with a custom terex lexer,
  we might want to restrict ^~ to the beginning of the pattern.
  Currently, however it can be anywhere, so you can mix TECO patterns with regular expressions.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Allows searching by regular expressions.
  We will never support all ARE constructs in TECO patterns, so this is useful to have available.
* Can only be typed upcaret.
  This leaves ^E~q available as an escape-regexp string building construct.
* Once we replace the pattern2regexp converter with a custom terex lexer,
  we might want to restrict ^~ to the beginning of the pattern.
  Currently, however it can be anywhere, so you can mix TECO patterns with regular expressions.
</pre>
</div>
</content>
</entry>
<entry>
<title>fixed block-wise backwards searches</title>
<updated>2026-06-28T15:30:15+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-28T15:30:15+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=f08dea5fead2f9ef9e0fa114b2e09aa94908d629'/>
<id>f08dea5fead2f9ef9e0fa114b2e09aa94908d629</id>
<content type='text'>
The calculation of the block start was faulty and could cause underflows
resulting in unpredictable behavior.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The calculation of the block start was faulty and could cause underflows
resulting in unpredictable behavior.
</pre>
</div>
</content>
</entry>
<entry>
<title>fixup: fixed searching for `|` - it must also be escaped when converted to a regular expression</title>
<updated>2026-06-28T15:10:55+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-28T15:10:55+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=baf4c173aafc0f225d5eda1f4f4db3d92db02f0e'/>
<id>baf4c173aafc0f225d5eda1f4f4db3d92db02f0e</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>fixed ^EGq (character class) pattern construct for embedded null bytes and `-`</title>
<updated>2026-06-28T11:44:41+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-28T11:44:41+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=7bd7bdad687e5f790afda6f0f22444f3a169a6b1'/>
<id>7bd7bdad687e5f790afda6f0f22444f3a169a6b1</id>
<content type='text'>
This was using g_regex_escape_string() which always translates a null byte
to `\0`, which is ambiguous if followed by other digits, so a null byte followed
by a digit would result in a wrong regular expression.
Actually the same could happen outside of character classes, ie. `@S/^@1/` was also broken.
Also it does not escape `-`, so the result cannot be used in character classes.
This is fixed now in a new custom implementation teco_regex_escape().
Once moving to a custom terex lexer, we won't need any of this of course
unless we want to provide a regex escaping string building construct.

We are now completely free of GRegex.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This was using g_regex_escape_string() which always translates a null byte
to `\0`, which is ambiguous if followed by other digits, so a null byte followed
by a digit would result in a wrong regular expression.
Actually the same could happen outside of character classes, ie. `@S/^@1/` was also broken.
Also it does not escape `-`, so the result cannot be used in character classes.
This is fixed now in a new custom implementation teco_regex_escape().
Once moving to a custom terex lexer, we won't need any of this of course
unless we want to provide a regex escaping string building construct.

We are now completely free of GRegex.
</pre>
</div>
</content>
</entry>
<entry>
<title>teco_globber_t is ported from GRegex to terex</title>
<updated>2026-06-28T00:35:17+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-28T00:35:17+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=0dfb113b47d958093e6ae086c9695e5be83b24b8'/>
<id>0dfb113b47d958093e6ae086c9695e5be83b24b8</id>
<content type='text'>
It's now a private struct, so we can include the regex_t wihout
having to draw in the terex headers everywhere.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
It's now a private struct, so we can include the regex_t wihout
having to draw in the terex headers everywhere.
</pre>
</div>
</content>
</entry>
<entry>
<title>terex is the new regular expression engine now and replaces PCRE (GRegex)</title>
<updated>2026-06-27T22:39:51+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-27T22:39:51+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=4fe5bc6f3867096965270c90f2e1e5df77b8825f'/>
<id>4fe5bc6f3867096965270c90f2e1e5df77b8825f</id>
<content type='text'>
* terex is based on Henry Spencer's regular expression engine for Tcl.
  It is a hybrid NFA/DFA design which has better worst-time runtimes than
  the backtracking PCRE. Memory usage is also limited and can no longer
  increase catastrophically.
* It should no longer be possible to crash SciTECO with pathological
  searches.
* Since it reliably supports partial matches (REG_EXPECT) we can
  now enable the new backwards-search algorithm by default.
  This used to be broken because of a glib bug, which I already
  fixed. It would however take a long time until this ends up
  on the majority of glib installations.
* Regexp executions can still be quite slow if you are looking
  for a pattern at the end of a huge file, which can hang the editor,
  but this can now at least theoretically be solved by adding
  hooks into terex to poll for interruptions.
* We can now also get rid of a TECO-pattern to regexp translation
  step by directly generating terex tokens (TODO).
* Performance-wise terex appears to be slower than PCRE for simple
  forward searches even when linking everything with optimzations (FIXME).
* Having a stand-alone regular expression engine is also a huge
  step in getting rid of glib.

See also: https://git.fmsbw.de/terex/about/
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* terex is based on Henry Spencer's regular expression engine for Tcl.
  It is a hybrid NFA/DFA design which has better worst-time runtimes than
  the backtracking PCRE. Memory usage is also limited and can no longer
  increase catastrophically.
* It should no longer be possible to crash SciTECO with pathological
  searches.
* Since it reliably supports partial matches (REG_EXPECT) we can
  now enable the new backwards-search algorithm by default.
  This used to be broken because of a glib bug, which I already
  fixed. It would however take a long time until this ends up
  on the majority of glib installations.
* Regexp executions can still be quite slow if you are looking
  for a pattern at the end of a huge file, which can hang the editor,
  but this can now at least theoretically be solved by adding
  hooks into terex to poll for interruptions.
* We can now also get rid of a TECO-pattern to regexp translation
  step by directly generating terex tokens (TODO).
* Performance-wise terex appears to be slower than PCRE for simple
  forward searches even when linking everything with optimzations (FIXME).
* Having a stand-alone regular expression engine is also a huge
  step in getting rid of glib.

See also: https://git.fmsbw.de/terex/about/
</pre>
</div>
</content>
</entry>
</feed>
