<feed xmlns='http://www.w3.org/2005/Atom'>
<title>sciteco/src/search.c, branch lsp</title>
<subtitle>Scintilla-based Text Editor and COrrector</subtitle>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/'/>
<entry>
<title>don't crash on `0S...$`, but always throw an error</title>
<updated>2026-08-07T19:23:59+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-08-07T18:47:22+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=8a8edaa2daf37f7c12e21b9199755b5de8e489f5'/>
<id>8a8edaa2daf37f7c12e21b9199755b5de8e489f5</id>
<content type='text'>
* This has been broken for some time, but it didn't crash in v2.5.2.
* Even when it did not crash, the search just failed.
  In TECO-11 (judging by TECOC), we must throw an error, though.
  Video TECO on the other hand always succeeds in such cases.
* Added test case
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* This has been broken for some time, but it didn't crash in v2.5.2.
* Even when it did not crash, the search just failed.
  In TECO-11 (judging by TECOC), we must throw an error, though.
  Video TECO on the other hand always succeeds in such cases.
* Added test case
</pre>
</div>
</content>
</entry>
<entry>
<title>fixed dot after an unsuccessful interactive search</title>
<updated>2026-08-04T22:05:44+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-08-04T21:58:13+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=df87077309d06207b234b1231128c205ec11a155'/>
<id>df87077309d06207b234b1231128c205ec11a155</id>
<content type='text'>
If you had a partial success during interactive searching, dot would be
left at that position since the failure handling only reset via SCI_GOTOPOS.
This is now fixed by restoring the initial dot in glyphs as well.
All existing occurrences of "dot" have been renamed to "pos" for consistency.
(Nowadays, "dot" should be reserved for glyph positions since this is what
`.` returns. Especially if both kinds of positions are used in the same code.)

A test case has been added.

This has been broken since 685507922b0b75da5935076395a5b1ec1ef58356.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
If you had a partial success during interactive searching, dot would be
left at that position since the failure handling only reset via SCI_GOTOPOS.
This is now fixed by restoring the initial dot in glyphs as well.
All existing occurrences of "dot" have been renamed to "pos" for consistency.
(Nowadays, "dot" should be reserved for glyph positions since this is what
`.` returns. Especially if both kinds of positions are used in the same code.)

A test case has been added.

This has been broken since 685507922b0b75da5935076395a5b1ec1ef58356.
</pre>
</div>
</content>
</entry>
<entry>
<title>get rid of teco_qreg_vtable_t::undo_set_integer()</title>
<updated>2026-07-25T00:37:17+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-07-25T00:37:17+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=a53fa85dabdfc2c47ec0c9e6626ddda2379a8e38'/>
<id>a53fa85dabdfc2c47ec0c9e6626ddda2379a8e38</id>
<content type='text'>
It was never required as a separate callback/method,
but was kept merely for consistency.
Since we now call teco_current_doc_set_dot() in
teco_qreg_dot_set_integer(), we'd have to split it up into
an "undo" method as well. I decided to get rid of the
superfluous Q-Reg method instead.

It's quite likely we could get rid of the remaining
undo_set_string(), undo_exchange_string() and undo_edit()
callbacks as well (TODO).
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
It was never required as a separate callback/method,
but was kept merely for consistency.
Since we now call teco_current_doc_set_dot() in
teco_qreg_dot_set_integer(), we'd have to split it up into
an "undo" method as well. I decided to get rid of the
superfluous Q-Reg method instead.

It's quite likely we could get rid of the remaining
undo_set_string(), undo_exchange_string() and undo_edit()
callbacks as well (TODO).
</pre>
</div>
</content>
</entry>
<entry>
<title>revised and improved the Unicode glyph-to-byte conversion heuristics</title>
<updated>2026-07-24T23:55:28+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-07-24T23:55:28+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=685507922b0b75da5935076395a5b1ec1ef58356'/>
<id>685507922b0b75da5935076395a5b1ec1ef58356</id>
<content type='text'>
Previously almost all glyph-to-byte offset conversions consulted
Scintilla's line index and counted characters on the resulting line.
For instance a simple expression like `.+1J` would scan the same line
twice completely, which would be very slow on pathologically long lines.
Even insertions did that due to having to update the ^Y ranges.
If you repeat such an operation over all characters as in `&lt;.+1:J;&gt;`
you would have complexity O(n^2) for n = line length.
Only commands with an explicit relative nature like `C` and `A` would
use teco_view_glyph2bytes_relative() which scans beginning at dot
as long as the relative movement is less than 1024 glyphs.

Wit the new heuristics almost all glyph-to-byte and byte-to-glyph
conversions can make use of that optimization.
This requires that dot must at all times be known in glyphs as well -
the byte position is managed by Scintilla (SCI_GETCURRENTPOS).
We therefore introduced teco_current_doc_set_dot() and
teco_current_doc_get_dot() to update dot in the current buffer or
Q-Register -- it cannot be stored along with the view since
Q-Registers share a single view.
A number of auxiliary functions have been introduced for
converting relative to a known (glyphs,bytes) offset pair
and for converting absolute and relative positions with regard
to the current doc and SCI_GETCURRENTPOS position.
Of course this is error-prone since the glyph and dot positions
are interdependant - they must always be kept in sync.

With these new optimizations even pathologically long lines can
(usually) be managed even in UTF-8 documents.
It does not address slow-downs in Scintilla's line layout, yet.
grosciteco.tes for instance runs twice as fast now.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Previously almost all glyph-to-byte offset conversions consulted
Scintilla's line index and counted characters on the resulting line.
For instance a simple expression like `.+1J` would scan the same line
twice completely, which would be very slow on pathologically long lines.
Even insertions did that due to having to update the ^Y ranges.
If you repeat such an operation over all characters as in `&lt;.+1:J;&gt;`
you would have complexity O(n^2) for n = line length.
Only commands with an explicit relative nature like `C` and `A` would
use teco_view_glyph2bytes_relative() which scans beginning at dot
as long as the relative movement is less than 1024 glyphs.

Wit the new heuristics almost all glyph-to-byte and byte-to-glyph
conversions can make use of that optimization.
This requires that dot must at all times be known in glyphs as well -
the byte position is managed by Scintilla (SCI_GETCURRENTPOS).
We therefore introduced teco_current_doc_set_dot() and
teco_current_doc_get_dot() to update dot in the current buffer or
Q-Register -- it cannot be stored along with the view since
Q-Registers share a single view.
A number of auxiliary functions have been introduced for
converting relative to a known (glyphs,bytes) offset pair
and for converting absolute and relative positions with regard
to the current doc and SCI_GETCURRENTPOS position.
Of course this is error-prone since the glyph and dot positions
are interdependant - they must always be kept in sync.

With these new optimizations even pathologically long lines can
(usually) be managed even in UTF-8 documents.
It does not address slow-downs in Scintilla's line layout, yet.
grosciteco.tes for instance runs twice as fast now.
</pre>
</div>
</content>
</entry>
<entry>
<title>regular expression matching can now be reliably interrupted using CTRL+C</title>
<updated>2026-06-29T17:20:55+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-29T17:20:55+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=49587788101dce76111a8b16dfed0889c26b0e61'/>
<id>49587788101dce76111a8b16dfed0889c26b0e61</id>
<content type='text'>
* The previous checks for interruptions only helped in a few corner cases
  like for very high search-repeat counts or during backwards searches across
  the entire buffer.
* But even with terex' more predictable runtime properties
  a single regex execution can hang quite a long time.
  E.g. `S^EM^X$` on a huge buffer or even more so with backreferences as in
  `S^~(.*)\1$`.
* We now use the new tere_set_is_interrupted_cb() to register
  teco_interface_is_interrupted(). Types should be compatible as long
  as gboolean resolves to int.
* It's no longer necessary to manually check for teco_interface_is_interrupted()
  since tere_exec() now returns REG_EINTR in case the callback returned TRUE
  in which case it's handled by teco_error_regex_set().
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* The previous checks for interruptions only helped in a few corner cases
  like for very high search-repeat counts or during backwards searches across
  the entire buffer.
* But even with terex' more predictable runtime properties
  a single regex execution can hang quite a long time.
  E.g. `S^EM^X$` on a huge buffer or even more so with backreferences as in
  `S^~(.*)\1$`.
* We now use the new tere_set_is_interrupted_cb() to register
  teco_interface_is_interrupted(). Types should be compatible as long
  as gboolean resolves to int.
* It's no longer necessary to manually check for teco_interface_is_interrupted()
  since tere_exec() now returns REG_EINTR in case the callback returned TRUE
  in which case it's handled by teco_error_regex_set().
</pre>
</div>
</content>
</entry>
<entry>
<title>fixed searching for `)`</title>
<updated>2026-06-29T15:31:25+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-29T15:31:25+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=8601ccfb905a5cc6fe10f7d81a491155f9236f35'/>
<id>8601ccfb905a5cc6fe10f7d81a491155f9236f35</id>
<content type='text'>
It for some strange reason had to be escaped for AREs
even though a single freestanding `)` cannot mean anything.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
It for some strange reason had to be escaped for AREs
even though a single freestanding `)` cannot mean anything.
</pre>
</div>
</content>
</entry>
<entry>
<title>implemented the ^~ pattern match construct: the rest of the pattern will be an Advanced Regular Expression</title>
<updated>2026-06-28T22:32:13+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-28T22:15:46+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=3522966d9584ec16e2f469acd0fe8727857a9d25'/>
<id>3522966d9584ec16e2f469acd0fe8727857a9d25</id>
<content type='text'>
* Allows searching by regular expressions.
  We will never support all ARE constructs in TECO patterns, so this is useful to have available.
* Can only be typed upcaret.
  This leaves ^E~q available as an escape-regexp string building construct.
* Once we replace the pattern2regexp converter with a custom terex lexer,
  we might want to restrict ^~ to the beginning of the pattern.
  Currently, however it can be anywhere, so you can mix TECO patterns with regular expressions.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Allows searching by regular expressions.
  We will never support all ARE constructs in TECO patterns, so this is useful to have available.
* Can only be typed upcaret.
  This leaves ^E~q available as an escape-regexp string building construct.
* Once we replace the pattern2regexp converter with a custom terex lexer,
  we might want to restrict ^~ to the beginning of the pattern.
  Currently, however it can be anywhere, so you can mix TECO patterns with regular expressions.
</pre>
</div>
</content>
</entry>
<entry>
<title>fixed block-wise backwards searches</title>
<updated>2026-06-28T15:30:15+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-28T15:30:15+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=f08dea5fead2f9ef9e0fa114b2e09aa94908d629'/>
<id>f08dea5fead2f9ef9e0fa114b2e09aa94908d629</id>
<content type='text'>
The calculation of the block start was faulty and could cause underflows
resulting in unpredictable behavior.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The calculation of the block start was faulty and could cause underflows
resulting in unpredictable behavior.
</pre>
</div>
</content>
</entry>
<entry>
<title>fixup: fixed searching for `|` - it must also be escaped when converted to a regular expression</title>
<updated>2026-06-28T15:10:55+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-28T15:10:55+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=baf4c173aafc0f225d5eda1f4f4db3d92db02f0e'/>
<id>baf4c173aafc0f225d5eda1f4f4db3d92db02f0e</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>fixed ^EGq (character class) pattern construct for embedded null bytes and `-`</title>
<updated>2026-06-28T11:44:41+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-28T11:44:41+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=7bd7bdad687e5f790afda6f0f22444f3a169a6b1'/>
<id>7bd7bdad687e5f790afda6f0f22444f3a169a6b1</id>
<content type='text'>
This was using g_regex_escape_string() which always translates a null byte
to `\0`, which is ambiguous if followed by other digits, so a null byte followed
by a digit would result in a wrong regular expression.
Actually the same could happen outside of character classes, ie. `@S/^@1/` was also broken.
Also it does not escape `-`, so the result cannot be used in character classes.
This is fixed now in a new custom implementation teco_regex_escape().
Once moving to a custom terex lexer, we won't need any of this of course
unless we want to provide a regex escaping string building construct.

We are now completely free of GRegex.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This was using g_regex_escape_string() which always translates a null byte
to `\0`, which is ambiguous if followed by other digits, so a null byte followed
by a digit would result in a wrong regular expression.
Actually the same could happen outside of character classes, ie. `@S/^@1/` was also broken.
Also it does not escape `-`, so the result cannot be used in character classes.
This is fixed now in a new custom implementation teco_regex_escape().
Once moving to a custom terex lexer, we won't need any of this of course
unless we want to provide a regex escaping string building construct.

We are now completely free of GRegex.
</pre>
</div>
</content>
</entry>
</feed>
