<feed xmlns='http://www.w3.org/2005/Atom'>
<title>sciteco/src/doc.h, branch lsp</title>
<subtitle>Scintilla-based Text Editor and COrrector</subtitle>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/'/>
<entry>
<title>revised and improved the Unicode glyph-to-byte conversion heuristics</title>
<updated>2026-07-24T23:55:28+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-07-24T23:55:28+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=685507922b0b75da5935076395a5b1ec1ef58356'/>
<id>685507922b0b75da5935076395a5b1ec1ef58356</id>
<content type='text'>
Previously almost all glyph-to-byte offset conversions consulted
Scintilla's line index and counted characters on the resulting line.
For instance a simple expression like `.+1J` would scan the same line
twice completely, which would be very slow on pathologically long lines.
Even insertions did that due to having to update the ^Y ranges.
If you repeat such an operation over all characters as in `&lt;.+1:J;&gt;`
you would have complexity O(n^2) for n = line length.
Only commands with an explicit relative nature like `C` and `A` would
use teco_view_glyph2bytes_relative() which scans beginning at dot
as long as the relative movement is less than 1024 glyphs.

Wit the new heuristics almost all glyph-to-byte and byte-to-glyph
conversions can make use of that optimization.
This requires that dot must at all times be known in glyphs as well -
the byte position is managed by Scintilla (SCI_GETCURRENTPOS).
We therefore introduced teco_current_doc_set_dot() and
teco_current_doc_get_dot() to update dot in the current buffer or
Q-Register -- it cannot be stored along with the view since
Q-Registers share a single view.
A number of auxiliary functions have been introduced for
converting relative to a known (glyphs,bytes) offset pair
and for converting absolute and relative positions with regard
to the current doc and SCI_GETCURRENTPOS position.
Of course this is error-prone since the glyph and dot positions
are interdependant - they must always be kept in sync.

With these new optimizations even pathologically long lines can
(usually) be managed even in UTF-8 documents.
It does not address slow-downs in Scintilla's line layout, yet.
grosciteco.tes for instance runs twice as fast now.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Previously almost all glyph-to-byte offset conversions consulted
Scintilla's line index and counted characters on the resulting line.
For instance a simple expression like `.+1J` would scan the same line
twice completely, which would be very slow on pathologically long lines.
Even insertions did that due to having to update the ^Y ranges.
If you repeat such an operation over all characters as in `&lt;.+1:J;&gt;`
you would have complexity O(n^2) for n = line length.
Only commands with an explicit relative nature like `C` and `A` would
use teco_view_glyph2bytes_relative() which scans beginning at dot
as long as the relative movement is less than 1024 glyphs.

Wit the new heuristics almost all glyph-to-byte and byte-to-glyph
conversions can make use of that optimization.
This requires that dot must at all times be known in glyphs as well -
the byte position is managed by Scintilla (SCI_GETCURRENTPOS).
We therefore introduced teco_current_doc_set_dot() and
teco_current_doc_get_dot() to update dot in the current buffer or
Q-Register -- it cannot be stored along with the view since
Q-Registers share a single view.
A number of auxiliary functions have been introduced for
converting relative to a known (glyphs,bytes) offset pair
and for converting absolute and relative positions with regard
to the current doc and SCI_GETCURRENTPOS position.
Of course this is error-prone since the glyph and dot positions
are interdependant - they must always be kept in sync.

With these new optimizations even pathologically long lines can
(usually) be managed even in UTF-8 documents.
It does not address slow-downs in Scintilla's line layout, yet.
grosciteco.tes for instance runs twice as fast now.
</pre>
</div>
</content>
</entry>
<entry>
<title>updated copyright to 2026</title>
<updated>2026-01-01T06:59:49+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-01-01T06:59:49+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=c2feb2a6f71fc9adb20226fb3c2260c236e974e0'/>
<id>c2feb2a6f71fc9adb20226fb3c2260c236e974e0</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>fixed teco_doc_undo_exchange(): use reference counting now</title>
<updated>2025-05-03T08:33:01+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-05-03T08:33:01+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=850000492cd43e40109b3a11c8613b5fff577755'/>
<id>850000492cd43e40109b3a11c8613b5fff577755</id>
<content type='text'>
* When popping from the Q-Register stack to local Q-Registers in macro calls,
  problems would arise when rubbing out that macro call since we would eventually restore
  a pointer that has been permanently freed along with the local Q-Registers.
* Naturally this could result in everything from Valgrind warnings to crashes.
* Added test case.
* There is still a test case that fails when run under `--valgrind`,
  but it's apparently due to a glib-internal memory leak.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* When popping from the Q-Register stack to local Q-Registers in macro calls,
  problems would arise when rubbing out that macro call since we would eventually restore
  a pointer that has been permanently freed along with the local Q-Registers.
* Naturally this could result in everything from Valgrind warnings to crashes.
* Added test case.
* There is still a test case that fails when run under `--valgrind`,
  but it's apparently due to a glib-internal memory leak.
</pre>
</div>
</content>
</entry>
<entry>
<title>updated copyright to 2025</title>
<updated>2025-01-12T23:39:34+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-01-12T23:39:34+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=d842eaee19e2723f845d4b8314a230cf68e82653'/>
<id>d842eaee19e2723f845d4b8314a230cf68e82653</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>added raw ANSI mode to facilitate 8-bit clean editing (refs #5)</title>
<updated>2024-09-09T16:22:21+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-09-09T14:54:26+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=4f231871a0208ec9bcc2679fce25d3b9795d1597'/>
<id>4f231871a0208ec9bcc2679fce25d3b9795d1597</id>
<content type='text'>
* When enabled with bit 2 in the ED flags (0,4ED),
  all registers and buffers will get the raw ANSI encoding (as if 0EE had been
  called on them).
  You can still manually change the encoding, eg. by calling 65001EE afterwards.
* Also the ANSI mode sets up character representations for all bytes &gt;= 0x80.
  This is currently done only depending on the ED flag, not when setting 0EE.
* Since setting 16,4ED for 8-bit clean editing in a macro can be tricky -
  the default unnamed buffer will still be at UTF-8 and at least a bunch
  of environment registers as well - we added the command line option
  `--8bit` (short `-8`) which configures the ED flags very early on.
  As another advantage you can mung the profile in 8-bit mode as well
  when using SciTECO as a sort of interactive hex editor.
* Disable UTF-8 checks in 8-bit clean mode (sample.teco_ini).
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* When enabled with bit 2 in the ED flags (0,4ED),
  all registers and buffers will get the raw ANSI encoding (as if 0EE had been
  called on them).
  You can still manually change the encoding, eg. by calling 65001EE afterwards.
* Also the ANSI mode sets up character representations for all bytes &gt;= 0x80.
  This is currently done only depending on the ED flag, not when setting 0EE.
* Since setting 16,4ED for 8-bit clean editing in a macro can be tricky -
  the default unnamed buffer will still be at UTF-8 and at least a bunch
  of environment registers as well - we added the command line option
  `--8bit` (short `-8`) which configures the ED flags very early on.
  As another advantage you can mung the profile in 8-bit mode as well
  when using SciTECO as a sort of interactive hex editor.
* Disable UTF-8 checks in 8-bit clean mode (sample.teco_ini).
</pre>
</div>
</content>
</entry>
<entry>
<title>Xq and ]q inherit the document encoding from the source document (refs #5)</title>
<updated>2024-09-09T16:22:21+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-09-08T22:03:33+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=41ab5cf0289dab60ac1ddc97cf9680ee2468ea6c'/>
<id>41ab5cf0289dab60ac1ddc97cf9680ee2468ea6c</id>
<content type='text'>
* ^Uq however always sets an UTF8 register as the source
  is supposed to be a SciTECO macro which is always UTF-8.
* :^Uq preserves the register's encoding
* teco_doc_set_string() now also sets the encoding
* instead of trying to restore the encoding in teco_doc_undo_set_string(),
  we now swap out the document in a teco_doc_t and pass it to an undo token.
* The get_codepage() Q-Reg method has been removed as the same
  can now be done with teco_doc_get_string() and the get_string() method.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* ^Uq however always sets an UTF8 register as the source
  is supposed to be a SciTECO macro which is always UTF-8.
* :^Uq preserves the register's encoding
* teco_doc_set_string() now also sets the encoding
* instead of trying to restore the encoding in teco_doc_undo_set_string(),
  we now swap out the document in a teco_doc_t and pass it to an undo token.
* The get_codepage() Q-Reg method has been removed as the same
  can now be done with teco_doc_get_string() and the get_string() method.
</pre>
</div>
</content>
</entry>
<entry>
<title>Unicode support for the Q-Register commands (refs #5)</title>
<updated>2024-09-09T16:22:21+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-08-30T02:15:36+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=7507ad3e1816f3bc9004dceb39bb303804287438'/>
<id>7507ad3e1816f3bc9004dceb39bb303804287438</id>
<content type='text'>
* this required adding several Q-Register vtable methods
* it should still be investigated whether the repeated calling of
  SCI_ALLOCATELINECHARACTERINDEX causes any overhead.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* this required adding several Q-Register vtable methods
* it should still be investigated whether the repeated calling of
  SCI_ALLOCATELINECHARACTERINDEX causes any overhead.
</pre>
</div>
</content>
</entry>
<entry>
<title>updated copyright to 2024</title>
<updated>2024-01-21T11:45:05+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-01-21T11:07:29+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=1cecf04656532e94e1fe9fe25460774324b2197c'/>
<id>1cecf04656532e94e1fe9fe25460774324b2197c</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>updated copyright to 2023</title>
<updated>2023-04-05T15:11:32+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2023-04-05T15:11:32+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=0f7424e9b041646d782fb4c8b2f81a2f74856731'/>
<id>0f7424e9b041646d782fb4c8b2f81a2f74856731</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>updated copyright to 2022 and updated TODO</title>
<updated>2022-06-21T01:41:16+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2022-06-21T01:41:16+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=70619bb88918e9cd057dbbc6a87e890cbce49a08'/>
<id>70619bb88918e9cd057dbbc6a87e890cbce49a08</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
</feed>
