<feed xmlns='http://www.w3.org/2005/Atom'>
<title>sciteco/src/ring.h, branch lsp</title>
<subtitle>Scintilla-based Text Editor and COrrector</subtitle>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/'/>
<entry>
<title>support lookups via language servers (LSP)</title>
<updated>2026-08-10T22:38:31+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-08-02T13:24:45+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=44f5bf677282308b959411c416da2b5b08db2062'/>
<id>44f5bf677282308b959411c416da2b5b08db2062</id>
<content type='text'>
* The main interface is the `FT` command.
  `FT` was an undocumented Video TECO command for etags/ctags lookups.
  I don't want to exactly copy its interface, though.
* `FT` allows looking up symbol names,
  definitions and references.
* ctags will be supported via ctags-lsp.
  LSP support is more powerful though and works without
  regenerating TAGS files all the time.
  The LSP will also allow you to customize auto-completions
  using SciTECO itself (i.e. by writing a language server
  in SciTECO).
* For multiple results, `FT$` can be used to cycle through
  results - this should mimic repeated `S$` or `N$`.
* `:FT...$` does a fuzzy search. IMHO it's not important
  to return a status integer instead. `FT` will only
  really be used in interactive mode.
* Document synchronization is supported via hooks from
  ring.c and via Scintilla notifications.
* Currently, the LSP communication is based on blocking
  GIOChannels. This means that a misbehaving hanging
  server could "lock up" the entire editor (FIXME).
  Only on ncurses you can always kill the subprocess by
  pressing CTRL+C.
  We need helper functions in spawn.c to read and write
  with interruptions.
* The textDocument/didChange notification transmits
  not only all edits, but all files' contents as well
  during initial synchronization.
  Therefore it is optimized to write and JSON-escape
  data without copying them around in memory and without
  destroying the buffer gap.
* Use $SCITECO_LSP to configure the language server.
  You can also use `tee` to capture stdin and stdout.
  Perhaps $SCITECO_LSP should be saved in .teco_session,
  so you can change it between projects?
* $SCITECO_LSP_ROOT is used to point to the project's
  root directory. session.tes will set it up automatically.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* The main interface is the `FT` command.
  `FT` was an undocumented Video TECO command for etags/ctags lookups.
  I don't want to exactly copy its interface, though.
* `FT` allows looking up symbol names,
  definitions and references.
* ctags will be supported via ctags-lsp.
  LSP support is more powerful though and works without
  regenerating TAGS files all the time.
  The LSP will also allow you to customize auto-completions
  using SciTECO itself (i.e. by writing a language server
  in SciTECO).
* For multiple results, `FT$` can be used to cycle through
  results - this should mimic repeated `S$` or `N$`.
* `:FT...$` does a fuzzy search. IMHO it's not important
  to return a status integer instead. `FT` will only
  really be used in interactive mode.
* Document synchronization is supported via hooks from
  ring.c and via Scintilla notifications.
* Currently, the LSP communication is based on blocking
  GIOChannels. This means that a misbehaving hanging
  server could "lock up" the entire editor (FIXME).
  Only on ncurses you can always kill the subprocess by
  pressing CTRL+C.
  We need helper functions in spawn.c to read and write
  with interruptions.
* The textDocument/didChange notification transmits
  not only all edits, but all files' contents as well
  during initial synchronization.
  Therefore it is optimized to write and JSON-escape
  data without copying them around in memory and without
  destroying the buffer gap.
* Use $SCITECO_LSP to configure the language server.
  You can also use `tee` to capture stdin and stdout.
  Perhaps $SCITECO_LSP should be saved in .teco_session,
  so you can change it between projects?
* $SCITECO_LSP_ROOT is used to point to the project's
  root directory. session.tes will set it up automatically.
</pre>
</div>
</content>
</entry>
<entry>
<title>revised and improved the Unicode glyph-to-byte conversion heuristics</title>
<updated>2026-07-24T23:55:28+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-07-24T23:55:28+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=685507922b0b75da5935076395a5b1ec1ef58356'/>
<id>685507922b0b75da5935076395a5b1ec1ef58356</id>
<content type='text'>
Previously almost all glyph-to-byte offset conversions consulted
Scintilla's line index and counted characters on the resulting line.
For instance a simple expression like `.+1J` would scan the same line
twice completely, which would be very slow on pathologically long lines.
Even insertions did that due to having to update the ^Y ranges.
If you repeat such an operation over all characters as in `&lt;.+1:J;&gt;`
you would have complexity O(n^2) for n = line length.
Only commands with an explicit relative nature like `C` and `A` would
use teco_view_glyph2bytes_relative() which scans beginning at dot
as long as the relative movement is less than 1024 glyphs.

Wit the new heuristics almost all glyph-to-byte and byte-to-glyph
conversions can make use of that optimization.
This requires that dot must at all times be known in glyphs as well -
the byte position is managed by Scintilla (SCI_GETCURRENTPOS).
We therefore introduced teco_current_doc_set_dot() and
teco_current_doc_get_dot() to update dot in the current buffer or
Q-Register -- it cannot be stored along with the view since
Q-Registers share a single view.
A number of auxiliary functions have been introduced for
converting relative to a known (glyphs,bytes) offset pair
and for converting absolute and relative positions with regard
to the current doc and SCI_GETCURRENTPOS position.
Of course this is error-prone since the glyph and dot positions
are interdependant - they must always be kept in sync.

With these new optimizations even pathologically long lines can
(usually) be managed even in UTF-8 documents.
It does not address slow-downs in Scintilla's line layout, yet.
grosciteco.tes for instance runs twice as fast now.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Previously almost all glyph-to-byte offset conversions consulted
Scintilla's line index and counted characters on the resulting line.
For instance a simple expression like `.+1J` would scan the same line
twice completely, which would be very slow on pathologically long lines.
Even insertions did that due to having to update the ^Y ranges.
If you repeat such an operation over all characters as in `&lt;.+1:J;&gt;`
you would have complexity O(n^2) for n = line length.
Only commands with an explicit relative nature like `C` and `A` would
use teco_view_glyph2bytes_relative() which scans beginning at dot
as long as the relative movement is less than 1024 glyphs.

Wit the new heuristics almost all glyph-to-byte and byte-to-glyph
conversions can make use of that optimization.
This requires that dot must at all times be known in glyphs as well -
the byte position is managed by Scintilla (SCI_GETCURRENTPOS).
We therefore introduced teco_current_doc_set_dot() and
teco_current_doc_get_dot() to update dot in the current buffer or
Q-Register -- it cannot be stored along with the view since
Q-Registers share a single view.
A number of auxiliary functions have been introduced for
converting relative to a known (glyphs,bytes) offset pair
and for converting absolute and relative positions with regard
to the current doc and SCI_GETCURRENTPOS position.
Of course this is error-prone since the glyph and dot positions
are interdependant - they must always be kept in sync.

With these new optimizations even pathologically long lines can
(usually) be managed even in UTF-8 documents.
It does not address slow-downs in Scintilla's line layout, yet.
grosciteco.tes for instance runs twice as fast now.
</pre>
</div>
</content>
</entry>
<entry>
<title>updated copyright to 2026</title>
<updated>2026-01-01T06:59:49+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-01-01T06:59:49+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=c2feb2a6f71fc9adb20226fb3c2260c236e974e0'/>
<id>c2feb2a6f71fc9adb20226fb3c2260c236e974e0</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>avoid unnecessary cleanups of recovery files</title>
<updated>2025-12-28T23:42:41+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2025-12-28T23:31:20+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=713462dfdf3c46a998b43525cbbf5ae0ec8ea84b'/>
<id>713462dfdf3c46a998b43525cbbf5ae0ec8ea84b</id>
<content type='text'>
* After the last commit 0b593eb7d0e6907b19cdbb605caf1becae351004
  we tried to clean up (unlink) recovery files for all dirty buffers.
  This resulted in superfluous file deletions before any recovery
  file was dumped; after disabling file recovery and even in
  batch mode.
  It's not tolerable that SciTECO scripts try to unlink files
  as a side effect e.g. of EW.
  Also, sometimes you may have to clean up recovery dumps even
  in batch mode, e.g. in Quit hooks.
* Also, it was broken for dirty unnamed buffers, which would
  cause glib errors.
* That's why we had to add another buffer state for
  dirty files with outdated recovery dumps (TECO_BUFFER_DIRTY_OUTDATED_DUMP).
  Once a dump was written, a buffer never directly transitions
  into the TECO_BUFFER_DIRTY_NO_DUMP state again.
  We can now reliably unlink() only where we'd expect a
  recovery file to exist in the first place.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* After the last commit 0b593eb7d0e6907b19cdbb605caf1becae351004
  we tried to clean up (unlink) recovery files for all dirty buffers.
  This resulted in superfluous file deletions before any recovery
  file was dumped; after disabling file recovery and even in
  batch mode.
  It's not tolerable that SciTECO scripts try to unlink files
  as a side effect e.g. of EW.
  Also, sometimes you may have to clean up recovery dumps even
  in batch mode, e.g. in Quit hooks.
* Also, it was broken for dirty unnamed buffers, which would
  cause glib errors.
* That's why we had to add another buffer state for
  dirty files with outdated recovery dumps (TECO_BUFFER_DIRTY_OUTDATED_DUMP).
  Once a dump was written, a buffer never directly transitions
  into the TECO_BUFFER_DIRTY_NO_DUMP state again.
  We can now reliably unlink() only where we'd expect a
  recovery file to exist in the first place.
</pre>
</div>
</content>
</entry>
<entry>
<title>fixed left-over recovery files</title>
<updated>2025-12-28T20:22:46+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2025-12-28T20:22:46+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=0b593eb7d0e6907b19cdbb605caf1becae351004'/>
<id>0b593eb7d0e6907b19cdbb605caf1becae351004</id>
<content type='text'>
* It was possible to provoke left-over recovery files even if the editor does *not* crash:
  1. If you dirtified the buffer (state = TECO_BUFFER_DIRTY), it would be dumped to a
     recovery file (TECO_BUFFER_DIRTY_DUMPED).
  2. If you dirtify the buffer again, the state will become TECO_BUFFER_DIRTY again,
     so it's up for dumping in the next cycle.
  3. If you now save and exit (e.g. `:EX`) the recovery file is not deleted since
     the state is not TECO_BUFFER_DIRTY_DUMPED.
* A buffer can have a recovery file both for TECO_BUFFER_DIRTY and TECO_BUFFER_DIRTY_DUMPED,
  so we must clean up afterwards in both states.
* Of course, it may __not__ yet have a recovery file in the TECO_BUFFER_DIRTY state.
  The g_unlink() might therefore be superfluous on those files.
  Moreover, if you disable recovery files, SciTECO will now still try to unlink the
  recovery file.
  These operations could only be avoided by adding yet another state, e.g.
  TECO_BUFFER_DIRTY_OUTDATED_DUMP, so that after the first dump you will never switch
  back into TECO_BUFFER_DIRTY.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* It was possible to provoke left-over recovery files even if the editor does *not* crash:
  1. If you dirtified the buffer (state = TECO_BUFFER_DIRTY), it would be dumped to a
     recovery file (TECO_BUFFER_DIRTY_DUMPED).
  2. If you dirtify the buffer again, the state will become TECO_BUFFER_DIRTY again,
     so it's up for dumping in the next cycle.
  3. If you now save and exit (e.g. `:EX`) the recovery file is not deleted since
     the state is not TECO_BUFFER_DIRTY_DUMPED.
* A buffer can have a recovery file both for TECO_BUFFER_DIRTY and TECO_BUFFER_DIRTY_DUMPED,
  so we must clean up afterwards in both states.
* Of course, it may __not__ yet have a recovery file in the TECO_BUFFER_DIRTY state.
  The g_unlink() might therefore be superfluous on those files.
  Moreover, if you disable recovery files, SciTECO will now still try to unlink the
  recovery file.
  These operations could only be avoided by adding yet another state, e.g.
  TECO_BUFFER_DIRTY_OUTDATED_DUMP, so that after the first dump you will never switch
  back into TECO_BUFFER_DIRTY.
</pre>
</div>
</content>
</entry>
<entry>
<title>TECO_DEFINE_STATE() no longer constructs callback names for mandatory callbacks, but tries to use static assertions</title>
<updated>2025-12-26T17:10:42+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2025-12-26T17:10:42+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=c2114fa0af73b42bc1ef302f7511ef87690cc0b1'/>
<id>c2114fa0af73b42bc1ef302f7511ef87690cc0b1</id>
<content type='text'>
* Requiring state callbacks by generating their names (e.g. NAME##_input) has several disadvantages:
  * The callback is not explicitly referenced when the state is defined.
    So an unintroduced reader will see some static function, which is nowhere referenced and still
    doesn't cause "unused" warnings.
  * You cannot choose the name of function that implements the callback freely.
  * In "substates" you need to generate a callback function if you want to provide a default.
    You also need to provide dummy wrapper functions whenever you want to reuse some existing
    function as the implementation.
* Instead, we are now using static assertions to check whether certain callbacks have been
  implemented.
  Unfortunately, this does not work on all compilers. In particular GCC won't consider
  references to state objects fully constant (even though they are) and does not allow
  them in _Static_assert (G_STATIC_ASSERT). This could only be made to work in newer GCC
  with -std=c2x or -std=gnu23 in combination with constexpr.
  It does work on Clang, though.
  So I introduced TECO_ASSERT_SAFE() which also passes if the expression is *not* constant.
  These static assertions are not crucial - they do not check anything that can differ between
  systems. So we can always rely on the checks performed by FreeBSD CI for instance.
  Also, you will of course quickly notice missing callbacks at runtime - with and without
  additional runtime assertions.
* All mandatory callbacks must still be explicitly initialized in the TECO_DEFINE_STATE calls.
* After getting rid of generated callback implementations, the TECO_DEFINE_STATE macros
  can finally be qualified with `static`.
* The TECO_DECLARE_STATE() macro has been removed. It no longer abstracts anything
  and cannot be used to declare static teco_state_t anyway.
  Also TECO_DEFINE_UNDO_CALL() also doesn't have a DECLARE counterpart.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Requiring state callbacks by generating their names (e.g. NAME##_input) has several disadvantages:
  * The callback is not explicitly referenced when the state is defined.
    So an unintroduced reader will see some static function, which is nowhere referenced and still
    doesn't cause "unused" warnings.
  * You cannot choose the name of function that implements the callback freely.
  * In "substates" you need to generate a callback function if you want to provide a default.
    You also need to provide dummy wrapper functions whenever you want to reuse some existing
    function as the implementation.
* Instead, we are now using static assertions to check whether certain callbacks have been
  implemented.
  Unfortunately, this does not work on all compilers. In particular GCC won't consider
  references to state objects fully constant (even though they are) and does not allow
  them in _Static_assert (G_STATIC_ASSERT). This could only be made to work in newer GCC
  with -std=c2x or -std=gnu23 in combination with constexpr.
  It does work on Clang, though.
  So I introduced TECO_ASSERT_SAFE() which also passes if the expression is *not* constant.
  These static assertions are not crucial - they do not check anything that can differ between
  systems. So we can always rely on the checks performed by FreeBSD CI for instance.
  Also, you will of course quickly notice missing callbacks at runtime - with and without
  additional runtime assertions.
* All mandatory callbacks must still be explicitly initialized in the TECO_DEFINE_STATE calls.
* After getting rid of generated callback implementations, the TECO_DEFINE_STATE macros
  can finally be qualified with `static`.
* The TECO_DECLARE_STATE() macro has been removed. It no longer abstracts anything
  and cannot be used to declare static teco_state_t anyway.
  Also TECO_DEFINE_UNDO_CALL() also doesn't have a DECLARE counterpart.
</pre>
</div>
</content>
</entry>
<entry>
<title>fixup: renamed "backups" to "recovery files"</title>
<updated>2025-12-19T22:25:48+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2025-12-19T22:25:48+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=2592ef74ab2eba57c32fe21993ce01e9698b106f'/>
<id>2592ef74ab2eba57c32fe21993ce01e9698b106f</id>
<content type='text'>
* Other editors call "backup files" previous copies of saved files.
  This role would be served by savepoint files in SciTECO.
* Likewise filename~ would point to such a backup file.
  It therefore makes sense that savepoint files also end in tildes (.teco-n-filename~).
* Security copies of modified buffers would be called "auto-saves" (Emacs) or
  "swap files" (Vim).
  Both of these terms is IMHO misleading, so SciTECO now uses the
  term "recovery file".
* "Recovery files" are now named #filename# just like in Emacs.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Other editors call "backup files" previous copies of saved files.
  This role would be served by savepoint files in SciTECO.
* Likewise filename~ would point to such a backup file.
  It therefore makes sense that savepoint files also end in tildes (.teco-n-filename~).
* Security copies of modified buffers would be called "auto-saves" (Emacs) or
  "swap files" (Vim).
  Both of these terms is IMHO misleading, so SciTECO now uses the
  term "recovery file".
* "Recovery files" are now named #filename# just like in Emacs.
</pre>
</div>
</content>
</entry>
<entry>
<title>implemented backup file mechanism</title>
<updated>2025-12-17T00:17:11+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2025-12-17T00:17:11+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=deed71ac895451041359d7b18e58eca0a0972bc3'/>
<id>deed71ac895451041359d7b18e58eca0a0972bc3</id>
<content type='text'>
* The backup mechanism is supposed to guard against crashes of SciTECO and
  unexpected program terminations (e.g. power cycling, etc.)
* In a given interval (no matter whether busy or idlying on the prompt)
  SciTECO saves all modified buffers with the filename~ (like most other editors).
  As an optimization files are not backed up if they have been backed up
  previously to avoid pointless and possibly slow file system writes.
* While the backup mechanism exists outside of the usual undo-paradigm -
  backup file creating is not bound to character input and it makes no sense
  to restore the exact state of backup files - there are some interesting
  interactions:
  * When a buffer is dirtyfied or saved that was previously backed up, it must always
    be reset to the DIRTY state on rubout, so backups are eventually recreated.
  * When a buffer is dirtyfied first (was clean), the backup file must be
    removed on rubout as well - we don't expect backup files for clean buffers.
* There is currently no automatic way to restore backup files.
  This could potentially be done by opener.tes and session.tes in the future,
  although you couldn't currently always get meaningful user feedback
  (whether he wants to restore the file).
  Perhaps we should at least log a message when detecting backup files that
  are newer than the file that is being opened.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* The backup mechanism is supposed to guard against crashes of SciTECO and
  unexpected program terminations (e.g. power cycling, etc.)
* In a given interval (no matter whether busy or idlying on the prompt)
  SciTECO saves all modified buffers with the filename~ (like most other editors).
  As an optimization files are not backed up if they have been backed up
  previously to avoid pointless and possibly slow file system writes.
* While the backup mechanism exists outside of the usual undo-paradigm -
  backup file creating is not bound to character input and it makes no sense
  to restore the exact state of backup files - there are some interesting
  interactions:
  * When a buffer is dirtyfied or saved that was previously backed up, it must always
    be reset to the DIRTY state on rubout, so backups are eventually recreated.
  * When a buffer is dirtyfied first (was clean), the backup file must be
    removed on rubout as well - we don't expect backup files for clean buffers.
* There is currently no automatic way to restore backup files.
  This could potentially be done by opener.tes and session.tes in the future,
  although you couldn't currently always get meaningful user feedback
  (whether he wants to restore the file).
  Perhaps we should at least log a message when detecting backup files that
  are newer than the file that is being opened.
</pre>
</div>
</content>
</entry>
<entry>
<title>if EX falis because of a dirty buffer, the buffer's id is now included in the error message</title>
<updated>2025-07-22T22:15:45+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-07-22T22:15:45+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=c2cdafdd5c92f8caa4701257ba91dd7915054fbe'/>
<id>c2cdafdd5c92f8caa4701257ba91dd7915054fbe</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>implemented &lt;ER&gt; command for reading a file into the current buffer</title>
<updated>2025-07-13T15:35:32+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-07-13T15:35:32+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=fbab5e252f22de37d42cc6c2a014d690a9312565'/>
<id>fbab5e252f22de37d42cc6c2a014d690a9312565</id>
<content type='text'>
* This command exists in Video TECO.
  In Video TECO it also supports reading multiple files with a glob pattern -- we do not support that
  as I am not convinced of its usefulness.
* teco_view_load() has been extended, so it can read into dot without
  discarding the existing document.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* This command exists in Video TECO.
  In Video TECO it also supports reading multiple files with a glob pattern -- we do not support that
  as I am not convinced of its usefulness.
* teco_view_load() has been extended, so it can read into dot without
  discarding the existing document.
</pre>
</div>
</content>
</entry>
</feed>
