<feed xmlns='http://www.w3.org/2005/Atom'>
<title>sciteco/src/Makefile.am, branch lsp</title>
<subtitle>Scintilla-based Text Editor and COrrector</subtitle>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/'/>
<entry>
<title>support lookups via language servers (LSP)</title>
<updated>2026-08-10T22:38:31+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-08-02T13:24:45+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=44f5bf677282308b959411c416da2b5b08db2062'/>
<id>44f5bf677282308b959411c416da2b5b08db2062</id>
<content type='text'>
* The main interface is the `FT` command.
  `FT` was an undocumented Video TECO command for etags/ctags lookups.
  I don't want to exactly copy its interface, though.
* `FT` allows looking up symbol names,
  definitions and references.
* ctags will be supported via ctags-lsp.
  LSP support is more powerful though and works without
  regenerating TAGS files all the time.
  The LSP will also allow you to customize auto-completions
  using SciTECO itself (i.e. by writing a language server
  in SciTECO).
* For multiple results, `FT$` can be used to cycle through
  results - this should mimic repeated `S$` or `N$`.
* `:FT...$` does a fuzzy search. IMHO it's not important
  to return a status integer instead. `FT` will only
  really be used in interactive mode.
* Document synchronization is supported via hooks from
  ring.c and via Scintilla notifications.
* Currently, the LSP communication is based on blocking
  GIOChannels. This means that a misbehaving hanging
  server could "lock up" the entire editor (FIXME).
  Only on ncurses you can always kill the subprocess by
  pressing CTRL+C.
  We need helper functions in spawn.c to read and write
  with interruptions.
* The textDocument/didChange notification transmits
  not only all edits, but all files' contents as well
  during initial synchronization.
  Therefore it is optimized to write and JSON-escape
  data without copying them around in memory and without
  destroying the buffer gap.
* Use $SCITECO_LSP to configure the language server.
  You can also use `tee` to capture stdin and stdout.
  Perhaps $SCITECO_LSP should be saved in .teco_session,
  so you can change it between projects?
* $SCITECO_LSP_ROOT is used to point to the project's
  root directory. session.tes will set it up automatically.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* The main interface is the `FT` command.
  `FT` was an undocumented Video TECO command for etags/ctags lookups.
  I don't want to exactly copy its interface, though.
* `FT` allows looking up symbol names,
  definitions and references.
* ctags will be supported via ctags-lsp.
  LSP support is more powerful though and works without
  regenerating TAGS files all the time.
  The LSP will also allow you to customize auto-completions
  using SciTECO itself (i.e. by writing a language server
  in SciTECO).
* For multiple results, `FT$` can be used to cycle through
  results - this should mimic repeated `S$` or `N$`.
* `:FT...$` does a fuzzy search. IMHO it's not important
  to return a status integer instead. `FT` will only
  really be used in interactive mode.
* Document synchronization is supported via hooks from
  ring.c and via Scintilla notifications.
* Currently, the LSP communication is based on blocking
  GIOChannels. This means that a misbehaving hanging
  server could "lock up" the entire editor (FIXME).
  Only on ncurses you can always kill the subprocess by
  pressing CTRL+C.
  We need helper functions in spawn.c to read and write
  with interruptions.
* The textDocument/didChange notification transmits
  not only all edits, but all files' contents as well
  during initial synchronization.
  Therefore it is optimized to write and JSON-escape
  data without copying them around in memory and without
  destroying the buffer gap.
* Use $SCITECO_LSP to configure the language server.
  You can also use `tee` to capture stdin and stdout.
  Perhaps $SCITECO_LSP should be saved in .teco_session,
  so you can change it between projects?
* $SCITECO_LSP_ROOT is used to point to the project's
  root directory. session.tes will set it up automatically.
</pre>
</div>
</content>
</entry>
<entry>
<title>terex is the new regular expression engine now and replaces PCRE (GRegex)</title>
<updated>2026-06-27T22:39:51+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2026-06-27T22:39:51+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=4fe5bc6f3867096965270c90f2e1e5df77b8825f'/>
<id>4fe5bc6f3867096965270c90f2e1e5df77b8825f</id>
<content type='text'>
* terex is based on Henry Spencer's regular expression engine for Tcl.
  It is a hybrid NFA/DFA design which has better worst-time runtimes than
  the backtracking PCRE. Memory usage is also limited and can no longer
  increase catastrophically.
* It should no longer be possible to crash SciTECO with pathological
  searches.
* Since it reliably supports partial matches (REG_EXPECT) we can
  now enable the new backwards-search algorithm by default.
  This used to be broken because of a glib bug, which I already
  fixed. It would however take a long time until this ends up
  on the majority of glib installations.
* Regexp executions can still be quite slow if you are looking
  for a pattern at the end of a huge file, which can hang the editor,
  but this can now at least theoretically be solved by adding
  hooks into terex to poll for interruptions.
* We can now also get rid of a TECO-pattern to regexp translation
  step by directly generating terex tokens (TODO).
* Performance-wise terex appears to be slower than PCRE for simple
  forward searches even when linking everything with optimzations (FIXME).
* Having a stand-alone regular expression engine is also a huge
  step in getting rid of glib.

See also: https://git.fmsbw.de/terex/about/
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* terex is based on Henry Spencer's regular expression engine for Tcl.
  It is a hybrid NFA/DFA design which has better worst-time runtimes than
  the backtracking PCRE. Memory usage is also limited and can no longer
  increase catastrophically.
* It should no longer be possible to crash SciTECO with pathological
  searches.
* Since it reliably supports partial matches (REG_EXPECT) we can
  now enable the new backwards-search algorithm by default.
  This used to be broken because of a glib bug, which I already
  fixed. It would however take a long time until this ends up
  on the majority of glib installations.
* Regexp executions can still be quite slow if you are looking
  for a pattern at the end of a huge file, which can hang the editor,
  but this can now at least theoretically be solved by adding
  hooks into terex to poll for interruptions.
* We can now also get rid of a TECO-pattern to regexp translation
  step by directly generating terex tokens (TODO).
* Performance-wise terex appears to be slower than PCRE for simple
  forward searches even when linking everything with optimzations (FIXME).
* Having a stand-alone regular expression engine is also a huge
  step in getting rid of glib.

See also: https://git.fmsbw.de/terex/about/
</pre>
</div>
</content>
</entry>
<entry>
<title>command-line arguments are no longer passed via the unnamed buffer, but via special Q-registers ^Ax</title>
<updated>2025-08-06T13:46:37+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-08-06T13:46:37+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=280cb9da39fc7b5357f6071926d511394f6d0152'/>
<id>280cb9da39fc7b5357f6071926d511394f6d0152</id>
<content type='text'>
* The unnamed buffer is also used for reading from --stdin, so you couldn't practically combine
  --stdin with passing command-line arguments to macros.
* The old approach of passing command-line arguments via lines in the
  unnamed buffer was flawed anyway as it wouldn't work with filenames containing LF.
  This is just a very ancient feature, written when there weren't even long Q-reg names in SciTECO.
* You can now e.g. pipe into SciTECO and edit what was read interactively, e.g. `dmesg | sciteco -i`.
  You can practically use SciTECO as a pager.
* htbl.tes is now a command-line filter (uses -qio).
* grosciteco.tes reads Troff intermediate code from stdin, so we no longer need
  "*.intermediate" temporary files.
* added a getopt.tes test case to the testsuite.
* This change unfortunately breaks most macros accepting command-line arguments,
  even if they used getopt.tes.
  It also requires updating ~/.teco_ini - see fallback.teco_ini.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* The unnamed buffer is also used for reading from --stdin, so you couldn't practically combine
  --stdin with passing command-line arguments to macros.
* The old approach of passing command-line arguments via lines in the
  unnamed buffer was flawed anyway as it wouldn't work with filenames containing LF.
  This is just a very ancient feature, written when there weren't even long Q-reg names in SciTECO.
* You can now e.g. pipe into SciTECO and edit what was read interactively, e.g. `dmesg | sciteco -i`.
  You can practically use SciTECO as a pager.
* htbl.tes is now a command-line filter (uses -qio).
* grosciteco.tes reads Troff intermediate code from stdin, so we no longer need
  "*.intermediate" temporary files.
* added a getopt.tes test case to the testsuite.
* This change unfortunately breaks most macros accepting command-line arguments,
  even if they used getopt.tes.
  It also requires updating ~/.teco_ini - see fallback.teco_ini.
</pre>
</div>
</content>
</entry>
<entry>
<title>simplified the htbl.tes preprocessor and the SUBST_MACRO using new --quiet, --stdin and --stdout options</title>
<updated>2025-08-03T15:01:34+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-08-03T14:56:04+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=c5510d684e4879ab9a5295b4a1981888a4268627'/>
<id>c5510d684e4879ab9a5295b4a1981888a4268627</id>
<content type='text'>
* htbl.tes now reads from stdin and writes to stdout.
  Allows avoiding temporary `*.htbl` files
* grosciteco.tes still cannot be simplified since --stdin cannot be combined with
  passing command-line arguments (FIXME).
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* htbl.tes now reads from stdin and writes to stdout.
  Allows avoiding temporary `*.htbl` files
* grosciteco.tes still cannot be simplified since --stdin cannot be combined with
  passing command-line arguments (FIXME).
</pre>
</div>
</content>
</entry>
<entry>
<title>refactored =/==/=== command into stdio-commands.c</title>
<updated>2025-07-21T22:31:37+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-07-21T22:31:37+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=63a314144a36691a57c2e82fe2f3da008c403aba'/>
<id>63a314144a36691a57c2e82fe2f3da008c403aba</id>
<content type='text'>
There will be a lot more commands for terminal/message
input and output soon.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
There will be a lot more commands for terminal/message
input and output soon.
</pre>
</div>
</content>
</entry>
<entry>
<title>factored out all cursor movement and deletion commands into the new compilation unit move-commands.c</title>
<updated>2025-03-22T12:00:51+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-03-22T11:58:12+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=fe69fdb29f994db3bbcaffcb202856618b03b9b0'/>
<id>fe69fdb29f994db3bbcaffcb202856618b03b9b0</id>
<content type='text'>
* This made sense to include both `W` and `V`, so we also included `D` and `K`.
  `^Q` is included since it converts between lines and glyphs.
* These are all single-letter commands, so they aren't complete parser states
  but callbacks to be referenced in teco_machine_main_transition_t.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* This made sense to include both `W` and `V`, so we also included `D` and `K`.
  `^Q` is included since it converts between lines and glyphs.
* These are all single-letter commands, so they aren't complete parser states
  but callbacks to be referenced in teco_machine_main_transition_t.
</pre>
</div>
</content>
</entry>
<entry>
<title>implemented Scintilla lexer for SciTECO code, i.e. TECO syntax highlighting</title>
<updated>2024-12-12T21:58:14+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-12-09T09:58:25+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=244a54a18b7db6af177c9d10f3224772f08d7484'/>
<id>244a54a18b7db6af177c9d10f3224772f08d7484</id>
<content type='text'>
* this works by embedding the SciTECO parser and driving it always (exclusively)
  in parse-only mode.
* A new teco_state_t::style determines the Scintilla style for any character
  accepted in the given state.
* Therefore, the SciTECO lexer is always 100% exact and corresponds to the current
  SciTECO grammer - it does not have to be maintained separately.
  There are a few exceptions and tweaks, though.
* The contents of curly-brace escapes (`@^Uq{...}`) are rendered as ordinary
  code using a separate parser instance.
  This can be disabled with the lexer.sciteco.macrodef property.
  Unfortunately, SciTECO does not currently allow setting lexer properties (FIXME).
* Labels and comments are currently styled the same.
  This could change in the future once we introduce real comments.
* Lexers are usually implemented in C++, but I did not want to draw in C++.
  Especially not since we'd have to include parser.h and other SciTECO headers,
  that really do not want to keep C++-compatible.
  Instead, the lexer is implemented "in the container".
  @ES/SCI_SETILEXER/sciteco/ is internally translated to SCI_SETILEXER(NULL)
  and we get Scintilla notifications when styling the view becomes necessary.
  This is then centrally forwarded to the teco_lexer_style() which
  uses the ordinary teco_view_ssm() API for styling.
* Once the command line becomes a Scintilla view even on Curses,
  we can enabled syntax highlighting of the command line macro.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* this works by embedding the SciTECO parser and driving it always (exclusively)
  in parse-only mode.
* A new teco_state_t::style determines the Scintilla style for any character
  accepted in the given state.
* Therefore, the SciTECO lexer is always 100% exact and corresponds to the current
  SciTECO grammer - it does not have to be maintained separately.
  There are a few exceptions and tweaks, though.
* The contents of curly-brace escapes (`@^Uq{...}`) are rendered as ordinary
  code using a separate parser instance.
  This can be disabled with the lexer.sciteco.macrodef property.
  Unfortunately, SciTECO does not currently allow setting lexer properties (FIXME).
* Labels and comments are currently styled the same.
  This could change in the future once we introduce real comments.
* Lexers are usually implemented in C++, but I did not want to draw in C++.
  Especially not since we'd have to include parser.h and other SciTECO headers,
  that really do not want to keep C++-compatible.
  Instead, the lexer is implemented "in the container".
  @ES/SCI_SETILEXER/sciteco/ is internally translated to SCI_SETILEXER(NULL)
  and we get Scintilla notifications when styling the view becomes necessary.
  This is then centrally forwarded to the teco_lexer_style() which
  uses the ordinary teco_view_ssm() API for styling.
* Once the command line becomes a Scintilla view even on Curses,
  we can enabled syntax highlighting of the command line macro.
</pre>
</div>
</content>
</entry>
<entry>
<title>Win32: fixed Unicode commandlines with newer MinGW runtimes</title>
<updated>2024-11-10T20:16:52+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-11-10T20:16:52+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=605bd59516b0868cc73ed01f913eeb331033a84b'/>
<id>605bd59516b0868cc73ed01f913eeb331033a84b</id>
<content type='text'>
* should also fix Win32 nightly builds
* Even though we weren't using main's argv, but were using glib
  API for retrieving the command line in UTF-8, newer MinGW runtimes
  would fail when converting the Unicode command line into the system codepage
  would be lossy.
* Most people seem to compile in a "manifest" to work around this issue.
  But this requires newer Windows versions and using some Microsoft tool which isn't
  even in $PATH.
  Instead, we now link with -municode and define wmain() instead, even though we still
  ignore argv. wmain() proabably get's the command line in UTF-16 and we'd have to
  convert it anyway.
* See https://github.com/msys2/MINGW-packages/issues/22462
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* should also fix Win32 nightly builds
* Even though we weren't using main's argv, but were using glib
  API for retrieving the command line in UTF-8, newer MinGW runtimes
  would fail when converting the Unicode command line into the system codepage
  would be lossy.
* Most people seem to compile in a "manifest" to work around this issue.
  But this requires newer Windows versions and using some Microsoft tool which isn't
  even in $PATH.
  Instead, we now link with -municode and define wmain() instead, even though we still
  ignore argv. wmain() proabably get's the command line in UTF-16 and we'd have to
  convert it anyway.
* See https://github.com/msys2/MINGW-packages/issues/22462
</pre>
</div>
</content>
</entry>
<entry>
<title>symbols-extract.tes works in 8-bit mode now (refs #5)</title>
<updated>2024-09-09T17:24:18+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-09-09T17:21:22+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=a7d28d3fed7748dcaedbcac8902cb9e1339cb078'/>
<id>a7d28d3fed7748dcaedbcac8902cb9e1339cb078</id>
<content type='text'>
* significantly speeds up build time
* Scintilla and Lexilla headers and symbols are all-ASCII anyway.
* We should probably have a look at the quicksort implementation
  in string.tes, as it can probably be optimized in UTF-8 documents as well.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* significantly speeds up build time
* Scintilla and Lexilla headers and symbols are all-ASCII anyway.
* We should probably have a look at the quicksort implementation
  in string.tes, as it can probably be optimized in UTF-8 documents as well.
</pre>
</div>
</content>
</entry>
<entry>
<title>fully support out of tree builds</title>
<updated>2024-08-23T02:51:55+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-08-23T02:13:15+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=ee9cf43587d5fef3a0f6d97ef50b8cf848945bcb'/>
<id>ee9cf43587d5fef3a0f6d97ef50b8cf848945bcb</id>
<content type='text'>
* You no longer have to copy contrib/scintilla, contrib/scinterm and contrib/lexilla
  manually to the build directory.
* It turns out, that Scintilla/Lexilla was supporting this since 2016.
  Scintilla allows pointing to a source directory (srdir) and Lexilla to a binary directory (DIR_O).
* For Scinterm I opened a pull request in order to add srcdir/basedir variables:
  https://github.com/orbitalquark/scinterm/pull/21
* `make distcheck` is therefore now also fixed.
* The FreeBSD package is now allowed to build out of source.
  I haven't tested it yet.
* See also https://github.com/ScintillaOrg/lexilla/issues/266
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* You no longer have to copy contrib/scintilla, contrib/scinterm and contrib/lexilla
  manually to the build directory.
* It turns out, that Scintilla/Lexilla was supporting this since 2016.
  Scintilla allows pointing to a source directory (srdir) and Lexilla to a binary directory (DIR_O).
* For Scinterm I opened a pull request in order to add srcdir/basedir variables:
  https://github.com/orbitalquark/scinterm/pull/21
* `make distcheck` is therefore now also fixed.
* The FreeBSD package is now allowed to build out of source.
  I haven't tested it yet.
* See also https://github.com/ScintillaOrg/lexilla/issues/266
</pre>
</div>
</content>
</entry>
</feed>
