<feed xmlns='http://www.w3.org/2005/Atom'>
<title>sciteco/lib/lexer.tes, branch v2.5.2</title>
<subtitle>Scintilla-based Text Editor and COrrector</subtitle>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/'/>
<entry>
<title>added ED flag 2048 to redirect Scintilla messages to the command line view: enables syntax highlighting on the command line</title>
<updated>2025-11-08T12:00:47+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>rhaberkorn@fmsbw.de</email>
</author>
<published>2025-11-07T20:52:15+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=397554a6091f4a8d337cc4935638bf736bef23be'/>
<id>397554a6091f4a8d337cc4935638bf736bef23be</id>
<content type='text'>
* M[lexer.set.cmdline] can be used to set up syntax highlighting on the command line
  (if desired).
* Color schemes with light-dark themes (solarized.tes) are now responsible
  to update the command line view as well.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* M[lexer.set.cmdline] can be used to set up syntax highlighting on the command line
  (if desired).
* Color schemes with light-dark themes (solarized.tes) are now responsible
  to update the command line view as well.
</pre>
</div>
</content>
</entry>
<entry>
<title>&lt;EI&gt; has been repurposed and is the macro file inclusion (indirect file) command now</title>
<updated>2025-05-24T14:22:52+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-05-24T13:24:49+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=6e3da17a2fae11af9ae00d9b59bd0d752022e16b'/>
<id>6e3da17a2fae11af9ae00d9b59bd0d752022e16b</id>
<content type='text'>
* Improves DEC TECO-11 compatibility.
* &lt;EM&gt; is still supported as a synonym, but considered deprecated and is no longer documented.
  A warning is printed when invoked.
  It can be repurposed at any time in the future.
* `EI$` is not yet supported.
  I am unsure whether this makes any sense.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Improves DEC TECO-11 compatibility.
* &lt;EM&gt; is still supported as a synonym, but considered deprecated and is no longer documented.
  A warning is printed when invoked.
  It can be repurposed at any time in the future.
* `EI$` is not yet supported.
  I am unsure whether this makes any sense.
</pre>
</div>
</content>
</entry>
<entry>
<title>fixup: fixed monospace sections in womanpages when lexer.font is not customized (refs #34)</title>
<updated>2025-04-22T23:07:59+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2025-04-22T23:07:59+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=01419ac9b5c8961e240feaa9f5d160a633f801ed'/>
<id>01419ac9b5c8961e240feaa9f5d160a633f801ed</id>
<content type='text'>
* In the grosciteco-generated .woman.tec files, the fonts were only
  set on the monospaced styles if lexer.font was set since it was
  undefined by default.
  We'd need a lare IF-ELSE statement to handle that in womanpages
  since woman.tes changes the default font to a variable-width font.
  Just leaving the default font, therefore won't work.
* We now always initialize lexer.font in lexer.tes.
  You no longer have to check for its existance.
* Consequently, you can no longer set lexer.font *before* munging
  lexer.tes because it will be overwritten.
  At least not without additional checks.
  Such a design is not suggested by fallback.teco_ini, though.
* Fixes monospaced sections in the tutorial and other womanpages on Gtk.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* In the grosciteco-generated .woman.tec files, the fonts were only
  set on the monospaced styles if lexer.font was set since it was
  undefined by default.
  We'd need a lare IF-ELSE statement to handle that in womanpages
  since woman.tes changes the default font to a variable-width font.
  Just leaving the default font, therefore won't work.
* We now always initialize lexer.font in lexer.tes.
  You no longer have to check for its existance.
* Consequently, you can no longer set lexer.font *before* munging
  lexer.tes because it will be overwritten.
  At least not without additional checks.
  Such a design is not suggested by fallback.teco_ini, though.
* Fixes monospaced sections in the tutorial and other womanpages on Gtk.
</pre>
</div>
</content>
</entry>
<entry>
<title>introduced true block and EOL comments</title>
<updated>2024-12-24T10:29:32+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-12-24T10:29:32+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=ef897b418a4487196e1dbc18a97046f8f0aea2e8'/>
<id>ef897b418a4487196e1dbc18a97046f8f0aea2e8</id>
<content type='text'>
* The previous convention of !* ... *! are now true block comments,
  i.e. they are parsed faster, don't spam the goto table and allow
  embedding of exclamation marks - only "*!" terminates the comment.
* It is therefore now forbidden to have goto labels beginning with "*".
* Also support "!!" to introduce EOL comments (like C++'s //).
  This disallows empty labels, but they weren't useful anyway.
  This is the shortest way to begin a comment.
* All comment labels have been converted to true comments, to ensure
  that syntax highlighting works correctly.
  EOL comments are used for single line commented-out code, since it's
  easiest to uncomment - you don't have to jump to the line end.
  This is a pure convention / coding style.
  Other people might do it differently.
* It's of course still possible to abuse goto labels as comments
  as TECO did for ages.
* In lexing / syntax highlighting, labels and comments are highlighted differently.
* When syntax highlighting, a single "!" will first be highlighted as a label
  since it's not yet unambiguous. Once you type the second character (* or !),
  the first character is retroactively styled as a comment as well.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* The previous convention of !* ... *! are now true block comments,
  i.e. they are parsed faster, don't spam the goto table and allow
  embedding of exclamation marks - only "*!" terminates the comment.
* It is therefore now forbidden to have goto labels beginning with "*".
* Also support "!!" to introduce EOL comments (like C++'s //).
  This disallows empty labels, but they weren't useful anyway.
  This is the shortest way to begin a comment.
* All comment labels have been converted to true comments, to ensure
  that syntax highlighting works correctly.
  EOL comments are used for single line commented-out code, since it's
  easiest to uncomment - you don't have to jump to the line end.
  This is a pure convention / coding style.
  Other people might do it differently.
* It's of course still possible to abuse goto labels as comments
  as TECO did for ages.
* In lexing / syntax highlighting, labels and comments are highlighted differently.
* When syntax highlighting, a single "!" will first be highlighted as a label
  since it's not yet unambiguous. Once you type the second character (* or !),
  the first character is retroactively styled as a comment as well.
</pre>
</div>
</content>
</entry>
<entry>
<title>fixed lexer.checkheader: restore dot in case of success</title>
<updated>2024-12-13T15:20:03+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-12-13T15:20:03+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=cb1cb619cd6cc6ab0c2bc3ebedfe50d01db351b7'/>
<id>cb1cb619cd6cc6ab0c2bc3ebedfe50d01db351b7</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>support the ::S anchored search (string comparison) command (and ::FD, ::FR, ::FS as well)</title>
<updated>2024-12-06T14:20:52+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-12-06T14:20:52+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=e5884ab2166ab5a03294baa54601b8785e6d9727'/>
<id>e5884ab2166ab5a03294baa54601b8785e6d9727</id>
<content type='text'>
* The colon modifier can now occur 2 times.
  Specifying `@` more than once or `:` more than twice is an error now.
* Commands do not check for excess colon modifiers - almost every command would have
  to check it. Instead, a double colon will simply behave like a single colon on most
  commands.
* All search commands inherit the anchored semantics, but it's not very useful in some combinations
  like -::S, ::N or ::FK.
  That's why the `::` variants are not documented everywhere.
* The lexer.checkheader macro could be simplified and should also be faster now,
  speeding up startup.
  Eventually this macro can be made superfluous, e.g. by using 1:FB or 0,1^Q::S.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* The colon modifier can now occur 2 times.
  Specifying `@` more than once or `:` more than twice is an error now.
* Commands do not check for excess colon modifiers - almost every command would have
  to check it. Instead, a double colon will simply behave like a single colon on most
  commands.
* All search commands inherit the anchored semantics, but it's not very useful in some combinations
  like -::S, ::N or ::FK.
  That's why the `::` variants are not documented everywhere.
* The lexer.checkheader macro could be simplified and should also be faster now,
  speeding up startup.
  Eventually this macro can be made superfluous, e.g. by using 1:FB or 0,1^Q::S.
</pre>
</div>
</content>
</entry>
<entry>
<title>lexer.auto: use case-sensitive searches</title>
<updated>2024-11-24T02:27:02+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-11-24T02:27:02+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=8905a333c9666be628798c04af98a2fdeeb75db5'/>
<id>8905a333c9666be628798c04af98a2fdeeb75db5</id>
<content type='text'>
* lexer.checkheader is therefore case-sensitive now as well
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* lexer.checkheader is therefore case-sensitive now as well
</pre>
</div>
</content>
</entry>
<entry>
<title>Xq and ]q inherit the document encoding from the source document (refs #5)</title>
<updated>2024-09-09T16:22:21+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-09-08T22:03:33+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=41ab5cf0289dab60ac1ddc97cf9680ee2468ea6c'/>
<id>41ab5cf0289dab60ac1ddc97cf9680ee2468ea6c</id>
<content type='text'>
* ^Uq however always sets an UTF8 register as the source
  is supposed to be a SciTECO macro which is always UTF-8.
* :^Uq preserves the register's encoding
* teco_doc_set_string() now also sets the encoding
* instead of trying to restore the encoding in teco_doc_undo_set_string(),
  we now swap out the document in a teco_doc_t and pass it to an undo token.
* The get_codepage() Q-Reg method has been removed as the same
  can now be done with teco_doc_get_string() and the get_string() method.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* ^Uq however always sets an UTF8 register as the source
  is supposed to be a SciTECO macro which is always UTF-8.
* :^Uq preserves the register's encoding
* teco_doc_set_string() now also sets the encoding
* instead of trying to restore the encoding in teco_doc_undo_set_string(),
  we now swap out the document in a teco_doc_t and pass it to an undo token.
* The get_codepage() Q-Reg method has been removed as the same
  can now be done with teco_doc_get_string() and the get_string() method.
</pre>
</div>
</content>
</entry>
<entry>
<title>lexer.checkheader is Unicode-aware now (refs #5)</title>
<updated>2024-09-09T16:22:21+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2024-09-02T13:30:23+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=e466218d6c608ec4456384dc94aefafdb5b60586'/>
<id>e466218d6c608ec4456384dc94aefafdb5b60586</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>implemented &lt;$$&gt; command for returning from a macro</title>
<updated>2016-02-15T14:00:55+00:00</updated>
<author>
<name>Robin Haberkorn</name>
<email>robin.haberkorn@googlemail.com</email>
</author>
<published>2016-02-13T17:43:00+00:00</published>
<link rel='alternate' type='text/html' href='https://git.fmsbw.de/sciteco/commit/?id=f08187e454f56954b41d95615ca2e370ba19667e'/>
<id>f08187e454f56954b41d95615ca2e370ba19667e</id>
<content type='text'>
 * &lt;$$&gt; is faster than jumping to the end of the macro
   and enables shorter code for returning values from macros.
 * this also replaces $$ as an immediate editing command.
   In other words, command line termination is an ordinary command
   now. The old behaviour was similar to what classic TECO did.
   Classic TECO however had no choice than to track key presses
   directly for command line termination as it did not keep track
   about the parser state as input was typed.
   This led to some glitches in the language. For instance
   "FS$$" would terminate the command line, unless the second escape
   was typed after backspace, etc. This behaviour is not worth copying
   and SciTECO did a better job than that by making sure that at least the
   second escape is only effective if it is not part of language syntax.
   This still lead to some undesirable cases like "ES...$$$" that would
   terminate the command line unexpectedly.
   To terminate the command line after something like "FS$$", you will
   now have to type "FS$$$$".
 * As it is a regular command now - just executed immediately - and
   its properties stay close to the macro return behaviour, command line
   termination may now not always be performed when $$ is typed even
   as a standalone command. E.g. "Ofoo$ !bar!$$ !foo!Obar$" will
   curiously terminate the command line now.
 * This also means that macros can finally terminate command lines
   by using the command line editing commands ({ and }) to insert
   $$ into the command line macro.
   This is also of interest for function key macros.
 * This implementation showed some serious shortcoming in SciTECO's
   current parser that yet have to be fixed.
   E.g. the macro "@^Ua{&lt;$$&gt;}" is currently unsafe since
   loops abuse the expression stack for storing their state and $$
   does not touch the expression stack. Calling "Ma&gt;" would actually
   continue the loop jumping to the beginning of the command line
   since program counters referring to the macro A will be reused!
   This cannot be easily solved by checking for loop termination
   since being able to return that way from loops is a useful
   feature. This is a problem even without loops and $$, e.g. as
   in "@^Ua{1,2,3(4,5} Ma)".
   Instead, a kind of expression stack frame pointer must be
   added to macro invocation stack frames, pointing to the beginning
   of the expression stack for the current frame.
   At the end of macros or on return, the stack contents of
   corresponding to the frame can be discarded while preserving the
   immediate arguments at the time of the return or end-of-macro.
   This would stabilize SciTECO's macro semantics.
 * When a top-level macro returns in batch mode, it would
   be a good idea to use the last argument to calculate the
   process return code, so it can be set by SciTECO scripts (TODO).
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
 * &lt;$$&gt; is faster than jumping to the end of the macro
   and enables shorter code for returning values from macros.
 * this also replaces $$ as an immediate editing command.
   In other words, command line termination is an ordinary command
   now. The old behaviour was similar to what classic TECO did.
   Classic TECO however had no choice than to track key presses
   directly for command line termination as it did not keep track
   about the parser state as input was typed.
   This led to some glitches in the language. For instance
   "FS$$" would terminate the command line, unless the second escape
   was typed after backspace, etc. This behaviour is not worth copying
   and SciTECO did a better job than that by making sure that at least the
   second escape is only effective if it is not part of language syntax.
   This still lead to some undesirable cases like "ES...$$$" that would
   terminate the command line unexpectedly.
   To terminate the command line after something like "FS$$", you will
   now have to type "FS$$$$".
 * As it is a regular command now - just executed immediately - and
   its properties stay close to the macro return behaviour, command line
   termination may now not always be performed when $$ is typed even
   as a standalone command. E.g. "Ofoo$ !bar!$$ !foo!Obar$" will
   curiously terminate the command line now.
 * This also means that macros can finally terminate command lines
   by using the command line editing commands ({ and }) to insert
   $$ into the command line macro.
   This is also of interest for function key macros.
 * This implementation showed some serious shortcoming in SciTECO's
   current parser that yet have to be fixed.
   E.g. the macro "@^Ua{&lt;$$&gt;}" is currently unsafe since
   loops abuse the expression stack for storing their state and $$
   does not touch the expression stack. Calling "Ma&gt;" would actually
   continue the loop jumping to the beginning of the command line
   since program counters referring to the macro A will be reused!
   This cannot be easily solved by checking for loop termination
   since being able to return that way from loops is a useful
   feature. This is a problem even without loops and $$, e.g. as
   in "@^Ua{1,2,3(4,5} Ma)".
   Instead, a kind of expression stack frame pointer must be
   added to macro invocation stack frames, pointing to the beginning
   of the expression stack for the current frame.
   At the end of macros or on return, the stack contents of
   corresponding to the frame can be discarded while preserving the
   immediate arguments at the time of the return or end-of-macro.
   This would stabilize SciTECO's macro semantics.
 * When a top-level macro returns in batch mode, it would
   be a good idea to use the last argument to calculate the
   process return code, so it can be set by SciTECO scripts (TODO).
</pre>
</div>
</content>
</entry>
</feed>
