implemented Scintilla lexer for SciTECO code, i.e. TECO syntax highlighting

* this works by embedding the SciTECO parser and driving it always (exclusively)
  in parse-only mode.
* A new teco_state_t::style determines the Scintilla style for any character
  accepted in the given state.
* Therefore, the SciTECO lexer is always 100% exact and corresponds to the current
  SciTECO grammer - it does not have to be maintained separately.
  There are a few exceptions and tweaks, though.
* The contents of curly-brace escapes (`@^Uq{...}`) are rendered as ordinary
  code using a separate parser instance.
  This can be disabled with the lexer.sciteco.macrodef property.
  Unfortunately, SciTECO does not currently allow setting lexer properties (FIXME).
* Labels and comments are currently styled the same.
  This could change in the future once we introduce real comments.
* Lexers are usually implemented in C++, but I did not want to draw in C++.
  Especially not since we'd have to include parser.h and other SciTECO headers,
  that really do not want to keep C++-compatible.
  Instead, the lexer is implemented "in the container".
  @ES/SCI_SETILEXER/sciteco/ is internally translated to SCI_SETILEXER(NULL)
  and we get Scintilla notifications when styling the view becomes necessary.
  This is then centrally forwarded to the teco_lexer_style() which
  uses the ordinary teco_view_ssm() API for styling.
* Once the command line becomes a Scintilla view even on Curses,
  we can enabled syntax highlighting of the command line macro.

1 files changed, 9 insertions, 5 deletions

diff --git a/TODO b/TODO
index cacb782..5692c48 100644
--- a/TODO
+++ b/TODO
@@ -106,6 +106,8 @@ Known Bugs:
    This feature would especially be important in order to support
    e.g. the SCI_SETPROPERTY message with two strings in order to
    set lexer properties.
+ * The lexer.sciteco.macrodef lexer property cannot currently
+   be set/disabled (see above).
  * Mac OS: The colors are screwed up with the terminal.tes color scheme
    (and with --no-profile) under Mac OS terminal emulators.
    This does not happen under Linux with Darling.
@@ -144,6 +146,7 @@ Known Bugs:
    files around, it does not produce a core dump.
 
 Features:
+ * Perhaps ^S should return a positive number after <FK>.
  * Auto-indention could be implemented via context-sensitive
    immediate editing commands similar to tab-expansion.
    Avoids having to make LF a magic character in insertion
@@ -202,8 +205,8 @@ Features:
      integer and floating point types internally.
      The operator decides how to interpret the arguments
      and the return type.
-   * Having a separate number parser state will simplify
-     number syntax highlighting.
+   * Having a separate number parser state will slightly simplify
+     number syntax highlighting (see teco_lexer_getstyle())..
  * Key macro masking flag for the beginning of the command
    line. May be useful e.g. for solarized's F5 key (i.e. function
    key macros that need to terminate the command line as they
@@ -346,9 +349,6 @@ Features:
  * Command to free Q-Register (remove from table).
    e.g. FQ (free Q). :FQ could free by QRegister prefix name for
    the common use case of Q-Register subtables and lists.
- * TECO syntax highlighting.
-   This should now be relatively easy to implement by reusing
-   the parser.
  * multiline commandline
    * Perhaps use Scintilla view as mini buffer.
      This means patching Scintilla, so it does not break lines
@@ -591,6 +591,10 @@ Features:
    So --1 == 1 and --C == C.
    There is probably no need for this "feature" and an error
    should be thrown instead on the second `-`.
+ * The command line replacement register (Escape) should be automatically
+   syntax highlighted.
+   I am not sure however how that could be done without a special
+   ED hook.
 
 Optimizations:
  * Use SC_DOCUMENTOPTION_STYLES_NONE in batch mode.