| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Analoguous to :EX, but always saves the file like EW$, not only if it's dirty.
|
|
|
|
|
|
as a separate command)
|
|
* We don't actually have to negate ^S results after FK.
For deleting the matched pattern, you can use ^YD or -^SD.
|
|
* this works by embedding the SciTECO parser and driving it always (exclusively)
in parse-only mode.
* A new teco_state_t::style determines the Scintilla style for any character
accepted in the given state.
* Therefore, the SciTECO lexer is always 100% exact and corresponds to the current
SciTECO grammer - it does not have to be maintained separately.
There are a few exceptions and tweaks, though.
* The contents of curly-brace escapes (`@^Uq{...}`) are rendered as ordinary
code using a separate parser instance.
This can be disabled with the lexer.sciteco.macrodef property.
Unfortunately, SciTECO does not currently allow setting lexer properties (FIXME).
* Labels and comments are currently styled the same.
This could change in the future once we introduce real comments.
* Lexers are usually implemented in C++, but I did not want to draw in C++.
Especially not since we'd have to include parser.h and other SciTECO headers,
that really do not want to keep C++-compatible.
Instead, the lexer is implemented "in the container".
@ES/SCI_SETILEXER/sciteco/ is internally translated to SCI_SETILEXER(NULL)
and we get Scintilla notifications when styling the view becomes necessary.
This is then centrally forwarded to the teco_lexer_style() which
uses the ordinary teco_view_ssm() API for styling.
* Once the command line becomes a Scintilla view even on Curses,
we can enabled syntax highlighting of the command line macro.
|
|
* As known from DEC TECO, but extended to convert absolute positions to line numbers as well.
:^Q returns the current line.
* Especially useful in macros that accept line arguments,
as it is much shorter than something like
^E@ES/LINEFROMPOSITION//+Q.l@ES/POSITIONFROMLINE//:^E-.
* On the other hand, the fact that ^Q checks the line range means we cannot
easily replace lexer.checkheader with something like
[:J 0,^Q::S...$ ]:
Using SCI_POSITIONFROMLINE still has the advantage that it returns `Z` for out-of-bounds ranges
which would be cumbersome to write with the current ^Q.
* Perhaps there should be a separate command for converting between absolute lines and positions
and :^Q should be repurposed to return a failure boolean for out-of-range values?
* fnkeys.tes could be simplified.
|
|
::FS as well)
* The colon modifier can now occur 2 times.
Specifying `@` more than once or `:` more than twice is an error now.
* Commands do not check for excess colon modifiers - almost every command would have
to check it. Instead, a double colon will simply behave like a single colon on most
commands.
* All search commands inherit the anchored semantics, but it's not very useful in some combinations
like -::S, ::N or ::FK.
That's why the `::` variants are not documented everywhere.
* The lexer.checkheader macro could be simplified and should also be faster now,
speeding up startup.
Eventually this macro can be made superfluous, e.g. by using 1:FB or 0,1^Q::S.
|
|
* Can be freely combined with the colon-modifier as well.
:@Xq cut-appends to register q.
* This simply deletes the given buffer range after the copy or append operation
as if followed by another <K> command.
* This has indeed been a very annoying missing feature, as you often have to retype the
range for a K or D command.
At the same time, this cannot be reasonably solved with a macro since macros
do not accept Q-Register arguments -- so we would have to restrict ourselves to one or a few
selected registers.
I was also considering to solve this with a special stack operation that duplicates the
top values, so that Xq leaves arguments for K, but this couldn't work for cutting lines
and would also be longer to type.
* It's the first non-string command that accepts @.
Others may follow in the future.
We're approaching ITS TECO madness levels.
|
|
lengths (refs #27)
* Allows storing pattern matches into Q-Registers (^YXq).
* You can also refer to subpatterns marked by ^E[...] by passing a number > 0.
This is equivalent to \0-9 references in many programming languages.
* It's especially useful for supporting TECO's equivalent of structural regular expressions.
This will be done with additional macros.
* You can also simply back up to the beginning of an insertion or search.
So I...$^SC leaves dot at the beginning of the insertion.
S...$^SC leaves dot before the found pattern.
This has been previously requested by users.
* Perhaps there should be ^Y string building characters as well to backreference
in search-replacement commands (TODO).
This means that the search commands would have to store the matched text itself
in teco_range_t structures since FR deletes the matched text before
processing the replacement string.
It could also be made into a FR/FS-specific construct,
so we don't fetch the substrings unnecessarily.
* This differs from DEC TECO in always returning the same range even after dot movements,
since we are storing start/end byte positions instead of only the length.
Also DEC TECO does not support fetching subpattern ranges.
|
|
* We cannot call it "." since that introduces a local register
and we don't want to add an unnecessary syntactic exception.
* Allows the idiom [: ... ]: to temporarily move around.
Also, you can now write ^E\: without having to store dot in a register first.
* In the future we might add an ^E register as well for byte offsets.
However, there are much fewer useful applications.
* Of course, you can now also write nU: instead of nJ, Q: instead of "." and
n%: instead of "nC.". However it's all not really useful.
|
|
|
|
* This would actually causes crashes when trying to format numbers.
* The ^R local register has a custom set_integer() method now,
so that the check is performed also when using nU.^X.
|
|
^R now (refs #17)
* This way the search mode and radix are local to the current macro frame,
unless the macro was invoked with :Mq.
If colon-modified, you can reproduce the same effect by calling
[.^X 0^X ... ].^X
* The radix register is cached in the Q-Reg table as an optimization.
This could be done with the other "special" registers as well, but at the
cost of larger stack frames.
* In order to allow constructs like [.^X typed with upcarets,
the Q-Register specification syntax has been extended:
^c is the corresponding control code instead of the register "^".
|
|
* Usually you will only want -^X for enabling case sensitive searches
and 0^X for case-insensitive searches (which is also the default).
* An open question is what happens if the user sets -^X and then calls
a macro. The search mode flag should probably be stacked away along
with the search-string. This means we'd need a ^X special Q-Reg as well,
so you can write [^X[_ 0^X S...$ ]_]^X.
Alternatively, the search mode flag should be a property of the
macro frame, along with the radix.
|
|
jumping to the beginning of the macro)
* I am not sure whether this feature is really that useful...
* teco_machine_main_t::macro_pc is now pointing to the __next__ character to execute,
therefore it's easier to manipulate by flow control commands.
Also, it can now be unsigned (gsize) like all other program counters.
* Detected thanks to running the testsuite under Valgrind.
|
|
current state machine
* The previous solution was not wrong, but unnecessarily complex. We already have a flag
for exactly this purpose.
* Avoid redundancies by introducing teco_machine_stringbuilding_set_codepage().
|
|
teco_machine_stringbuilding_t::codepage
* It's contained in teco_machine_main_t which is created per macro call frame.
So after macro calls, the machine no longer exists.
It is therefore unsafe to undo its members indiscriminately.
* On the other hand, we must undo the codepage setting when run interactively,
so it is now only undone when belonging to the commandline macro frame.
* This was actually causing memory corruptions on every fnkeys cursor movement, but never
caused crashes - probably because the invalid pointers are always pointing to unused
parts of the C call stack.
* Initially broken in b31b8871.
|
|
allow breaking from within braces
For instance, you can now write <23(1;)> without leaving anything on the stack.
|
|
* The XTerm version is still checked if we detect running under XTerm.
* Actually, the XTerm implementation is broken for Unicode clipboard contents.
* Kitty supports OSC-52, but you __must__ enable read-clipboard.
With read-clipboard-ask, there will be a timeout.
But we cannot read without a timeout since otherwise we would hang indefinitely
if the escape sequence turns out to not work.
* For urxvt, I have hacked an existing extension:
https://gist.github.com/rhaberkorn/d7406420b69841ebbcab97548e38b37d
* st currently supports only setting the clipboard, but not querying it.
|
|
all expansions of ^EQq, ^EUq and so on
* Previously, there was no way to enter upper-case mode in interactive commands since
the Ctrl+W immediate editing command is interpreted everywhere.
* Without the case folding of ^EQq/^EUq results, the upper and lower case modes are actually pretty useless
considering that modern keyboards have caps lock.
So it was clear we need this, regardless of what the classic TECOs did.
The TECO-11 manual is not very clear on this.
tecoc apparently does not case-fold ^EQq results.
* This opens up new idioms, for instance
`EUq^W^W^EQq$` in order to upper case register q.
It's also the only way you can currently upper-case Unicode codepoints.
|
|
* Practically requires one of the "Nerd Font" fonts,
so it's disabled by default.
Add 0,512ED to the profile to enable them.
* The new ED flag could be used to control Gtk icons as well,
but they are left always-enabled for the time being.
Is there any reason anybody would like to disable icons in Gtk?
* The list of icons has been adapted and extended from exa:
https://github.com/ogham/exa/blob/master/src/output/icons.rs
* The icons are hardcoded as presorted lists,
so we can binary search them.
This could change in the future. If there is any demand,
they could be made configurable via Q-Registers as well.
|
|
* ALL keypresses (the UTF-8 sequences resulting from key presses) can now be remapped.
* This is especially useful with Unicode support, as you might want to alias
international characters to their corresponding latin form in the start state,
so you don't have to change keyboard layouts so often.
This is done automatically in Gtk, where we have hardware key press information,
but has to be done with key macros in Curses.
There is a new key mask 4 (bit 3) for that purpose now.
* Also, you might want to define non-ANSI letters to perform special functions in
the start state where it won't be accepted by the parser anyway.
Suppose you have a macro M→, you could define
@^U[^K→]{m→} 1^_U[^K→]
This effectively "extends" the parser and allow you to call macro "→" by a single
key press. See also #5.
* The register prefix has been changed from ^F (for function) to ^K (for key).
This is the only thing you have to change in order to migrate existing
function key macros.
* Key macros are enabled by default. There is no longer any way to disable
function key handling in curses, as I never found any reason or need to disable it.
Theoretically, the default ESCDELAY could turn out to be too small and function
keys don't get through. I doubt that's possible unless on extremely slow serial lines.
Even then, you'd have to increase ESCDELAY and instead of disabling function keys
simply define an escape surrogate.
* The ED flag has been removed and its place is reserved for a future mouse support flag
(which does make sense to disable in curses sometimes).
fnkeys.tes is consequently also enabled by default in sample.teco_ini.
* Key macros are handled as an unit. If one character results in an error,
the entire string is rubbed out.
This fixes the "CLOSE" key on Gtk.
It also makes sure that the original error message is preserved and not overwritten
by some subsequent syntax error.
It was never useful that we kept inserting characters after the first error.
|
|
The following rules apply:
* All SciTECO macros __must__ be in valid UTF-8, regardless of the
the register's configured encoding.
This is checked against before execution, so we can use glib's non-validating
UTF-8 API afterwards.
* Things will inevitably get slower as we have to validate all macros first
and convert to gunichar for each and every character passed into the parser.
As an optimization, it may make sense to have our own inlineable version of
g_utf8_get_char() (TODO).
Also, Unicode glyphs in syntactically significant positions may be case-folded -
just like ASCII chars were. This is is of course slower than case folding
ASCII. The impact of this should be measured and perhaps we should restrict
case folding to a-z via teco_ascii_toupper().
* The language itself does not use any non-ANSI characters, so you don't have to
use UTF-8 characters.
* Wherever the parser expects a single character, it will now accept an arbitrary
Unicode/UTF-8 glyph as well.
In other words, you can call macros like M§ instead of having to write M[§].
You can also get the codepoint of any Unicode character with ^^x.
Pressing an Unicode character in the start state or in Ex and Fx will now
give a sane error message.
* When pressing a key which produces a multi-byte UTF-8 sequence, the character
gets translated back and forth multiple times:
1. It's converted to an UTF-8 string, either buffered or by IME methods (Gtk).
On Curses we could directly get a wide char using wget_wch(), but it's
not currently used, so we don't depend on widechar curses.
2. Parsed into gunichar for passing into the edit command callbacks.
This also validates the codepoint - everything later on can assume valid
codepoints and valid UTF-8 strings.
3. Once the edit command handling decides to insert the key into the command line,
it is serialized back into an UTF-8 string as the command line macro has
to be in UTF-8 (like all other macros).
4. The parser reads back gunichars without validation for passing into
the parser callbacks.
* Flickering in the Curses UI and Pango warnings in Gtk, due to incompletely
inserted and displayed UTF-8 sequences, are now fixed.
|
|
teco_interface_glyphs2bytes() and teco_interface_bytes2glyphs() (refs #5)
* for consistency with all the other teco_view wrappers in interface.h
|
|
* When enabled with bit 2 in the ED flags (0,4ED),
all registers and buffers will get the raw ANSI encoding (as if 0EE had been
called on them).
You can still manually change the encoding, eg. by calling 65001EE afterwards.
* Also the ANSI mode sets up character representations for all bytes >= 0x80.
This is currently done only depending on the ED flag, not when setting 0EE.
* Since setting 16,4ED for 8-bit clean editing in a macro can be tricky -
the default unnamed buffer will still be at UTF-8 and at least a bunch
of environment registers as well - we added the command line option
`--8bit` (short `-8`) which configures the ED flags very early on.
As another advantage you can mung the profile in 8-bit mode as well
when using SciTECO as a sort of interactive hex editor.
* Disable UTF-8 checks in 8-bit clean mode (sample.teco_ini).
|
|
* ^Uq however always sets an UTF8 register as the source
is supposed to be a SciTECO macro which is always UTF-8.
* :^Uq preserves the register's encoding
* teco_doc_set_string() now also sets the encoding
* instead of trying to restore the encoding in teco_doc_undo_set_string(),
we now swap out the document in a teco_doc_t and pass it to an undo token.
* The get_codepage() Q-Reg method has been removed as the same
can now be done with teco_doc_get_string() and the get_string() method.
|
|
or codepoints) (refs #5)
* This is trickier than it sounds because there isn't one single place to consult.
It depends on the context.
If the string argument relates to buffer contents - as in <I>, <S>, <FR> etc. -
the buffer's encoding is consulted.
If it goes into a register (EU), the register's encoding is consulted.
Everything else (O, EN, EC, ES...) expects only Unicode codepoints.
* This is communicated through a new field teco_machine_stringbuilding_t::codepage
which must be set in the states' initial callback.
* Seems overkill just for ^EUq, but it can be used for context-sensitive
processing of all the other string building constructs as well.
* ^V and ^W cannot be supported for Unicode characters for the time being without an Unicode-aware parser
|
|
|
|
* This will naturally work with both ASCII characters and various
non-English scripts.
* Unfortunately, it cannot work with the other non-ANSI single-byte codepages.
* If we'd like to support scripts working with all sorts of codepoints,
we'd have to introduce a new command for translating individual codepoints
from the current codepage (as reported by EE) to Unicode.
|
|
* I decoded the Scintilla charset values into codepages, at least
those used on Gtk.
* make sure that the line character index is not allocated or released
too often, as it is actually internally reference counted, which
could result in it missing when we really need it.
* The line character index still appears to be released whenever
the document pointer changes, which will happen after using
a different Q-Register.
This could be a performance bottleneck (FIXME).
|
|
teco_state_start_get() (refs #5)
|
|
There is a widespread myth that they could take up to 6 bytes.
|
|
between glyph and byte offsets (refs #5)
* ^E is heavily overloaded and can also be used to check whether a given index is valid
(as it is the same that most movement commands to internally).
Besides that, it is mainly useful for interfacing with Scintilla messages.
* EE takes a code page or 0 for ANSI/ASCII.
Currently all documents and new registers are UTF-8.
There will have to be some kind of codepage inheritance and a single-byte-only mode.
|
|
* this required adding several Q-Register vtable methods
* it should still be investigated whether the repeated calling of
SCI_ALLOCATELINECHARACTERINDEX causes any overhead.
|
|
* This works reasonably well unless lines are exceedingly long
(as on a line we always count characters).
The following test case is still slow (on Unicode buffers):
10000<@I/XX/> <%a-1:J;>
While the following is now also fast:
10000<@I/X^J/> <%a-1:J;>
* Commands with relative character offsets (C, R, A, D) have
a special optimization where they always count characters beginning
at dot, as long as the argument is now exceedingly large.
This means they are fast even on exceedingly long lines.
* The remaining commands (search, EC/EG, Xq) now accept glyph indexes.
|
|
(refs #5)
certain test cases are still way too slow:
10000<@I/X^J/> 20000<R>
or
10000<@I/X^J/> 20000<%a-1J>
SCI_ALLOCATELINECHARACTERINDEX does not help much here.
It probably speeds up only SCI_LINEFROMINDEXPOSITION and SCI_INDEXPOSITIONFROMLINE.
|
|
unsigned integer
* SCI_GETCHARAT is internally casted to `char`, which may be signed.
Characters > 127 therefore become negative and stay so when casted to sptr_t.
We therefore cast it back to guchar (unsigned char).
* The same is true whenever returning a string's character to SciTECO (teco_int_t)
as our string type is `gchar *`.
* <^^x> now also works for those characters.
Eventually, the parser will probably become UTF8-aware and this will
have to be done differently.
|
|
* It's generally a bad idea to pass backslashes as a glyph in macro arguments, even as `\\`
since this could easily be interpreted as an escape.
* Instead we now always use `\[rs]`.
|
|
* Horizontal movements (left/right cursor keys) establish the current column
and vertical movements (up/down) will try to keep on that column.
* This has long been problematic in SciTECO as it requires state that gets
reversed when the command line replacement takes place.
* I experimented with encoding the current horizontal position into the braced
movement operations as in (123C5U$), but I decided that this was clumsy and
I generally did not want these expressions to become even larger.
* Instead I decided to add some minimal support to the C core in the form of 4EJ
which is like a number register only that it does NOT get reversed on rubout.
This is exploited by the fnkeys.tes macros by storing the current position
beyond replacements.
* In theory, this should be a property of the document, but we cannot easily
store custom parameters per document.
So instead, there is just one global variable.
When editing another buffer, it gets reset to .ESGETCOLUMN$$.
sample.teco_ini has been updated.
* The current X position only makes sense in the context of fnkeys.tes, as
TECO commands like <C> are not necessarily "horizonal" movements.
For the same reason, the core does not try to initialize 4EJ automatically
when editing new buffers.
It's entirely left to the TECO macros.
* The commandline replacement is more robust now as it checks braced
expressions at the end of the command line more thorougly.
It will no longer swallow all preceding braced expressions.
Only if they are at least 4 characters in length and end in `C)` or `R)`.
|
|
|
|
Scintilla now
* The patch avoids all automatic scrolling consistently, including in SCI_UNDO.
This speads up Undo (especially after interruptions).
* Also, the patch disables a very costly and pointless (in SciTECO) algorithm that
effectively made <Ix$> uninterruptible.
* Effectively reverts large parts of 8ef010da59743fcc4927c790f585ba414ec7b129.
I have never liked using unintuitive Scintilla messages to avoid scrolling.
|
|
|
|
* fixes test cases like 3<%a:>
* you can now use :F< in pass-through loops as well
* F> outside of loops will now exit the current macro level.
This is analogous to what TECO-11 did.
In interactive mode, F> is currently also equivalent to $$
(terminates command line).
|
|
registers
* An empty but valid teco_string_t can contain NULL pointers.
More precisely, a state's done_cb() can be invoked with such empty strings
in case of empty string arguments.
Also a registers get_string() can return the NULL pointer
for existing registers with uninitialized string parts.
* In all of these cases, the language should treat "uninitialized" strings
exactly like empty strings.
* Not doing so, resulted in a number of vulnerabilities.
* EN$$ crashed if "_" was uninitialized
* The ^E@q and ^ENq string building constructs would crash for existing but
uninitialized registers q.
* ?$ would crash
* ESSETILEXER$$ would crash
* This is now fixed.
Test cases have been added.
* I cannot guarantee that I have found all such cases.
Generally, it might be wise to change our definitions and make sure that
every teco_string_t must have an associated heap object to be valid.
All functions returning pointer+length pairs should consequently also never
return NULL pointers.
|
|
|
|
|
|
* avoid emitting SCI_UNDO undo tokens if the Scintilla undo action would actually be empty
|
|
once per keypress
* Esp. costly since Scintilla 5.
* We now avoid any Scintilla message that automatically scrolls the caret (makes the
caret visible) and instead call SCI_SCROLLCARET only once after every keypress in the
interface implementation.
* From nowon, use
* SCI_SETEMPTYSELECTION instead of SCI_GOTOPOS
* SCI_SETEMPTYSELECTION(SCI_POSITIONFROMLINE(...)) instead of SCI_GOTOLINE
* SCI_SETSELECTIONSTART and SCI_SETSELECTIONEND instead of SCI_SETSEL
* With these optimizations we are significantly faster than before
the Scintilla upgrade (6e67f5a682ff46d69888fec61b94bf45cec46721).
It is now even safe to execute the Gtk test suite during CI.
|
|
file systems
* There is a "Scintilla.h" as well.
* should fix macOS and builds on native Windows hosts
* It wasn't practical to refer to the Scintilla includes using paths since
the Scintilla location is configurable (--with-scintilla).
So we'd have to write something like #include <include/Scintilla.h>.
For Scinterm we cannot avoid collisions neither as its path is also
configurable (--with-scinterm).
Effectively, we must prevent name clashes across SciTECO and all
of Scintilla and Scinterm.
|
