| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
statement
* this is a SciTECO extension - it's not in TECO-11
* Allows for select-case-like constructs with default-clauses as in
:Os.^EQa$
!* default *!
!s.foo!
!* ... *!
!s.bar!
!* ... *!
* Consistent with nOlabel0,label1,...$ if <n> is out of range.
Unfortunately this form of computed goto is not applicable when
"selecting" by strings or non-consecutive integers.
* In order to continue after the <:O> statement, we must keep the
program counter along with the label we were looking for.
At the end of the macro, the PC is restored instead of throwing
an error.
* Since that would be very inefficient in loops - where potentially
all iterations would result in rescanning till the end of the
macro - we now store a completed-flag in the goto table.
If it is set while trying to :O to an unknown label, we can
just continue execution.
|
|
stdin or the user
* n:^T always prints bytes (cf. :^A)
* ^T without arguments returns a codepoint or byte from stdin.
In interactive mode, this currentply places a cursor in the message line and waits for a keypress.
|
|
* regression introduced in 2baa14add6d9976c29b27cf4470bb458a0198694
|
|
document's boundaries
|
|
* Instead of separate stand-alone commands, they are now allowed only immediately
in front of the commands that accept them.
* The order is still insignificant if both `@` and `:` are accepted.
* The number of colon modifiers is now also checked.
We basically get this for free.
* `@` has syntactic significance, so it could not be set conditionally anyway.
Still, it was possible to provoke bugs were `@` was interpreted conditionally
as in `@ 2<I/foo/$>`.
* Even when not causing bugs, a mistyped `@` would often influence the
__next__ command, causing unexpected behavior, for instance when
typing `@(233C)W`.
* While it was theoretically possible to set `:` conditionally, it could also
be "passed through" accidentally to some command where it wasn't expected as in
`:Ifoo$ C`.
I do not know of any real useful application or idiom of a conditionally set `:`.
If there would happen to be some kind of useful application, `:'` and `:|` could
be re-allowed easily, though.
* I was condidering introducing a common parser state for modified commands,
but that would have been tricky and introduce a lot of redundant command lists.
So instead, we now simply everywhere check for excess modifiers.
To simplify this task, teco_machine_main_transition_t now contains flags
signaling whether the transition is allowed with `@` or `:` modifiers set.
It currently only has to be checked in the start state, after `E` and `F`.
|
|
|
|
* @ES/SCI_SETILEXER/lib^@name/ now opens the lexer <name> in library <lib>.
* You need to define the environment variable $SCITECO_SCINTILLUA_LEXERS to point
to the lexers/ subdirectory (containing the *.lua files).
Perhaps this should default to the dirname of <lib>?
* The semantics of SCI_NAMEOFSTYLE have been changed:
It now returns style ids when given style names, so you can actually write
Scintillua lexer *.tes files.
This will be superfluous if we had a way to return strings from Scintilla messages into
Q-Registers, e.g. 23@EPq/SCI_NAMEOFSTYLE/.
* We now depend on gmodule as well, but it should always be part of glib.
It does not change the library dependencies of any package.
It might result in gmodule shared libraries to be bundled in the Win32 and Mac OS
packages if they weren't already.
|
|
::FS as well)
* The colon modifier can now occur 2 times.
Specifying `@` more than once or `:` more than twice is an error now.
* Commands do not check for excess colon modifiers - almost every command would have
to check it. Instead, a double colon will simply behave like a single colon on most
commands.
* All search commands inherit the anchored semantics, but it's not very useful in some combinations
like -::S, ::N or ::FK.
That's why the `::` variants are not documented everywhere.
* The lexer.checkheader macro could be simplified and should also be faster now,
speeding up startup.
Eventually this macro can be made superfluous, e.g. by using 1:FB or 0,1^Q::S.
|
|
lengths (refs #27)
* Allows storing pattern matches into Q-Registers (^YXq).
* You can also refer to subpatterns marked by ^E[...] by passing a number > 0.
This is equivalent to \0-9 references in many programming languages.
* It's especially useful for supporting TECO's equivalent of structural regular expressions.
This will be done with additional macros.
* You can also simply back up to the beginning of an insertion or search.
So I...$^SC leaves dot at the beginning of the insertion.
S...$^SC leaves dot before the found pattern.
This has been previously requested by users.
* Perhaps there should be ^Y string building characters as well to backreference
in search-replacement commands (TODO).
This means that the search commands would have to store the matched text itself
in teco_range_t structures since FR deletes the matched text before
processing the replacement string.
It could also be made into a FR/FS-specific construct,
so we don't fetch the substrings unnecessarily.
* This differs from DEC TECO in always returning the same range even after dot movements,
since we are storing start/end byte positions instead of only the length.
Also DEC TECO does not support fetching subpattern ranges.
|
|
terminating process groups on Win32
* Sometimes already the job assignment failed in CI builds.
We now check whether the process is still alive before throwing an error.
* We now set the JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE flag.
This theoretically shouldn't be necessary when using TerminateJobObject(),
but who knows.
|
|
* added TECO_ERROR_CLIPBOARD for all clipboard-related errors
|
|
characters
* Some characters like LF wouldn't be displayed in the message line correctly.
* In fact the Gtk UI cannot display any of the control characters correctly.
* I was considering deferring all echoing/formatting to the UIs, so they can use
TecoGtkLabel or teco_curses_format_str().
This is not possible since messages transmitted via GError must not contain null-bytes,
so these need to be sorted out earlier anyway.
* This should also fix syntax errors in PDCurses for Windows where "%C" apparently doesn't
work with non-ANSI codepoints.
|
|
* This was unsafe and could easily result in crashes, since teco_qreg_current
would afterwards point to an already freed Q-Register.
* Since automatically editing another register or buffer is not easy to do right,
we throw an error instead.
|
|
* This is used for error messages (TECO macro stackframes),
so it's important to display columns in characters.
* Program counters are in bytes and therefore everywhere gsize.
This is by glib convention.
|
|
The following rules apply:
* All SciTECO macros __must__ be in valid UTF-8, regardless of the
the register's configured encoding.
This is checked against before execution, so we can use glib's non-validating
UTF-8 API afterwards.
* Things will inevitably get slower as we have to validate all macros first
and convert to gunichar for each and every character passed into the parser.
As an optimization, it may make sense to have our own inlineable version of
g_utf8_get_char() (TODO).
Also, Unicode glyphs in syntactically significant positions may be case-folded -
just like ASCII chars were. This is is of course slower than case folding
ASCII. The impact of this should be measured and perhaps we should restrict
case folding to a-z via teco_ascii_toupper().
* The language itself does not use any non-ANSI characters, so you don't have to
use UTF-8 characters.
* Wherever the parser expects a single character, it will now accept an arbitrary
Unicode/UTF-8 glyph as well.
In other words, you can call macros like M§ instead of having to write M[§].
You can also get the codepoint of any Unicode character with ^^x.
Pressing an Unicode character in the start state or in Ex and Fx will now
give a sane error message.
* When pressing a key which produces a multi-byte UTF-8 sequence, the character
gets translated back and forth multiple times:
1. It's converted to an UTF-8 string, either buffered or by IME methods (Gtk).
On Curses we could directly get a wide char using wget_wch(), but it's
not currently used, so we don't depend on widechar curses.
2. Parsed into gunichar for passing into the edit command callbacks.
This also validates the codepoint - everything later on can assume valid
codepoints and valid UTF-8 strings.
3. Once the edit command handling decides to insert the key into the command line,
it is serialized back into an UTF-8 string as the command line macro has
to be in UTF-8 (like all other macros).
4. The parser reads back gunichars without validation for passing into
the parser callbacks.
* Flickering in the Curses UI and Pango warnings in Gtk, due to incompletely
inserted and displayed UTF-8 sequences, are now fixed.
|
|
(refs #5)
certain test cases are still way too slow:
10000<@I/X^J/> 20000<R>
or
10000<@I/X^J/> 20000<%a-1J>
SCI_ALLOCATELINECHARACTERINDEX does not help much here.
It probably speeds up only SCI_LINEFROMINDEXPOSITION and SCI_INDEXPOSITIONFROMLINE.
|
|
|
|
|
|
|
|
This is a total conversion of SciTECO to plain C (GNU C11).
The chance was taken to improve a lot of internal datastructures,
fix fundamental bugs and lay the foundations of future features.
The GTK user interface is now in an useable state!
All changes have been squashed together.
The language itself has almost not changed at all, except for:
* Detection of string terminators (usually Escape) now takes
the string building characters into account.
A string is only terminated outside of string building characters.
In other words, you can now for instance write
I^EQ[Hello$world]$
This removes one of the last bits of shellisms which is out of
place in SciTECO where no tokenization/lexing is performed.
Consequently, the current termination character can also be
escaped using ^Q/^R.
This is used by auto completions to make sure that strings
are inserted verbatim and without unwanted sideeffects.
* All strings can now safely contain null-characters
(see also: 8-bit cleanliness).
The null-character itself (^@) is not (yet) a valid SciTECO
command, though.
An incomplete list of changes:
* We got rid of the BSD headers for RB trees and lists/queues.
The problem with them was that they used a form of metaprogramming
only to gain a bit of type safety. It also resulted in less
readble code. This was a C++ desease.
The new code avoids metaprogramming only to gain type safety.
The BSD tree.h has been replaced by rb3ptr by Jens Stimpfle
(https://github.com/jstimpfle/rb3ptr).
This implementation is also more memory efficient than BSD's.
The BSD list.h and queue.h has been replaced with a custom
src/list.h.
* Fixed crashes, performance issues and compatibility issues with
the Gtk 3 User Interface.
It is now more or less ready for general use.
The GDK lock is no longer used to avoid using deprecated functions.
On the downside, the new implementation (driving the Gtk event loop
stepwise) is even slower than the old one.
A few glitches remain (see TODO), but it is hoped that they will
be resolved by the Scintilla update which will be performed soon.
* A lot of program units have been split up, so they are shorter
and easier to maintain: core-commands.c, qreg-commands.c,
goto-commands.c, file-utils.h.
* Parser states are simply structs of callbacks now.
They still use a kind of polymorphy using a preprocessor trick.
TECO_DEFINE_STATE() takes an initializer list that will be
merged with the default list of field initializers.
To "subclass" states, you can simply define new macros that add
initializers to existing macros.
* Parsers no longer have a "transitions" table but the input_cb()
may use switch-case statements.
There are also teco_machine_main_transition_t now which can
be used to implement simple transitions. Additionally, you
can specify functions to execute during transitions.
This largely avoids long switch-case-statements.
* Parsers are embeddable/reusable now, at least in parse-only mode.
This does not currently bring any advantages but may later
be used to write a Scintilla lexer for TECO syntax highlighting.
Once parsers are fully embeddable, it will also be possible
to run TECO macros in a kind of coroutine which would allow
them to process string arguments in real time.
* undo.[ch] still uses metaprogramming extensively but via
the C preprocessor of course. On the downside, most undo
token generators must be initiated explicitly (theoretically
we could have used embedded functions / trampolines to
instantiate automatically but this has turned out to be
dangereous).
There is a TECO_DEFINE_UNDO_CALL() to generate closures for
arbitrary functions now (ie. to call an arbitrary function
at undo-time). This simplified a lot of code and is much
shorter than manually pushing undo tokens in many cases.
* Instead of the ridiculous C++ Curiously Recurring Template
Pattern to achieve static polymorphy for user interface
implementations, we now simply declare all functions to
implement in interface.h and link in the implementations.
This is possible since we no longer hace to define
interface subclasses (all state is static variables in
the interface's *.c files).
* Headers are now significantly shorter than in C++ since
we can often hide more of our "class" implementations.
* Memory counting is based on dlmalloc for most platforms now.
Unfortunately, there is no malloc implementation that
provides an efficient constant-time memory counter that
is guaranteed to decrease when freeing memory.
But since we use a defined malloc implementation now,
malloc_usable_size() can be used safely for tracking memory use.
malloc() replacement is very tricky on Windows, so we
use a poll thread on Windows. This can also be enabled
on other supported platforms using --disable-malloc-replacement.
All in all, I'm still not pleased with the state of memory
limiting. It is a mess.
* Error handling uses GError now. This has the advantage that
the GError codes can be reused once we support error catching
in the SciTECO language.
* Added a few more test suite cases.
* Haiku is no longer supported as builds are instable and
I did not manage to debug them - quite possibly Haiku bugs
were responsible.
* Glib v2.44 or later are now required.
The GTK UI requires Gtk+ v3.12 or later now.
The GtkFlowBox fallback and sciteco-wrapper workaround are
no longer required.
* We now extensively use the GCC/Clang-specific g_auto
feature (automatic deallocations when leaving the current
code block).
* Updated copyright to 2021.
SciTECO has been in continuous development, even though there
have been no commits since 2018.
* Since these changes are so significant, the target release has
been set to v2.0.
It is planned that beginning with v3.0, the language will be
kept stable.
|
|
|
|
* we were basing the glib allocators on throwing std::bad_alloc just like
the C++ operators. However, this always was unsafe since we were throwing
exceptions across plain-C frames (Glib).
Also, the memory vtable has been deprecated in Glib, resulting in
ugly warnings.
* Instead, we now let the C++ new/delete operators work like Glib
by basing them on g_malloc/g_slice.
This means they will assert and the application will terminate
abnormally in case of OOM. OOMs cannot be handled properly anyway, so it is
more important to have a good memory limiting mechanism.
* Memory limiting has been completely revised.
Instead of approximating undo stack sizes using virtual methods
(which is unprecise and comes with a performance penalty),
we now use a common base class SciTECO::Object to count the memory
required by all objects allocated within SciTECO.
This is less precise than using global replacement new/deletes
which would allow us to control allocations in all C++ code including
Scintilla, but they are only supported as of C++14 (GCC 5) and adding compile-time
checks would be cumbersome.
In any case, we're missing Glib allocations (esp. strings).
* As a platform-specific extension, on Linux/glibc we use mallinfo()
to count the exact memory usage of the process.
On Windows, we use GetProcessMemoryInfo() -- the latter implementation
is currently UNTESTED.
* We use g_malloc() for new/delete operators when there is
malloc_trim() since g_slice does not free heap chunks properly
(probably does its own mmap()ing), rendering malloc_trim() ineffective.
We've also benchmarked g_slice on Linux/glib (malloc_trim() shouldn't
be available elsewhere) and found that it brings no significant
performance benefit.
On all other platforms, we use g_slice since it is assumed
that it at least does not hurt.
The new g_slice based allocators should be tested on MSVCRT
since I assume that they bring a significant performance benefit
on Windows.
* Memory limiting does now work in batch mode as well and is still
enabled by default.
* The old UndoTokenWithSize CRTP hack could be removed.
UndoStack operations should be a bit faster now.
But on the other hand, there will be an overhead due to repeated
memory limit checking on every processed character.
|
|
* mapped to different registers beginning with "~"
* on supported platforms accessing the clipboard is as easy as
X~ or G~.
Naturally this also allows clipboards to be pasted in
string arguments/insertions (^EQ~).
* Currently, Gtk+, PDCurses and ncurses/XTerm are supported.
For XTerm clipboard support, users must set 0,256ED to enable
it since we cannot check for XTerm window ops programmatically
(at least without libX11).
* When clipboard regs exist, the clipboard can also be deemed functional.
This allows macros to fall back to xclip(1) if necessary.
* EOL handling has been moved into a new file eol.c and eol.h.
EOL translation no longer depends on GIOChannels but can be
memory-backed as well.
|
|
optimizationa and additional checks
* undo tokens emitted by the expression stack no longer waste
memory by pointing to the stack implementation.
This uses some ugly C++ constant template arguments but
saves 4 or 8 byte per undo token depending on the architecture.
* Round braces are counted now and the return command $$ will
use this information to discard all non-relevant brace levels.
* It is an error to close a brace when none have been opened.
* The bracing rules are still very liberal, allowing you to
close braces in macros belonging to a higher call frame
or leave them open at the end of a macro.
While this is technically possible, it is perhaps a good
idea to stricten these rules in some future release.
* Loops no longer (ab)use the expression stack to store
program counters and loop counters.
This removes flow control from the responsibility of the
expression stack which is much safer now since we can control
where we jump to.
This also eased implemented proper semantics for $$.
* It is an error to leave loops open at the end of a macro
or trying to close a loop opened in the caller of the macro.
Similarily it is only possible to close a loop from the
current invocation frame.
This means it is now impossible to accidentally jump to invalid
PCs.
* Even though loop context stacks could be attached directly to
the macro invocation frame, this would be inefficient.
Instead there's a loop frame pointer now that is part of the
invocation frame. All frames will reuse the same stack structure.
* Loops are automatically discarded when returning using $$.
* Special aggregating forms of the loop start (":<") and loop
end (":>") commands are possible now and have been implemented.
This improves SciTECO's capability as a stack-oriented language.
It is no longer necessary to write recursive macros to generate
stack values of arbitrary length dynamically or to process them.
* All expression and loop stacks are still fixed-size.
It may be a good idea to implement dynamic resizing (TODO).
* Added some G_UNLIKELYs to Execute::macro(). Should improve
the branch prediction of modern CPUs.
* Local Q-Register tables are allocated on the stack now instead
of on the heap (the bulk of a table is stored on the heap anyway).
Should improve performance of macro invocations.
* Document that "F<" will jump to the beginning of the macro
if there is no loop.
This is not in standard TECO, but I consider it a useful feature.
|
|
* <$$> is faster than jumping to the end of the macro
and enables shorter code for returning values from macros.
* this also replaces $$ as an immediate editing command.
In other words, command line termination is an ordinary command
now. The old behaviour was similar to what classic TECO did.
Classic TECO however had no choice than to track key presses
directly for command line termination as it did not keep track
about the parser state as input was typed.
This led to some glitches in the language. For instance
"FS$$" would terminate the command line, unless the second escape
was typed after backspace, etc. This behaviour is not worth copying
and SciTECO did a better job than that by making sure that at least the
second escape is only effective if it is not part of language syntax.
This still lead to some undesirable cases like "ES...$$$" that would
terminate the command line unexpectedly.
To terminate the command line after something like "FS$$", you will
now have to type "FS$$$$".
* As it is a regular command now - just executed immediately - and
its properties stay close to the macro return behaviour, command line
termination may now not always be performed when $$ is typed even
as a standalone command. E.g. "Ofoo$ !bar!$$ !foo!Obar$" will
curiously terminate the command line now.
* This also means that macros can finally terminate command lines
by using the command line editing commands ({ and }) to insert
$$ into the command line macro.
This is also of interest for function key macros.
* This implementation showed some serious shortcoming in SciTECO's
current parser that yet have to be fixed.
E.g. the macro "@^Ua{<$$>}" is currently unsafe since
loops abuse the expression stack for storing their state and $$
does not touch the expression stack. Calling "Ma>" would actually
continue the loop jumping to the beginning of the command line
since program counters referring to the macro A will be reused!
This cannot be easily solved by checking for loop termination
since being able to return that way from loops is a useful
feature. This is a problem even without loops and $$, e.g. as
in "@^Ua{1,2,3(4,5} Ma)".
Instead, a kind of expression stack frame pointer must be
added to macro invocation stack frames, pointing to the beginning
of the expression stack for the current frame.
At the end of macros or on return, the stack contents of
corresponding to the frame can be discarded while preserving the
immediate arguments at the time of the return or end-of-macro.
This would stabilize SciTECO's macro semantics.
* When a top-level macro returns in batch mode, it would
be a good idea to use the last argument to calculate the
process return code, so it can be set by SciTECO scripts (TODO).
|
|
|
|
appending to "$"
* these operations are unsupported and there is no benefit
in ignoring them silently. It only confused the user.
|
|
cleanup/refactoring
* characters rubbed out are not totally removed from the command line,
but only from the *effective* command line.
* The rubbed out command line is displayed after the command line cursor.
On Curses it is grey and underlined.
* When characters are inserted that are on the rubbed out part of the command line,
the cursor simply moves forward.
NOTE: There's currently no immediate editing command for reinserting the
next character/word from the rubbed out command line.
* Characters resulting in errors are no longer simply discarded but rubbed out,
so they will stay in the rubbed out part of the command line, reminding you
which character caused the error.
* Improved Cmdline formatting on Curses UI:
* Asterisk is printed bold
* Control characters are printed in REVERSE style, similar to what
Scinterm does. The controll character formatting has thus been moved
from macro_echo() in cmdline.cpp to the UI implementations.
* Updated the GTK+ UI (UNTESTED): I did only, the most important API
adaptions. The command line still does not use any colors.
* Refactored entire command line handling:
* The command line is now a class (Cmdline), and most functions
in cmdline.cpp have been converted to methods.
* Esp. process_edit_cmd() (now Cmdline::process_edit_cmd()) has been
simplified. There is no longer the possibility of a buffer overflow
because of static insertion buffer sizes
* Cleaned up usage of the cmdline_pos variable (now Cmdline::pc) which
is really a program counter that used a different origin as macro_pc
which was really confusing.
* The new Cmdline class is theoretically 8-bit clean. However all of this
will change again when we introduce Scintilla views for the command line.
* Added 8-bit clean (null-byte aware) versions of QRegisterData::set_string()
and QRegisterData::append_string()
|
|
string-utils.cpp and string-utils.h
* also improved performance of String::append() by using g_realloc()
* added String::append() variant for non-null-terminated strings
|
|
|
|
* when throwing GlibError(), this is taken care of automatically.
* fixes a memleak since there may be resources associated with the
GError.
|
|
the QUIT hook is actually not that trivial and required
some architectural changes.
First, the QUIT hook execution and any error that might
occurr cannot always be attached to an existing error stack
frame. Thereforce, to give a stack frame for QUIT hooks and
to improve the readability of error traces for ED hooks in general,
a special EDHookFrame is added to every ED hook execution error.
Secondly, since QUIT hooks can themselves throw errors, we cannot
run it from an atexit() handler. Instead it's always called manually
before __successful__ program termination. An error in a QUIT hook
will result in a failure return code nevertheless.
Thirdly, errors in QUIT hooks should not prevent program termination
(in interactive mode), therefore they are only invoked from main()
and always in batch mode. I.e. if the interactive mode is terminated
(EX$$), SciTECO will switch back to batch mode and run the QUIT
hook there. This is also symmetric to program startup, which is
always in batch mode.
This means that Interface::event_loop() no longer runs indefinitely.
If it returns, this signals that the interface shut down and
batch mode may be restored by SciTECO.
|
|
introduced Error::set_coord()
|
|
* the GError expection has been renamed to GlibError, to avoid
nameclashes when working from the SciTECO namespace
|
