| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
* should also fix Win32 nightly builds
* Even though we weren't using main's argv, but were using glib
API for retrieving the command line in UTF-8, newer MinGW runtimes
would fail when converting the Unicode command line into the system codepage
would be lossy.
* Most people seem to compile in a "manifest" to work around this issue.
But this requires newer Windows versions and using some Microsoft tool which isn't
even in $PATH.
Instead, we now link with -municode and define wmain() instead, even though we still
ignore argv. wmain() proabably get's the command line in UTF-16 and we'd have to
convert it anyway.
* See https://github.com/msys2/MINGW-packages/issues/22462
|
|
restrictions
|
|
|
|
* You can now specify `--with-scitecodatadir` as a relative path,
that will be interpreted relative to the binary's location.
* Win32 binaries already were relocatable, but this was a Windows-specific
hack. Win32 binaries are now built with `--with-scitecodatadir=.`
since everything is in a single directory.
* Ubuntu packages are now also built `--with-scitecodatadir=../share/sciteco`.
This is not crucial for ordinary installations, but is meant for AppImage creation.
* Since AppImages are now built from relocatable packages,
we no longer need the unionfs-workaround from pkg2appimage.
This should fix the strange root contents when autocompleting in
AppImage builds.
* This might also fix the appimage.github.io CI issues.
I assume that because I could reproduce the issue on FreeBSD's
Linuxulator in dependence of pkg2appimage's "union"-setting.
See https://github.com/AppImage/appimage.github.io/pull/3402
* Determining the binary location actually turned out be hard and
very platform-dependant. There are now implementations for Windows
(which could also read argv[0]), Linux and generic UNIX (which
works on FreeBSD, but I am not sure about the others).
I believe this could also be useful on Mac OS to create app bundles,
but this needs to be tested - currently the Mac OS binaries are
installed into fixed locations and don't use relocation.
|
|
|
|
* Supports all immediate editing commands.
Naturally it cannot emulate arbitrary key presses since there is no
canonic ASCII-encoding of function keys.
Key macros are not consequently also not testable.
The --fake-cmdline parameter is instead treated very similar to
a key macro expansion.
* Most importantly this allows adding test cases for rubout behavior
and bugs that are quite common.
* Added regression test cases for the last two rubout bugs.
* It's not easy to pass control codes in command line arguments in
a portable manner, so the test cases will often use { and }.
Control codes could be used e.g. by defining variables like
RUBOUT=`printf '\b'`
and referencing them with ${RUBOUT}.
|
|
* We now recreate the event loop with every call since
it turned out that the idle watcher wouldn't be invoked
after the event loop has been quit once.
This at least fixes interruption of ECbash -c 'while true; do true; done'$.
* Unfortunately, ECping -t 8.8.8.8$ still cannot be interrupted
(unless you manually kill the process from the task manager).
|
|
terminating process groups on Win32
* Sometimes already the job assignment failed in CI builds.
We now check whether the process is still alive before throwing an error.
* We now set the JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE flag.
This theoretically shouldn't be necessary when using TerminateJobObject(),
but who knows.
|
|
* For instance, when using the Solarized color scheme,
comments will be in italics.
* The underline style could also be supported in color.set,
but currently it's not required.
* Unfortunately, this does *not* fix the INDIC_PLAIN indicator style for
underlining, so you still cannot use that eg. for spell checking.
* See https://github.com/orbitalquark/scinterm/issues/22
|
|
|
|
all expansions of ^EQq, ^EUq and so on
* Previously, there was no way to enter upper-case mode in interactive commands since
the Ctrl+W immediate editing command is interpreted everywhere.
* Without the case folding of ^EQq/^EUq results, the upper and lower case modes are actually pretty useless
considering that modern keyboards have caps lock.
So it was clear we need this, regardless of what the classic TECOs did.
The TECO-11 manual is not very clear on this.
tecoc apparently does not case-fold ^EQq results.
* This opens up new idioms, for instance
`EUq^W^W^EQq$` in order to upper case register q.
It's also the only way you can currently upper-case Unicode codepoints.
|
|
* Ctrl+^ (30) and Caret+caret (^^) were both translated to a single caret.
While there might be some reason to keep this behavior for double-caret,
it is certainly pointless for Ctrl+^.
* That gives you an easy way to insert Ctrl+^ (code 30) into documents with <I>.
Perviously, you either had to insert a double-caret, typing 4 carets in a row,
or you had to use <EI> or 30I$.
* The special handling of double-caret could perhaps be abolished altogether,
as we also have ^Q^ to escape plain carets.
The double-caret syntax is very archaic from the time that there was no proper
^Q as far as I recall correctly.
|
|
* @EQ$/.../ sets the current directory from the contents of the given file.
@E%$/.../ stores the currend directory in the given file.
* @EQ*/.../ will fail, just like ^U*...$.
@E%*/.../ stores the current buffer's name in the given file.
* It's especially useful with the clipboard registers.
There could still be a minor bug in @E%~/.../ with regard to EOL normalization
as teco_view_save() will use the EOL style of the current document, which
may not be the style of the Q-Reg contents.
Conversions can generally be avoided for these particular commands.
But without teco_view_save() we'd have to care about save point creation.
|
|
* This was unsafe and could easily result in crashes, since teco_qreg_current
would afterwards point to an already freed Q-Register.
* Since automatically editing another register or buffer is not easy to do right,
we throw an error instead.
|
|
|
|
|
|
|
|
|
|
* Some platforms like Ubuntu actually ship widechar and non-widechar versions
of ncurses with different pkg-config files.
Other platforms like FreeBSD will ship an "ncursesw" and "ncurses" pkg-config file
but both point to the same wide-char library anyway.
* Currently we are not using wide-char APIs to ensure maximum compatibility even with
embedded systems where ncurses might be built without widechar support.
But in order to handle Unicode correctly, we still need to link against the widechar version
of ncurses (if available).
* Compilation on platforms without a widechar ncurses is now handled by the common
AC_CHECK_LIB() fallback (which might actually find a widechar version anyway if it just
didn't install the pkg-config file).
If necessary, we could also check for the "ncurses" package if "ncursesw" is not found.
* This fixes Unicode display and input on Ubuntu.
|
|
* You no longer have to copy contrib/scintilla, contrib/scinterm and contrib/lexilla
manually to the build directory.
* It turns out, that Scintilla/Lexilla was supporting this since 2016.
Scintilla allows pointing to a source directory (srdir) and Lexilla to a binary directory (DIR_O).
* For Scinterm I opened a pull request in order to add srcdir/basedir variables:
https://github.com/orbitalquark/scinterm/pull/21
* `make distcheck` is therefore now also fixed.
* The FreeBSD package is now allowed to build out of source.
I haven't tested it yet.
* See also https://github.com/ScintillaOrg/lexilla/issues/266
|
|
|
|
* The scinterm submodule temporarily points to my own fork.
* This is necessary for the aspell macro on Curses.
See https://github.com/rhaberkorn/sciteco/wiki/Useful-Macros#spell-checker
* See also https://github.com/orbitalquark/scinterm/pull/19
|
|
* This is optimized for Groff, but works for Heirloom Troff and Neatroff as well.
Currently, the Heirloom and Neatroff requests are just added ontop of the Groff
ones. Theoretically, we could also try to separate the keyword lists into
a base K&R set with Groff, Heirloom and Neatroff ontop.
* The lexer necessarily has many restrictions, as Troff is fundamentally unparseable
(like classic TECO) and needs a lot of per-request knowledge.
* The "*.mm" extension has been removed from the lexers/cpp.tes.
I don't know what language this was for, and I prefer `*.mm` files
to be considered Troff.
* Temporarily changed the lexilla submodule URL.
The corresponding Lexila lexer is in the process of being upstreamed.
Once it is, I will probably revert the submodule to the official repository,
as the "troff" branch is not stable (can be rebased).
|
|
|
|
* MacOS packages are now built on macos-12 since macos-11 has been deprecated.
|
|
|
|
|
|
for debug builds
* There is cleanup that is not strictly necessary, because it only frees memory
which is freed on program termination anyway.
* However, it helps to explicitly free everything for debugging memory leaks via Valgrind.
* The new macro reduces the number of #ifdef statements.
* On NDEBUG, the code of these functions will still be eliminated.
* If functions are referenced only from the destructor, there will be no unused function
warnings, even in NDEBUG.
|
|
* This is supposed to allow new users without any prior exposure to SciTECO
to pick up the basics of practical usage of SciTECO as an editor.
It almost does not elaborate on scripting-side of things.
* This requires a full Groff installation, so the document is not built by default.
|
|
|
|
Scintilla now
* The patch avoids all automatic scrolling consistently, including in SCI_UNDO.
This speads up Undo (especially after interruptions).
* Also, the patch disables a very costly and pointless (in SciTECO) algorithm that
effectively made <Ix$> uninterruptible.
* Effectively reverts large parts of 8ef010da59743fcc4927c790f585ba414ec7b129.
I have never liked using unintuitive Scintilla messages to avoid scrolling.
|
|
* This would sometimes rub out more than expected due to
reading undefined memory.
Actually even crashes were not impossible.
* This is because SCI_GETWORDCHARS does not null-terminate the buffer
it writes but this was assumed.
In effect, we could easily read beyond the allocated memory in wchars
if there doesn't happen to be a null-char following the buffer.
* Consequently, null-chars in word chars were also not supported,
although this would hardly trouble anybody.
* Instead, we now store the word chars in a teco_string_t which
supports non-null-terminated strings natively.
Still we null-terminate the string to keep teco_string_t's promises
about degrading to null-terminated char *.
This is currently not necessary.
* teco_is_wordchar() has been replaced by teco_string_contains().
|
|
|
|
|
|
* This has always been broken as Gtk will not hide the
window before suspending.
* It has been deemed to complicated to implement at the moment.
Even if we can catch SIGTSTP (not that trivial), it seems to be
impossible - at least without some lower level Xlib interaction -
to hide the program window before raising SIGTSTP.
* Even if everything worked, it is unclear whether it is actually
desirable to suspend a GUI application - ^Z may be pressed accidentally
and it will be inconvenient to resume the job.
So we would additionally have to check for the existence of
an attached console.
|
|
* This was surprisingly easy to implement as Gtk+ 3 already
supports it via GtkPlug.
* Allows embedding SciTECO into other Xembed-aware applications.
* Unfortunately there are very few generic Xembed hosts.
tabbed (https://tools.suckless.org/tabbed/) would be one of them.
It could be used to add tabs to SciTECO even on non-tiling window managers:
$ tabbed sciteco --xembed
* Unfortunately, it does not seem to be possible to use this feature
to let SciTECO replace the contents of a terminal window even though
many terminal emulators provide $WINDOWID.
|
|
|
|
|
|
This is a total conversion of SciTECO to plain C (GNU C11).
The chance was taken to improve a lot of internal datastructures,
fix fundamental bugs and lay the foundations of future features.
The GTK user interface is now in an useable state!
All changes have been squashed together.
The language itself has almost not changed at all, except for:
* Detection of string terminators (usually Escape) now takes
the string building characters into account.
A string is only terminated outside of string building characters.
In other words, you can now for instance write
I^EQ[Hello$world]$
This removes one of the last bits of shellisms which is out of
place in SciTECO where no tokenization/lexing is performed.
Consequently, the current termination character can also be
escaped using ^Q/^R.
This is used by auto completions to make sure that strings
are inserted verbatim and without unwanted sideeffects.
* All strings can now safely contain null-characters
(see also: 8-bit cleanliness).
The null-character itself (^@) is not (yet) a valid SciTECO
command, though.
An incomplete list of changes:
* We got rid of the BSD headers for RB trees and lists/queues.
The problem with them was that they used a form of metaprogramming
only to gain a bit of type safety. It also resulted in less
readble code. This was a C++ desease.
The new code avoids metaprogramming only to gain type safety.
The BSD tree.h has been replaced by rb3ptr by Jens Stimpfle
(https://github.com/jstimpfle/rb3ptr).
This implementation is also more memory efficient than BSD's.
The BSD list.h and queue.h has been replaced with a custom
src/list.h.
* Fixed crashes, performance issues and compatibility issues with
the Gtk 3 User Interface.
It is now more or less ready for general use.
The GDK lock is no longer used to avoid using deprecated functions.
On the downside, the new implementation (driving the Gtk event loop
stepwise) is even slower than the old one.
A few glitches remain (see TODO), but it is hoped that they will
be resolved by the Scintilla update which will be performed soon.
* A lot of program units have been split up, so they are shorter
and easier to maintain: core-commands.c, qreg-commands.c,
goto-commands.c, file-utils.h.
* Parser states are simply structs of callbacks now.
They still use a kind of polymorphy using a preprocessor trick.
TECO_DEFINE_STATE() takes an initializer list that will be
merged with the default list of field initializers.
To "subclass" states, you can simply define new macros that add
initializers to existing macros.
* Parsers no longer have a "transitions" table but the input_cb()
may use switch-case statements.
There are also teco_machine_main_transition_t now which can
be used to implement simple transitions. Additionally, you
can specify functions to execute during transitions.
This largely avoids long switch-case-statements.
* Parsers are embeddable/reusable now, at least in parse-only mode.
This does not currently bring any advantages but may later
be used to write a Scintilla lexer for TECO syntax highlighting.
Once parsers are fully embeddable, it will also be possible
to run TECO macros in a kind of coroutine which would allow
them to process string arguments in real time.
* undo.[ch] still uses metaprogramming extensively but via
the C preprocessor of course. On the downside, most undo
token generators must be initiated explicitly (theoretically
we could have used embedded functions / trampolines to
instantiate automatically but this has turned out to be
dangereous).
There is a TECO_DEFINE_UNDO_CALL() to generate closures for
arbitrary functions now (ie. to call an arbitrary function
at undo-time). This simplified a lot of code and is much
shorter than manually pushing undo tokens in many cases.
* Instead of the ridiculous C++ Curiously Recurring Template
Pattern to achieve static polymorphy for user interface
implementations, we now simply declare all functions to
implement in interface.h and link in the implementations.
This is possible since we no longer hace to define
interface subclasses (all state is static variables in
the interface's *.c files).
* Headers are now significantly shorter than in C++ since
we can often hide more of our "class" implementations.
* Memory counting is based on dlmalloc for most platforms now.
Unfortunately, there is no malloc implementation that
provides an efficient constant-time memory counter that
is guaranteed to decrease when freeing memory.
But since we use a defined malloc implementation now,
malloc_usable_size() can be used safely for tracking memory use.
malloc() replacement is very tricky on Windows, so we
use a poll thread on Windows. This can also be enabled
on other supported platforms using --disable-malloc-replacement.
All in all, I'm still not pleased with the state of memory
limiting. It is a mess.
* Error handling uses GError now. This has the advantage that
the GError codes can be reused once we support error catching
in the SciTECO language.
* Added a few more test suite cases.
* Haiku is no longer supported as builds are instable and
I did not manage to debug them - quite possibly Haiku bugs
were responsible.
* Glib v2.44 or later are now required.
The GTK UI requires Gtk+ v3.12 or later now.
The GtkFlowBox fallback and sciteco-wrapper workaround are
no longer required.
* We now extensively use the GCC/Clang-specific g_auto
feature (automatic deallocations when leaving the current
code block).
* Updated copyright to 2021.
SciTECO has been in continuous development, even though there
have been no commits since 2018.
* Since these changes are so significant, the target release has
been set to v2.0.
It is planned that beginning with v3.0, the language will be
kept stable.
|
|
|
|
|
|
been shown to be unacceptably broken, so the fallback implementation has been improved
* mallinfo() is not only broken on 64-bit systems but slows things
down linearilly to the memory size of the process.
E.g. after 500000<%A>, SciTECO will act sluggish! Shutting down
afterwards can take minutes...
mallinfo() was thus finally discarded as a memory measurement
technique.
* Evaluating /proc/self/statm? has also been evaluated and discarded
because doing this frequently is even slower.
* Instead, the fallback implementation has been drastically improved:
* If possible use C++14 global sized deallocators, allowing memory measurements
across the entire C++ code base with minimal runtime overhead.
Since we only depend on C++11, a lengthy Autoconf check had to be introduced.
* Use malloc_usable_size() with global non-sized deallocators to
measure the approx. memory usage of the entire process (at least
the ones done via C++).
The cheaper C++11 sized deallocators implemented via SciTECO::Object still
have precedence, so this affects Scintilla code only.
* With both improvements the test case
sciteco -e '<@EU[X^E\a]"^E\a"%a>'
is handled sufficiently well now on glibc and performance is much better
now.
* The jemalloc-specific technique has been removed since it no longer
brings any benefits compared to the improved fallback technique.
Even the case of using malloc_usable_size() in strict C++ mode is
up to 3 times faster.
* The new fallback implementation might actually be good enough for
Windows as well if some MSVCRT-specific support is added, like
using _msize() instead of malloc_usable_size().
This must be tested and benchmarked, so we keep the Windows-specific
implementation for the time being.
|
|
* in a flat list of undo tokens, we need to store
the program counter (ie. command line position)
that the undo token corresponds to.
Since in general there is more than one undo token per
input character, this stored PCs redundantly.
* For input characters with no undo tokens
(only applies to NOPs like space in the command line
macro), this needs one more pointer than before.
* In case of 1 undo token per input character,
the new implementation uses approx. the same memory.
* In the most common case of more than one undo token
per input character, this saves at least 4 bytes per
undo token.
* In large macros and long loops the effect is especially
pronounced. E.g. 500000<%A> will use 8MB less memory
with the new implementation.
|
|
as State::process_edit_cmd() virtual methods
* Cmdline::process_edit_cmd() was much too long and deeply nested.
It used RTTI excessively to implement the state-specific behaviour.
It became apparent that the behaviour is largely state-specific and could be
modelled much more elegantly as virtual methods of State.
* Basically, a state can now implement a method to customize its
commandline behaviour.
In the case that the state does not define custom behaviour for
the key pressed, it can "chain" to the parent class' process_edit_cmd().
This can be optimized to tail calls by the compiler.
* The State::process_edit_cmd() implementations are still isolated in
cmdline.cpp. This is not strictly necessary but allows us keep the
already large compilations units like parser.cpp small.
Also, the edit command processing has little to do with the rest of
a state's functionality and is only used in interactive mode.
* As a result, we have many small functions now which are much easier to
maintain.
This makes adding new and more complex context sensitive editing behaviour
easier.
* State-specific function key masking has been refactored by introducing
State::get_fnmacro_mask().
* This allowed us to remove the States::is_*() functions which have
always been a crutch to support context-sensitive key handling.
* RTTI is almost completely eradicated, except for exception handling
and StdError(). Both remaining cases can probably be avoided in the
future, allowing us to compile smaller binaries.
|
|
and added a FreeBSD/jemalloc-specific implementation
* largely reverts 39cfc573, but leaves in minor and documentation
changes.
* further experimentation of memory limiting using malloc() wrapping
has shown additional problems, like dlsym() calling malloc-functions,
further reducing the implementation to glibc-specific means.
This means there had been no implementation for FreeBSD and checks
would have to rely on undocumented internal implementation details
of different libcs, which is not a good thing.
* Other problems have been identified, like having to wrap calloc(),
guarding against underruns and multi-thread safety had been identified
but could be worked around.
* A technique by calculating the memory usage as sbrk(0) - &end
has been shown to be effective enough, at least on glibc.
However even on glibc it has shortcomings since malloc() will
somtimes use mmap() for allocations and the technique
relies on implementation details of the libc.
Furthermore another malloc_trim(0) had to be added to the error
recovery in interactive mode, since glibc does not adjust the program break
automatically (to avoid syscalls I presume).
* On FreeBSD/jemalloc, the sbrk(0) method totally fails because jemalloc
exclusively allocates via mmap() -> that solution was discarded as well.
* Since all evaluated techniques turn out to be highly platform
specific, I reverted to the simple and stable platform-specific
mallinfo() API on Linux.
* On FreeBSD/jemalloc, it's possible to use mallctl("stats.allocated")
for the same purpose - so it works there, too now.
It's slower than the other techniques, though.
* A lengthy discussion has been added to memory.cpp, so that we
do not repeat the previous mistakes.
|
|
portable and faster hack
* Works by "hooking" into malloc() and friends and counting the
usable heap object sizes with malloc_usable_size().
Thus, it has no memory-overhead.
* Will work at least on Linux and (Free)BSD.
Other UNIXoid systems may work as well - this is tested by ./configure.
* Usually faster than even the fallback implementation since the
memory limit is hit earlier.
* A similar approach could be tried on Windows (TODO).
* A proper memory-limiting counting all malloc()s in the system can make
a huge difference as this test case shows:
sciteco -e '<@EU[X^E\a]"^E\a"%a>'
It will allocate gigabytes before hitting the 500MB memory limit...
* Fixed the UNIX-function checks on BSDs.
|
|
|
|
* some classic TECOs have this
* just like ^[, dollar works as a command only, not as a string terminator
* it improves the readability of macros using printable characters only
* it closes a gap in the language by allowing $$ (double-dollar) and
^[$ as printable ways to write the return from macro command.
^[^[ was not and is not possible.
* since command line termination is a regular interactive return-command
in SciTECO, double-dollar will also terminate the command line now.
This will be allowed unless it turns out to be a cause of trouble.
* The handling of unterminated commands has been cleaned up by
introducing State::end_of_macro().
Most commands (and thus states) except the start state cannot be
valid at the end of a macro since this indicates an unterminated/incomplete
command.
All lookahead-commands (currently only ^[) will end implicitly
at the end of a macro and so will need a way to perform their action.
The virtual method allows these actions to be defined with the rest
of the state's implementation.
|
|
performance issues with memory measurements
* Fixed build problems on Windows
* g_slice on Windows has been shown to be of little use either
and it does not work well with the GetProcessMemoryInfo()
measurements.
Also, it brings the same problem as on Glibc: Not even command-line
termination returns the memory to the OS.
Therefore, we don't use g_slice at all and commented on it.
* The custom Linux and Windows memory measurement approaches
have been shown to be inefficient.
As a workaround, scripts disable memory limiting.
* A better approach -- but it will only work on Glibc -- might
be to hook into malloc(), realloc() and free() globally
and use the malloc_usable_size() of a heap object for
memory measurements. This will be relatively precise and cheap.
* We still need the "Object" base class in order to measure
memory usage as a fallback approach.
|
|
* we were basing the glib allocators on throwing std::bad_alloc just like
the C++ operators. However, this always was unsafe since we were throwing
exceptions across plain-C frames (Glib).
Also, the memory vtable has been deprecated in Glib, resulting in
ugly warnings.
* Instead, we now let the C++ new/delete operators work like Glib
by basing them on g_malloc/g_slice.
This means they will assert and the application will terminate
abnormally in case of OOM. OOMs cannot be handled properly anyway, so it is
more important to have a good memory limiting mechanism.
* Memory limiting has been completely revised.
Instead of approximating undo stack sizes using virtual methods
(which is unprecise and comes with a performance penalty),
we now use a common base class SciTECO::Object to count the memory
required by all objects allocated within SciTECO.
This is less precise than using global replacement new/deletes
which would allow us to control allocations in all C++ code including
Scintilla, but they are only supported as of C++14 (GCC 5) and adding compile-time
checks would be cumbersome.
In any case, we're missing Glib allocations (esp. strings).
* As a platform-specific extension, on Linux/glibc we use mallinfo()
to count the exact memory usage of the process.
On Windows, we use GetProcessMemoryInfo() -- the latter implementation
is currently UNTESTED.
* We use g_malloc() for new/delete operators when there is
malloc_trim() since g_slice does not free heap chunks properly
(probably does its own mmap()ing), rendering malloc_trim() ineffective.
We've also benchmarked g_slice on Linux/glib (malloc_trim() shouldn't
be available elsewhere) and found that it brings no significant
performance benefit.
On all other platforms, we use g_slice since it is assumed
that it at least does not hurt.
The new g_slice based allocators should be tested on MSVCRT
since I assume that they bring a significant performance benefit
on Windows.
* Memory limiting does now work in batch mode as well and is still
enabled by default.
* The old UndoTokenWithSize CRTP hack could be removed.
UndoStack operations should be a bit faster now.
But on the other hand, there will be an overhead due to repeated
memory limit checking on every processed character.
|
