diff options
Diffstat (limited to 'TODO')
| -rw-r--r-- | TODO | 310 |
1 files changed, 226 insertions, 84 deletions
@@ -4,8 +4,16 @@ Tasks: "edit" hook. Known Bugs: - * Using fnkeys.tes still flickers on PDCurses/WinGUI. - Apparently a PDCurses bug. + * Editing very large files, or at least files with very long lines, is painstakingly slow. + Try for instance openrussian-custom-2023-10-09.sql. + For some strange reason, this affects both Curses and GTK. + In UTF-8 mode, this doesn't even load anytime soon. + * PDCurses/WinGUI: Pressing Ctrl+Shift+6 on an US keyboard layout + inserts "6^^". This looks like a PDCurses/WinGUI bug. + https://github.com/Bill-Gray/PDCursesMod/issues/323 + * PDCurses/WinGUI: There is still some flickering, but it got better + since key macros update the command line only once. + https://github.com/Bill-Gray/PDCursesMod/issues/322 * Win32: Interrupting <EC> will sometimes hang. Affects both PDCurses/WinGUI and Gtk. In this case you have to kill the subprocess using the task manager. @@ -16,17 +24,14 @@ Known Bugs: after hitting the OOM limit, eg. after <%a>. Apparently an effect of HAVE_MORECORE (sbrk()) - some allocation is always left at the end. - * S<LF>^ES^N<$ does not find the first line that does not begin with "<" - ^ES is apparently not greedy. - * fnkeys.tes: Cursor movements will swallow all preceding braced - expressions - there should be more checks. + * S<LF>^ES^N<$ does not find the first line that does not begin with "<". + This is because \s+ backtracks and can match shorter sequences. + Perhaps ^ES should always be translated to \s++ (possessive quantifier)? * Colors are still wrong in Linux console even if TERM=linux-16color when using Solarized. Affects e.g. the message line which uses the reverse of STYLE_DEFAULT. Perhaps we must call init_color() before initializing color pairs (currently done by Scinterm). - * Scinterm: The underline attribute is not applied properly - even on Urxvt where it obviously works. * session.save should save and reset ^R. Perhaps ^R should be mapped to a Q-Reg to allow [^R. Currently, saving the buffer session fails if ^R != 10. @@ -42,17 +47,15 @@ Known Bugs: and b) the file mode and ownership of re-created files can be preserved. We should fall back silently to an (inefficient) memory copy or temporary file strategy if this is detected. - * crashes on large files: S^EM^X$ (regexp: .*) - Happens because the Glib regex engine is based on a recursive + * Crashes on large files: S^EM^X$ (regexp: (.)+) + Happens because the Glib regex engine is based on a recursive (backtracking) Perl regex library. - The Homebrew and MinGW versions of glib no longer suffer from this. + I can provoke the problem only on Ubuntu 20.04. This is apparently impossible to fix as long as we do not have control over the regex engine build. - We should therefore switch the underlying Regex engine. - Oniguruma looks promising and is also packed for Ubuntu (libonig2). - It would also directly allow globbing by tweaking the syntax. - TRE also looks promising and is smaller than Oniguruma. - GRegEx (PCRE) could still be supported as a fallback. + We need something based on a non-backtracking Thompson's NFA with Unicode (UTF-8), see + https://swtch.com/~rsc/regexp/ + Basically only RE2 would check all the boxes. * It is still possible to crash SciTECO using recursive functions, since they map to the C program's call stack. It is perhaps best to use another stack of @@ -67,25 +70,30 @@ Known Bugs: automatically. Unfortunately, neither Linux, nor FreeBSD allow the re-linking based on file descriptors. On Linux this fails because the - file's link count will be 0; on BSD, /dev/fd/X is on a different - volume and cannot be linked to its old path. - Thus, we'd have to copy the file. + file's link count will be 0; + On BSD linkat(AT_EMPTY_PATH) requires root privileges and does not + work with unlinked files anyway. * Windows NT has hard links as well, but they don't work with file handles either. However, it could be possible to call CreateFile(FILE_FLAG_DELETE_ON_CLOSE) on the savepoint file, ensuring cleanup even on abnormal program termination. + However, this flag cannot be cleared once we restore a save point, + so we'd have to copy its contents just like in the UNIX case. There is also MoveFileEx(file, NULL, MOVEFILE_DELAY_UNTIL_REBOOT). * Windows has file system forks, but they can be orphaned just like ordinary files but are harder to locate and clean up manually. * Setting window title is broken on ncurses/XTerm. + The necessary capabilities are usually not in the Terminfo database. Perhaps do some XTerm magic here. We can also restore window titles on exit using XTerm. + * The XTerm OSC-52 clipboard feature appears to garble Unicode characters. + This is apparently an XTerm bug, probably due to 8-bit-uncleanliness. + It was verified by `printf "\e]52;c;?\a"` on the command line. * Glib (error) messages are not integrated with SciTECO's logging system. - * Out-of-tree builds are broken. - See contrib/scintilla.am. * Gtk on Unix: On ^Z, we do not suspend properly. The window is still shown. + This would be a useful feature especially with --xembed on st. Perhaps we should try to catch SIGTSTP? This does not work with g_unix_signal_add(), though, so any workaround would be tricky. @@ -97,7 +105,7 @@ Known Bugs: the window has really been hidden/unrealized. Even if everything worked, it might well be annoying if you accidentally suspend your instance while not being - connected to a terminal. + connected to a terminal. Although this could be checked at runtime. Suspension from the command-line has therefore been disabled on Gtk for the time being. * Many Scintilla commands <ES> can easily crash the editor. @@ -113,6 +121,34 @@ Known Bugs: (and with --no-profile) under Mac OS terminal emulators. This does not happen under Linux with Darling. See https://github.com/rhaberkorn/sciteco/issues/12 + * File name autocompletion should take glob patterns into account. + The simple reason is that if a filename really contains glob characters + and you are trying to open it with EB, you might end up not being + able to autocomplete it if a previous autocompletion inserted + escaped glob patterns. + Unfortunately, this would be very tricky to do right. + * The git.blame macro is broken, at least on Git v2.45.2 and v2.25.1. Compare + cat sample.teco_ini | git blame --incremental --contents - -- sample.teco_ini | grep -E '^[a-f0-9]{40}' + (which is wrong and does not even contain all commits) with + git blame --incremental --contents sample.teco_ini -- sample.teco_ini | grep -E '^[a-f0-9]{40}' + which is correct. Without --incremental even the formatting is broken. + This could well be a Git bug. + * Margins, identions and the like are not configured on the unnamed + buffer by sample.teco_ini. + And this is probably correct. + However when saving a new unnamed file for the first time, + nothing will change either and it's tricky to apply the correct + settings. You have to EF and EB the file after the initial + save to get everything set up correctly. + Or manually run 0M#ED, but this cannot be rubbed out. + We need a "save" ED hook to get this right. + For instance, you could check whether the margin is set up + as a sign of whether lexing has already been applied. + On the other hand, this Save also can't be properly rubbed out and + will not restore the original margins and styling unless we add native + commands for ALL of the Scintilla messages involved. + Automatically running EF EB...$ in the "save" hook could + also have unwanted side effects. Features: * Auto-indention could be implemented via context-sensitive @@ -171,51 +207,57 @@ Features: and the return type. * Having a separate number parser state will simplify number syntax highlighting. - * Function key masking flag for the beginning of the command + * Key macro masking flag for the beginning of the command line. May be useful e.g. for solarized's F5 key (i.e. function key macros that need to terminate the command line as they cannot be rubbed out properly). - * fnkeys.tec could preserve the column more reliably when - moving up and down by encoding a character offset into the - command line. E.g. (100-3C) would tell us that we have to add - 3 to the real column when moving up/down because the current - line is too short. - * Function key macros should behave more like regular macros: - If inserting a character results in an error, the entire - macro should be rubbed out. This means it would be OK to - let commands in function key macros fail and would fix, e.g. - ^FCLOSE. - * Function key macros could support special escape sequences + * Key macros could support special escape sequences that allow us to modify the parser state reliably. E.g. one construct could expand to the current string argument's termination character (which may not be Escape). - In combination with a special function key macro state + In combination with a special key macro state effective only in the start state of the string building state machine, perhaps only in insertion commands, this could be used to make the cursor movement keys work in insertion commands by automatically terminating the command. Even more simple, the function key flag could be effective only when the termination character is $. - * Function key handling should always be enabled. This was - configurable because of the way escape was handled in ncurses. - Now that escape is always immediate, there is little benefit - in having this still configurable. In fact if turned off, - SciTECO would try to execute escape sequences. - The ED flag could still exist and tell whether the function - key macros are used at all (i.e. this is how Gtk behaves currently). + * Support more function keys. + We can define more function keys via define_key(3NCURSES). + Unfortunately they are not really standardized - st and urxvt for instance + have different escape sequences for Ctrl+Up or Alt+Up. + It seems they can be looked up with tigetstr() and + then passed to define_key(). + Alternatively call use_extended_names(TRUE) and look up the + key codes with key_defined(). + At the very least PDCurses and Gtk could support much more + keys and Alt and Ctrl modifiers. + See also https://stackoverflow.com/questions/31379824/how-to-get-control-characters-for-ctrlleft-from-terminfo-in-zsh + https://gist.github.com/rkumar/1237091 * Mouse support. Not that hard to implement. Mouse events - use a pseudo function key macro as in Curses. + use a pseudo key macro as in Curses. Using some special command, macros can query the current mouse state (this maps to an Interface method). + This should be configurable via an ED flag as it changes + the behavior of the terminal. + * GTK currently allows mouse scrolling as it is not under control + of SciTECO. + Once we got mouse support in Curses, we could either also + automatically scroll or (better) expose the scroll events + as key macros. * Support loading from stdin (--stdin) and writing to the current buffer to stdout on exit (--stdout). This will make it easy to write command line filters, - We will need flags like --8-bit-clean and --quiet with + We will need flags like and --quiet with single-letter forms to make it possible to write hash-bang lines like #!...sciteco -q8iom Command line arguments should then also be handled differently, passing them in an array or single string register, so they no longer affect the unnamed buffer. + * Once we've got --stdout, it makes sense to ship a version of + tecat written in SciTECO. + This is useful as a git diff textconv filter. + See https://gist.github.com/rhaberkorn/6534ecf1b05de6216d0a9c33f31ab5f8 * For third-party macro authors, it is useful to know the standard library path (e.g. to install new lexers). There could be a --print-path option, or with the --quiet @@ -249,17 +291,28 @@ Features: of EI that considers $SCITECOPATH. Current use of EI (insert without string building) will have to move, but it might vanish anyway once we can disable string building - with a special character. - * ::S for string "comparisons" (anchored search). + with a special character, eg. you could write I^C instead. + * <I> doesn't have string building enabled in classic TECO. + Changing this would perhaps be a change too radical. + Also, we would then need a string-building variant like <:I>. + * ::S for string "comparisons" (anchored search) and + ::FS for anchored search-replace. This is supposed to be an alias for .,.:FB which would be .,.:S in SciTECO. Apparanetly, the bounded search is still incompatible in SciTECO, as it is allowed to match beyond the bounds. Either the semantics of m,n:S should be changed or an FB command with classic TECO semantics should be - introduced.. + introduced. * ^S (-(length) of last referenced string), ^Y as .+^S,. * ^Q convert line arg into character arg * ^A, T and stdio in general + * nA returned -1 in case of invalid positions (similar to SciTECO's ^E) + instead of failing. + * ^W was an immediate action command to repaint the screen. + This could be a regular command to allow refreshing in long loops. + Video TECO had ET for the same purpose. + TECO 10 had a ^W regular command for case folding all strings, + but I don't think it's worth supporting. * Search for beginning of string; i.e. a version of S that leaves dot before the search string, similar to FK (request of N.M.). @@ -289,10 +342,8 @@ Features: * instead of 0EB to show the list of buffers, there should perhaps be a special TAB-completion (^G mode?) that completes only buffers in the ring. It should also display the numeric buffer ids. - * properly support Unicode encodings and the character-based model - * link against libncursesw if possible - * translate documents to Unicode strings - * a position refers to a character/codepoint + * Gtk: Unicode IME support for asiatic languages. + Have a look how it is done in Scintilla. * Progress indication in commandline cursor: Perhaps blinking or invisible? * Command to free Q-Register (remove from table). @@ -380,6 +431,7 @@ Features: all of SciTECO can be run against nodejs as a runtime. I'm not aware of any (working) alternatives, like cross-compiling for the JVM. + See also https://gist.github.com/VitoVan/92ba4f2b68fec31cda803119686295e5 * Windows supports virtual terminals now. See: https://docs.microsoft.com/en-us/windows/console/classic-vs-vt Perhaps we should transition from PDCurses to something using @@ -387,28 +439,6 @@ Features: * Improve the message line so it can log multiple messages. Especially important on GUI platforms and Win32 so we can get rid of the attached console window. - * On Unix/X11, it may be possible to draw the Gtk+ UI into the - same window as the current terminal. - SciTECO now supports Xembed via the --xembed option. - Urxvt and many other terminal emulators have $WINDOWID. - But the two won't work together. Urxvt and XTerm are apparently - not real Xembed hosts. - * Currently, you cannot pass UTF-8 parameters to SciTECO macros. - This is not critical since we don't support Unicode anyway. - Sooner or later however we should use g_win32_get_command_line(). - * Some platforms like MinGW and some Linux flavours have - native Scintilla packages. - Perhaps it makes sense to be able to build against them - using --with-scintilla. - * Write a Scintilla/Lexilla lexer for Roff - * There is an Urxvt extension 52-osc for implementing the - xterm-like clipboard control sequences. - If may therefore make sense to support that even without checking - the xterm version and update documentation accordingly. - * Write standard library module for spell checking. - This can eg. use hunspell/aspell/ispell `-a` mode. - See my SciTE implementation on how to do this. - The resulting macro can be added to a save hook. * A dirtify-hook would be useful and could be used for spell checking. Naturally it could only be exected at the end of executing interactive commands) and it should be triggered @@ -432,11 +462,12 @@ Features: A new private key has already been registered on Launchpad and Github. We just need to integrate with CI. See also https://github.com/marketplace/actions/import-gpg - * AppImage for Linux * 64-bit Windows builds * Mac OS Arm64 builds either separately or via universal binary. See https://codetinkering.com/switch-homebrew-arm-x86/ Target flag: `-target arm64-apple-macos11` + * Get into AppImageHub. + * Get meta-rhaberkorn into https://layers.openembedded.org * Linux: Relocatable binaries instead of hardcoding the library path. This makes it possible to run builds installed via `make install DESTDIR=...` and will aid in creating AppImages. @@ -449,6 +480,81 @@ Features: or grep -n results. * <:EF> for saving and closing a buffer, similar to <:EX>. * Bash completions. + * Case-sensitive search command (modifier or flag). + * FreeBSD: rctl(8) theoretically allows setting up per-process actions + when exceeding the memory limit. + This however requires special system settings. + * Auto-completions customization via external programs. + This among other things could be used to integrate LSPs-driven + autocompletions. + * Whereever we take buffer positions (nJ; n,mD; nQ...), + negative numbers could refer to the end of the buffer or + Q-Register string. + * Support extended operators like in TECO-64: + https://github.com/fpjohnston/TECO-64/blob/master/doc/oper.md + However, instead of introducing a separate parser state, better + use operators like ~=, ~< etc. + * It should be possible to disable auto-completions of one-character + register names, so that we can map the idention macro to M<TAB>. + * Add a configure-switch for LTO (--enable-lto). + * There should be a string building character for including + a character by code. Currently, there is only ^EUq where + q must be set earlier. + This would be useful when searching in binary files or + to include Unicode characters by code point. + Unfortunately its syntax cannot depend on the string argument's + encoding, as that could confuse parse-only mode. + Perhaps ^E!xxxx or overload ^Qx0123. + TECO-11 has ^Ennn (octal), but only for searching? + * There should be a string building construct for escaping + search patterns. + Since string building is performed immediately before + search pattern translation, you cannot currently + search for a Q-Register verbatim. + * Tweak the Makefile lexer for Automake support. + In the simplest case, just add the *.am file extension. + * Add an fnkeys.tes alternative where moving cursor keys + leaves you in the insert (I) command. + That will behave very similar to classical editors. + * Lexing via SciTECO macros? + They would have to be in their own parser instance since Scintilla + could ask us to restyle at any time and within string arguments, + which would confuse the parser as it is. + Also, parsers are not fully embeddable right now. + At the same time, it would need access to the view/document it's + supposed to style. Tricky, but not impossible. + * Support external lexers. + Eg. @ES/SCI_SETILEXER/scintillua:APL/ + automatically loads libscintillua.so or scintillua.dll, caches + the library handle and creates the "APL" lexer. + There would also have to be some kind of SCI_NAMEDSTYLE wrapper, + so we can look up style ids by name (specifically for Scintillua). + * ^^ in string building expanding to a single caret is not + consistent. Perhaps we should allow only ^Q^ as a way to insert a + single caret? + * Support for non-ANSI single byte encodings is still incomplete. + You can set them with EE and they will be correctly + displayed (on Gtk at least), but there is no way + to insert text in these codepages, at least outside + of the latin range of course. + There are two ways this could be implemented: + * Either all sorts of commands automatically iconv + from/to the configured encoding. + * Or we iconv once to UTF-8 when loading the file + and iconv back when saving. + This is probably easier but means, you have to + somehow specify the codepage on EB as you cannot + change it later on. + We could say that nEB...$ specifies the code page + if the string argument is nonempty. + On the other hand, iconv uses symbolic identifiers. + * Perhaps the Unicode "icons" should be configurable via TECO. + In the easiest case there could simply be 2 Q-Reg namespaces: + ^F... for filenames and ^E... for extensions. + The numeric part could be used to store the codepoints. + * Scinterm: The underline and italic styles are not applied properly + even on Urxvt where they obviously work. + https://github.com/orbitalquark/scinterm/issues/22 Optimizations: * Use SC_DOCUMENTOPTION_STYLES_NONE in batch mode. @@ -458,7 +564,8 @@ Optimizations: requested explicitly per ED flag or command line option. * teco_interface_cmdline_update() should be called only once after inserting an entire command line macro. - * There is should be a common error code for null-byte tests. + * There should be a common error code for null-byte tests + instead of TECO_ERROR_FAILED. * teco_string_append() could be optimized by ORing a padding into the realloc() size (e.g. 0xFF). However, this has not proven effective on Linux/glibc @@ -468,24 +575,59 @@ Optimizations: * commonly used (special) Q-Registers could be cached, saving the q-reg table lookup * refactor search commands (create proper base class) - * Add a configure-switch for LTO (--enable-lto). * undo__teco_interface_ssm() could always include the check for teco_current_doc_must_undo(). - * Define TECO_DESTRUCTOR depending on NDEBUG instead - of ifdefing all __attribute__((destructors)). - This also avoids warning for symbols referenced only in - the constructors. + * Avoid Scintilla Undo actions entirely. + This will make undo token creation of deletions harder, + since we must first query the text to be reinserted again, + but will probably save lots of memory. + * Newer GCC and C23 features: + * Perhaps teco_bool_t usage could be simplified using + __attribute__((hardbool)). + * Use `#elifdef` instead of `#elif defined`. + * Use `[[gnu::foo]]` instead of `__attribute__((foo))`. + * The TECO_FOR_EACH() hack could be simplified at least marginally + using __VA_OPT__(). + * Parsing might be optimized with libc's Unicode handling + functions and/or custom optimized versions. + * The new Scintilla IDocumentEditable interface could be + used to speed up Q-Register string accesses. + That is, when it actually supports anything useful. + This is a provisional feature and supported only via C++, + so we would need a small wrapper to call it from C world. + * Perhaps replace glib with gnulib. Could improve + portability and reduce binary size on platforms where + I have to ship the glib shared library. + Also we could control memory allocations more tightly. + We could also make use of the fnmatch and canonicalize + modules. + On the other hand, this imports tons of sh*t into the + repository and chains us to Autotools. + * Does it make sense to import glib-2.0.m4? + * According to ChatGPT (sic) the glibc and jemalloc malloc_usable_size() + do not change during the lifetime of an object, + although this is an implementation detail. + Perhaps we should support these system allocators by default + if malloc_usable_size() and the original functions as __malloc() + are defined. + But how to even test for glibc's ptmalloc? + Linux could use musl as well for instance. Documentation: - * Code docs (Doxygen). It's slowly getting better... + * Doxygen docs could be deployed on Github pages + automatically. * The ? command could be extended to support looking up help terms at dot in the current document (e.g. if called ?$). Furthermore, womanpages could contain "hypertext" links to help topics using special Troff markup and grosciteco support. * The command reference should include an overview. - * Write a cheat sheet. Either on www.cheatography.com, or - using Groff and include with SciTECO. * Write some tutorials for the Wiki, e.g. about paragraph reflowing... Object-oriented SciTECO ideoms etc. ;-) - * Record a "video" on https://asciinema.org/ + * What to do with `--xembed`: tabbed, st + when used as the git editor, etc. + * Perhaps there should be a Getting Started document, + that is automatically opened by sample.teco_ini. + * The HTML manuals lack monospaced fonts. + This is partly because an.tmac removes the Courier family + in nroff mode, but it still doesn't work if you undo this. |