implemented ncurses clipboard support via external processes

* As an alternative to OSC-52, which is rarely supported by terminal emulators.
* Makes the new mouse support much more useful since you rely on good builtin
  clipboard support. You can no longer e.g. just double-click a word to copy it into
  the "primary" selection as terminal emulators do by default.
* Set $SCITECO_CLIPBOARD_SET/GET e.g. to xclip, way-copy, pbcopy or some wrapper script.
* This is currently using POSIX-specific popen() API, so it behaves a bit different
  to command execution via EC/EG.
  I am not sure if it's worth rewriting with the GSpawn-API, since it will be used
  only on POSIX anyway and a GSpawn-based implementation is likely to be a bit larger.
* Should there be some small command-line utility for interacting (esp. pasting) via OSC-52,
  built-in OSC-52 support could well be removed from SciTECO.
  Currently, I know only of https://github.com/theimpostor/osc/ and it requires
  very recent Go compilers. (I still haven't tested it. Quite possibly, pasting when run as
  a piped command is impossible.)

6 files changed, 254 insertions, 53 deletions

diff --git a/TODO b/TODO
index cbe1714..289e724 100644
--- a/TODO
+++ b/TODO
@@ -390,11 +390,6 @@ Features:
    Clipboards are not flexible enough and not supported everywhere.
    I am not sure how to elegantly address instances, though.
    Especially without some kind of central name registry.
- * Generic clipboard support via optional libclipboard support.
-   This would mainly benefit the ncurses version, especially when enabling
-   mouse support.
-   But it may be beneficial to support it on all other Curses-variants
-   and on Gtk as well.
  * Touch restored save point files - should perhaps be configurable.
    This is important when working with Makefiles, as make looks
    at the modification times of files.
diff --git a/configure.ac b/configure.ac
index e355c24..69e98e8 100644
--- a/configure.ac
+++ b/configure.ac
@@ -173,7 +173,7 @@ AC_CHECK_FUNCS([memset setlocale strchr strrchr fstat sscanf], , [
 # glib defines G_OS_UNIX instead...
 case $host in
 *-*-linux* | *-*-*bsd* | *-*-darwin* | *-*-cygwin* | *-*-haiku*)
-	AC_CHECK_FUNCS([realpath readlink pathconf fchown dup dup2 getpid open read kill mmap], , [
+	AC_CHECK_FUNCS([realpath readlink pathconf fchown dup dup2 getpid open read kill mmap popen pclose], , [
 		AC_MSG_ERROR([Missing libc function])
 	])
 	AC_SEARCH_LIBS(dladdr, [dl], , [
diff --git a/doc/sciteco.1.in b/doc/sciteco.1.in
index cc17237..b7084c5 100644
--- a/doc/sciteco.1.in
+++ b/doc/sciteco.1.in
@@ -324,6 +324,30 @@ environment before initializing Curses, so these variables
 can be modified in the profile macro.
 .
 .LP
+.SCITECO_TOPIC "$SCITECO_CLIPBOARD_SET" "$SCITECO_CLIPBOARD_GET"
+On ncurses, in addition to the OSC-52 protocol, you can use external
+processes to drive the built-in clipboard Q-Registers (\(lq~\(rq and so on).
+For that you can set the \fBSCITECO_CLIPBOARD_SET\fP and \fBSCITECO_CLIPBOARD_GET\fP
+environment variables or their corresponding Q-Registers to shell commands,
+that receive the clipboard contents on stdin or output the requested clipboard on stdout.
+In the configured commands, the string \(lq{}\(rq is replaced with a single
+letter code of the clipboard to set:
+\(lqc\(rq, \(lqp\(rq or \(lqs\(rq as in the clipboard register names.
+The given commands will always be executed by \fB/bin/sh\fP, regardless of
+the \fBSHELL\fP environment variable or
+the value of bit 8 (128) in the \fBED\fP flags.
+The spawned processes also do not currently inherit the environment from the
+Q-Register environment variables, i.e. you cannot change the process environment
+via \*(ST code.
+\# That would only be possible by rewriting everything with GSpawn.
+.SCITECO_TOPIC xclip
+See
+.B @scitecodatadir@/sample.teco_ini
+for an example of how to integrate the X11 clipboard via
+.BR xclip (1).
+Integrating with Wayland and the Mac OS clipboards is of course also possible.
+.
+.LP
 .SCITECO_TOPIC "$GTK_CSD"
 On GTK+, you may turn off the infamous client-side window decorations
 by setting the environment variable \fBGTK_CSD\fP to \(lq0\(rq.
diff --git a/doc/sciteco.7.template b/doc/sciteco.7.template
index 95c3503..b274715 100644
--- a/doc/sciteco.7.template
+++ b/doc/sciteco.7.template
@@ -1580,10 +1580,10 @@ The existence of a clipboard register can thus be checked
 in macros to determine whether getting and modifying that
 particular clipboard is supported natively.
 .br
+\*(ST supports two ways of driving the clipboard on ncurses.
 .SCITECO_TOPIC OSC-52 xterm
-\*(ST does \fBnot\fP generally support clipboards on ncurses,
-but has special support for OSC-52 escape sequences, as were
-introduced by sufficiently recent versions of
+First of all, there is built-in support for OSC-52 escape sequences,
+as were introduced by sufficiently recent versions of
 .BR xterm (1)
 and have since been adopted by several other terminal emulators.
 Since the operability of OSC-52 clipboards cannot be tested
@@ -1593,6 +1593,7 @@ configured.
 .BR xterm (1)
 for instance must be configured for allowing
 the \fISetSelection\fP and \fIGetSelection\fP window operations.
+It is nevertheless observed to be a very buggy and unreliable feature.
 If running under
 .BR xterm (1),
 \*(ST will still check whether the XTerm version is sufficient.
@@ -1601,10 +1602,16 @@ Other terminal emulators like Kitty may ask for permission to read the
 clipboard (\fBread-clipboard-ask\fP).
 This is not supported by \*(ST and must be disabled
 (use \fBread-clipboard\fP instead).
-.SCITECO_TOPIC xclip
-If native clipboard support is unavailable, users may
-still fall back to using external tools like \fBxclip\fP(1)
-with the \fBEC\fP command.
+.br
+Alternatively, if OSC-52 clipboards are disabled, you can set the
+\fB$SCITECO_CLIPBOARD_SET\fP and \fB$SCITECO_CLIPBOARD_GET\fP
+environment variables (or corresponding Q-Registers) to shell
+commands, that receive the clipboard
+contents on stdin and output the requested clipboard on stdout.
+This allows integrating with various windowing environments.
+See \fBENVIRONMENT\fP in
+.BR sciteco (1)
+for more details.
 .br
 Setting the string part of a clipboard register will set that
 clipboard. \*(ST will perform automatic EOL-translation according
diff --git a/sample.teco_ini b/sample.teco_ini
index 0debdcc..13ab76b 100644
--- a/sample.teco_ini
+++ b/sample.teco_ini
@@ -62,9 +62,13 @@ EMQ[$SCITECOPATH]/fnkeys.tes
 !* Comment out to disable mouse interaction on Curses *!
 0,64ED
 
-!* Uncomment if terminal supports OSC-52 clipboards *!
+!* Uncomment if terminal emulator supports OSC-52 clipboards *!
 !!0,256ED
 
+!* For integrating with xclip on ncurses *!
+[$SCITECO_CLIPBOARD_SET]xclip -in -selection {}
+[$SCITECO_CLIPBOARD_GET]xclip -out -selection {} || true
+
 !* Uncomment to enable Unicode icons in the Curses UI *!
 !!0,512ED
 
diff --git a/src/interface-curses/interface.c b/src/interface-curses/interface.c
index a80a7ef..42ffdc6 100644
--- a/src/interface-curses/interface.c
+++ b/src/interface-curses/interface.c
@@ -44,6 +44,10 @@
 #include <glib/gprintf.h>
 #include <glib/gstdio.h>
 
+#ifdef G_OS_UNIX
+#include <sys/wait.h>
+#endif
+
 #include <curses.h>
 
 #ifdef HAVE_TIGETSTR
@@ -1263,39 +1267,7 @@ teco_interface_get_clipboard(const gchar *name, gchar **str, gsize *len, GError
 	return TRUE;
 }
 
-#elif defined(CURSES_TTY)
-
-static void
-teco_interface_init_clipboard(void)
-{
-	/*
-	 * At least on XTerm, there are escape sequences
-	 * for modifying the clipboard (OSC-52).
-	 * This is not standardized in terminfo, so we add special
-	 * XTerm support here. Unfortunately, it is pretty hard to find out
-	 * whether clipboard operations will actually work.
-	 * XTerm must be at least at v203 and the corresponding window operations
-	 * must be enabled.
-	 * There is no way to find out if they are but we must
-	 * not register the clipboard registers if they aren't.
-	 * Still, XTerm clipboards are broken with Unicode characters.
-	 * Also, there are other terminal emulators supporting OSC-52,
-	 * so the XTerm version is only checked if the terminal identifies as XTerm.
-	 * Also, a special clipboard ED flag must be set by the user.
-	 *
-	 * NOTE: Apparently there is also a terminfo entry Ms, but it's probably
-	 * not worth using it since it won't always be set and even if set, does not
-	 * tell you whether the terminal will actually answer to the escape sequence or not.
-	 */
-	if (!(teco_ed & TECO_ED_OSC52) ||
-	    (teco_xterm_version() >= 0 && teco_xterm_version() < 203))
-		return;
-
-	teco_qreg_table_insert(&teco_qreg_table_globals, teco_qreg_clipboard_new(""));
-	teco_qreg_table_insert(&teco_qreg_table_globals, teco_qreg_clipboard_new("P"));
-	teco_qreg_table_insert(&teco_qreg_table_globals, teco_qreg_clipboard_new("S"));
-	teco_qreg_table_insert(&teco_qreg_table_globals, teco_qreg_clipboard_new("C"));
-}
+#elif defined(G_OS_UNIX) && defined(CURSES_TTY)
 
 static inline gchar
 get_selection_by_name(const gchar *name)
@@ -1310,9 +1282,48 @@ get_selection_by_name(const gchar *name)
 	return g_ascii_tolower(*name) ? : 'c';
 }
 
-gboolean
-teco_interface_set_clipboard(const gchar *name, const gchar *str, gsize str_len,
-                             GError **error)
+/*
+ * OSC-52 clipboard implementation.
+ *
+ * At least on XTerm, there are escape sequences
+ * for modifying the clipboard (OSC-52).
+ * This is not standardized in terminfo, so we add special
+ * XTerm support here. Unfortunately, it is pretty hard to find out
+ * whether clipboard operations will actually work.
+ * XTerm must be at least at v203 and the corresponding window operations
+ * must be enabled.
+ * There is no way to find out if they are but we must
+ * not register the clipboard registers if they aren't.
+ * Still, XTerm clipboards are broken with Unicode characters.
+ * Also, there are other terminal emulators supporting OSC-52,
+ * so the XTerm version is only checked if the terminal identifies as XTerm.
+ * Also, a special clipboard ED flag must be set by the user.
+ *
+ * NOTE: Apparently there is also a terminfo entry Ms, but it's probably
+ * not worth using it since it won't always be set and even if set, does not
+ * tell you whether the terminal will actually answer to the escape sequence or not.
+ *
+ * This is a rarely used feature and could theoretically also be handled
+ * by the $SCITECO_CLIPBOARD_SET/GET feature.
+ * Unfortunately, there is no readily available command-line utility allowing both
+ * copying and pasting via OSC-52.
+ * That's really the only reason we keep built-in OSC-52 clipboard support.
+ *
+ * FIXME: This is the only thing here requiring CURSES_TTY.
+ * On the other hand, there is hardly any non-PDCurses on UNIX, which is not
+ * on a TTY, so we shouldn't be loosing much by requiring both.
+ */
+
+static inline gboolean
+teco_interface_osc52_is_enabled(void)
+{
+	return teco_ed & TECO_ED_OSC52 &&
+	       (teco_xterm_version() < 0 || teco_xterm_version() >= 203);
+}
+
+static gboolean
+teco_interface_osc52_set_clipboard(const gchar *name, const gchar *str, gsize str_len,
+                                   GError **error)
 {
 	fputs("\e]52;", teco_interface.screen_tty);
 	fputc(get_selection_by_name(name), teco_interface.screen_tty);
@@ -1351,8 +1362,8 @@ teco_interface_set_clipboard(const gchar *name, const gchar *str, gsize str_len,
 	return TRUE;
 }
 
-gboolean
-teco_interface_get_clipboard(const gchar *name, gchar **str, gsize *len, GError **error)
+static gboolean
+teco_interface_osc52_get_clipboard(const gchar *name, gchar **str, gsize *len, GError **error)
 {
 	gboolean ret = TRUE;
 
@@ -1446,7 +1457,167 @@ cleanup:
 	return ret;
 }
 
-#else /* !PDCURSES && !CURSES_TTY */
+/*
+ * Implementation using external processes.
+ *
+ * NOTE: This could be done with the portable GSpawn API as well,
+ * but this implementation is much simpler.
+ * We don't really need it on Windows anyway as long as we are using
+ * only PDCurses.
+ * This might only be of interest on Windows if building for the Win32 version
+ * of ncurses.
+ * As a downside, compared to GSpawn, this cannot inherit the environment
+ * variables from the global Q-Register table.
+ */
+
+static void
+teco_interface_init_clipboard(void)
+{
+	if (!teco_interface_osc52_is_enabled() &&
+	    (!teco_qreg_table_find(&teco_qreg_table_globals, "$SCITECO_CLIPBOARD_SET", 22) ||
+	     !teco_qreg_table_find(&teco_qreg_table_globals, "$SCITECO_CLIPBOARD_GET", 22)))
+		return;
+
+	teco_qreg_table_insert(&teco_qreg_table_globals, teco_qreg_clipboard_new(""));
+	teco_qreg_table_insert(&teco_qreg_table_globals, teco_qreg_clipboard_new("P"));
+	teco_qreg_table_insert(&teco_qreg_table_globals, teco_qreg_clipboard_new("S"));
+	teco_qreg_table_insert(&teco_qreg_table_globals, teco_qreg_clipboard_new("C"));
+}
+
+gboolean
+teco_interface_set_clipboard(const gchar *name, const gchar *str, gsize str_len,
+                             GError **error)
+{
+	if (teco_interface_osc52_is_enabled())
+		return teco_interface_osc52_set_clipboard(name, str, str_len, error);
+
+	static const gchar *reg_name = "$SCITECO_CLIPBOARD_SET";
+
+	teco_qreg_t *reg = teco_qreg_table_find(&teco_qreg_table_globals, reg_name, strlen(reg_name));
+	if (!reg) {
+		/* Q-Register could have been removed in the meantime */
+		g_set_error(error, TECO_ERROR, TECO_ERROR_FAILED,
+		            "Cannot set clipboard. %s is undefined.", reg_name);
+		return FALSE;
+	}
+
+	g_auto(teco_string_t) command;
+	if (!reg->vtable->get_string(reg, &command.data, &command.len, NULL, error))
+		return FALSE;
+	if (teco_string_contains(&command, '\0')) {
+		teco_error_qregcontainsnull_set(error, reg_name, strlen(reg_name), FALSE);
+		return FALSE;
+	}
+
+	gchar *sel = g_strstr_len(command.data, command.len, "{}");
+	if (sel) {
+		*sel++ = ' ';
+		*sel = get_selection_by_name(name);
+	}
+
+	FILE *pipe = popen(command.data, "w");
+	if (!pipe) {
+		g_set_error(error, TECO_ERROR, TECO_ERROR_FAILED,
+		            "Cannot spawn process from %s", reg_name);
+		return FALSE;
+	}
+
+	size_t len = fwrite(str, 1, str_len, pipe);
+
+	int status = pclose(pipe);
+	if (status < 0 || !WIFEXITED(status)) {
+		g_set_error(error, TECO_ERROR, TECO_ERROR_FAILED,
+		            "Error reaping process from %s", reg_name);
+		return FALSE;
+	}
+	if (WEXITSTATUS(status) != 0) {
+		g_set_error(error, TECO_ERROR, TECO_ERROR_FAILED,
+		            "Process from %s returned with exit code %d",
+		            reg_name, WEXITSTATUS(status));
+		return FALSE;
+	}
+
+	if (len < str_len) {
+		g_set_error(error, TECO_ERROR, TECO_ERROR_FAILED,
+		            "Error writing to process from %s", reg_name);
+		return FALSE;
+	}
+
+	return TRUE;
+}
+
+gboolean
+teco_interface_get_clipboard(const gchar *name, gchar **str, gsize *len, GError **error)
+{
+	if (teco_interface_osc52_is_enabled())
+		return teco_interface_osc52_get_clipboard(name, str, len, error);
+
+	static const gchar *reg_name = "$SCITECO_CLIPBOARD_GET";
+
+	teco_qreg_t *reg = teco_qreg_table_find(&teco_qreg_table_globals, reg_name, strlen(reg_name));
+	if (!reg) {
+		/* Q-Register could have been removed in the meantime */
+		g_set_error(error, TECO_ERROR, TECO_ERROR_FAILED,
+		            "Cannot get clipboard. %s is undefined.", reg_name);
+		return FALSE;
+	}
+
+	g_auto(teco_string_t) command;
+	if (!reg->vtable->get_string(reg, &command.data, &command.len, NULL, error))
+		return FALSE;
+	if (teco_string_contains(&command, '\0')) {
+		teco_error_qregcontainsnull_set(error, reg_name, strlen(reg_name), FALSE);
+		return FALSE;
+	}
+
+	gchar *sel = g_strstr_len(command.data, command.len, "{}");
+	if (sel) {
+		*sel++ = ' ';
+		*sel = get_selection_by_name(name);
+	}
+
+	FILE *pipe = popen(command.data, "r");
+	if (!pipe) {
+		g_set_error(error, TECO_ERROR, TECO_ERROR_FAILED,
+		            "Cannot spawn process from %s", reg_name);
+		return FALSE;
+	}
+
+	gchar buffer[1024];
+	size_t read_len;
+
+	g_auto(teco_string_t) ret = {NULL, 0};
+
+	do {
+		read_len = fread(buffer, 1, sizeof(buffer), pipe);
+		teco_string_append(&ret, buffer, read_len);
+	} while (read_len == sizeof(buffer));
+
+	int status = pclose(pipe);
+	if (status < 0 || !WIFEXITED(status)) {
+		g_set_error(error, TECO_ERROR, TECO_ERROR_FAILED,
+		            "Error reaping process from %s", reg_name);
+		return FALSE;
+	}
+	/*
+	 * You may have to add a `|| true` for instance to xclip if it
+	 * could fail for empty selections.
+	 */
+	if (WEXITSTATUS(status) != 0) {
+		g_set_error(error, TECO_ERROR, TECO_ERROR_FAILED,
+		            "Process from %s returned with exit code %d",
+		            reg_name, WEXITSTATUS(status));
+		return FALSE;
+	}
+
+	*str = ret.data;
+	*len = ret.len;
+	memset(&ret, 0, sizeof(ret));
+
+	return TRUE;
+}
+
+#else /* !PDCURSES && !G_OS_UNIX && !CURSES_TTY */
 
 static void
 teco_interface_init_clipboard(void)