partially reversed/fixed-up b7ff56db631: avoid g_slice allocators and performance issues with memory measurements

 * Fixed build problems on Windows
 * g_slice on Windows has been shown to be of little use either
   and it does not work well with the GetProcessMemoryInfo()
   measurements.
   Also, it brings the same problem as on Glibc: Not even command-line
   termination returns the memory to the OS.
   Therefore, we don't use g_slice at all and commented on it.
 * The custom Linux and Windows memory measurement approaches
   have been shown to be inefficient.
   As a workaround, scripts disable memory limiting.
 * A better approach -- but it will only work on Glibc -- might
   be to hook into malloc(), realloc() and free() globally
   and use the malloc_usable_size() of a heap object for
   memory measurements. This will be relatively precise and cheap.
 * We still need the "Object" base class in order to measure
   memory usage as a fallback approach.

7 files changed, 47 insertions, 14 deletions

diff --git a/TODO b/TODO
index e32bf31..d9418a6 100644
--- a/TODO
+++ b/TODO
@@ -234,6 +234,21 @@ Features:
    Macros may retrieve the code and string of the last error.
 
 Optimizations:
+ * The Linux-specific memory limiting using mallinfo() is
+   very slow (50% to 150% slower than the fallback implementation
+   measured in batch mode).
+   The fallback implementation does not come with much of a
+   runtime penalty.
+   Still I've found no faster way of measuring the process heap.
+   A soft resource limit would be ideal but unfortunately,
+   it lets malloc() return NULL and we're not in control of all
+   the mallocs, so glib could abort before we have a chance to
+   react on it.
+   Since the slow-down affects interactive mode as well, disabling
+   limiting in batch mode is merely a workaround.
+   Perhaps Linux should simply use the fallback limiting as well
+   (or support this via a configure option).
+   On Windows (2000), the overhead is approx. the same.
  * Add G_UNLIKELY to all error throws.
  * Instead of using RTTI to implement the immediate editing command
    behaviours in Cmdline::process_edit_cmd() depending on the current
diff --git a/configure.ac b/configure.ac
index 42094ec..c8dab97 100644
--- a/configure.ac
+++ b/configure.ac
@@ -126,6 +126,8 @@ case $host in
 *-mingw*)
 	AC_CHECK_HEADERS([windows.h psapi.h], , [
 		AC_MSG_ERROR([Missing Windows headers!])
+	], [
+		#include <windows.h>
 	])
 
 	# Make sure we get GetProcessMemoryInfo():
diff --git a/doc/grosciteco.tes b/doc/grosciteco.tes
index 262f2c0..e94603e 100755
--- a/doc/grosciteco.tes
+++ b/doc/grosciteco.tes
@@ -1,6 +1,8 @@
 #!/usr/local/bin/sciteco -m
 !* grosciteco.tes [-t <output-tec>] [--] <output-woman> <input> *!
 
+0,2EJ !* FIXME: Memory limiting is too slow *!
+
 :EMQ[$SCITECOPATH]/getopt.tes
 
 !* Process command-line options *!
diff --git a/doc/htbl.tes b/doc/htbl.tes
index fced7e7..11851ae 100755
--- a/doc/htbl.tes
+++ b/doc/htbl.tes
@@ -2,6 +2,8 @@
 !* htbl.tes <input> <output> *!
 !* Troff tbl "drop-in" replacement *!
 
+0,2EJ !* FIXME: Memory limiting is too slow *!
+
 LR 0X#in 2LR 0X#ou EBQ#in EB -EF
 
 <
diff --git a/doc/tedoc.tes b/doc/tedoc.tes
index 623be3a..f89c45f 100755
--- a/doc/tedoc.tes
+++ b/doc/tedoc.tes
@@ -1,6 +1,8 @@
 #!/usr/local/bin/sciteco -m
 !* tedoc.tes [-C] [--] <output> <input> <sources> *!
 
+0,2EJ !* FIXME: Memory limiting is too slow *!
+
 :EMQ[$SCITECOPATH]/getopt.tes
 
 @[format_header]{
diff --git a/src/memory.cpp b/src/memory.cpp
index 853e52f..329a9af 100644
--- a/src/memory.cpp
+++ b/src/memory.cpp
@@ -51,9 +51,12 @@ MemoryLimit::get_usage(void)
 	/*
 	 * NOTE: `uordblks` is an int and thus prone
 	 * to wrap-around issues.
+	 *
 	 * Unfortunately, the only other machine readable
 	 * alternative is malloc_info() which prints
-	 * into a FILE * stream [sic!]
+	 * into a FILE * stream [sic!] and is unspeakably
+	 * slow even if writing to an unbuffered fmemopen()ed
+	 * stream.
 	 */
 	return info.uordblks;
 }
@@ -72,7 +75,7 @@ MemoryLimit::get_usage(void)
 	 * happens, we can just as well terminate abnormally.
 	 */
 	if (G_UNLIKELY(!GetProcessMemoryInfo(GetCurrentProcess(),
-	                                     &info, sizeof(info))) {
+	                                     &info, sizeof(info)))) {
 		gchar *msg = g_win32_error_message(GetLastError());
 		g_error("Cannot get memory usage: %s", msg);
 		/* shouldn't be reached */
@@ -108,7 +111,7 @@ MemoryLimit::set_limit(gsize new_limit)
 
 		Error err("Cannot set undo memory limit (%s): "
 		          "Current usage too large (%s).",
-		          usage_str, limit_str);
+		          limit_str, usage_str);
 
 		g_free(limit_str);
 		g_free(usage_str);
@@ -136,32 +139,37 @@ void *
 Object::operator new(size_t size) noexcept
 {
 #ifdef USE_MEMORY_COUNTING
-	SciTECO::memory_usage += size;
+	memory_usage += size;
 #endif
 
-#ifdef HAVE_MALLOC_TRIM
 	/*
-	 * Using g_slice would render malloc_trim()
+	 * Since we've got the sized-delete operator
+	 * below, we could allocate via g_slice.
+	 *
+	 * Using g_slice however would render malloc_trim()
 	 * ineffective. Also, it has been shown to be
 	 * unnecessary on Linux/glibc.
+	 * Glib is guaranteed to use the system malloc(),
+	 * so g_malloc() cooperates with malloc_trim().
+	 *
+	 * On Windows (even Windows 2000), the slice allocator
+	 * did not show any significant performance boost
+	 * either. Also, since g_slice never seems to return
+	 * memory to the OS and we cannot force it to do so,
+	 * it will not cooperate with the Windows-specific
+	 * memory measurement and it is hard to recover
+	 * from memory limit exhaustions.
 	 */
 	return g_malloc(size);
-#else
-	return g_slice_alloc(size);
-#endif
 }
 
 void
 Object::operator delete(void *ptr, size_t size) noexcept
 {
-#ifdef HAVE_MALLOC_TRIM
 	g_free(ptr);
-#else
-	g_slice_free1(size, ptr);
-#endif
 
 #ifdef USE_MEMORY_COUNTING
-	SciTECO::memory_usage -= size;
+	memory_usage -= size;
 #endif
 }
 
diff --git a/src/symbols-extract.tes b/src/symbols-extract.tes
index a88731d..7d12ea8 100755
--- a/src/symbols-extract.tes
+++ b/src/symbols-extract.tes
@@ -4,6 +4,8 @@
  *                       <output file> <input header>
  *!
 
+0,2EJ !* FIXME: Memory limiting is too slow *!
+
 :EMQ[$SCITECOPATH]/getopt.tes
 EMQ[$SCITECOPATH]/string.tes