| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
* it turned out to be possible to provoke memory_usage
overflows or underruns, resulting in unrecoverable states
* a possible reason can be that at least with G++ 5.4.0,
the compiler would sometimes call the (default) non-sized
delete followed by our custom sized delete/deallocator.
* This was true even after compiling Scintilla with -fsized-deallocation.
* therefore we provide an empty non-sized delete now.
* memory_usage counting can now be debugged by uncommenting
DEBUG_MAGIC in memory.cpp. This uses a magic value to detect
instrumented allocations being mixed with non-instrumented
allocations.
* simplified the global sized-deallocation functions
(they are identical to the Object-class allocators).
|
|
and added a FreeBSD/jemalloc-specific implementation
* largely reverts 39cfc573, but leaves in minor and documentation
changes.
* further experimentation of memory limiting using malloc() wrapping
has shown additional problems, like dlsym() calling malloc-functions,
further reducing the implementation to glibc-specific means.
This means there had been no implementation for FreeBSD and checks
would have to rely on undocumented internal implementation details
of different libcs, which is not a good thing.
* Other problems have been identified, like having to wrap calloc(),
guarding against underruns and multi-thread safety had been identified
but could be worked around.
* A technique by calculating the memory usage as sbrk(0) - &end
has been shown to be effective enough, at least on glibc.
However even on glibc it has shortcomings since malloc() will
somtimes use mmap() for allocations and the technique
relies on implementation details of the libc.
Furthermore another malloc_trim(0) had to be added to the error
recovery in interactive mode, since glibc does not adjust the program break
automatically (to avoid syscalls I presume).
* On FreeBSD/jemalloc, the sbrk(0) method totally fails because jemalloc
exclusively allocates via mmap() -> that solution was discarded as well.
* Since all evaluated techniques turn out to be highly platform
specific, I reverted to the simple and stable platform-specific
mallinfo() API on Linux.
* On FreeBSD/jemalloc, it's possible to use mallctl("stats.allocated")
for the same purpose - so it works there, too now.
It's slower than the other techniques, though.
* A lengthy discussion has been added to memory.cpp, so that we
do not repeat the previous mistakes.
|
|
|
|
* we were basing the glib allocators on throwing std::bad_alloc just like
the C++ operators. However, this always was unsafe since we were throwing
exceptions across plain-C frames (Glib).
Also, the memory vtable has been deprecated in Glib, resulting in
ugly warnings.
* Instead, we now let the C++ new/delete operators work like Glib
by basing them on g_malloc/g_slice.
This means they will assert and the application will terminate
abnormally in case of OOM. OOMs cannot be handled properly anyway, so it is
more important to have a good memory limiting mechanism.
* Memory limiting has been completely revised.
Instead of approximating undo stack sizes using virtual methods
(which is unprecise and comes with a performance penalty),
we now use a common base class SciTECO::Object to count the memory
required by all objects allocated within SciTECO.
This is less precise than using global replacement new/deletes
which would allow us to control allocations in all C++ code including
Scintilla, but they are only supported as of C++14 (GCC 5) and adding compile-time
checks would be cumbersome.
In any case, we're missing Glib allocations (esp. strings).
* As a platform-specific extension, on Linux/glibc we use mallinfo()
to count the exact memory usage of the process.
On Windows, we use GetProcessMemoryInfo() -- the latter implementation
is currently UNTESTED.
* We use g_malloc() for new/delete operators when there is
malloc_trim() since g_slice does not free heap chunks properly
(probably does its own mmap()ing), rendering malloc_trim() ineffective.
We've also benchmarked g_slice on Linux/glib (malloc_trim() shouldn't
be available elsewhere) and found that it brings no significant
performance benefit.
On all other platforms, we use g_slice since it is assumed
that it at least does not hurt.
The new g_slice based allocators should be tested on MSVCRT
since I assume that they bring a significant performance benefit
on Windows.
* Memory limiting does now work in batch mode as well and is still
enabled by default.
* The old UndoTokenWithSize CRTP hack could be removed.
UndoStack operations should be a bit faster now.
But on the other hand, there will be an overhead due to repeated
memory limit checking on every processed character.
|
