# Bugs * The MIDIStream should be flushed when starting via Stream:play(). * There are lots of limitations with Jupyter servers (see README). Perhaps it would be better to use xeus-lua, but that would require us to refactor everything into a proper Lua library and it's harder to build. https://github.com/jupyter-xeus/xeus-lua * CTRL+C interruption on the REPL loop does not always work. Test case: while true do end This is a known bug in LuaJIT, see https://luajit.org/faq.html#ctrlc. As a workaround, add checkint() to all tight loops. # Features * RTNeural support. Should probably be a separate library or an optional dependency. This is important to justify the thesis as related to AI. Since it is a C++ library making extensive use of templates, there will have to be a small C wrapper. Since build time options are important (to choose the backend), it will have to be added as a submodule. Port the official RTNeural-example to Applause. * Real-time input. See inputstream branch. * Line Continuations on the CLI (like Lua's CLI) * CLI auto completions via libreadline. This could directly introspect the Stream object for instance. All of this should be ported to Lua code naturally. * OSC support * Audio input (see inputstream branch) * File writing during live playback to avoid having to use jack_rec. Stream:save() could be adapted and an immediate save could be achieved using a Stream:drain() method or with a Stream:savenow() shortcut. * Multi-core support via GStreamer-like queue: https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-plugins/html/gstreamer-plugins-queue.html * If it turns out, we cannot cleanly fork() after starting the Jack client, it might be possible to send Streams as byte code dumps as well. * Stream optimizer * Automatic caching (finding identical subtrees). * Removing contraproductive caching, e.g. after plain number streams, MIDIStream, EvdevStream... * Automatic paralellization * Automatic eager evaluation of small Streams independant of real-time input. * Moving Stream:evabs() and Stream:sub() closer down the source, so we can make use of per-class optimizations. * ... * Allow stringified timestamps everywhere, where we currently accept lengths in samples. tosamples()? * Github pages (LDoc documentation). They can be automatically pushed by a Github action. Can we change what index.html points to? Alternatively, we could make a simple browser-side forwarding. * Allow limited operation without a running Jack server. * Package everything including Jupyter in Docker container. This requires mounting via -v /dev/shm:/dev/shm:rw -u $(id -u):$(id -g). Ideally, this will work with X11 windows as well. Perhaps bundle https://github.com/Xpra-org/xpra/. The HTML client might even allow integrating Tk windows into Jupyter cells? * Widget toolkit integration. We could optionally integrate Gtk (lgi) or Tcl and drive their main loops from Stream:foreach() or Stream:play() whenever their modules are loaded. This would allow creating GUIs to control streams without having to go via external MIDI applications. There would also have to be a new Stream class that reads values always from the class table, so you can actually change values from Lua callbacks. Or a TclStream for reading Tcl variables. The Tcl/Tk C-API is easy enough to interface directly from FFI. It would also be possible to replace GNU plot with drawing into Tk canvases both in stand-alone windows and for sending them to Jupyter. E.g. this here allows exporting Tk canvases to SVG: https://wiki.tcl-lang.org/page/can2svg There is also http://www.gtk-server.org/, which would be really easy to integrate, runs in its own process (no need to drive the main loop), can potentially work with Broadway (Web) backend that could be hacked into Jupyter as well. On the other hand, to get values back into Lua, you will have to poll a TCP socket or FIFO anyway. * Unit Tests. We could use LuaUnit (https://github.com/bluebird75/luaunit) and directly import luaunit.lua or add a Git submodule. # Improvements * Stream:gtick() is currently not real-time safe but called from tick functions (either to reset the stream or to start is whenever this cannot be prepared for in the parent gtick()). This is especially a problem for SndfileStream. We must somehow avoid all costly operations in gtick(), doing them in ctor() instead. Furthermore, we could theoretically avoid table lookups in gtick() by binding Stream attributes to local variables. This could be achieved most easily by assigning self.gtick to a lambda in ctor(). Furthermore, tick() functions should always cache self.gtick, so as to avoid table lookups. * It could be useful to pass a timestamp (in samples) into the tick function. This will simplify some calculations and allows resetting a stream elegantly (for instance in RepeatStream). It will also speed up seeks, as in SubStream. IndexStream could be without memory limits. Unfortunately, it will not resolve all non-realtime-safe gtick-invocations. It's also questionable what happens when the timestamp wraps. Whether a wrap is safe or not, will depend on the generator. * The JIT compiler currently does not emit SIMD instructions (see simd-test.lua). See also https://github.com/LuaJIT/LuaJIT/issues/40 * Perhaps always enable Fused multiply-add (-O+fma). The reduced accuracy shouldn't hurt us much. * Allow building LuaJIT with -DLUAJIT_ENABLE_LUA52COMPAT. This will enable #Stream and we may implement an __ipairs metamethod. * Per-class optimizations. * SndfileStream:sub() for seeking in sound files. * EvdevStream:evabs() for retrieving the minimum/maximum values automatically.