1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
|
# Bugs
* The MIDIStream should be flushed when starting via Stream:play().
* There are lots of limitations with Jupyter servers (see README).
Perhaps it would be better to use xeus-lua, but that would require us to refactor everything
into a proper Lua library and it's harder to build.
https://github.com/jupyter-xeus/xeus-lua
* CTRL+C interruption on the REPL loop does not always work.
Test case: while true do end
This is a known bug in LuaJIT, see https://luajit.org/faq.html#ctrlc.
As a workaround, add checkint() to all tight loops.
# Features
* RTNeural support. Should probably be a separate library or an optional dependency.
Since it is a C++ library making extensive use of templates, there will have to be
a small C wrapper.
Since build time options are important (to choose the backend), it will have to be
added as a submodule.
Port the official RTNeural-example to Applause.
* Real-time input. See inputstream branch.
* Line Continuations on the CLI (like Lua's CLI)
* CLI auto completions via libreadline.
This could directly introspect the Stream object for instance.
All of this should be ported to Lua code naturally.
* OSC support
* Audio input (see inputstream branch)
* File writing during live playback to avoid having to use jack_rec.
Stream:save() could be adapted and an immediate save could be achieved using a Stream:drain() method
or with a Stream:savenow() shortcut.
* Multi-core support via GStreamer-like queue:
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-plugins/html/gstreamer-plugins-queue.html
* If it turns out, we cannot cleanly fork() after starting the Jack client,
it might be possible to send Streams as byte code dumps as well.
* Stream optimizer
* Automatic caching (finding identical subtrees).
* Removing contraproductive caching, e.g. after plain number streams, MIDIStream, EvdevStream...
* Automatic paralellization
* Automatic eager evaluation of small Streams independant of real-time input.
* Moving Stream:evabs() and Stream:sub() closer down the source, so we can make
use of per-class optimizations.
* ...
* Allow stringified timestamps everywhere, where we currently accept lengths in samples. tosamples()?
* Github pages (LDoc documentation).
They can be automatically pushed by a Github action.
Can we change what index.html points to?
Alternatively, we could make a simple browser-side forwarding.
* Allow limited operation without a running Jack server.
* Package everything including Jupyter in Docker container.
This requires mounting via -v /dev/shm:/dev/shm:rw -u $(id -u):$(id -g).
Ideally, this will work with X11 windows as well.
Perhaps bundle https://github.com/Xpra-org/xpra/.
The HTML client might even allow integrating Tk windows into Jupyter cells?
* Widget toolkit integration.
We could optionally integrate Gtk (lgi) or Tcl and drive their main loops from Stream:iter() or Stream:play()
whenever their modules are loaded.
This would allow creating GUIs to control streams without having to go via external MIDI
applications. There would also have to be a new Stream class that reads values always from the
class table, so you can actually change values from Lua callbacks.
Or a TclStream for reading Tcl variables.
The Tcl/Tk C-API is easy enough to interface directly from FFI.
It would also be possible to replace GNU plot with drawing into Tk canvases both in stand-alone
windows and for sending them to Jupyter.
E.g. this here allows exporting Tk canvases to SVG: https://wiki.tcl-lang.org/page/can2svg
There is also http://www.gtk-server.org/, which would be really easy to integrate,
runs in its own process (no need to drive the main loop), can potentially work with Broadway (Web)
backend that could be hacked into Jupyter as well.
On the other hand, to get values back into Lua, you will have to poll a TCP socket or FIFO anyway.
* Spectrogram output via ffmpeg.
This can be made to work in a window, Jupyter notebook and in the console.
Theoretically, we can even support realtime analysis and render into Kitty by piping into mpv.
* Built-in tracker-like sequencer, ie. stream compiler.
Will read a simple DSL where each tab corresponds to an output stream.
Rows will be triggered in a given tempo. In each cell, you can place a MIDI note name
or directly a number.
The result is an automatic note-on/off signal. This wouldn't be compatible with Stream:mvelocity(),
though.
* Unit Tests.
We could use LuaUnit (https://github.com/bluebird75/luaunit) and directly import luaunit.lua
or add a Git submodule.
# Improvements
* Stream:gtick() is currently not real-time safe but called from tick functions
(either to reset the stream or to start is whenever this cannot be prepared for in the parent gtick()).
This is especially a problem for SndfileStream.
We must somehow avoid all costly operations in gtick(), doing them in ctor() instead.
Furthermore, we could theoretically avoid table lookups in gtick() by binding
Stream attributes to local variables.
This could be achieved most easily by assigning self.gtick to a lambda in ctor().
Furthermore, tick() functions should always cache self.gtick, so as to avoid
table lookups.
* It could be useful to pass a timestamp (in samples) into the tick function.
This will simplify some calculations and allows resetting a stream elegantly (for instance in RepeatStream).
It will also speed up seeks, as in SubStream.
IndexStream could be without memory limits.
Unfortunately, it will not resolve all non-realtime-safe gtick-invocations.
It's also questionable what happens when the timestamp wraps. Whether a wrap is safe or not, will depend
on the generator.
* The JIT compiler currently does not emit SIMD instructions (see simd-test.lua).
See also https://github.com/LuaJIT/LuaJIT/issues/40
* Perhaps always enable Fused multiply-add (-O+fma).
The reduced accuracy shouldn't hurt us much.
* Allow building LuaJIT with -DLUAJIT_ENABLE_LUA52COMPAT.
This will enable #Stream and we may implement an __ipairs metamethod.
* Per-class optimizations.
* SndfileStream:sub() for seeking in sound files.
* EvdevStream:evabs() for retrieving the minimum/maximum values automatically.
* More options for plotting into terminals via Stream:gnuplot().
Theoretically we can support xterm and others via braille characters (gnuplot's block terminal).
The problem is that we cannot detect automatically whether this is safe and would have to rely on some kind of
configuration.
It might also be viable to do our own graphics via Cairo, which allows SVG output in contrast to
Tk canvases.
* Automatically detect whether terminal supports the Kitty graphics protocol:
https://sw.kovidgoyal.net/kitty/graphics-protocol/#querying-support-and-available-transmission-mediums
* Two C non-inlineable function calls per sample due to applause_push_sample() could be avoided by including our
own lockless ring-buffer implementation.
* Document ZipStream semantics - see chapter Stream Algebra in my thesis.
|