From 1baab88cf12eb985e456b259920ed2102c67be62 Mon Sep 17 00:00:00 2001 From: Dave Griffiths Date: Wed, 7 Sep 2022 11:18:06 +0100 Subject: [PATCH] sorting docs out a bit --- README.md | 230 ++++--------------------------------------------- docs/manual.md | 199 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 217 insertions(+), 212 deletions(-) create mode 100644 docs/manual.md diff --git a/README.md b/README.md index 11a35e1..66361ce 100644 --- a/README.md +++ b/README.md @@ -4,17 +4,15 @@ A custom sample smashing/re-aggregating app designed by Aphex Twin -*Basic concept* - Samplebrain chops samples up into a 'brain' of interconnected small -samples (blocks) connected into a network by similarity. It reads a +sections (blocks) connected into a network by similarity. It reads a target sample, chops it up into blocks in the same way, and tries to -match each one with one of the sample blocks in it's brain to play in +match each one with one of the blocks in it's brain to play in realtime. -This, in theory, allows you to interpret one type of sound with -another. We gradually added more and more controls until it became -slightly out of control and difficult to explain. +This (in theory) allows you to interpret one type of sound with +another. Over time we gradually added more and more controls until it +became slightly out of control and increasingly difficult to explain. Quick start: @@ -24,206 +22,14 @@ Quick start: 4. Click (re)generate target 5. Press play -## Manual +# [Manual](docs/manual.md) + +# Binaries + +* [Windows]() +* [Mac]() +* Linux - see building below -Brain tweaks: - -These settings control how the block search works. - -### fft / mfcc - -Choose whether to search using FFT (raw frequency analysis) or MFCC -(Mel-frequency cepstral coefficients) which are higher order paramters -that attempt to model perception of sound. MFCC is usually a bit -better, but it depends on what you are doing, you can blend between -them to use a mix. Setting this to 0% or 100% switches off the other -search option, so is a bit more CPU friendly. - -### freq & dynamics / freq only - -Search using both frequency (pitch) and dynamics (volume changes over -time), or only frequency - which uses normalised blocks. Generally you -want the first option. - -### fft subsection - -When using FFT mode you can select a subrange of the (100) frequency -bins to use for scoring potential blocks, potentially allowing you to -target a specific frequency range. Not terribly useful in practice. - -### novelty - -One thing that tends to happen is that the same block or set of blocks -can be overused if there isn't enough variation in the brain -blocks. Sometimes we want to bias the selection against reuse, so -novelty biases the selection away from similarity - if you turn it all -the way up it will ignore the target completely and just play the -least used ones in some odd semi-random order. - -### boredom - -This increases the speed at which novelty wears off, creating a wider -spread of possible blocks. Not quite clear exactly why this is -different to increasing novelty, but it sounds different. - -### stickyness - -If the error is under this threshold, play the next block in the brain -rather than the closest. This will have the effect of elongating -chunks of brain samples that you hear. - -### search stretch - -Repeats blocks in the target a fixed amount, like a simple timestretch -- in synaptic mode this gives the system repeated attempts to find a -closer match. - -### algorithm - -* basic - -Searches all samples in the brain, and uses the closest match. - -* reversed - -Searches all samples in the brain, and uses the least closest -match. In practice this needs work, as it tends to select silent or -very quiet blocks. - -* synaptic - -As brains get larger, we get more blocks, and they get slower to -search. This mode provides a constant search time over arbitrarily -huge brains. When generating the brains we connect them together into -a network via similarity (via connections called synapses). We keep a -position in the network and only search the nearby blocks - this -assumes that sounds tend to change gradually, or at least more -gradually than the small block lengths. - -* slide - -Similar to synaptic but if we can't find a close enough match (based -on synaptic slide error) we stretch the target, repeating blocks until -we land on a block that is close enough. This mode warps the timing of -the target. - -### num synapses - -How many connections to check in synaptic or slide mode. - -### synaptic slide error - -The acceptable error to consider a block as "close enought" in slide mode. - -## Target sound: - -These settings control how the target sound is broken up into blocks. - -### load target - -Load a target sound to try and match - -### block size - -The size of the blocks in samples. This does not need to match the -brain block size, but it probably should. - -### block overlap - -Percentage overlap in blocks. - -### window shape - -The shape of the window - "dodgy" is actually box. - -### (re)generate blocks - -Compute the target blocks. - -### use mic input - -Attempts to stream blocks live from the microphone. I think this is -broken at present. - -## Mix: - -These are settings that happen after the search. - -### autotune - -Attempt to pitch bend the chosen brain block to better match the target. - -### normalised - -Mix in normalised brain blocks - removing all dynamics. Might work -with frequency only search. - -### brain / target - -Mix in the target blocks to the output - for cheating, or testing purposes. - -### stereo mode - -Run everything once for left and again for right speaker. - -## Brain contents - -These settings allow you to build a brain of samples, and switch in -and out specific samples during playback. - -### all/none - -You can select which samples to use without regenerating the -brain. This selects all or none of the samples. - -### load sound/directory/clear - -Load sounds into the brain, either individually or entire directories -in one go. - -### block size - -The size of the blocks in samples. This does not need to match the -target block size, but it probably should. - -### block overlap - -Percentage overlap in blocks. - -### window shape - -The shape of the window - "dodgy" is actually box. - -### (re)generate blocks - -Compute the brain blocks. - -### load brain/save brain - -You can save and load brains separately to the targets. - -## Lower bar - -General playback settings - -### play/pause/record/stop - -Start/stop and record - -### volume - -Global volume - -### load/save session - -Load and save the entire session. - -## Net tab - -This allows you to control multiple instances of samplebrain over the -network all running their own brains simultaneously. This feature has -not been tested well! - # Installing/building ## Linux install: @@ -245,18 +51,18 @@ Build $ run it: ## What's here -1. app: - * main app code -2. brain: - * sample granulation engine code +1. brain: + * samplebrain engine code +2. app: + * code to build the Qt GUI app 3. gui: * qt designer project files 4. cooking: * sketches and ideas * proof of concept written in python - * initial attept at clojure version (abandoned) + * initial (abandoned) attempt at clojure version -# Todo: +# Todos: * Write VST version? diff --git a/docs/manual.md b/docs/manual.md new file mode 100644 index 0000000..97e67b6 --- /dev/null +++ b/docs/manual.md @@ -0,0 +1,199 @@ +# Samplebrain Manual + +## Brain tweaks: + +These settings control how the block search works. + +### fft / mfcc + +Choose whether to search using FFT (raw frequency analysis) or MFCC +(Mel-frequency cepstral coefficients) which are higher order paramters +that attempt to model perception of sound. MFCC is usually a bit +better, but it depends on what you are doing, you can blend between +them to use a mix. Setting this to 0% or 100% switches off the other +search option, so is a bit more CPU friendly. + +### freq & dynamics / freq only + +Search using both frequency (pitch) and dynamics (volume changes over +time), or only frequency - which uses normalised blocks. Generally you +want the first option. + +### fft subsection + +When using FFT mode you can select a subrange of the (100) frequency +bins to use for scoring potential blocks, potentially allowing you to +target a specific frequency range. Not terribly useful in practice. + +### novelty + +One thing that tends to happen is that the same block or set of blocks +can be overused if there isn't enough variation in the brain +blocks. Sometimes we want to bias the selection against reuse, so +novelty biases the selection away from similarity - if you turn it all +the way up it will ignore the target completely and just play the +least used ones in some odd semi-random order. + +### boredom + +This increases the speed at which novelty wears off, creating a wider +spread of possible blocks. Not quite clear exactly why this is +different to increasing novelty, but it sounds different. + +### stickyness + +If the error is under this threshold, play the next block in the brain +rather than the closest. This will have the effect of elongating +chunks of brain samples that you hear. + +### search stretch + +Repeats blocks in the target a fixed amount, like a simple timestretch +- in synaptic mode this gives the system repeated attempts to find a +closer match. + +### algorithm + +* basic + +Searches all samples in the brain, and uses the closest match. + +* reversed + +Searches all samples in the brain, and uses the least closest +match. In practice this needs work, as it tends to select silent or +very quiet blocks. + +* synaptic + +As brains get larger, we get more blocks, and they get slower to +search. This mode provides a constant search time over arbitrarily +huge brains. When generating the brains we connect them together into +a network via similarity (via connections called synapses). We keep a +position in the network and only search the nearby blocks - this +assumes that sounds tend to change gradually, or at least more +gradually than the small block lengths. + +* slide + +Similar to synaptic but if we can't find a close enough match (based +on synaptic slide error) we stretch the target, repeating blocks until +we land on a block that is close enough. This mode warps the timing of +the target. + +### num synapses + +How many connections to check in synaptic or slide mode. + +### synaptic slide error + +The acceptable error to consider a block as "close enought" in slide mode. + +## Target sound: + +These settings control how the target sound is broken up into blocks. + +### load target + +Load a target sound to try and match + +### block size + +The size of the blocks in samples. This does not need to match the +brain block size, but it probably should. + +### block overlap + +Percentage overlap in blocks. + +### window shape + +The shape of the window - "dodgy" is actually box. + +### (re)generate blocks + +Compute the target blocks. + +### use mic input + +Attempts to stream blocks live from the microphone. I think this is +broken at present. + +## Mix: + +These are settings that happen after the search. + +### autotune + +Attempt to pitch bend the chosen brain block to better match the target. + +### normalised + +Mix in normalised brain blocks - removing all dynamics. Might work +with frequency only search. + +### brain / target + +Mix in the target blocks to the output - for cheating, or testing purposes. + +### stereo mode + +Run everything once for left and again for right speaker. + +## Brain contents + +These settings allow you to build a brain of samples, and switch in +and out specific samples during playback. + +### all/none + +You can select which samples to use without regenerating the +brain. This selects all or none of the samples. + +### load sound/directory/clear + +Load sounds into the brain, either individually or entire directories +in one go. + +### block size + +The size of the blocks in samples. This does not need to match the +target block size, but it probably should. + +### block overlap + +Percentage overlap in blocks. + +### window shape + +The shape of the window - "dodgy" is actually box. + +### (re)generate blocks + +Compute the brain blocks. + +### load brain/save brain + +You can save and load brains separately to the targets. + +## Lower bar + +General playback settings + +### play/pause/record/stop + +Start/stop and record + +### volume + +Global volume + +### load/save session + +Load and save the entire session. + +## Net tab + +This allows you to control multiple instances of samplebrain over the +network all running their own brains simultaneously. This feature has +not been tested well!