some manual work and fixed crash bug with cancel loading a session

This commit is contained in:
Dave Griffiths 2022-09-07 11:03:23 +01:00
parent 4a2092339b
commit d394b0c182
6 changed files with 236 additions and 27 deletions

240
README.md
View File

@ -2,10 +2,230 @@
## What is this?
A custom sample granulation app designed by Aphex Twin
A custom sample smashing/re-aggregating app designed by Aphex Twin
*Basic concept*
Samplebrain chops samples up into a 'brain' of interconnected small
samples (blocks) connected into a network by similarity. It reads a
target sample, chops it up into blocks in the same way, and tries to
match each one with one of the sample blocks in it's brain to play in
realtime.
This, in theory, allows you to interpret one type of sound with
another. We gradually added more and more controls until it became
slightly out of control and difficult to explain.
Quick start:
1. Load a bunch of samples into the brain
2. Click (re)generate brain
3. Load a loop sample into the target
4. Click (re)generate target
5. Press play
## Manual
Brain tweaks:
These settings control how the block search works.
### fft / mfcc
Choose whether to search using FFT (raw frequency analysis) or MFCC
(Mel-frequency cepstral coefficients) which are higher order paramters
that attempt to model perception of sound. MFCC is usually a bit
better, but it depends on what you are doing, you can blend between
them to use a mix. Setting this to 0% or 100% switches off the other
search option, so is a bit more CPU friendly.
### freq & dynamics / freq only
Search using both frequency (pitch) and dynamics (volume changes over
time), or only frequency - which uses normalised blocks. Generally you
want the first option.
### fft subsection
When using FFT mode you can select a subrange of the (100) frequency
bins to use for scoring potential blocks, potentially allowing you to
target a specific frequency range. Not terribly useful in practice.
### novelty
One thing that tends to happen is that the same block or set of blocks
can be overused if there isn't enough variation in the brain
blocks. Sometimes we want to bias the selection against reuse, so
novelty biases the selection away from similarity - if you turn it all
the way up it will ignore the target completely and just play the
least used ones in some odd semi-random order.
### boredom
This increases the speed at which novelty wears off, creating a wider
spread of possible blocks. Not quite clear exactly why this is
different to increasing novelty, but it sounds different.
### stickyness
If the error is under this threshold, play the next block in the brain
rather than the closest. This will have the effect of elongating
chunks of brain samples that you hear.
### search stretch
Repeats blocks in the target a fixed amount, like a simple timestretch
- in synaptic mode this gives the system repeated attempts to find a
closer match.
### algorithm
* basic
Searches all samples in the brain, and uses the closest match.
* reversed
Searches all samples in the brain, and uses the least closest
match. In practice this needs work, as it tends to select silent or
very quiet blocks.
* synaptic
As brains get larger, we get more blocks, and they get slower to
search. This mode provides a constant search time over arbitrarily
huge brains. When generating the brains we connect them together into
a network via similarity (via connections called synapses). We keep a
position in the network and only search the nearby blocks - this
assumes that sounds tend to change gradually, or at least more
gradually than the small block lengths.
* slide
Similar to synaptic but if we can't find a close enough match (based
on synaptic slide error) we stretch the target, repeating blocks until
we land on a block that is close enough. This mode warps the timing of
the target.
### num synapses
How many connections to check in synaptic or slide mode.
### synaptic slide error
The acceptable error to consider a block as "close enought" in slide mode.
## Target sound:
These settings control how the target sound is broken up into blocks.
### load target
Load a target sound to try and match
### block size
The size of the blocks in samples. This does not need to match the
brain block size, but it probably should.
### block overlap
Percentage overlap in blocks.
### window shape
The shape of the window - "dodgy" is actually box.
### (re)generate blocks
Compute the target blocks.
### use mic input
Attempts to stream blocks live from the microphone. I think this is
broken at present.
## Mix:
These are settings that happen after the search.
### autotune
Attempt to pitch bend the chosen brain block to better match the target.
### normalised
Mix in normalised brain blocks - removing all dynamics. Might work
with frequency only search.
### brain / target
Mix in the target blocks to the output - for cheating, or testing purposes.
### stereo mode
Run everything once for left and again for right speaker.
## Brain contents
These settings allow you to build a brain of samples, and switch in
and out specific samples during playback.
### all/none
You can select which samples to use without regenerating the
brain. This selects all or none of the samples.
### load sound/directory/clear
Load sounds into the brain, either individually or entire directories
in one go.
### block size
The size of the blocks in samples. This does not need to match the
target block size, but it probably should.
### block overlap
Percentage overlap in blocks.
### window shape
The shape of the window - "dodgy" is actually box.
### (re)generate blocks
Compute the brain blocks.
### load brain/save brain
You can save and load brains separately to the targets.
## Lower bar
General playback settings
### play/pause/record/stop
Start/stop and record
### volume
Global volume
### load/save session
Load and save the entire session.
## Net tab
This allows you to control multiple instances of samplebrain over the
network all running their own brains simultaneously. This feature has
not been tested well!
# Installing/building
## Linux install:
Install libraries for the sample engine:
@ -16,19 +236,27 @@ Install dependancies for the graphical user interface:
$ sudo apt install build-essential qtcreator qt5-default
Build it:
Build $ run it:
$ cd app
$ qmake
$ make
$ samplebrain
## What's here
1. app:
main app and QT gui code
* main app code
2. brain:
sample granulation engine code
3. cooking:
* sample granulation engine code
3. gui:
* qt designer project files
4. cooking:
* sketches and ideas
* proof of concept written in python
* initial attept at clojure version (abandoned)
# Todo:
* Write VST version?

View File

@ -101,7 +101,8 @@ void MainWindow::init_from_session(const string &filename) {
// pull the bits out of the file to set the interface...
// is this easier than direct access? no idea??
ifstream ifs(filename.c_str(),ios::binary);
if (!ifs) return;
brain s,t;
u32 version=0;
ifs||version;

View File

@ -1,20 +0,0 @@
#!/usr/bin/env python
import re
import sys
fn = sys.argv[1]
test = open(fn, 'r').read()
pattern = re.compile('[A-Z](?=[a-z])')
def format_term(term):
return '_%s' % (term.lower())
test = pattern.sub(lambda m: format_term(m.group(0)), test)
test = re.sub(r'(?<=[^a-z])_(?=[a-z])','',test)
o = open(fn,'w')
o.write(test)
o.close()

View File

Before

Width:  |  Height:  |  Size: 4.5 KiB

After

Width:  |  Height:  |  Size: 4.5 KiB