Sensomusic

#1 2018-09-28 22:21:40

x.iso
Platinum Member

Suddenly idea came to mind when I remembered how in professional TV studios you can find oscilloscopes measuring RBG signal from video or HSL or other parameters derived from the current frame of the video. Shouldn't be technically a problem to do the same in digital, but is it possible to get any wavefrom derived from video (or bitmap) out with the tools Usine have? I suspect it should be with bitwise conversions and math modules, but I'm not really good at that stuff.

In the end the signals could be used as oscillators for synths or LFO's and it could get really interesting from there.

Offline

 

#2 2018-09-29 09:53:59

oli_lab
Platinum Member

very interesting.
could you point us towards such hardware oscilloscopes ?


http://oli-lab.org

Win10 I7/16GB RAM  - RME MADIFACE - SSL alpha link 4-16 - OSC capable interfaces

follow OLI_LAB adventures on Facebook
http://www.facebook.com/pages/OLI_LAB/3 … 506?v=wall

Offline

 

#3 2018-09-29 10:09:26

23fx23
Community

there are converters in \color, usine color to separated rgb, or hsl and vice versa, and in video\ video to color array,
could scan a line, find  grid median, build and histogram or something with that.

but the main question i think is how to translate that to a continuous sound waveform. a typical video framerate is only 25fps, vs 44100 needed audio,
there would need something to generate blocs size arrays that have a making sense  'continuity' from one frame to the next this sound more tricky reflexion
but very interesting.

Online

 

#4 2018-09-29 22:28:49

x.iso
Platinum Member

23fx23 wrote:

there are converters in \color, usine color to separated rgb, or hsl and vice versa, and in video\ video to color array,
could scan a line, find  grid median, build and histogram or something with that.

but the main question i think is how to translate that to a continuous sound waveform. a typical video framerate is only 25fps, vs 44100 needed audio,
there would need something to generate blocs size arrays that have a making sense  'continuity' from one frame to the next this sound more tricky reflexion
but very interesting.


well that's not really much of a problem since you can just use Smooth to blend choppy data.

Offline

 

#5 2018-09-29 22:50:33

x.iso
Platinum Member

oli_lab wrote:

very interesting.
could you point us towards such hardware oscilloscopes ?


https://www.youtube.com/watch?v=PQQZlSyF2-0
here's typical example how it looks

Offline

 

#6 2018-09-29 23:28:28

oli_lab
Platinum Member

x.iso wrote:
oli_lab wrote:

very interesting.
could you point us towards such hardware oscilloscopes ?


https://www.youtube.com/watch?v=PQQZlSyF2-0
here's typical example how it looks


Ok
what you see is the analog signal of an old CRT TV.
it is sync to the signal that make the spot going from bottom rigtnt to top left at the end of each frame.
I'm not sure we get similar signals with digital video.

will need to scan the entire content of the pixel array line by line
and asuming we managed to interpolate those datas to ply them at desired audiorate. we'll get probably something very noisy !

let say we have a 800x600 videostream
that is 480 000 pixels

Now we want to play this image at 100Hz, We need to read the entire image 100 times per second, that is (if we assume the sample rate is 48 000Hz) to read a sample that is of 48000/100 = 480 samples.
so we need to get the mean value of each subsequent 1000 pixels so the 480 000 pixel will be représented in the 480 audio samples. Or simply read every 1000 pixels in the array and interpolate

we need a new module !


http://oli-lab.org

Win10 I7/16GB RAM  - RME MADIFACE - SSL alpha link 4-16 - OSC capable interfaces

follow OLI_LAB adventures on Facebook
http://www.facebook.com/pages/OLI_LAB/3 … 506?v=wall

Offline

 

#7 2018-09-30 23:42:38

x.iso
Platinum Member

So yeah, with some patching I was able to get some waveforms derived from video, now I'm going to make some simple synth out of it.
http://www.sensomusic.org/forums/uploads.php?file=2018-10-01.png

Offline

 

#8 2018-09-30 23:45:59

x.iso
Platinum Member

Actually, there's so much data in those arrays that I guess it can be chopped to chunks for independent voices

Offline

 

#9 2018-10-01 02:27:53

x.iso
Platinum Member

here's results so far. looks promising
https://youtu.be/Eza6AtpbSlA

Offline

 

#10 2018-10-02 09:53:04

23fx23
Community

Cool! Gonna check tonight

Online

 

#11 2018-10-02 18:54:57

x.iso
Platinum Member

here's a patch I've made so far. You might need to swap VST limiter for stock one or something else.
http://www.sensomusic.org/forums/upload … dSynth.pat

Last edited by x.iso (2018-10-02 18:55:06)

Offline

 

#12 2018-10-03 01:17:36

x.iso
Platinum Member

By the way, is it possible to 'smooth' array values? By that I mean lessen the gap between adjacent values by a factor, could help make smoother waveforms.

Offline

 

#13 2018-10-03 08:17:16

23fx23
Community

As far as i remember, not directly via build in modules. Could use a lowpass filter but that will limited to fixed blocs size. Sadly very buzy at work till this weekend, but if wanna investigate i had an addon that used a script procedure to blur/smooth an array. Its called automizer if still accessible in addons

Online

 

#14 2018-10-03 14:37:47

x.iso
Platinum Member

It works better with oscillator module from Oli_Lab. here's some progress, quite usable at this point.
https://www.youtube.com/watch?v=HPsv44YiLhs

Offline

 

#15 2018-10-03 16:07:06

oli_lab
Platinum Member

x.iso wrote:

It works better with oscillator module from Oli_Lab. here's some progress, quite usable at this point.
https://www.youtube.com/watch?v=HPsv44YiLhs


I was  about  to tell you that.
I did some experiments from the picture of your patch.
With draw lut you don't need so much to compress the array.
I did not used the concat array but used separate oscillators for each color.


http://oli-lab.org

Win10 I7/16GB RAM  - RME MADIFACE - SSL alpha link 4-16 - OSC capable interfaces

follow OLI_LAB adventures on Facebook
http://www.facebook.com/pages/OLI_LAB/3 … 506?v=wall

Offline

 

#16 2018-10-03 18:12:46

x.iso
Platinum Member

oli_lab wrote:
x.iso wrote:

It works better with oscillator module from Oli_Lab. here's some progress, quite usable at this point.
https://www.youtube.com/watch?v=HPsv44YiLhs


I was  about  to tell you that.
I did some experiments from the picture of your patch.
With draw lut you don't need so much to compress the array.
I did not used the concat array but used separate oscillators for each color.


yeah, next thing I was going to try is to get 3 OSC's for each color and try modulations between them, then there's also option to modulate vertical VS horizontal arrays, HSL instead of RGB and so on. many options to try out.

Offline

 

#17 2018-10-05 22:34:15

99Letters
Member

hello, can I ask you?

now some audio sounds into usinehollyhock,
but don't know why sounds can't inside to its??

ummmmm.something help me??
use motu828. and I can see audio coming ...but no sounds.
I watched this tutorial video.

can u help me??
http://www.sensomusic.org/forums/upload … .53.33.png

actually use Ableton 10
can sounds from audio interface through come inside 3/4 channel.
http://www.sensomusic.org/forums/upload … .56.01.png

and actually I put audio from Mac book pro inside microphone.
also cant come to stereo out too.

Last edited by 99Letters (2018-10-05 23:25:54)

Offline

 

#18 2018-10-06 06:30:09

x.iso
Platinum Member

99Letters wrote:

hello, can I ask you?

now some audio sounds into usinehollyhock,
but don't know why sounds can't inside to its??

ummmmm.something help me??
use motu828. and I can see audio coming ...but no sounds.
I watched this tutorial video.

can u help me??
http://www.sensomusic.org/forums/upload … .53.33.png

actually use Ableton 10
can sounds from audio interface through come inside 3/4 channel.
http://www.sensomusic.org/forums/upload … .56.01.png

and actually I put audio from Mac book pro inside microphone.
also cant come to stereo out too.


When you open Devices window, look at the left menu, it should have options to add audio inputs to devices.
http://www.sensomusic.org/forums/uploads.php?file=usine-addinput.JPG
If you don't see that, then click on 'In' section in Devices window. Also, those 'devices' are actually patches that route internal signals to hardware ones, so you can open existing or added input or output in patch panel and see what actual hardware I/O is used there and reroute as desired, if needed.

Offline

 

#19 2018-10-27 04:05:09

x.iso
Platinum Member

So I've basically done most things I wanted with this synth with what I have. But I've been hesitant on releasing this as add-on for few reasons, it's quite CPU intensive (I think without hardware acceleration on it's not even going to work on some machines) and it seems to have some bugs for which I can't figure out the cause yet. So I'm putting it here first kinda as beta test, maybe someone will have ideas on how to optimize it or find what's not working right. Here it is, VidSynth:

http://www.sensomusic.org/forums/uploads.php?file=vidsynth.png

it can use either external video signal or internal video player (you can use still pictures as well in it), there's a switch for input in UI. I haven't figured out why drag-n-drop doesn't work through separate list object, but there's a button to show internal list for the player.
P.S.: this patch depends on quite a few different oli_lab modules! Without his modules, it would be quite a pain to make.

VidSynth

Last edited by x.iso (2018-10-27 04:07:43)

Offline

 

#20 2018-10-28 01:30:53

x.iso
Platinum Member

I've reuploaded the patch since I found some mistakes here and there.

Offline