No internet connection
  1. Home
  2. How to

Creating macros for audio/midi clip movement in Nuendo/Cubase with midi knobs

By Mark Dawson @Mark_Dawson
    2019-12-15 16:11:23.198Z

    Hi guys, new user here just asking for a bit of advice.

    After watching a video by @JesperA on Facebook regarding audio clip control using midi in Cubase, I decided to join up and try and see if I can replicate what Jesper has accomplished.

    Here is a link to the video:
    https://m.facebook.com/groups/321302182044281?view=permalink&id=415657332608765&ref=group_browse

    I’ve downloaded soundflow and I’m very new to the whole concept, but I’m struggling on where to start to be honest.

    After a friendly email chat with Christian he has recommended I come here and ask for help. So any help is appreciated.

    Regarding my set up, I use Nuendo 10, and I don’t have the volume controllers Jesper uses but I do have a Steinberg CMC QC controller, and an AKAI apc40 which I’m sure I could use instead?

    As I said, if you can help put me in the right direction on how to use sound flow in order for me (and others) to control clips in this way I’ll be hugely greatful. It seems like such a game changer in workflow, and would be so much faster than using the mouse.

    Thanks,

    Mark

    Reply
    • 39 replies

    There are 39 replies. Estimated reading time: 29 minutes

    1. Christian Scheuer @chrscheuer2019-12-15 16:12:18.086Z

      Oops sorry Mark I overlapped with this - I'll move my post into this as a reply:

      Reply
      1. Christian Scheuer @chrscheuer2019-12-15 16:11:09.351Z

        Let's use this thread for the discussion on how to best implement this.

        To get started, we should first address the overall architecture of making this work.

        Generally speaking, you would use ControllerMate to receive input from your HID device and convert it into for example MIDI messages.
        Then, in SoundFlow you would have a script that receives these MIDI messages and then performs actions in Cubase/Nuendo according to what it receives.

        It's a good idea to split the process of implementing this up into those 2 different workflows. I would start out by using a regular MIDI device to control SoundFlow and then set up the HID->ControllerMate->SF link once you have figured out how to use MIDI via SoundFlow to control Nuendo.

        To get started with controlling Nuendo you have to install the Steinberg SKI Remote plugin. You can find the installer here:
        https://www.steinberg.net/en/support/content_and_accessories/steinberg_ski_remote.html

        Once you have it installed, you should verify that the setup is correct in your device setup in Cubase/Nuendo. Jesper can help verify that if you post a screenshot of your setup here.

        Once Steinberg SKI Remote is set up, you will now be able to control Nuendo from SoundFlow.
        First step to get started is to do the following in SoundFlow:

        • Use the "New" button to create a new Macro
        • In the macro editor, click "Add Action"
        • Now choose the "Perform Action" item in the Cubase/Nuendo folder.
        • If you have SKI correctly installed, you can now choose any of the thousands of Cubase/Nuendo actions in the drop down menu inside the "Perform Action" action.
        • Here, for example choose "Nudge Right".
        • Test that it works by clicking the "Run Command" button.

        Ok so now you have a macro in SoundFlow that can control Cubase/Nuendo.

        Next step is to convert this into a script so we can do more advanced stuff.
        To convert the macro action into Javascript click the three small dots in the top right corner of the action, and choose "Copy as Javascript".

        Now click the "+ New" button and choose "Script" to create a new command script.

        In this script paste the code you copied from the macro.

        Test that the script works just like the macro by clicking "Run Command".

        Now we can start mapping this script to be triggered by MIDI input.
        Go up into the Trigger section (above the script) and add a MIDI trigger.

        Once you've gotten this far, let's talk again about how to script the various different MIDI commands.

        Reply
        1. Christian Scheuer @chrscheuer2019-12-15 16:13:48.285Z

          Correction to the above: If you'll be using an Akai MIDI controller you won't need ControllerMate at all. In any case the above still holds true for how to get started.

          Reply
          1. Christian Scheuer @chrscheuer2019-12-15 16:54:12.079Z

            For the SKI setup it's important to turn off authentication since SoundFlow needs to connect without a password.

            Reply
      2. M
        In reply toMark_Dawson⬆:
        Mark Dawson @Mark_Dawson
          2019-12-15 17:04:39.255Z

          Thanks a lot Christian, so far so good! I've installed the Steinberg SKI plugin, managed to link the CMC QC to move the audio file! Here's a screenshot so far.

          I mapped the 1st knob on the QC to nudge right the audio file in Nuendo, it only moves to the right no matter which way I turn the knob, obviously becuase I need to now assign a nudge left macro?

          Thanks

          Reply
          1. MMark Dawson @Mark_Dawson
              2019-12-15 17:11:24.070Z

              I've tried to do the same thing with 'Nudge left' and assigned it to the same knob on the QC but I've ran into a warning.

              Conflict...This trigger is also being used in other commands...

              Reply
              1. Christian Scheuer @chrscheuer2019-12-15 17:17:13.934Z

                Great progress!
                Yes ok so our MIDI bindings map either to a full device or to the first two bytes of the MIDI message.
                For being able to create more advanced stuff we recommend that you choose "ALL EVENTS" so that the same script will process all input from your device, and then you will make the script filter on the input.
                Try to choose "ALL EVENTS" on the script to make sure that no matter what you do on your MIDI device the action runs.

                Then you'll need to start filtering on the input, like this:

                var m = event.trigger.midiBytes;
var m0 = m[0], m1 = m[1], m2 = m[2];

//If note ON on channel 1:
if (m0 == 0x90) {
    if (m1 == 48) {
        //We received a Note ON message on C3.
        //Do something here...
    } else if (m1 == 50) {
        //We received a Note ON message on D3.
        //Do something else here...
    }
}
                Reply
                1. Christian Scheuer @chrscheuer2019-12-15 17:20:52.750Z

                  Here the m0 variable is the first midi byte, m1 is the 2nd, etc.
                  0x90 is the hexidecimal representation of a note on on channel 1.
                  0x91 would be note on on channel 2.

                  CC values start at 0xB0.
                  So m0 == 0xB0 would be for a CC value received on channel 1, m0 == 0xB1 would be a CC value received on channel 2, etc.
                  To figure out which CC, you'd test m1 (the 2nd byte), and to get the CC value for that CC, you'd test the 3rd byte (m2).

                  So for example if you want to filter on a VPot that sends CC values on CC 11 on channel 0, you'd do:

                  if (m0 == 0xB0) {
    //Match CC on channel 1
    if (m1 == 11) {
        //Match CC11
        if (m2 > 64) {
            //Turned right
        } else {
            //Turned left
        }
    }
}
                  Reply
                  1. Christian Scheuer @chrscheuer2019-12-15 17:22:57.495Z

                    To test what you're actually receiving to better debug what you should look for, you can also insert this at the top:

                    var m = event.trigger.midiBytes;
var m0 = m[0], m1 = m[1], m2 = m[2];

log(`m0: ${m0}, m1: ${m1}, m2: ${m2}`);

                    This will log the m0, m1 and m2 values when the script receives MIDI input.
                    You can always comment out (temporarily disable) this log line when you don't need it by prepending the line with //:

                    //log(`m0: ${m0}, m1: ${m1}, m2: ${m2}`);

                    This will again disable the logging, but it allows you to quickly enable it again by removing the //.

                    Reply
                    1. MMark Dawson @Mark_Dawson
                        2019-12-15 17:51:12.893Z

                        Thanks Christain, Ive managed to change the script to All events and now every knob i turn makes the clip nudge right, however, you have lost me after that!

                        Filtering on the input, is that something I have to add under source, under the previous java script that I copied from the 1st part? Or do I have to create a new command name?

                        Reply
                        1. Christian Scheuer @chrscheuer2019-12-15 18:25:38.652Z

                          See this as an example. The if blocks is basically the logic that determines, based on the input, which action to take.
                          So you would move the performAction call into the if blocks at the correct positions.

                          For example this would work if your VPot sends CC11 messages on channel 1:

                          var m = event.trigger.midiBytes;
var m0 = m[0], m1 = m[1], m2 = m[2];

log(`m0: ${m0}, m1: ${m1}, m2: ${m2}`);

if (m0 == 0xB0) {

    //Match CC on channel 1
    if (m1 == 11) {
        //Match CC11

        if (m2 > 64) {
            //Turned right
            sf.ui.nuendo.performAction({
                categoryName: 'Nudge',
                actionName: 'Right',
            });
        } else {
            //Turned left
            sf.ui.nuendo.performAction({
                categoryName: 'Nudge',
                actionName: 'Left',
            });
        }

    }

}
                          Reply
                          1. Christian Scheuer @chrscheuer2019-12-15 18:26:33.614Z

                            You can read more about if blocks here:

                            https://www.w3schools.com/jsref/jsref_if.asp
                            Reply
              2. M
                In reply toMark_Dawson⬆:
                Mark Dawson @Mark_Dawson
                  2019-12-15 19:55:04.443Z

                  Hi Christian, in the end I just tried to copy in your commands because I just don't understand this at all I'm afraid, nothing seems to be working.

                  I've never attempted code before and all this is really confusing to me, I'll keep trying and hopefully it will click. Thanks for your help so far

                  Reply
                  1. Christian Scheuer @chrscheuer2019-12-15 20:07:39.160Z

                    That's totally understandable. Which CC does your vpots use?

                    Reply
                    1. MMark Dawson @Mark_Dawson
                        2019-12-15 22:06:16.805Z

                        In SoundFlow under the midi tab of main command trigger, when I press record and turn the knob on the controller it says: b0,10,*

                        Using the Midi monitor app on my computer, its says that my Steinberg QC is Port2 and is sending data on channel 1. The data column says: General purpose 1 (coarse)
                        and in the last column when I turn the knob left its says 65, and 1 when I turn right.

                        When you say cc and vpots Im assuming thats what you mean? Again, sorry for being such a noob on this, its all new to me!

                        Reply
                        1. Christian Scheuer @chrscheuer2019-12-15 22:12:03.414Z

                          Alright so that's a fairly simple fix. Try this (make sure to copy everything including the last } characters):

                          var m = event.trigger.midiBytes;
var m0 = m[0], m1 = m[1], m2 = m[2];

if (m0 == 0xB0) {

    //Match CC on channel 1
    if (m1 == 10) {
        //Match CC10

        if (m2 < 64) {
            //Turned right
            sf.ui.nuendo.performAction({
                categoryName: 'Nudge',
                actionName: 'Right',
            });
        } else if (m2 >= 65) {
            //Turned left
            sf.ui.nuendo.performAction({
                categoryName: 'Nudge',
                actionName: 'Left',
            });
        }

    }

}
                          Reply
                          1. Christian Scheuer @chrscheuer2019-12-15 22:12:43.944Z

                            note the m1 == 10 here is because your knob sends out CC10 messages.

                            Reply
                            1. MMark Dawson @Mark_Dawson
                                2019-12-15 22:57:50.824Z

                                Thanks Christian, It still wont work unfortunately but I think it maybe because the QC is by default controlling the Quick controls in Nuendo? I will try this tomorrow with the APC40 and see if I can make any progress.

                                Thanks for your help so far, I wont give up yet lol

                                Reply
                                1. MMark Dawson @Mark_Dawson
                                    2019-12-15 23:10:21.226Z

                                    Ok, ignore my last message because I simplified it for now on the QC:
                                    So far I have managed to, over the 1st four knobs, added:

                                    1. nudge left
                                    2. nudge right
                                    3. Increase clip volume
                                    4. Decrease clip volume

                                    So Im getting there, slowly and obviously struggling with the concept of using both the left and right knob movements of each pot at the moment, so using 2 knobs when I should use . But slowly getting there!

                                    Reply1 Like
                                    1. Jesper Ankarfeldt @JesperA2019-12-16 09:24:18.987Z

                                      Hi Mark. So I'm just in the middle of flying to Copenhagen from LA, so haven't had a moment to really sit down.
                                      Never owned a CMC controller. was always curious about the AI one.

                                      It should be able to fairly easily optimize what you already have setup, so you can use a knob in both directions :)

                                      If the last script Christian shared didn't work, it might be because the QC sends some different message. If you could let us know what values you are getting if you put the script in, and have the QC as trigger set to ALL EVENTS:

                                      var m = event.trigger.midiBytes;
var m0 = m[0], m1 = m[1], m2 = m[2];

if (m0 == 0xB0) {
    //Match CC on channel 1
    if (m1 == 10) {
        //Match CC10
        log(m2)
        }
    }
}

                                      And then let us know what values you get when you do a left and a right movement. Potentially they will also send different values if you turn it faster or slower.

                                      Best

                                      Reply
                                      1. Christian Scheuer @chrscheuer2019-12-16 17:25:06.489Z

                                        Yea I based my code on Marks' reply here:
                                        https://forum.soundflow.org/-1407#post-17

                                        Where it says he gets 65 and 1 as m2.

                                        Reply
                        2. M
                          In reply toMark_Dawson⬆:
                          Mark Dawson @Mark_Dawson
                            2019-12-16 20:51:55.548Z

                            Hi Jesper, thanks for getting involved Im looking forward to learning this once I get my head around the basics.

                            Ive decided to scrap the QC for now, and Ive started fresh with the APC40 mainly because it isnt mapped to anything already and might be a little easier to understand.

                            When I use midi monitor app with the apc40 and turn the 1st knob of 8 in device control, I get the following:

                            Reply
                            1. MMark Dawson @Mark_Dawson
                                2019-12-16 20:55:23.502Z

                                So, the furthest left the knob, the value is 0, going up to 127 as I turn to the right.

                                I have made the 2 macros for nudge left, and nudge right again in soundflow and added them to the 1st 2 knobs and they work as they did with the QC, still trying to figure out the next part!!

                                Reply
                                1. MMark Dawson @Mark_Dawson
                                    2019-12-16 21:09:52.991Z

                                    Ive used this in the script, that Christian sent me yesterday:
                                    var m = event.trigger.midiBytes;
                                    var m0 = m[0], m1 = m[1], m2 = m[2];

                                    log(m0: ${m0}, m1: ${m1}, m2: ${m2});

                                    and mapped that script to the 3rd knob, and recorded these values from left turn to right turn

                                    Reply
                                    1. MMark Dawson @Mark_Dawson
                                        2019-12-16 21:10:52.537Z

                                        Hopefully you now know what midi channels Im using properly? :)

                                        Reply1 Like
                                        1. Christian Scheuer @chrscheuer2019-12-17 10:52:54.419Z

                                          Thanks Mark, this is very helpful indeed.
                                          Let's simplify the script so hopefully you can follow along what's going on.
                                          Quite clearly you get [176, 18, 0] when turning left and [176, 18, 127] when turning right.
                                          Let's use that in the script. Now each if (...) block simply checks against those same numbers

                                          var m = event.trigger.midiBytes;
var m0 = m[0], m1 = m[1], m2 = m[2];

//log(`m0: ${m0}, m1: ${m1}, m2: ${m2}`);

if (m0 == 176 && m1 == 18 && m2 == 0) {
    //Turned left
    sf.ui.nuendo.performAction({
        categoryName: 'Nudge',
        actionName: 'Left',
    });
} else if (m0 == 176 && m1 == 18 && m2 == 127) {
    //Turned right
    sf.ui.nuendo.performAction({
        categoryName: 'Nudge',
        actionName: 'Right',
    });
}
                                          Reply
                                          1. Christian Scheuer @chrscheuer2019-12-17 10:55:20.539Z

                                            An if (...) statement checks if the value inside the parentheses is true (ish). If it is, then it executes the code inside the following block (the stuff inside the curly brackets { ... }.
                                            The && means logical AND.
                                            So this code says - if m0 is 176 and m1 is 18 and m2 is 0, then do the stuff inside the {...} which here is to nudge left.
                                            ELSE (if the above was not true)
                                            check for a different condition...
                                            if m0 is 176 and m1 is 18 and m2 is 127, then do the stuff inside the {...} which here is to nudge right.

                                            Reply
                                            1. MMark Dawson @Mark_Dawson
                                                2019-12-17 19:46:01.724Z

                                                Hi Christian, Thats working!! Thank You, its all starting to make sense a bit now!

                                                The only issue I have, is that I have to do a few full turns to get the apc40 knob to go all the way left or right in order for the clip to move if you follow what I mean.

                                                I think thats because the values range from 0 all the way to 127 (in midi monitor the more i move the knob the values rise from 0-127 untill ive got to a full right turn)

                                                Is there a way to make the clip move at any point in between the 0-127 range depending on which way I turn?

                                                Thanks for your help so far, this is difficult but starting to make sense at least

                                                Reply
                                                1. Christian Scheuer @chrscheuer2019-12-17 20:41:42.570Z

                                                  Hi Mark. That's great.

                                                  Okay so what you're running into now, is that there are different types of MIDI knobs (often also called Pots or VPots, for virtual potentiometer, or rotary encoder).
                                                  They can be either relative or absolute. The relative kind are those that can spin endlessly in any direction, there are no start/stop position. The absolute kind are those that can go all the way to the left to a certain point, and then go all the way to the right, until a certain point.

                                                  Absolute rotary encoders typically send out CC messages with an absolute value ranging from 0 to 127 depending on where on the circle you are.
                                                  Relative rotary encoders typically send out CC messages with a relative value indicating how far you moved left or right in a given time frame (ie. how fast you were moving minus or plus). This can be encoded in different ways depending on the MIDI controller, but often 64 is the center, and values below 64 indicate a left turn whereas values above indicate a right turn.

                                                  For the thing we're building a relative rotary encoder makes most sense, since what we want is to move something relatively in Cubase/Nuendo. We shouldn't be limited when getting all the way down to 0 and then be unable to move any further (your clip could be anywhere on the timeline, our script has no knowledge of the clips absolute position, we only care about the relative one).

                                                  To make things even more complex, some MIDI controllers can make their own mapping of relative to absolute data, meaning relative (endless) rotary encoders can spit out absolute data. If the MIDI controller does that, it's hard to use in SF.

                                                  It sounds like what you're trying to do in this latest attempt is to use a MIDI controller that sends absolute values. This will give you the exact problem you're describing.
                                                  The solution is to use an endless (virtual) relative rotary encoder that spits out delta values (and then change the numbers in the script to match the input you're receiving).

                                                  Reply
                                                  1. MMark Dawson @Mark_Dawson
                                                      2019-12-17 22:32:38.381Z

                                                      Hi Christian. Yes, this makes perfect sense thanks for explaining!

                                                      So, after learning this about the apc40 I went back to the Steinberg QC and have got it working! Now left and right turns of knob 1 on the QC shifts the clip left and right. Fantastic

                                                      To be honest I wasnt sure what I was doing earlier when you and Jesper told me to get the values, but once I copied this into the script:
                                                      var m = event.trigger.midiBytes;
                                                      var m0 = m[0], m1 = m[1], m2 = m[2];

                                                      //log(m0: ${m0}, m1: ${m1}, m2: ${m2});

                                                      I was able to tell that the pots were m2 1 for right, and m2 65 for left. Then I just changed the if and else if values to match the QC and its all working well!

                                                      Thanks for helping me get this far :)

                                                      Reply1 Like
                                      • In reply toMark_Dawson⬆:
                                        Jesper Ankarfeldt @JesperA2020-01-19 06:14:29.997Z

                                        Hey Mark.

                                        It seems like you've already had a lot of progress. I was away on Holiday so wasn't keeping track on the internet :)
                                        Let me know if you want more help.

                                        best

                                        Reply
                                        1. Progress
                                        2. M
                                          Mark Dawson @Mark_Dawson
                                            2019-12-17 22:48:19.862Z

                                            I've just successfully added volume up and down on knob 2 of the QC! Very chuffed haha. Definitely getting the hang of it now!

                                            Reply
                                            1. M
                                              Mark Dawson @Mark_Dawson
                                                2019-12-17 22:49:59.680Zreplies toMark_Dawson⬆:

                                                Pot 2 is virtually the same values as pot one but the m1 value is 17 rather than 16 :)

                                                Reply1 Like
                                                1. Christian Scheuer @chrscheuer2019-12-17 22:53:24.234Zreplies toMark_Dawson⬆:

                                                  That's awesome! Now you're getting it :D
                                                  Couple of final tips:

                                                  • Hit the "Format Code" button to make the code editor automatically indent the code lines. This will help read the code when having if blocks etc, since it will indent the code in a logical sense.

                                                  • Comment the log(...) line by putting // in the start of the line to hide the messages.

                                                  • Your trigger seems to be event specific. You can experiment with using ALL EVENTS and by that you'll be able to make more complex scripts down the road. Not necessary for this use case but important to remember if stuff is not working that essentially you're already filtering MIDI input on your trigger right now.

                                                  Reply
                                                  1. M
                                                    Mark Dawson @Mark_Dawson
                                                      2019-12-18 19:29:03.685Zreplies tochrscheuer⬆:

                                                      Thanks Christian! Im working late tonight but will try these out a little later!!

                                                      Reply1 Like
                                                      1. M
                                                        Mark Dawson @Mark_Dawson
                                                          2019-12-18 22:49:31.467Zreplies tochrscheuer⬆:

                                                          Hi Christian, the format code tip is very helpful and now I can turn off the midi drop down messages by using //. Great.

                                                          I'm not sure I follow you with the 3rd point though...but compared to when I first started I think I'm really getting to grips with the basics now so thanks!

                                                          The latest thing I'm trying is I've mapped 'fade in and fade out' to the 3rd knob, which I feel is a very helpful tool to have because I use fades all the time and having it mapped to the knob rather than using the mouse is fantastic, however, I would like to try and set up the end fade in and out of the same audio clip, but on the same knob with a keyswitch.

                                                          example, the normal knob 3 will fade the start of the clip in with a right turn , and back out with a left turn....then if I press, say, command and turn the same knob, the end of the same clip fades out and in etc?

                                                          I'm assuming I need to add a keytrigger instead of midi trigger in this instance?

                                                          Reply
                                                          1. Christian Scheuer @chrscheuer2019-12-18 22:52:40.362Zreplies toMark_Dawson⬆:

                                                            Great to see your quick progress on this :)

                                                            In relation to the state of keyboard modifiers while reacting to MIDI input, you don't have to have a separate trigger for that. You can keep your script, and then at the right spot add:

                                                            if (event.keyboardState.hasCommand)
{
    
}
else 
{

}

                                                            This provides an if/ else statement for you to split behavior depending on if the Command key is held down or not. Feel free to post your entire script if you need help on where to fit this in.

                                                            Reply
                                                            1. M
                                                              Mark Dawson @Mark_Dawson
                                                                2019-12-19 20:42:55.573Zreplies tochrscheuer⬆:

                                                                Hi Christian, I've almost got it working with this script. The fade in/out works fine, and once I press command the fade out decreases with a left turn but the right turn does the same exact thing and the clip is just constantly fading out rather than switching from 'Increment fade out length' to decrement fade out length'

                                                                Any idea what I've done wrong? I first tried it with just 'else' at the bottom but nothing was responding so I added 'else if' which got me this far!

                                                                cheers

                                                                Reply
                                                                1. Christian Scheuer @chrscheuer2019-12-20 18:37:06.594Zreplies toMark_Dawson⬆:

                                                                  Hi Mark

                                                                  Please note you can quote scripts inside your comments here, then it's easier to copy/paste. You'll need to make a line in your editor with 3 backticks: ``` before and another line like that after your code.
                                                                  If you paste it in like that I'll edit it for you.

                                                                  Reply
                                                                  Reply (discussion)Add progress note