No internet connection
  1. Home
  2. How to

Mouse position in relation to interface elements

By Andrew Scheps @Andrew_Scheps
    2020-04-02 08:59:27.427Z

    Is there a way to get the id of the interface element the mouse is hovered over? I have a few scripts that instantiate plugins on particular insert slots, but I'd like to see if I can take the destination insert slot from the current mouse position so I could move the mouse to the insert slot , then hit the stream deck button. Is that possible?

    Solved in post #2, click to view
    • 16 replies

    There are 16 replies. Estimated reading time: 15 minutes

    1. You can do something like this:

      var mousePos = sf.mouse.getPosition().position;
      var hoveredElementTitle = sf.ui.root.getElementAtPosition(mousePos.x, mousePos.y).title.value;
      log(hoveredElementTitle);
      

      Should give you Insert Assignment A, B, C, etc.

      ReplySolution
      1. Andrew Scheps @Andrew_Scheps
          2020-04-02 09:06:48.167Z

          Awesome, I'll try this later today, thanks!

          1. In reply tochrscheuer:

            Other ways you could do this:

            • When you click the Stream Deck button, a dynamic Deck is displayed with the available insert slots, so it's just two clicks on the SD (and it could show which ones are already in use).
            • When you click the Stream Deck button, a popup is displayed where you can select/search for the slot.
            1. Andrew Scheps @Andrew_Scheps
                2020-04-02 09:30:04.108Z

                This sounds like it could be the way to go. I'll go back and have a look at your plugin settings dynamic deck and see if I can figure it out. Thanks!

                1. In reply tochrscheuer:
                  Andrew Scheps @Andrew_Scheps
                    2020-04-02 14:56:28.694Z2020-04-02 17:28:56.987Z

                    I've been working on this and have gotten the bulk of it working but had a couple of questions. Here's the code so far (I'm sure you'l recognise most of it from one of your other posts):

                    I'm deleting a lot of this post because I've figured some of it out below.

                    My questions are:

                    1. There's got to be a better way to fill the items for the stream deck using one of your magic arrow functions - Better below but I'm sure there's a much more efficient way to do it.

                    2. I'd love to test the text so unassigned inserts could have a different background. Would I do that in an external function and just build an array of RGB values or is there a way to do it while building the modal items directly - Solved below

                    3. And this is the one I don't get at all, the first 8 buttons (the top row of the deck) correctly return values 0-7, but buttons 9 and 10 on the second row return 5 and 6. The names are displaying correctly so I know they are actually buttons 8 and 9. I'm sure it's something stupid I'm not seeing...

                    This is still eluding me.

                    1. Andrew Scheps @Andrew_Scheps
                        2020-04-02 17:22:40.518Z

                        Ok, I've refined (and made a mess) and things are closer: This code reads the inserts, builds two rows of buttons (A-E in row one, then 3 blank buttons, then F-J on row 2) and the backgrounds are set by whether it's assigned or not. That all works, but the return values are still screwy. I'm sure they're set to 0,1,2,3,4,10,10,10,5,6,7,8,9 but I get back 0,1,2,3,4,10,10,10,10,10,10,5,6. Hmmmmm

                        const blueBtn = [0, 100, 255], greenBtn = [0, 255, 100], blackBtn = [0, 0, 0], greyBtn = [100, 100, 100];
                        function formatTitle(s) {
                            var res = '', i = 0;
                            var numDisplayLines = 0;
                            var maxLineLength = 7;
                            if (s == "unassigned") { return "<empty>" };
                            var splitPreset = s.split(" ");
                            if (splitPreset.length < 5) {
                                for (i = 0; i < splitPreset.length - 1; i++) {
                                    res += splitPreset[i] + "\n";
                                }
                                res += splitPreset[splitPreset.length - 1];
                            } else {
                                while (numDisplayLines < 5 && i < splitPreset.length - 1) {
                                    if (splitPreset[i].length + splitPreset[i + 1].length < maxLineLength) {
                                        res += splitPreset[i] + " " + splitPreset[i + 1] + "\n";
                                        i = i + 2;
                                    } else {
                                        res += splitPreset[i];
                                        i++;
                                    }
                                    numDisplayLines++
                                }
                            }
                            return res;
                        }
                        
                        
                        var btns = []
                        btns = sf.ui.proTools.selectedTrack.invalidate().insertButtons;         //Existing Inserts
                        
                        var btnsForDeck=[];                                                       //array for building the buttons
                        btnsForDeck.length=13
                        
                        for (i=0; i<5; i++){
                            btnsForDeck[i] = btns[i].value.invalidate().value
                        }
                        for (i=5; i<8; i++){
                            btnsForDeck[i] = ""
                        }
                        for (i=8; i<13; i++){
                            btnsForDeck[i] = btns[i-3].value.invalidate().value
                        }
                        
                        
                        var btnCols = new Array(13);                                                // Array for Deck button colours                                                
                        for (var i = 0; i < btnCols.length; i++) {
                            btnCols[i] = [];
                        }
                        for (var i = 0; i < btnCols.length; i++) {
                            if (i < 5) {
                                if (btnsForDeck[i] == "unassigned") {
                                    btnCols[i] = greyBtn
                                } else (btnCols[i] = blueBtn)
                            } else {
                                if (i > 4 && i < 8) {
                                    btnCols[i] = blackBtn
                                } else {
                                    if (btnsForDeck[i] == "unassigned") {
                                        btnCols[i] = greyBtn
                                    } else (btnCols[i] = greenBtn)
                                }
                            }
                        }
                        var returnValues = [0, 1, 2, 3, 4, 10, 10, 10, 5, 6, 7, 8, 9];
                        
                        var sd = sf.devices.streamDeck.firstDevice;
                        
                        var btnItems = [];
                        
                        for (i = 0; i < 13; i++) {
                            btnItems[i] = {
                                title: formatTitle(btnsForDeck[i]),
                                color: { r: btnCols[i][0], g: btnCols[i][1], b: btnCols[i][2] },
                                value: returnValues[i],
                            }
                        }
                        
                        var selectedValue = sd.showModal({
                            items: btnItems
                        }).selectedItem.value;
                        
                        
                        
                        log(selectedValue + '');
                        
                        1. Andrew - thanks for your research on this. I'm assuming you're doing this on an XL stream deck right? Writing a script for you now.

                          1. Christian Scheuer @chrscheuer2020-04-02 18:25:55.176Z2020-04-02 18:40:18.724Z

                            This is a partial re-write of the script using functions to create the buttons which makes everything a little more configurable/readable:

                            function selectInsert({ device }) {
                                const config = {
                                    emptyBtn: { r: 0, g: 0, b: 0 },
                                    assignedBtnFirstRow: { r: 0, g: 100, b: 255 },
                                    assignedBtnSecondRow: { r: 0, g: 255, b: 100 },
                                    unassignedBtn: { r: 100, g: 100, b: 100 },
                                };
                            
                                function formatTitle(s) {
                                    var res = '', i = 0;
                                    var numDisplayLines = 0;
                                    var maxLineLength = 7;
                                    if (s == "unassigned") { return "<empty>" };
                                    var splitPreset = s.split(" ");
                                    if (splitPreset.length < 5) {
                                        for (i = 0; i < splitPreset.length - 1; i++) {
                                            res += splitPreset[i] + "\n";
                                        }
                                        res += splitPreset[splitPreset.length - 1];
                                    } else {
                                        while (numDisplayLines < 5 && i < splitPreset.length - 1) {
                                            if (splitPreset[i].length + splitPreset[i + 1].length < maxLineLength) {
                                                res += splitPreset[i] + " " + splitPreset[i + 1] + "\n";
                                                i = i + 2;
                                            } else {
                                                res += splitPreset[i];
                                                i++;
                                            }
                                            numDisplayLines++
                                        }
                                    }
                                    return res;
                                }
                            
                                function main() {
                            
                                    function InsertButton(insertNumber) {
                                        var assignment = insertAssignments[insertNumber - 1];
                                        var isUnassigned = assignment === "unassigned";
                                        return {
                                            title: formatTitle(assignment),
                                            color: isUnassigned
                                                ? config.unassignedBtn
                                                : insertNumber <= 5
                                                    ? config.assignedBtnFirstRow
                                                    : config.assignedBtnSecondRow,
                                            value: insertNumber,
                                        };
                                    }
                            
                                    function NullButton() {
                                        return {
                                            title: '',
                                            color: config.emptyBtn,
                                            value: -1,
                                        };
                                    }
                            
                                    function NullButtons(count) {
                                        return Array.from(Array(count)).map(_ => NullButton());
                                    }
                            
                                    //Array of insert assignments ("unassigned" or name of plugin)
                                    var insertAssignments = sf.ui.proTools.selectedTrack.invalidate().insertButtons.map(b => b.value.invalidate().value);
                            
                                    //Define the layout of buttons
                                    var buttons;
                                    if (device.buttonCount === 32) {
                                        //Stream Deck XL layout
                                        buttons = [
                                            //First row
                                            InsertButton(1), InsertButton(2), InsertButton(3), InsertButton(4), InsertButton(5), NullButton(), NullButton(), NullButton(),
                                            //Second row
                                            InsertButton(6), InsertButton(7), InsertButton(8), InsertButton(9), InsertButton(10), NullButton(), NullButton(), NullButton(),
                                        ].concat(NullButtons(16));
                                    } else {
                                        //Stream Deck normal layout
                                        buttons = [
                                            //First row
                                            InsertButton(1), InsertButton(2), InsertButton(3), InsertButton(4), InsertButton(5),
                                            //Second row
                                            InsertButton(6), InsertButton(7), InsertButton(8), InsertButton(9), InsertButton(10),
                                        ].concat(NullButtons(5));
                                    }
                            
                                    //Show & select button
                                    var selectedValue = device.showModal({
                                        items: buttons,
                                    }).selectedItem.value;
                            
                                    return selectedValue;
                                }
                            
                                return main();
                            }
                            
                            var insertNumber = selectInsert({
                                device: sf.devices.streamDeck.firstDevice,
                            });
                            
                            log(insertNumber);
                            
                            1. Andrew Scheps @Andrew_Scheps
                                2020-04-02 18:56:56.812Z

                                Thanks Christian, but there's still a weird problem with the script, it's returning 1,2,3,4,5,-1,-1,-1,-1,-1,-1,6,7 instead of 1,2,3,4,5,-1,-1,-1,6,7,8,9,10

                                Could it have something to do with having scripts already assigned to some buttons or something like that?

                                1. Hm yea that's weird.
                                  I only tested the script here on my regular sized units. Let me double check on my XL (will do a little later).

                                  1. Andrew Scheps @Andrew_Scheps
                                      2020-04-02 19:00:26.537Z2020-04-02 19:28:51.831Z

                                      Thanks! This is going to a ridiculously useful script.

                                      Your code is so advanced and elegant, can you recommend any resources to learn Javascript? I've tried a couple online courses but they're so web centric they spend as much or more time on html and css as they do on javascript.

                                      A dumb question that will help me decipher things. In the following code:

                                              var insertAssignments = sf.ui.proTools.selectedTrack.invalidate().insertButtons.map(b => b.value.invalidate().value);
                                      

                                      is 'b' just a random variable name that you use to refer to each element of the original array and you don't have to define it since it's only referred to inside the arrow function?

                                      1. In reply tochrscheuer:
                                        Andrew Scheps @Andrew_Scheps
                                          2020-04-02 20:06:30.870Z

                                          Just to be specific about what's happening, the buttons all display correctly, but the return values are offset.
                                          Here are what's returned by row:

                                          Row 1:    1,2,3,4,5,-1,-1,-1
                                          Row 2:   -1,-1,-1,6,7,8,9,10
                                          Row 3:    8,9,10,-1,-1,-1,-1,-1
                                          Row 4:    -1,-1,-1,-1,-1,-1,-1,-1
                                          

                                          So for some reason it's copying the last three of each row to the first three of the row below it.

                                          1. Andrew Scheps @Andrew_Scheps
                                              2020-04-02 20:26:19.906Z

                                              Sorry to bombard you, but I just changed the script to return the title instead of the value and those are offset in the same way so the stream deck seems to be what is confused. I thought it might be the empty title that was doing it but putting text in the NullButton definition didn't change anything,

                                              1. Andrew Scheps @Andrew_Scheps
                                                  2020-04-02 20:38:08.803Z

                                                  Ok, definitely a stream deck issue. I just tried your plugin presets script which uses a modal dialog in the SD and it does exactly the same thing.

                                                  1. Yea I started to wonder if it may be a bug in the Stream Deck app or SF plugin or the installation of the plugin wrt how the buttons get organized. Let's take this part of the discussion on the beta forum - can I get you to file a bug report via Help/Issue so I get your log? :)