No internet connection
  1. Home
  2. How to

Script for clicking UI element in browser/web app

By Brendan Lyons @Brendan_Lyons
    2024-03-28 20:48:52.318Z

    Hi all

    I'm looking to write a script to click an element in the browser but having an issue figuring out how to write the code.

    Attached is a screenshot of the element and inspector showing the element. The element is the Microphone icon in the bottom left of the screenshot and the code block is highlighted.

    Any help would be greatly appreciated!

    Reply
    • 8 replies
    1. Christian Scheuer @chrscheuer2024-03-29 04:45:03.012Z

      If this is Chrome, you should be able to do something like this:

      
var chrome = sf.ui.app('com.google.Chrome');

function executeJs(script) {
    var res = sf.system.execAppleScript({
        script: 'tell application "Google Chrome"\n' +
            'set str to execute in active tab of the front window JavaScript "' + script.replace(/"/g, '\\"') + '"\n' +
            'str\n' +
            'end tell\n'
    });
    if (res.standardError) {
        log(res.standardError);
        throw 0;
    }
    //log(res.result);
    return res.result;
}

executeJs(`
document.querySelector('[aria-label="Mute Microphone"]').click();
`);
      Reply
      1. Christian Scheuer @chrscheuer2024-03-29 04:57:36.941Z

        This can also be simplified to:

        
function clickChromeButtonBy(ariaName, ariaValue) {
    sf.appleScript.googleChrome.windows.getByIndex(1).activeTab.execute(`
    document.querySelector('[aria-${ariaName}="${ariaValue}"]').click();
  `);
}

function clickChromeButtonByLabel(value) {
    clickChromeButtonBy('label', value);
}

clickChromeButtonByLabel('Mute Microphone');
        Reply1 Like
        1. Chris Shaw @Chris_Shaw2024-03-29 16:05:04.052Z

          Ooh, thanks for this. Been wanting to automate sending ref MP3s via WeTransfer for a while now. This should get me started.

          Reply
          1. In reply tochrscheuer⬆:
            BBrendan Lyons @Brendan_Lyons
              2024-03-29 16:44:38.843Z

              Hmmm - doesn't seem to be working. I don't have my Stream Deck with me so maybe Chrome has to be in focus?

              I'm working on building a small package of scripts for Sessionwire - it functions in Chrome, Brave, Edge - any Chromium based browser, and we also have a standalone application. Might be in touch to have a chat with you again about that.

              Reply
              1. Christian Scheuer @chrscheuer2024-03-29 16:59:17.158Z

                It's possible the "selector path" I'm using is not entirely correct. I would have to have the webpage open on my end to verify.

                You can also try running this in the Console in Chrome to test if the Chrome part of the code is correct:

                document.querySelector('[aria-label="Mute Microphone"]').click();
                Reply
                1. BBrendan Lyons @Brendan_Lyons
                    2024-03-30 20:16:11.503Z

                    No dice!

                    I'll have to get my dev team involved a bit I'm sure - but I've reached out to you - hopefully we can dive in. Once I get the hang of a script or two I can build out an entire library for our app/web app.

                    Reply
                    1. BBrendan Lyons @Brendan_Lyons
                        2024-03-30 20:16:24.273Z
                        Reply
                    2. In reply toBrendan_Lyons⬆:
                      Christian Scheuer @chrscheuer2024-03-29 16:59:58.053Z

                      Oh, nice! Yes, please reach out on support@soundflow.org - there is probably/most likely a better way to implement this :)

                      Reply
                ReplyAdd progress note