No internet connection
  1. Home
  2. How to

How to tell if a selection is on frame boundaries?

By Dustin Harris @Dustin_Harris
    2020-09-11 23:49:54.783Z2020-09-12 02:38:18.847Z

    Is there a way to tell in Sound Flow if a Pro Tools selection is on frame boundaries? (maybe something like get selection with subframes?)
    I have been exploring the math but Pro Tools seems to have calculated some of the combinations in ways that I can't figure out (44100 / 23.976 for example.

    The approach I've been working on is this:

    let sampleRate = 48000;
    let frameRate = 23.976;
    
    let valueInSamples = 265256992
    
    function frameBoundaryCheck(valueInSamples, sampleRate, frameRate) {
    
        let divisionFactor = Math.floor(sampleRate / frameRate)
    
        return (valueInSamples % divisionFactor == 0)
        
    }
    
    let result = frameBoundaryCheck(valueInSamples, sampleRate, frameRate)
    
    log(result)
    

    thoughts?

    • 16 replies

    There are 16 replies. Estimated reading time: 16 minutes

    1. Hm. Framerate / samplerate math is extremely tricky to get right, and many apps have slight variations in implementations.
      It would require a ton of testing to design an algorithm that completely matches Pro Tools'.
      Your approach definitely looks interesting, provided that Pro Tools also floors the ratio, which I would imagine could be wrong since it could cause drifting errors between video playback and audio playback. Effectively, in your case, by flooring the ratio (so instead of 2002.002002002002002 you have 2002), you're assuming the video playback speed would be 23,976023976, not 23,976000000. In other words, in this case, by eliminating the remainder of the division, you'd have 1 frame inaccuracy per 500 frames + 1 frame per 500000 frames, etc.
      You can check if that's the case by comparing your algorithms' conversion between samples and frames by going to a couple of thousands worth of frames into your PT session and see if your algorithm agrees with Pro Tools.
      Note however that you may run into even more issues getting the math right if you're also in drop-frame timecode, as obviously that drops frame names to fix these rounding errors so that the timecode matches more with wall-clock time.

      In short: It could take you a very long time to make this math right. And even if you get it exactly right, it may not be exactly like Pro Tools has implemented it (they could be ever so slightly wrong, or pragmatic, or however you'd put it).

      So, I'd go back and ask why you need this info in the first place and try to see if there was a different way of achieving it that may be easier/faster to implement.

      1. I just did a quick test here.
        23.976 fps, non-drop in Pro Tools. One hour in is 60*60*24 = 86400 frames, and Pro Tools says that is equal to 172972800 samples, which if you say 172972800 / 86400 you actually get the floored version as you put it - 2002. Interesting!
        Where did you get the idea to floor the ratio?

        1. For 44.1 kHz it appears to not floor but instead use a slightly incorrectly calculated version:

          Math:
          86400 frames
          44100 / 23,976 =  1839,339339339339339 samples per frame.
          
          From PT:
          100 frames: 183934
          1000 frames: 1839337
          10000 frames (6 min 56 sec 16 frames): 18393375
          
          1. It also appears that up to a certain resolution, Pro Tools will display the "correct" location of the timecode, with higher resolution than samples.
            Here I've set the cursor to 1000 frames (41 sec 16 frames) in 44.1 kHz with 23.976 fps timecode. The cursor is placed on a sample boundary while the timecode is located slightly off. It's not possible to be completely on the frame boundary since all PT selections are sample-accurate, and Pro Tools correctly visualizes that here.
            In other places, it appears to not correctly visualize it, which leads me to believe PT has all sorts of weirdness going on and probably a lot of patch-fixing over decades of real-world use.

          2. In reply tochrscheuer:
            Dustin Harris @Dustin_Harris
              2020-09-12 15:16:02.701Z

              I got the idea to Floor the ratio from the result, which was something like 2002.002... so I figured they might round down. I tested 48000/23.98 a lot and it always resulted predictably... the others not so much. Your investigation seems to suggest what I feared: guessing at Avid’s logic/liberties :)

              1. Yea exactly. I don't think they're following one standard but rather probably changed things over time based on bug reports for specific samplerate/framerate combinations without reimplementing it across the board.

                1. Ah wait. We're the ones that are wrong.
                  23.976 is not the correct actual framerate. The actual framerate is 24000/1001.
                  So the math holds up and ends up on an even 2002:
                  48000/(24000/1001) = 2002

                  1. Which also means their math is correct for 44100:

                    44100/(24000/1001) = 1839.3375

                    1. So you shouldn't floor anything. Instead you should make the calculation like this:

                      let sampleRate = 44100;
                      let frameRateNumerator = 24000;
                      let frameRateDenominator = 1001;
                      
                      let valueInSamples = 18393375
                      
                      function frameBoundaryCheck(valueInSamples, sampleRate, frameRateNumerator, frameRateDenominator) {
                      
                          //let numberOfFrames = valueInSamples * frameRateNumerator / (sampleRate * frameRateDenominator);
                          
                          let remainder = (valueInSamples * frameRateNumerator) % (sampleRate * frameRateDenominator);
                          return remainder == 0;
                      
                      }
                      
                      log(frameBoundaryCheck(valueInSamples, sampleRate, frameRateNumerator, frameRateDenominator));
                      
                      1. Dustin Harris @Dustin_Harris
                          2020-09-12 21:21:08.462Z2020-09-12 21:39:55.092Z

                          Thank you for all of that! I may have stumbled onto a useable method based on your math findings!!

                          There is rounding present at the sub-sample level, and this is either just how the math works out with .1% speed changes, or there is error introduced in the number of decimal places the system can represent for 24000/1001. Either way, this method rounds to the nearest sample and is providing a reliable result at 44.1/23.98. I'm going to build the model for all sample rates and frame rates and see how the system responds :)

                          let sampleRate = 44100;
                          let frameRate = 29.97;
                          
                          let valueInSamples = sf.ui.proTools.selectionGetInSamples().selectionStart
                          
                          function frameBoundaryCheck(valueInSamples, sampleRate) {
                          
                              let frameRateNumerator = frameRate * 1000
                              let frameRateDenominator = 1000
                          
                              if (Math.floor(frameRate) != frameRate) {
                                  frameRateNumerator = Math.ceil(frameRate) * 1000
                                  frameRateDenominator = 1001
                          
                              }
                          
                          
                              let divisionFactor = (sampleRate / (frameRateNumerator / frameRateDenominator))
                          
                              let remainder = (valueInSamples % divisionFactor)
                          
                              //alert(`Division Factor is ${divisionFactor}\nRemainder is ${remainder}`)
                          
                              if (divisionFactor - remainder < 1) {
                                  return true;
                          
                              } else if (remainder < 1) {
                          
                                  return true;
                          
                              } else {
                          
                                  return false;
                              }
                          }
                          
                          let result = frameBoundaryCheck(valueInSamples, sampleRate)
                          
                          alert(`${result}`) 
                          

                          EDIT: changed script to parse the incoming framerate to calculate the frameRateNumerator and frameRateDenominator variables automatically.

                          1. Gotcha. The reason why I refactored the modulo like this was precisely to deal with rounding errors (by only having one division/modulo):

                            let remainder = (valueInSamples * frameRateNumerator) % (sampleRate * frameRateDenominator);
                            

                            In your code, by having essentially 4 divisions, you're losing much more precision.

                            1. Dustin Harris @Dustin_Harris
                                2020-09-12 21:56:18.832Z2020-09-13 02:48:51.034Z

                                I'll come clean: I was getting 'undefined' returned from one of the variables in my script when I copied your math, because I made a mistake in my own code... so I sort of went back to the beginning to check and the extra steps added up to more error :) I'll adopt your more precise method.

                                Edit: I tried let remainder = (valueInSamples * frameRateNumerator) % (sampleRate * frameRateDenominator); in the script but for some reason I was getting large remainder values. The way I did the math resulted in subsample remainders in a way that I understood easily, and the precision appeared to be higher than needed to make the script work, so I kept it. But absolutely tell me if I misunderstood the whole idea/philosophy to your equation @chrscheuer :)

                                1. Dustin Harris @Dustin_Harris
                                    2020-09-13 02:41:42.678Z

                                    This is the use I had for the idea implemented:

                                    This script takes a selection in pro tools, checks if the selection is on frame boundaries, and if not, extends the selection to the nearest frame boundaries before the selection start and after the selection end.

                                    Comments and suggestions welcome! :)

                                    
                                    function getSessionInfo() { // Gets sample rate, bit rate, frame rate
                                    
                                        if (!sf.ui.proTools.windows.whoseTitle.is('Session Setup').exists) { //opens session setup window
                                            sf.ui.proTools.menuClick({ menuPath: ["Setup", "Session"] });
                                        }
                                    
                                        let sessionSetupWin = sf.ui.proTools.windows.whoseTitle.is('Session Setup').first;
                                    
                                        let sessionInfo = { //stores sample rate as a number, sample rate as a string name, frame rate, and bit depth as an object called sessionInfo
                                    
                                            sampleRateName: sessionSetupWin.groups.whoseTitle.is('Session Format').first.children.whoseRole.is("AXStaticText").allItems[2].value.value,
                                    
                                            sampleRateNumber: (+(sessionSetupWin.groups.whoseTitle.is('Session Format').first.children.whoseRole.is("AXStaticText").allItems[2].value.value.split(" ")[0]) * 1000),
                                    
                                            bitRate: sf.ui.proTools.windows.whoseTitle.is('Session Setup').first.groups.whoseTitle.is('Session Format').first.popupButtons.allItems[1].value.value,
                                    
                                            frameRate: sf.ui.proTools.windows.whoseTitle.is('Session Setup').first.groups.whoseTitle.is('Session Format').first.popupButtons.allItems[2].value.value.split(" ")[0]
                                    
                                    
                                    
                                        };
                                    
                                        sessionSetupWin.windowClose();
                                    
                                        return sessionInfo;
                                    }
                                    
                                    
                                    function frameBoundaryCheck(sessionInfo, valueInSamples) { //uses math to check if a selected sample coincides with a frame boundary with the session's sample rate and frame rate
                                    
                                        let sampleRate = sessionInfo.sampleRateNumber;
                                        let frameRate = +(sessionInfo.frameRate);
                                    
                                        let frameRateNumerator = frameRate * 1000;
                                        let frameRateDenominator = 1000;
                                    
                                        if (Math.floor(frameRate) != frameRate) {
                                    
                                            frameRateNumerator = Math.ceil(frameRate) * 1000;
                                            frameRateDenominator = 1001;
                                    
                                        }
                                    
                                        let divisionFactor = (sampleRate / (frameRateNumerator / frameRateDenominator));
                                        let remainder = (valueInSamples % divisionFactor);
                                        let multiplier = (valueInSamples / divisionFactor);
                                    
                                        let leftEdge = Math.round(Math.floor(multiplier) * divisionFactor);
                                        let rightEdge = Math.round(Math.ceil(multiplier) * divisionFactor);
                                    
                                        let isFrameEdge = false;
                                    
                                        if (divisionFactor - remainder < 1) {
                                            isFrameEdge = true;
                                    
                                        } else if (remainder < 1) {
                                    
                                            isFrameEdge = true;
                                    
                                        }
                                    
                                        let result = {
                                            isFrameEdge: isFrameEdge,
                                            leftEdge: leftEdge,
                                            rightEdge: rightEdge,
                                        };
                                    
                                        return result;
                                    }
                                    
                                    
                                    ////MAIN////
                                    
                                    function main() {
                                    
                                        //Get the session info in Pro Tools
                                        sf.ui.proTools.appActivateMainWindow();
                                        let sessionInfo = getSessionInfo();
                                    
                                        //get the timeline selection in samples
                                        let timelineSelection = sf.ui.proTools.selectionGetInSamples();
                                    
                                        //check if the selection start is on a frame edge, and if not, extend the selection start to the closest frame edge before the selection
                                        let checkSelectionStart = frameBoundaryCheck(sessionInfo, timelineSelection.selectionStart);
                                    
                                        if (checkSelectionStart.isFrameEdge != true) {
                                            timelineSelection.selectionStart = checkSelectionStart.leftEdge;
                                        }
                                    
                                        //check if the selection end is on a frame edge, and if not, extend the selection end to the closest frame edge after the selection
                                        let checkSelectionEnd = frameBoundaryCheck(sessionInfo, timelineSelection.selectionEnd);
                                    
                                        if (checkSelectionEnd.isFrameEdge != true) {
                                            timelineSelection.selectionEnd = checkSelectionEnd.rightEdge;
                                        }
                                    
                                        //apply the new selection with the boundaries set to frame edges
                                        sf.ui.proTools.selectionSetInSamples({ selectionStart: timelineSelection.selectionStart, selectionEnd: timelineSelection.selectionEnd })
                                    }
                                    
                                    main();
                                    
                                    1. Ahh... I think I would've just stayed in timecode and read what the current selection is and reentered it, forcing the selection onto frame boundaries.
                                      But then we wouldn't have learned all the math ;)

                                      1. Dustin Harris @Dustin_Harris
                                          2020-09-14 00:05:22.655Z

                                          Is there a way to do it in timecode? like reading subframes?

                                          EDIT: OOHHH... by reentering it you're assigning .00 to the subframes by default! Yeah... that is a much easier way to do it.

                                          1. Yea exactly. And if PT wouldn't force it onto .00 subframes since you were reentering an identical value (they may have a special case for that), then I'd just collapse the selection first (while remembering what to type in) before reentering it.