How to tell if a selection is on frame boundaries?
Is there a way to tell in Sound Flow if a Pro Tools selection is on frame boundaries? (maybe something like get selection with subframes?)
I have been exploring the math but Pro Tools seems to have calculated some of the combinations in ways that I can't figure out (44100 / 23.976 for example.
The approach I've been working on is this:
let sampleRate = 48000;
let frameRate = 23.976;
let valueInSamples = 265256992
function frameBoundaryCheck(valueInSamples, sampleRate, frameRate) {
let divisionFactor = Math.floor(sampleRate / frameRate)
return (valueInSamples % divisionFactor == 0)
}
let result = frameBoundaryCheck(valueInSamples, sampleRate, frameRate)
log(result)
thoughts?
- Christian Scheuer @chrscheuer2020-09-12 14:38:06.687Z
Hm. Framerate / samplerate math is extremely tricky to get right, and many apps have slight variations in implementations.
It would require a ton of testing to design an algorithm that completely matches Pro Tools'.
Your approach definitely looks interesting, provided that Pro Tools also floors the ratio, which I would imagine could be wrong since it could cause drifting errors between video playback and audio playback. Effectively, in your case, by flooring the ratio (so instead of 2002.002002002002002 you have 2002), you're assuming the video playback speed would be 23,976023976, not 23,976000000. In other words, in this case, by eliminating the remainder of the division, you'd have 1 frame inaccuracy per 500 frames + 1 frame per 500000 frames, etc.
You can check if that's the case by comparing your algorithms' conversion between samples and frames by going to a couple of thousands worth of frames into your PT session and see if your algorithm agrees with Pro Tools.
Note however that you may run into even more issues getting the math right if you're also in drop-frame timecode, as obviously that drops frame names to fix these rounding errors so that the timecode matches more with wall-clock time.In short: It could take you a very long time to make this math right. And even if you get it exactly right, it may not be exactly like Pro Tools has implemented it (they could be ever so slightly wrong, or pragmatic, or however you'd put it).
So, I'd go back and ask why you need this info in the first place and try to see if there was a different way of achieving it that may be easier/faster to implement.
Christian Scheuer @chrscheuer2020-09-12 14:59:06.039Z
I just did a quick test here.
23.976 fps, non-drop in Pro Tools. One hour in is60*60*24 = 86400 frames
, and Pro Tools says that is equal to 172972800 samples, which if you say 172972800 / 86400 you actually get the floored version as you put it - 2002. Interesting!
Where did you get the idea to floor the ratio?Christian Scheuer @chrscheuer2020-09-12 15:08:02.855Z
For 44.1 kHz it appears to not floor but instead use a slightly incorrectly calculated version:
Math: 86400 frames 44100 / 23,976 = 1839,339339339339339 samples per frame. From PT: 100 frames: 183934 1000 frames: 1839337 10000 frames (6 min 56 sec 16 frames): 18393375
Christian Scheuer @chrscheuer2020-09-12 15:11:06.344Z
It also appears that up to a certain resolution, Pro Tools will display the "correct" location of the timecode, with higher resolution than samples.
Here I've set the cursor to 1000 frames (41 sec 16 frames) in 44.1 kHz with 23.976 fps timecode. The cursor is placed on a sample boundary while the timecode is located slightly off. It's not possible to be completely on the frame boundary since all PT selections are sample-accurate, and Pro Tools correctly visualizes that here.
In other places, it appears to not correctly visualize it, which leads me to believe PT has all sorts of weirdness going on and probably a lot of patch-fixing over decades of real-world use.
- In reply tochrscheuer⬆:
Dustin Harris @Dustin_Harris
I got the idea to Floor the ratio from the result, which was something like 2002.002... so I figured they might round down. I tested 48000/23.98 a lot and it always resulted predictably... the others not so much. Your investigation seems to suggest what I feared: guessing at Avid’s logic/liberties :)
Christian Scheuer @chrscheuer2020-09-12 15:30:39.455Z
Yea exactly. I don't think they're following one standard but rather probably changed things over time based on bug reports for specific samplerate/framerate combinations without reimplementing it across the board.
Christian Scheuer @chrscheuer2020-09-12 15:32:23.493Z
Ah wait. We're the ones that are wrong.
23.976 is not the correct actual framerate. The actual framerate is 24000/1001.
So the math holds up and ends up on an even 2002:
48000/(24000/1001) = 2002Christian Scheuer @chrscheuer2020-09-12 15:33:09.382Z
Which also means their math is correct for 44100:
44100/(24000/1001) = 1839.3375
Christian Scheuer @chrscheuer2020-09-12 15:38:56.247Z
So you shouldn't floor anything. Instead you should make the calculation like this:
let sampleRate = 44100; let frameRateNumerator = 24000; let frameRateDenominator = 1001; let valueInSamples = 18393375 function frameBoundaryCheck(valueInSamples, sampleRate, frameRateNumerator, frameRateDenominator) { //let numberOfFrames = valueInSamples * frameRateNumerator / (sampleRate * frameRateDenominator); let remainder = (valueInSamples * frameRateNumerator) % (sampleRate * frameRateDenominator); return remainder == 0; } log(frameBoundaryCheck(valueInSamples, sampleRate, frameRateNumerator, frameRateDenominator));
Dustin Harris @Dustin_Harris
Thank you for all of that! I may have stumbled onto a useable method based on your math findings!!
There is rounding present at the sub-sample level, and this is either just how the math works out with .1% speed changes, or there is error introduced in the number of decimal places the system can represent for 24000/1001. Either way, this method rounds to the nearest sample and is providing a reliable result at 44.1/23.98. I'm going to build the model for all sample rates and frame rates and see how the system responds :)
let sampleRate = 44100; let frameRate = 29.97; let valueInSamples = sf.ui.proTools.selectionGetInSamples().selectionStart function frameBoundaryCheck(valueInSamples, sampleRate) { let frameRateNumerator = frameRate * 1000 let frameRateDenominator = 1000 if (Math.floor(frameRate) != frameRate) { frameRateNumerator = Math.ceil(frameRate) * 1000 frameRateDenominator = 1001 } let divisionFactor = (sampleRate / (frameRateNumerator / frameRateDenominator)) let remainder = (valueInSamples % divisionFactor) //alert(`Division Factor is ${divisionFactor}\nRemainder is ${remainder}`) if (divisionFactor - remainder < 1) { return true; } else if (remainder < 1) { return true; } else { return false; } } let result = frameBoundaryCheck(valueInSamples, sampleRate) alert(`${result}`)
EDIT: changed script to parse the incoming framerate to calculate the frameRateNumerator and frameRateDenominator variables automatically.
Christian Scheuer @chrscheuer2020-09-12 21:50:57.832Z
Gotcha. The reason why I refactored the modulo like this was precisely to deal with rounding errors (by only having one division/modulo):
let remainder = (valueInSamples * frameRateNumerator) % (sampleRate * frameRateDenominator);
In your code, by having essentially 4 divisions, you're losing much more precision.
Dustin Harris @Dustin_Harris
I'll come clean: I was getting 'undefined' returned from one of the variables in my script when I copied your math, because I made a mistake in my own code... so I sort of went back to the beginning to check and the extra steps added up to more error :) I'll adopt your more precise method.
Edit: I tried
let remainder = (valueInSamples * frameRateNumerator) % (sampleRate * frameRateDenominator);
in the script but for some reason I was getting large remainder values. The way I did the math resulted in subsample remainders in a way that I understood easily, and the precision appeared to be higher than needed to make the script work, so I kept it. But absolutely tell me if I misunderstood the whole idea/philosophy to your equation @chrscheuer :)Dustin Harris @Dustin_Harris
This is the use I had for the idea implemented:
This script takes a selection in pro tools, checks if the selection is on frame boundaries, and if not, extends the selection to the nearest frame boundaries before the selection start and after the selection end.
Comments and suggestions welcome! :)
function getSessionInfo() { // Gets sample rate, bit rate, frame rate if (!sf.ui.proTools.windows.whoseTitle.is('Session Setup').exists) { //opens session setup window sf.ui.proTools.menuClick({ menuPath: ["Setup", "Session"] }); } let sessionSetupWin = sf.ui.proTools.windows.whoseTitle.is('Session Setup').first; let sessionInfo = { //stores sample rate as a number, sample rate as a string name, frame rate, and bit depth as an object called sessionInfo sampleRateName: sessionSetupWin.groups.whoseTitle.is('Session Format').first.children.whoseRole.is("AXStaticText").allItems[2].value.value, sampleRateNumber: (+(sessionSetupWin.groups.whoseTitle.is('Session Format').first.children.whoseRole.is("AXStaticText").allItems[2].value.value.split(" ")[0]) * 1000), bitRate: sf.ui.proTools.windows.whoseTitle.is('Session Setup').first.groups.whoseTitle.is('Session Format').first.popupButtons.allItems[1].value.value, frameRate: sf.ui.proTools.windows.whoseTitle.is('Session Setup').first.groups.whoseTitle.is('Session Format').first.popupButtons.allItems[2].value.value.split(" ")[0] }; sessionSetupWin.windowClose(); return sessionInfo; } function frameBoundaryCheck(sessionInfo, valueInSamples) { //uses math to check if a selected sample coincides with a frame boundary with the session's sample rate and frame rate let sampleRate = sessionInfo.sampleRateNumber; let frameRate = +(sessionInfo.frameRate); let frameRateNumerator = frameRate * 1000; let frameRateDenominator = 1000; if (Math.floor(frameRate) != frameRate) { frameRateNumerator = Math.ceil(frameRate) * 1000; frameRateDenominator = 1001; } let divisionFactor = (sampleRate / (frameRateNumerator / frameRateDenominator)); let remainder = (valueInSamples % divisionFactor); let multiplier = (valueInSamples / divisionFactor); let leftEdge = Math.round(Math.floor(multiplier) * divisionFactor); let rightEdge = Math.round(Math.ceil(multiplier) * divisionFactor); let isFrameEdge = false; if (divisionFactor - remainder < 1) { isFrameEdge = true; } else if (remainder < 1) { isFrameEdge = true; } let result = { isFrameEdge: isFrameEdge, leftEdge: leftEdge, rightEdge: rightEdge, }; return result; } ////MAIN//// function main() { //Get the session info in Pro Tools sf.ui.proTools.appActivateMainWindow(); let sessionInfo = getSessionInfo(); //get the timeline selection in samples let timelineSelection = sf.ui.proTools.selectionGetInSamples(); //check if the selection start is on a frame edge, and if not, extend the selection start to the closest frame edge before the selection let checkSelectionStart = frameBoundaryCheck(sessionInfo, timelineSelection.selectionStart); if (checkSelectionStart.isFrameEdge != true) { timelineSelection.selectionStart = checkSelectionStart.leftEdge; } //check if the selection end is on a frame edge, and if not, extend the selection end to the closest frame edge after the selection let checkSelectionEnd = frameBoundaryCheck(sessionInfo, timelineSelection.selectionEnd); if (checkSelectionEnd.isFrameEdge != true) { timelineSelection.selectionEnd = checkSelectionEnd.rightEdge; } //apply the new selection with the boundaries set to frame edges sf.ui.proTools.selectionSetInSamples({ selectionStart: timelineSelection.selectionStart, selectionEnd: timelineSelection.selectionEnd }) } main();
Christian Scheuer @chrscheuer2020-09-13 23:18:07.730Z
Ahh... I think I would've just stayed in timecode and read what the current selection is and reentered it, forcing the selection onto frame boundaries.
But then we wouldn't have learned all the math ;)Dustin Harris @Dustin_Harris
Is there a way to do it in timecode? like reading subframes?
EDIT: OOHHH... by reentering it you're assigning .00 to the subframes by default! Yeah... that is a much easier way to do it.
Christian Scheuer @chrscheuer2020-09-14 08:41:42.119Z
Yea exactly. And if PT wouldn't force it onto .00 subframes since you were reentering an identical value (they may have a special case for that), then I'd just collapse the selection first (while remembering what to type in) before reentering it.