As you can guess, this results in a guaranteed waiting period to send that change through our pipeline even if an engineer has only made a small change.
Curious as to what other folks do to make this more efficient?
I'm working on a tool in my spare time to try and address this problem but I don't have a huge amount of time to work on it (https://github.com/colwill/ccc); it performs basic static analysis and code caching using tree-sitter and calculates the subset of test required.
This output is passed to the next step in the pipeline which runs that subset of tests.