Help with multiple object tracking example

I have a problem with an example code for multiple object tracking for computer vision library.

I added more things on top of that example to track velocities of each object. The script used to work for all of my videos, but it doesn't for another video that I have. The script detects objects, but can't focus on them. It 'jumps' from one object to another. I'm not sure why that happens. I'm not an expert on computer vision and I'm wondering if I need to adjust some parameters form the original example. I didn't touch any of the original code.

Thanks!