The landmarks come first, and absence is recorded
Every frame is searched for the points that describe how a horse is put together: poll, withers, croup, hip, and the limb chain down to the hoof. Where a joint is not clearly visible in a frame, that frame records it as absent. It is never filled in with a guess, because a guessed joint becomes a real-looking number three steps later and nobody can tell which one it was.