How soon can a rigorous exercise program be restarted?

The standard answer for many surgical procedures, nasal or otherwise, is six weeks. Contact sports should be avoided at least that long.