If the parts are close you may be able to nurse them on from within the fan housing.
Remove the components from within the fan housing and see if you can grab the duct from the inside and pull it to the fan.
But part of this depends on what the duct material is. Does it have the rigidity to hold shape and work with or is it the flexible hose?
Nothing waisted in trying this before cutting drywall