Use a wye instead of a tee to reduce backdraft. Also, most fans have an integral backdraft damper. If you have a 4" duct on the toilet fan, that is adquate for even a very large CFM bath fan.
Fans and lights directly over a tub or shower are required to be UL listed for that specific application. This may limit your choices.
In any event, such a fan must be connected to a GFCI protected circuit.