A tankless system may or may not have a modulating burner - some do, some don't. If it does have one, it takes it a little time to adjust (not much, but it does), if it doesn't, then the temperature of the outlet will change, sometimes radically, when the volume through it changes. Think about a pan of water on the stove. Put an 1/8-cup in the pan and let it sit for a minute on a hot pan, it may be boiling. Now, redo it and put in a quart of water. After a minute, it will be hotter, but no way as much.
In the shower, you might be much happier changing the shower control valve to a temperature controlled one. But, even those can take up to a second to follow the and respond to the change in water temperature.
I think what's happening, is any use of hot water elsewhere is not being responded to quick enough by the tankless system, and the temperature of the shower drops because the temperature of the hot water drops.
Some tankless systems are better than others. They work best as single use devices, some are okay for multipoint use.