X-rays and other medical scans do create significant short-term exposure levels to radiation. However, radiation is everywhere and whether the amount you get from medical may be small or large compared to your background exposure. For instance, if you live at a high elevation or do a lot of air travel for business, you are exposed to much higher levels than someone who lives at sealevel and/or does no business travel at all. Then there is ground source radiation (depending on your soil), radon, etc.
Typically, ingestion or inhalation of a radioactive material is worse than just being exposed to an exterior source. When you intake it into your body, it can stay with you. For instance, Plutonium behaves like Calcium, so if you ate something that had Pu in it, your body thinks it is Calcium and puts it into your bones. As you can imagine, this material will probably stay with you for life. Other things (like heavy water), will quickly move through your body (since it behave like normal water). If you are exposed to something like this, you drink lots of liquids to flush it out more quickly.
In the nuclear field, everyone wears dosimetry (measures dose) if you are working anywhere that could see increased radiation levels. These are tracked and a report (usually annually) is given to you showing what your exposure is over that year. The trick is that there are many kinds of radiation and many ways to measure it. One way is to look at a count of how many particles hit you (something like a Geiger counter just measures how much stuff is flying around). That only goes so far as some particles can only make it a short distance through the air and aren't even strong enough to make it past your skin. So, then you look at what can make it through the air. After this, you have to consider how much of what hits you is actually absorbed by your body (x-rays and gamma rays can go right through you (how an x-ray works), so not all of the energy that hits you is actually picked up by your body. The 4th thing is dose equivalent. The thought here is that different organs have different sensitivities to radiation, so it depends on how much radiation of what type hits where.
As you can imagine, dose equivalent is the most important because it links the radiation to possible medical effects. This is very different to measure, though. We don't have sensors in our organs to detect this, so it is mostly an educated guess how much actually hits that organ and how much damage it does. There are weighting factors (Quality Factor) that try to account for the sensitivity to different radiation types to different organs, but those factors might not be really accurate either.
We also don't really have good information that relates radiation levels to medical effects for lower levels of radiation. Most of the data we have is for large exposures (Atomic bomb suurvivors, etc.). So, the data we have says:
For these very high levels of radiation = dead in so many days. For really high levels (but not quite high enough to kill right away) = x% chance of getting fatal cancer, etc. in some amount of time. For more normal levels, we don't really know. If you are exposed to x amount of radiation, we can't say for sure that you'll get cancer. And if you do get cancer, we don't know if it was this radiation or if you would get it anyway. That is just the way cancer works. Some people who never smoke get lung cancer and some who smoke daily for 20 years never get it. Certain things can increase the probability that you will get cancer, but no one knows what probability they started out with when then were born. Some are more likely to get it than others.
In the nuclear field, we follow what is called ALARA (As Low As Reasonably Achievable). We can't avoid radiation altogether, but we do everything possible to minimize exposure (sheilding, getting the job done quickly to get out of there, etc.). If you have medical issues and may require mulitple scans, then you should follow this idea. Limit the scans if possible, use lower exposure types of scans (if possible), etc.