Some of the sludge could be from biological byproducts (i.e., stuff growing in the water). You may need to sanitize the pipes. The incoming water may need chlorine treatment. Does the water come from a well, or town water supply? If it is from a town, they (at least in the USA - don't know about Canada) are required to do periodic water tests and make the results available to anyone who asks. If it is a well, then it is up to the owner, after the initial home inspection when the house was built. I don't think that I'd drink the water until it can be proven safe, and even then, it sounds like it doesn't have the greatest taste, so may require some treatment or continued bottled water for at least drinking and cooking.