Someone was helping us remodel and used thinset as the base and mortar. The pavers are set and we are trying to clean off the excess thinset. Now we realize they should have used mortar. Grooves look good but there is splatters of thinset. Also it is milky on the brick where some had been wiped when wet.

Ideas? Inside house.