Try to find a solvent labeled as a mirror mastic remover. Spray it on and wait a bit. Then use a 5 in 1 glazing tool to scrape the wall.
Worse comes to worse, you'll just need to make several small drywall repairs. For textured walls like that, I prefer to use vinyl spackling. It doesn't shrink and you can use your drywall knife to swirl the texture, so you can fix the wall and put in the texture all in one step.