I understood "wall" to be my finished wall, but I believe you are saying it refers to the outer concrete wall. In that case, I would be in the yellow area.
So lets figure out how you might do that.
First, how close can you get a 2 inch pipe to the concrete wall? If the pipe is straight between two fittings, looking at
https://www.aetnaplastics.com/site_media/media/attachments/aetna_product_aetnaproduct/204/PVC Sch 40 Fittings Dimensions.pdf the fittings would be dimension M =2.75 at the thickest, and the pipe OD would be 2.375 inches. If the pipe is straight, that puts the inside of the pipe at 2.5625 inches from the wall. That would correspond to the minimum between the back of the breaker box and the concrete. If the pipe between the fittings could be bent toward the wall, we could reduce that needed space to 2.375. How to move the pipe side to the wall? Bend the pipe with heat, or stress it with clamps to bend it that little bit (0.1875 inch). So picking one of those numbers, we would see how much away from the wall the breaker panel needs to move.
The breaker box is probably held on with screws into the plywood. I would figure out how much more thickness I would need to add, and insert at least that much more plywood. Then screw it all together.
Before trying to slide the new wood into place, you would remove any cable clamps tying the NM too closely to the wall. You would unscrew the box, slide the new wood into place, screw into place, and replace any cable clamps needed.
I have not accounted for the wall perhaps not being completely vertical. So adding some margin would be good.
My comments on implementing this are the way I imagine it, and are not based on relevant experience.