For the section with the stairs right up against the foundation Vegas_sparky's solution is about as good as you can do.
But for the rest, in a Columbus OH climate it's
well worth insulating the wall rather than just drywalling it in. A CMU wall is worth at-best R3 (on a good day, with hollow cores and no wind, counting on the interior & exterior air films). Add to that the air films created by the sheet rock plus the sheet rock itself brings that to maybe R4 at best, which is a really CRUMMY R value, with a fairly high risk of condensation and mold in the cavity between the gypsum & CMU. When finishing a basement room it's an opportunity moment- if you don't insulated it then, you won't ever, unless it eventually turns into a mold farm or the furring rots. But adding the insulation NOW prevents those conditions.
Even 1" of EPS would roughly double that R-value, but 1.5" of foil-faced polyiso (R9-10) trapped to the wall with 1x furring would be even better, delivering about R12 performance due to the foil facer in the cavity. Subtracting off the thermal bridging of the furring & TapCons, and adding on the R-value of the CMU it performs at about R13-R14, which is about what a 2x6 wall with R19s performs at with all thermal bridging accounted for.
If using polyiso it's best to keep the bottom edge of the foam off the slab, since that type of foam will wick moisture, reducing it's performance. (EPS is fine resting on the slab.) Tape the seams with 2" FSK tape (purpose made aluminum duct tape) and seal the foam to the CMU with can-foam. If EPS, tape the seams with housewrap tape, then trowel on duct mastic over the tape to ensure the edges don't peel off the EPS over time.
It's also worth air sealing & insulating the band joist and foundation sill before you close it all in, if it isn't already. The air leakage between the foundation sill and CMU is usually a lot higher than you would expect just looking at it- more than all the window & door leakage in the rest of the house combined in most houses, a really long-skinny hole with a big cross sectional area.
To meet
IRC 2012 code min for Columbus (US climate zone 5A) the foundation would need R15 continuous insulation, but the CMU adds a bit, as well as the air-gap with the foil, so you'd be close. If you wanted to take it the whole way to that code min, 2" foil faced polyiso with a 3/4" air gap between the foil & gypsum is pretty close R15. With 3/4" thick furring that would take 3.5" TapCons 24" o.c. (or tighter, if the wall would look too wavy) to mount the foam & furring, which are available at box stores. Any thicker on the foam and it's sometimes hard to find the right screws. With thinner foam you can use shorter screws, of course, but try to get at least 3/4" screw penetration into the CMU.
With an insulated wall the wall temps will be warmer & more comfortable, and there is no condensation/adsorption potential on the furring or paper backing to the wall board. Even 1" EPS (R4) would mitigate the mold issue, but from a comfort and energy use point of view it's well worth taking it to at least R8 continuous insulation, if not the whole R15.