If you want to do a complete, nice job, and spend a little mula doing it, get a spray foam contractor to come out and fill the joist bays in the cantilever, and while they're there, also spray your entire basement rim joist, possibly your attic, etc if you see fit. Its going to be prohibitive to bring them out for a small job, but if you wanted to take care of a bunch of things at once, closed cell spray foam is the way to do it.
A DIY smaller scale solution would be the foam panels, cut almost to width. You're actually better off leaving about 1/4" on all sides, then use a can of Great Stuff to fill the remaining space. This will give you a better seal than you'll likely be able to do trying to do a tight fit with the panels. Cut out around the outlet box in the foam, then spray around it as well. For your duct, you might just want to push it up as high as you can, put some fiberglass or something up around the sides of it, and then install a couple panels of foam underneath it.
You can go too crazy here... the walls of your house are likely R19 at most, so putting 10 inches of foam in the floor (which loses much less heat than the walls/ceilings) would just be crazy.
If you want a bit more warmth and your basement is reasonably warm, you could take out the 2x10s closing off the openings between inside and out (make sure they're not in any way structural first), then install your first foam panel fitting tightly and leave a couple inches gap between it and the floor so that some warm air can get in there. But if your basement is fairly cool, this will not help at all.