I’m a beginner who has learned a lot from GBA. I’m trying to figure out how to improve air-sealing in my basement.

I have a 1912 “1.5 story” house in Zone 6 (Wisconsin). It is balloon framed. The basement has cement wall below grade. The studs sit directly on the sill plate and the floor joists that hold up the first floor sit a few feet above the sill plate. So there are stud walls on all sides of the basement.

At some time multiple owners ago, the walls were filled with cellulose, covered with cardboard-ish wall paneling (pic1), and the sill sealed with mortar(? pic2). This is clearly not airtight as evidenced by the blackened cellulose along edges and in other places, though the cellulose does seem to be slowing airflow. There are a few places where there is no insulation (pic3) mostly near to windows, so you can see what is behind the wall cavity. Is this sheathing or is that the back of the siding?

What course of action would you recommend? Two options seem apparent to me:

1) Remove the wall paneling but leave the cellulose and recover with drywall or maybe Thermax (so that it can be left exposed) and try to detail the Thermax so that it is as well sealed as possible. This may still allow air go up the wall studs, although at least where I took the paneling off there seems to be some blocking (? pic4).
2) Replace paneling and cellulose with rigid foam sealed with canned foam and mineral wool batt. This would waste the cellulose (though I could try to bag it and transport it to the attic). If I did this would I need to leave an air gap between the rigid foam and whats behind it? I’m not sure if what I am seeing is siding or some sort of sheathing (pic3).

Alternatively, I could remove the paneling and cellulose and have it spray foamed. I’d appreciate any/all suggestions!