For both, it is very likely that a wax or varnish was applied to them over the years.In order to paint them, you have to remove it or the paint will not adhere to the wood and thus, peel off.
Right now, I am doing some landscape in my backyard with ferns from the woods and other greenery that is perennial.
I know from experience that this is a lot of work but painter's caulk is inexpensive and so is sand paper and paint.
It is also a lot less expensive then replacing the paneling.
The money you save could then be used to help fix up the outside.
Deb Related: How to Prep Siding for Painting As far as the outside, you could start by putting a front porch the length of the home, if you can, with a roof on it and plant lots of flowers around it.