Line 9 x 13-inch pan with foil. Set aside. Combine the first 3 ingredients in a small saucepan; stir over low heat until melted and smooth. Spread into prepared pan. Refrigerate until set.
Filling
For filling, melt butter in a heavy saucepan over medium-high heat. Add sugar and milk; bring to a gentle boil. Reduce heat to medium-low; boil and stir for 5 minutes. Remove from heat. Stir in marshmallow creme, peanut butter, and vanilla. Add peanuts. Spread over first layer. Refrigerate until set.
Caramel Layer
Combine caramels and cream in a saucepan; stir over low heat until melted and smooth. Pour over the marshmallow layer. Refrigerate for at least 4 hours or overnight.
Icing
Melt 3 ingredients in a small saucepan; stir over low heat until melted and smooth. Spread over chilled caramel layer. Refrigerate until set.
Remove from refrigerator 20 minutes before cutting. Remove from pan and cut into 1-inch squares. Yields about 8 dozen.
Notes
These bars freeze well. I suggest cutting into desired bars before freezing them.