If you haven't already, slice your banana bread into ¼ inch slices. Set aside.
9x5 banana bread
In a large bowl, add the cold milk and sweetened condensed milk and stir to combine. The mixture will be slightly clumpy and thin — that's okay! Sprinkle the vanilla pudding mix over the top and whisk together until smooth, about 2-3 minutes. I prefer a hand whisk over a hand mixer to avoid clumping. Let the mixture sit for 5-10 minutes to thicken. While it sets, make your whipped cream.
In the bowl of a stand mixer fitted with the whisk attachment, or using a handheld mixer, add the heavy cream, powdered sugar, and vanilla extract. Beat until stiff peaks form. If you over-beat and it looks grainy, add a tablespoon or two of heavy cream and gently mix to smooth it out.
With a rubber spatula, gently fold the whipped cream into the pudding mixture. Place in the fridge for 1 hour.
To assemble, use a 9x13 casserole dish or a trifle bowl. Start with a thin layer of pudding on the bottom, then add a layer of sliced banana bread, followed by a layer of sliced bananas. Repeat until you have 2–3 layers total. Finish with a final layer of pudding that fully covers everything, making sure to seal the edges so the banana bread doesn't dry out.
6 to 8 bananas
Gently cover with plastic wrap and refrigerate for at least 6 hours and up to 24 hours to allow the banana bread to soften.
When ready to serve, remove the plastic wrap. Top with any remaining banana bread pieces and fresh banana slices. Serve cold!