What can I use to fill holes around radiator pipes?

I've taken the collars off our radiator pipes at the point where they enter the wall while redecorating and found that some of them have quite large holes surrounding the pipe.

What can I use to fill these holes? I have Soudal (gap filler) expanding foam but not sure if that will be OK with the heat? Is silicone or caulk a better option?