Comfortable headphones for sleep.

I would say in-ear earphones would probably be the best if you plan on wearing all night. But your milage will vary so much compared to everyone else, it's almost impossible to suggest a pair that would be guaranteed to be comfortable for you. I personally just use samsung earbuds and get through a night just fine with them in. I've also used over-head sony headphones but they get a bit much after a night or 2.

Have someone read to you, then you can still fall asleep.

Are you sleeping on your own? if so just buy an echo dot or something similar.

I tend to use my galaxy live buds under my sleep mask but it does have thick bands going around my head. Can be fun finding them in the morning sometimes if it slips but i don't listen to audio books that often when going to sleep these days.

Pillow speaker?

Skullcandy Jib. You can get both wired and wireless pairs, and are actually VERY cheap to buy.

They are in-ear but since they are so small they comfortably pop in like ear plugs and work fine even if you roll onto your side because there’s almost nothing sticking out.

If you go wireless the two are still attached by a short wire to a little box that houses the battery and Bluetooth receiver. Unfortunately that’s the trade-off for tiny earbuds that don’t stick out of your ears.