I used to feel like this, too. Some of my close friends are utterly lazy and unambitious. I became the same way when I'm around them, so I unconsciously distanced myself.
"I want to improve, but they're dragging me down."
That's what I thought at the time. And it's true to some extent. Some part of me may even still believe it until today.
But, then, despite their shortcomings, my friends are precious to me. I can't deny that.
I don't know. Maybe it's okay to retreat from our friends every now and then. But, yeah, I agree we should never burn bridges.