Meaning of Gandalf's quote "There never was much hope. Just a fool's hope"?

But it's only a fool's errand.

This is it....something so impossible that only a fool would think it even worth doing, only a bigger fool would attempt it....and only a fool would hope it could work.

The fool's hope here is the taking of the ring to Mordor for destruction.

Frodo’s mission is impossible. Even if the Fellowship had not been broken at the Falls of Rauros and Aragorn and Boromir, Legolas and Gimli had been at Frodo’s side on the journey to Mount Doom it would have remained impossible.

...and that's why it's a pivotal point of the story.

This short but very illuminating scene from the final Lord Of The Rings trilogy, “Return Of The King” hits an important theme, hope. When the young hobbit asks the ever wise Gandalf if there was any hope, he was initially shocked to hear the wizard tell him that there never really was much hope to begin with. And then he follows up by laying out the most profound portion of this speech/scene….”Just a fool’s hope”. The reason this moment serves as a pivotal part of the movie is because Gandalf refers to a “fool” as someone who technically is not “too smart for his own good” type of individual – an ironic characteristic the hobbit always had, that wasn’t necessarily ever really a bad thing. It’s often the ones who have this blind faith, because they don’t know any better

^source

Just to add to the answer from @Paulie_D - the 'fool' Gandalf is talking about here is himself. He is the one who dreamed up the scheme and hoped it would work.

I'm not sure about the books. Maybe it's too obvious and goes without saying but from the movie I always thought he was making comforting/affectionate jab at Pippin, whom he is always admonishing as "Fool of a Took."